The Exploration-Exploitation Dilemma

Imagine that after a crazy urge to gamble all your life savings you enter into a casino. You have in front of you a line of slot machines. To be capable of becoming a millionaire you should play the one that has the highest probability of winning, but obviously, you don’t know which one is the right one. So you need to try all of them an indefinite number of times to determine which one’s better. If you play too much in each slot you can end up wasting all your money in losing machines. If you decide to stay in one that seems the one after a few tries, but it isn’t, you’ll end up losing all your money to it.

This is a probabilistic problem also known as the exploration-exploitation dilemma. Each time you play you need to choose between repeating what you’ve done till now or to try out new movements that might get you better results.

This is one of the biggest problems you need to face when learning anything. When you’ve tried something that works, should you continue doing it and simply stay there? Or it would be better to try to discover new things, knowing it could be a waste of time or it could also get you to a better solution?

This kind of dichotomy is present in multiple subjects of life, not only when learning. The called mid-life crisis reflects the urge to explore new things after many years of exploiting what you know. Should you stay where you are or play something new? Continue working for your company or create a new one? Stay in your country or travel the world?

In the business world you need to question constantly if you should explore new things or exploit the ones that you know. With the available budget, what is the best strategy to distribute resources between the new projects of investigation, whose probabilities of generating benefits are yet unknown in large part?

Exploration works with the dynamic efficiency, searching new options, experimenting and carrying out investigations. Exploitation works with the static efficiency, polishing current processes, doing the same things but better, and getting the most out of the already known.

Of course there are several studies and mathematical theorems that try to model the problem and establish the optimal solution, like the Gittins index, but we’re not going to get that much into detail.

Sometimes the decision to explore or to exploit is really easy to make, but normally it hides small details that we don’t notice. What it’s clear is that determining the perfect equilibrium between exploration and exploitation is a challenge. If we don’t explore, the development gets stuck and results will be limited to the ones we know. And over-exploring usually ends up being counterproductive, because the resources at hand won’t be exploited enough.

So how is your personal productivity going? Does it works well? Do you need to refine your current method? Or do you think there must be something better out there that would be worth a try? What a dilemma!