Machine learning advances tend to evolve in bursts. Researchers publish a new paper with a newly discovered technique. It launches the industry forward & more researchers rapidly iterate to improve it further.

Progress looks like this - a series of aS curves one after another.

No one knows the time period between the rapid progress or the slope of the curves or how much progress we’ll make during one of these curves.

In the last few years, since the release of Attention is All You Need Paper, AI has boomed. The rapid pace of iteration has led to the release of model after model.

A collection of human evaluators marks our progress using a score called Elo collected by LMSyS, modeled after chess grandmaster rankings. If we plot Elo by Rank, we see an S-curve.

And if you look very closely at the upper right, which indicates the performance of the most recent models : GPT-4o, Claude 3.5 Sonnet, Gemini Pro, & Llama 3 - if you squint, perhaps you can imagine another elbow, another rapid advance of progress in performance.

OpenAI categorizes levels of AI similarly to self-driving cars. The five levels are

OpenAI has also said it’s close to moving from Level 1 to Level 2. What if that little orange elbow is the beginning of reasoning in AI?

In some software categories, first mover advantage exists. In search, last mover advantage (Google) won because they benefitted from the learnings of all who came before.

AI is characterized by waves of innovation & sudden change, a tecnological punctuated equilibrium.

Will the winners in this era be those who started early before the rapid advance or those who waited until afterwards?