A short history of AI

docs.scrimba.com

When ChatGPT arrived at the end of 2022, it felt like AI had appeared overnight. One week almost nobody you knew was talking to a computer in plain language; the next, everyone was. It is a tempting story, and it is also wrong in a way that matters, because it hides where these systems came from.

The chat box was new. The science under it was not. What you are using is the visible tip of roughly seventy years of work, a stack of ideas where each one broke the ceiling the last one hit. Knowing that road is not trivia. It tells you which parts of this field are durable and which are fashion, which is the difference between chasing every headline and understanding what you build on.

Telling computers the rules

The field has a birthday. In 1956 a small group of researchers met at Dartmouth College for a now-famous summer workshop and gave the idea its name: artificial intelligence. A few years earlier, in 1950, Alan Turing, the British mathematician who helped invent computing itself, had already posed the question that still hangs over the field, whether a machine could ever convince you it was thinking (an idea now known as the Turing test).

The first answer ran for decades and rested on a sensible-sounding idea: if you want a computer to act intelligent, write down the rules of intelligence by hand. Experts spelled out their knowledge as long lists of if-this-then-that, and the programs that ran them were called expert systems. For narrow, tidy problems, like diagnosing a specific kind of illness or configuring a machine to order, they worked.

Then they hit a wall, and that wall is the lesson of the era. The real world has too many rules, and most of them nobody can state. You tell a cat from a dog in an instant, but try to write the exact rule that separates them and you will be at it all week and still miss cases. Hand-coded knowledge does not scale to the mess of reality, and breaking that ceiling is what the next idea was for.

JunoTelling computers the rules So AI isn't new at all, the name dates to 1956, and Turing asked the big question back in 1950. The first idea was the natural one: if you want a computer to act smart, write down all the rules by hand. It worked for tidy little problems and fell apart on real ones, because try writing the exact rule for telling a cat from a dog. You can't, and that's the whole lesson here.

JunoTelling computers the rules The takeaway from the early decades: people tried to hand-write intelligence as if-then rules, the expert systems. Fine for narrow, tidy problems, useless for messy ones. The reason matters because it keeps coming back: most real-world knowledge is too vast and too fuzzy to spell out by hand. That wall is what every later idea was built to get past.

JunoTelling computers the rules AI got its name in 1956, on top of a question Turing posed in 1950, so anyone calling it brand new hasn't read much. The first approach was to write the rules of intelligence by hand. It held up on toy problems and collapsed on real ones, for the reason it always does: you cannot hand-write the messy parts of the world. Remember that the next time someone promises they've got it all mapped out.

Letting them learn instead

If you cannot write the rules, the alternative is to let the machine find them. Instead of telling a computer how to tell a cat from a dog, you show it thousands of labelled examples and let it work out the pattern itself. This is machine learning, and it flips the job: you stop programming the answer and start supplying the data, then let the program infer the rule.

The path there was not smooth. Twice the field promised far more than it could deliver, the money dried up, and progress stalled for years at a stretch. Those stretches are remembered as the AI winters, and they are the sharpest warning the history offers. AI has been overhyped before, more than once, and each cycle felt as certain as this one does. The technique that paid off in the end was rarely the one making the loudest promises at the time.

JunoLetting them learn instead Here's the clever flip: instead of writing the rules, you show the machine loads of examples and let it work out the pattern itself. That's machine learning. The road there wasn't smooth though, there were these long cold stretches called the AI winters where everyone got overexcited, the money dried up, and progress stalled. Worth remembering when this all feels too certain: it has felt certain before.

JunoLetting them learn instead Machine learning flips the job: stop writing the answer, supply the data, let the program infer the rule. Useful frame to carry. But the history has a warning baked in, the AI winters, two stretches where the field overpromised and funding collapsed. AI has been overhyped before, more than once, and the loudest promise at the time was rarely the technique that paid off.

JunoLetting them learn instead If you can't write the rules, let the machine find them from examples. That's the pivot, and it's a good one. What I'd actually have you keep is the other half: the AI winters. Twice the field promised the moon, the money evaporated, and everything froze for years. Each cycle felt as inevitable as this one does. I stopped trusting the loudest voice in the room for exactly that reason.

Deep learning breaks through

Early machine learning still leaned on people to hand-pick which features of the data mattered, its own smaller version of the hand-coding problem. The break came from an old idea finally given what it needed: neural networks, loosely inspired by how brain cells pass signals, that learn their own features straight from raw data.

That idea had been around since the 1950s and had never quite worked well enough. Two things changed around 2012. There was suddenly enough data, from a digitised internet, and enough cheap computing power, from the graphics chips built for video games, to train these networks at a size that paid off. A neural network (AlexNet) won a major image-recognition contest by a wide margin, and the field reorganised around the approach almost at once. This is deep learning, the engine under everything that followed.

The shape of that breakthrough repeats often enough to name: the idea was not new, the data and the compute were. A lot of AI progress looks like a sudden leap and is really an old idea meeting the resources it had been waiting for.

JunoDeep learning breaks through Neural networks are loosely inspired by how brain cells pass signals, and the neat thing is they figure out for themselves which bits of the data matter. The idea has been around since the 1950s, but it only really worked around 2012, once there was enough data and cheap computing power to train it properly. One network crushed an image contest and the whole field turned on a dime. Old idea, new fuel, a pattern you'll see again.

JunoDeep learning breaks through Deep learning's trick: neural networks learn their own features from raw data instead of having people hand-pick them. It clicked around 2012 when two things showed up, a digitised internet's worth of data and cheap GPU compute from gaming chips, and a network won a big image contest by a wide margin. The lesson to bank: a lot of "sudden" AI progress is an old idea finally meeting the resources it was waiting on.

JunoDeep learning breaks through Neural networks date to the 1950s and sat there not-quite-working for half a century. Then around 2012 the data and the cheap compute finally caught up, one model won an image contest by an embarrassing margin, and everyone rediscovered the idea overnight. That's the shape of most "breakthroughs" I've watched: the idea was old, the resources were new. Keeps me from getting starstruck by the demos.

The transformer

Deep learning was strong on images. Language was harder, because meaning depends on how words relate across a whole sentence or paragraph, and earlier networks read text one word at a time and lost the thread over any distance.

In 2017 a research paper introduced a design called the transformer, built around a mechanism called attention: instead of reading word by word, it weighs how every word relates to every other word at once. That sounds like a small architectural detail. It was the unlock. A transformer trains efficiently on enormous amounts of text, and it kept improving the more text and the bigger the network it was given. That last property, that scale kept paying off instead of flattening out, set up everything since.

Every model in this handbook, and the chat box behind tools like ChatGPT, is a transformer at heart. When you reach How LLMs work, the prediction loop you learn there is this architecture doing its one job.

JunoThe transformer Language was the hard one, because meaning depends on how words connect across a whole sentence, and older networks read one word at a time and lost the thread. Then in 2017 the transformer came along, with a trick called attention that looks at how every word relates to every other word all at once. That was the unlock. Every model you'll meet, the chat box included, is a transformer underneath. Pretty wild, right?

JunoThe transformer The 2017 transformer is the architecture under everything you'll touch here. Its move is attention: weigh how every word relates to every other word at once, instead of reading left to right and forgetting. The part that actually mattered is that it kept getting better the more text and compute you threw at it, instead of flattening out. That "scale keeps paying" property set up the whole era.

JunoThe transformer Images were solved; language wasn't, because meaning lives in how words relate across distance and the old networks read one word at a time. The 2017 transformer fixed that with attention, weighing every word against every other at once. Sounds like a footnote, was the unlock. The property that changed the game was dull on paper: it kept improving as you scaled it. Everything since is that one idea, scaled until it got expensive.

The LLM era and the ChatGPT moment

With the transformer in hand, one question took over: what happens if you make it bigger and feed it more? The answer, repeated across a line of models from 2018 on, was that it kept growing more capable in ways nobody had programmed by hand. Trained to do nothing but predict the next word across most of the public internet, these models picked up grammar, facts, translation, and a rough kind of reasoning as side effects of that single task. These are large language models, and the next chapter is about how that prediction actually works.

So why did it feel sudden in late 2022? Because what most people met was not a new model, it was a new door. The underlying ability had been building for years inside research labs and developer tools. ChatGPT wrapped it in a plain chat box that anyone could open, free, with no setup. The technology had been arriving for a decade; the interface arrived all at once, and that is the part the public lived as the beginning.

JunoThe LLM era and the ChatGPT moment Make the transformer bigger, feed it most of the internet, and something surprising happens: from nothing but learning to predict the next word, it picks up grammar, facts, even a rough kind of reasoning, all on its own. Those are large language models. And ChatGPT in 2022? That wasn't a new invention, it was a new door, a plain chat box that finally let everyone in. The tech had been arriving for a decade; the door opened all at once.

JunoThe LLM era and the ChatGPT moment Scale the transformer up, train it to predict the next word across the public internet, and grammar, facts, and rough reasoning fall out as side effects. That's an LLM. The reframe for late 2022: ChatGPT wasn't new technology, it was a new interface. The capability had been building in labs for years; the plain chat box put it in front of everyone overnight, and that's the part the public experienced as the start.

JunoThe LLM era and the ChatGPT moment Make it bigger, feed it the internet, and a next-word predictor quietly turns into something that writes, translates, and reasons after a fashion. Those are the LLMs. People think the world changed in November 2022. It didn't; the interface did. The ability had been compounding in research and dev tools for years, and ChatGPT wrapped it in a chat box anyone could open. New door, old building.

Standing on decades

Lay the road end to end and the shape is clear. Hand-written rules gave way to machines that learn from data, which gave way to networks that learn their own features, which gave way to the transformer and the discovery that scale keeps paying off. Each step broke the ceiling the step before it hit, across about seventy years. Today's models are the tip of that history, not its start.

This is not a history lesson for its own sake. It is the reason this handbook leads with fundamentals instead of headlines. The field moves in cycles of hype and winter, and the model on top this month will be replaced. What does not get replaced is an understanding of how these systems work, the same way the ideas from each era outlived the products built on them. Respect the road, learn the durable layer, and the next launch becomes something you can place instead of something that knocks you over.

The next chapter, Using AI well, turns from where these systems came from to how to actually work with them: where AI earns a place in your day, where it does not, and how to keep your judgement while the rest of the field loses its head.

JunoStanding on decades Lay it all out and the shape is lovely: hand-written rules gave way to learning from data, then to networks that learn their own features, then to the transformer and scale, each one breaking the ceiling the last one hit, over about seventy years. Today's models are the tip of all that, not the start. That's why we learn how things work here, not the headlines, because the understanding is the part that sticks around.

JunoStanding on decades End to end: rules, then learning from data, then features learned automatically, then the transformer and the discovery that scale keeps paying, each step past the previous wall, across seventy years. Today's models are the tip, not the origin. That's the whole argument for fundamentals over headlines: the model on top this month gets replaced, the understanding of how these systems work doesn't.

JunoStanding on decades Seventy years, four big steps, each one breaking the ceiling before it. Today's models are the tip of that history, not its beginning, whatever the launch posts tell you. I lead with the machine instead of the headlines for one reason: I've watched the product on top get replaced over and over, and the thing that never got replaced was understanding how it works. Learn the durable layer. The rankings are someone else's problem.