Author here. I’ve studied AI for about ten years and worked on professional-development software. One thing I keep noticing is that people learn best when curiosity is driving, but most of us end up learning AI out of worry about falling behind. That’s usually when we stall.
The essay opens with Feynman’s wobbling plate, a cafeteria tangent he chased for fun and later credited for his Nobel work. Then it covers eight threshold concepts, ideas that change how you see everything else once they click, roughly in the order the field found them. Each is paired with a surprising result from when it was discovered, like a 1.3B-parameter model beating the 175B GPT-3 once it was trained with human feedback. There’s also a study prompt you can use with any LLM.
Disclosure: I’m building a learning tool around these ideas, called Zairai. It gets one paragraph at the end.
My biggest lightbulb moment came from working out by hand how a neural net could handle a fixed goal, like deciding whether a piece of text fits a topic. I started with a neural version of bag-of-words, one weight per word, then realized I could encode arbitrary rules in the parameters. After that, “learned from data instead of programmed” stopped feeling abstract.
I’m curious what concept or exercise gave you a lightbulb moment, and what you’d swap into the list. Corrections on the history are welcome too.
Comments
The essay opens with Feynman’s wobbling plate, a cafeteria tangent he chased for fun and later credited for his Nobel work. Then it covers eight threshold concepts, ideas that change how you see everything else once they click, roughly in the order the field found them. Each is paired with a surprising result from when it was discovered, like a 1.3B-parameter model beating the 175B GPT-3 once it was trained with human feedback. There’s also a study prompt you can use with any LLM.
Disclosure: I’m building a learning tool around these ideas, called Zairai. It gets one paragraph at the end.
My biggest lightbulb moment came from working out by hand how a neural net could handle a fixed goal, like deciding whether a piece of text fits a topic. I started with a neural version of bag-of-words, one weight per word, then realized I could encode arbitrary rules in the parameters. After that, “learned from data instead of programmed” stopped feeling abstract.
I’m curious what concept or exercise gave you a lightbulb moment, and what you’d swap into the list. Corrections on the history are welcome too.