Deep Dive into LLMs like ChatGPT by Andrej Karpathy - TL;DW version

I think everything that Andrej Karpathy shares on X or YouTube is a goldmine of information, and his latest video, Deep Dive into LLMs like ChatGPT, is no exception. It provides an in-depth breakdown of the key mechanisms behind LLMs. At a whopping 3 hours and 30 minutes, it covers a lot of ground, and I’d highly recommend watching it. But in case you haven’t watched it because it’s too long, here’s a TL;DW version.

TL;DW (Too Long; Didn’t Watch)

• LLMs are trained in two main stages: pretraining (learning from vast datasets) and post-training (fine-tuning via human feedback).

• Pre-training involves "downloading and processing the internet," using datasets like Hugging Face’s FineWeb (44TB, 15 trillion tokens).

• Post-training refines the model with Supervised Fine-Tuning (SFT) and Reinforcement Learning with Human Feedback (RLHF).

• LLMs suffer from hallucinations but mitigations include model interrogation, web search integration, and structured reasoning techniques.

• Prompt engineering plays a crucial role in optimising LLM outputs and in-context learning allows adaptation without retraining.

• LLMs still have challenges like struggles with counting, spelling, and brittleness in certain scenarios, but improvements are ongoing.

Key Takeaways from the Talk

1. The Foundation: Pretraining on the Internet

The journey of building an LLM begins with pre-training on massive text datasets. Karpathy describes this step as effectively "downloading and preprocessing the internet." Large-scale datasets such as Hugging Face’s FineWeb (44TB, 15 trillion tokens) play a crucial role in training models to predict the next word in a sentence—a technique known as self-supervised learning.

Key processes in pretraining include:

• Downloading or using an existing large-scale dataset and pre-processing the data as follows:

  

• Tokenisation – Converting raw text into token sequences.

• Neural Network Training – Teaching models to generate coherent text based on input sequences.

• Inference

However, a base model at this stage is essentially an "internet document simulator"—good at predicting the next token but not optimised for useful conversations.

I really like the idea of “The psychology of the base model” as described here:

2. Post-Training: Making LLMs More Useful

Once a base model is trained, it undergoes two crucial post-training steps:

• Supervised Fine-Tuning (SFT): This involves feeding the model high-quality instruction-response pairs to refine its outputs and align them with human expectations.

• Reinforcement Learning with Human Feedback (RLHF): Karpathy likens this stage to a student practising problems after learning theory. RLHF helps shape the model’s behaviour, making it more aligned with human preferences, reducing harmful responses, and improving coherence.

3. Addressing Hallucinations

A significant challenge in LLMs is hallucination—where models confidently generate incorrect or nonsensical information. Karpathy highlights mitigation techniques such as:

• Model Interrogation – Teaching models to recognise gaps in their knowledge.

• Tool Use & Web Search – Allowing LLMs to query external sources to verify facts.

• Encouraging Reasoning – Asking models to "think step-by-step" before answering complex questions.

4. Prompt Engineering & In-Context Learning

Karpathy dives into prompt engineering, explaining how well-structured prompts can drastically improve model outputs. He also discusses in-context learning, where LLMs learn from the structure of prompts without needing weight adjustments—making them highly adaptable.

5. The Future of LLMs & Open Challenges

Despite their remarkable capabilities, Karpathy acknowledges that LLMs still face fundamental limitations, such as:

• Struggles with counting and spelling.

• Brittleness in certain use cases (Swiss cheese model of capabilities).

• The challenge of extending structured reasoning to open-ended creative tasks.

Yet, the field is advancing rapidly, and we’re only scratching the surface of what’s possible with AI-driven reasoning and problem-solving.

I don’t think LLMs were compared to Swiss Cheese before (or at least I’ve not come across this analogy before) but I like it.

Why You Should Watch the Talk

Karpathy’s presentation provides a non-technical, clear, and engaging deep dive into how LLMs are trained, fine-tuned, and improved over time. Whether you’re interested in building AI applications, understanding AI alignment, or just fascinated by the inner workings of ChatGPT, this talk is an essential watch.