Build a Large Language Model (From Scratch) by Sebastian Raschka

Not a book about large language models — a book in which you build one. Sebastian Raschka takes you from an empty file to a working GPT-style model in PyTorch, coding every component yourself, on hardware you already own.

What you build, in order

• Text preprocessing, byte-pair tokenisation, and data loaders with sliding windows
• The attention mechanism from first principles: simplified, self, causal, then multi-head
• The full transformer block — layer normalisation, GELU, feed-forward, residual connections
• Pretraining on unlabelled text, with loss computation, sampling strategies and temperature
• Loading OpenAI’s published GPT-2 weights into your own implementation
• Fine-tuning for classification, and instruction fine-tuning for a chat-style assistant

Who it’s for

Machine learning engineers, students, and developers who have used LLM APIs and want to understand the machinery underneath. Requires comfortable Python; PyTorch is taught as you go. The models are small enough to train on a laptop.

Edition details

• Author: Sebastian Raschka
• Publisher: Manning Publications
• Format: Paperback
• ISBN: 9781633437166
• Language: English

Ships free within the United States from Shelfcove.