What do you think?
Rate this book
Writing A Compiler In Go
This is the sequel to Writing An Interpreter In Go. We're picking up right where we left off and write a compiler and a virtual machine for Monkey.
Runnable and tested code front and center, built from the ground up, step by step — just like before.
But this time, we're going to define bytecode, compile Monkey and execute it in our very own virtual machine. It's the next step in Monkey's evolution.
It's the sequel to … a programming language
Writing A Compiler In Go is the sequel to Writing An Interpreter In Go. It starts right where the first one stopped, with a fully-working, fully-tested Monkey interpreter in hand, connecting both books seamlessly, ready to build a compiler and a virtual machine for Monkey.
In this book, we use the codebase (included in the book!) from the first part and extend it. We take the lexer, the parser, the AST, the REPL and the object system and use them to build a new, faster implementation of Monkey, right next to the tree-walking evaluator we built in the first book.
The approach is unchanged, too. Working, tested code is the focus, we build everything from scratch, do baby steps, write tests firsts, use no 3rd-party-libraries and see and understand how all the pieces fit together.
It's a continuation in prose and in code.
Do you need to read the first part before this one? If you're okay with treating the code from the first book as black box, then no. But that's not what these books are about; they're about opening up black boxes, looking inside and shining a light. You'll have the best understanding of where we're going in this book, if you know where we started.
Learn how to write a compiler and a virtual machine
Our main goal in in this book is to evolve Monkey. We change its architecture and turn it into a bytecode compiler and virtual machine.
We'll take the lexer, the parser, the AST and the object system we wrote in the first book and use them to build our own Monkey compiler and virtual machine … from scratch! We'll build them side-by-side so that we'll always have a running system we can steadily evolve.
What we end up with is not only much closer to the programming languages we use every day, giving us a better understanding of how they work, but also 3x faster. And that's without explicitly aiming for performance.
Here's what we'll do:
- We define our own bytecode instructions, specifying their operands and their encoding. Along the way, we also build a mini-disassembler for them.
- We write a compiler that takes in a Monkey AST and turns it into bytecode by emitting instructions
- At the same time we build a stack-based virtual machine that executes the bytecode in its main loop
We'll learn a lot about computers, how they work, what machine code and opcodes are, what the stack is and how to work with stack pointers and frame pointers, what it means to define a calling convention, and much more.
We also
- build a symbol table and a constant pool
- do stack arithmetic
- generate jump instructions
- build frames into our VM to execute functions with local bindings and arguments!
- add built-in functions to the VM
- get real closures working in the virtual machine and learn why closure-compilation is so tricky
Runnable and tested code front and center, built from the ground up, step by step — just like before.
But this time, we're going to define bytecode, compile Monkey and execute it in our very own virtual machine. It's the next step in Monkey's evolution.
It's the sequel to … a programming language
Writing A Compiler In Go is the sequel to Writing An Interpreter In Go. It starts right where the first one stopped, with a fully-working, fully-tested Monkey interpreter in hand, connecting both books seamlessly, ready to build a compiler and a virtual machine for Monkey.
In this book, we use the codebase (included in the book!) from the first part and extend it. We take the lexer, the parser, the AST, the REPL and the object system and use them to build a new, faster implementation of Monkey, right next to the tree-walking evaluator we built in the first book.
The approach is unchanged, too. Working, tested code is the focus, we build everything from scratch, do baby steps, write tests firsts, use no 3rd-party-libraries and see and understand how all the pieces fit together.
It's a continuation in prose and in code.
Do you need to read the first part before this one? If you're okay with treating the code from the first book as black box, then no. But that's not what these books are about; they're about opening up black boxes, looking inside and shining a light. You'll have the best understanding of where we're going in this book, if you know where we started.
Learn how to write a compiler and a virtual machine
Our main goal in in this book is to evolve Monkey. We change its architecture and turn it into a bytecode compiler and virtual machine.
We'll take the lexer, the parser, the AST and the object system we wrote in the first book and use them to build our own Monkey compiler and virtual machine … from scratch! We'll build them side-by-side so that we'll always have a running system we can steadily evolve.
What we end up with is not only much closer to the programming languages we use every day, giving us a better understanding of how they work, but also 3x faster. And that's without explicitly aiming for performance.
Here's what we'll do:
- We define our own bytecode instructions, specifying their operands and their encoding. Along the way, we also build a mini-disassembler for them.
- We write a compiler that takes in a Monkey AST and turns it into bytecode by emitting instructions
- At the same time we build a stack-based virtual machine that executes the bytecode in its main loop
We'll learn a lot about computers, how they work, what machine code and opcodes are, what the stack is and how to work with stack pointers and frame pointers, what it means to define a calling convention, and much more.
We also
- build a symbol table and a constant pool
- do stack arithmetic
- generate jump instructions
- build frames into our VM to execute functions with local bindings and arguments!
- add built-in functions to the VM
- get real closures working in the virtual machine and learn why closure-compilation is so tricky
271 pages, Kindle Edition
Published July 29, 2018
59 people are currently reading
643 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 12 of 12 reviews
June 28, 2019
(Disclaimer: I volunteered for proof-reading the book, and also provided technical feedback.)
This is a pragmatic book that successfully instructs a single person in building a compiler; it is not meant to be a textbook on compilers, compilation techniques, or compiler's theory. This book will get the reader building a compiler for a small language in one go (no pun intended), with code that can be easily extended to embrace more functionality. While the book focuses on the Go programming language, the ideas can be easily transported to different programming languages that allow an imperative programming paradigm (such as C, C++, Pascal, PHP, Python, and many others).
Thorsten employs an elegant programming style and test-driven approach in this book. The compiler is build in successions, handling larger problems at each step, but without overwhelming the reader. Some problems are much larger than others (e.g.: closures) by their nature, and yet Thorsten does an excellent job of both explaining and solving them.
The code is also continually tested for regressions because of the test-driven approach, so errors are always kept at bay (and sometimes the errors are expected, such as when introducing new functionality that intentionally breaks the code).
While the book focuses on compiling to bytecode for an accompanying virtual machine, it doesn't restrict the interested reader. A motivated reader will be able to adapt the compiler to produce an output that be further processed (e.g.: LLVM's intermediate format)---note that this is not covered in the book.
Overall this is an excellent hands-on and inspiring book for writing small compilers that a single person can manage. To my knowledge, the most closely-related book on this subject is Writing Compilers and Interpreters, though the style of that book is rather verbose in comparison to Thorsten's book.
This is a pragmatic book that successfully instructs a single person in building a compiler; it is not meant to be a textbook on compilers, compilation techniques, or compiler's theory. This book will get the reader building a compiler for a small language in one go (no pun intended), with code that can be easily extended to embrace more functionality. While the book focuses on the Go programming language, the ideas can be easily transported to different programming languages that allow an imperative programming paradigm (such as C, C++, Pascal, PHP, Python, and many others).
Thorsten employs an elegant programming style and test-driven approach in this book. The compiler is build in successions, handling larger problems at each step, but without overwhelming the reader. Some problems are much larger than others (e.g.: closures) by their nature, and yet Thorsten does an excellent job of both explaining and solving them.
The code is also continually tested for regressions because of the test-driven approach, so errors are always kept at bay (and sometimes the errors are expected, such as when introducing new functionality that intentionally breaks the code).
While the book focuses on compiling to bytecode for an accompanying virtual machine, it doesn't restrict the interested reader. A motivated reader will be able to adapt the compiler to produce an output that be further processed (e.g.: LLVM's intermediate format)---note that this is not covered in the book.
Overall this is an excellent hands-on and inspiring book for writing small compilers that a single person can manage. To my knowledge, the most closely-related book on this subject is Writing Compilers and Interpreters, though the style of that book is rather verbose in comparison to Thorsten's book.
October 13, 2018
I loved following through this book, just as I did with its predecessor (Writing an Interpreter in Go). Everything I said about that book applies here too, this was a really fun book to work through, building upon the interpreter and writing a bytecode generating and executing virtual machine.
I really hope there will be a follow-up called "Writing a JIT in Go" or similar to take this whole thing further.
I really hope there will be a follow-up called "Writing a JIT in Go" or similar to take this whole thing further.
September 21, 2023
This book has made the art of implementing compiler and bytecode virtual machines broadly accessible. It has a very engaging writing style and you are really carried through the techniques of implementing compiler and virtual machines.
I don't think there is any other book like this. Compilers is not a popular area and all other compiler books (except the interpreter book by the same author) have resulted in disappointment for me. But with this book, I filled a gap in my knowledge of programming language implementations: implementing compilers and bytecode VMs.
I don't think there is any other book like this. Compilers is not a popular area and all other compiler books (except the interpreter book by the same author) have resulted in disappointment for me. But with this book, I filled a gap in my knowledge of programming language implementations: implementing compilers and bytecode VMs.
August 14, 2020
A straightforward introduction to both Golang and compiler development. The compiler you build isn't very heavily engineered, as the author makes quite a few shortcuts and doesn't refactor much, emphasizing pedagogy over architecture. I thought, however, that the Pratt parsing technique was really interesting.
August 20, 2022
The book is impressive and gives a good insights into how the high-level logic is translated into the stack arithmetic and how the VM works.
I was always interested in that subject, but all the books and papers I found were very academic and full of theory, which was difficult to understand without prior knowledge.
Like the previous book, this one takes the opposite direction and dives directly into the practice. This helps and removes a lot of mystery from terms like Compiler, Stack arithmetic, and VM.
I sincerely enjoyed it.
I was always interested in that subject, but all the books and papers I found were very academic and full of theory, which was difficult to understand without prior knowledge.
Like the previous book, this one takes the opposite direction and dives directly into the practice. This helps and removes a lot of mystery from terms like Compiler, Stack arithmetic, and VM.
I sincerely enjoyed it.
September 6, 2024
“Writing a Compiler in Go” is a solid follow-up to “Writing an Interpreter in Go” and continues to showcase the author’s excellent teaching style. Like the first book, it offers clear explanations and practical coding examples. However, I found it a bit less engaging at times, possibly because I’m not currently writing any Go code. That said, it’s still a great resource if you’re interested in the topic, and it builds quite well on the foundation laid in the previous book.
August 19, 2023
An *excellent* tutorial on writing a bytecode compiler and VM for a modern language, with implementation code in Go.
Gets 5 stars for the execution, humour, language and code completeness.
Explanations were also very solid (though they were more layman based, rather than rigorous PLT explanations - refer to other books for those), which adds points to the book.
Gets 5 stars for the execution, humour, language and code completeness.
Explanations were also very solid (though they were more layman based, rather than rigorous PLT explanations - refer to other books for those), which adds points to the book.
May 11, 2025
Turn our tree-walking and on-the-fly-evaluating interpreter into a bytecode compiler and a virtual machine that executes the bytecode.
Besides providing a new layer of abstraction – the bytecode that’s passed from the compiler to the virtual machine – that makes the system more modular and three times faster than interpreters.
Besides providing a new layer of abstraction – the bytecode that’s passed from the compiler to the virtual machine – that makes the system more modular and three times faster than interpreters.
May 29, 2020
very well written, easy to follow code snippets, and a hands-on style one go through to complete a workable compiler project. Once I am done finishing it, feels like as if I went thru a course in Computer Science.
December 20, 2023
Just as the interpreter book I really enjoyed this one. It was definitely harder than the previous but I'm still very glad I went through it. Same as the previous one: very recommended if you're in any way interested in this sort of stuff, even if you don't know go
December 16, 2021
Very good book to understand about compiler, vm
May 2, 2024
I liked his first book more. However this was a great experience as well
Displaying 1 - 12 of 12 reviews