Translate everything up front, translate as you go, or start interpreting and compile the parts that run hot.
“Why is Python so slow?” and “why is PyTorch fast if it’s Python?” are two questions with the same answer, and the answer is the diagram above. How your code gets from source text to actually running on a CPU — compiled ahead of time, interpreted line by line, or JIT-compiled somewhere in between — shapes its speed, its startup latency, its portability, and its debuggability. This is a refresher on the three models, because the trade-offs between them are quietly everywhere in the tools you use.
Compilers: translate everything up front
A compiler (C, Go, Rust, C++) translates your entire source program into native machine code before it runs, as in the top row. The output is a binary the CPU executes directly. Because all the translation and optimization happen ahead of time, the running program is fast — the compiler had the whole program in view and could optimize aggressively (inlining, dead-code elimination, vectorization).
The costs: you pay a compile step every time you change code, the binary is tied to a target architecture (a compiled ARM binary won’t run on x86), and you lose runtime flexibility. You trade up-front work and portability for peak execution speed. When you need every cycle — systems software, databases, game engines — you compile.
Interpreters: translate as you go
An interpreter (Python, Ruby, classic JavaScript) reads your source and executes it statement by statement, every time it runs, with no separate machine-code artifact (middle row). This is wonderfully flexible: no build step, instant feedback, easy to run the same code anywhere there’s an interpreter, and dynamic features (runtime introspection, eval, monkey-patching) come naturally.
The cost is speed. The interpreter re-does the work of understanding your code on every execution, and it can’t see far enough ahead to optimize the way a compiler does. This is the direct reason Python is slow for pure-Python compute-heavy loops — every iteration pays interpreter overhead. It’s not that Python is badly built; it’s that interpreting is inherently more work at runtime than executing pre-compiled machine code.
JIT: start fast, then compile the hot parts
Just-In-Time compilation (the JVM, JavaScript’s V8, PyPy, .NET) is the clever hybrid in the bottom row. It starts by interpreting, so there’s no long up-front compile and startup is quick. But it watches the program run, identifies the “hot” code — functions and loops executed many times — and compiles those to optimized machine code on the fly. The 5% of your code that does 95% of the work gets compiled; the rest stays interpreted.
The payoff is real: the JVM and V8 can approach or match compiled-language speed on long-running workloads while keeping much of the flexibility of a managed runtime. The catch is warm-up — a JIT-compiled service is slow for its first moments and gets faster as the JIT kicks in, which is why JVM benchmarks discard the early runs and why serverless cold starts are painful for JIT languages. A JIT can also make smarter decisions than an ahead-of-time compiler in one respect: it can optimize for the actual runtime data and hardware, not just what was knowable at build time.
The trade, in one line
Ahead-of-time compilation buys peak speed at the cost of build time and portability. Interpreting buys flexibility and portability at the cost of speed. JIT tries to buy both, at the cost of warm-up and complexity. There’s no free lunch; there’s a choice about when you pay for translation.
The AI-era relevance: it’s the same story, one layer down
This isn’t trivia for AI engineers — it explains the performance model of the entire Python ML stack.
Python is the interface, not the engine. The reason it’s fine to write ML in “slow” Python is that the heavy lifting never happens in Python. NumPy, PyTorch, and TensorFlow are thin Python interfaces over compiled C++/CUDA kernels. Your Python loop isn’t crunching the matrix multiply — it’s dispatching to compiled code that does. Understanding this tells you exactly where Python overhead does and doesn’t matter: a tight Python loop over individual tensor elements is a performance disaster; a Python line that calls one big vectorized op is essentially free. This is the same lesson as the GIL and I/O-bound-vs-CPU-bound distinction, from a different angle.
torch.compile, XLA, and TorchScript are JIT compilers for tensor programs. Modern ML frameworks moved from executing operations one at a time (eager mode — interpreter-like, flexible, easy to debug) toward compiling the computation graph. torch.compile traces your model, fuses operations, and generates optimized kernels — a JIT for deep learning. XLA (behind JAX and TensorFlow) does ahead-of-time-style compilation of the whole graph. The performance gains are large precisely because compilation can see the whole computation and optimize across operations, exactly like a traditional compiler sees a whole program. When you hear “eager vs graph mode,” it’s “interpret vs compile.”
Cold starts and warm-up hit inference too. A freshly started model server, JIT-compiled kernels warming up, a container cold-starting — these are the warm-up costs of the diagram showing up in your p99 latency and your autoscaling behavior. Knowing that “the first requests after a deploy are slow, then it speeds up” is expected JIT/warm-up behavior saves you from chasing a phantom bug.
The takeaway
Every language and framework sits somewhere on the spectrum from “compile everything up front” to “interpret everything as you go,” with JIT bridging the middle. That position determines its speed, its startup cost, and its flexibility — and the trade is a when do we pay for translation decision, not a good-vs-bad one. In the AI world, the same spectrum explains why Python is an acceptable driver for GPU workloads, why torch.compile makes models faster, and why your inference latency has a warm-up shape. Fundamentals like this don’t age; they just get new front-ends.