Do you Ship your Entire Dev Environment

There is much discussion about whether x86 or ARM CPUs are the more power efficient ones. As the common wisdom goes, RISC instruction sets, like ARM's, can be more power-efficient than the more complex, CISC x86.

As it happens, this doesn't happen to be accurate. In fact, the amount of area dedicated to decoding instructions on a modern CPU is only a fraction of the total amount. Most of the transistors these dayse are spent on caches & trying to guess what the next instruction will fetch from memory, not just decoding the bytes of the instruction. Nevertheless, there is some truth to the matter: decades ago, this was all true. There were orders of magnitude fewer transistors to spend, while the complexity of instruction decoding didn't change a lot... so simplifying the decoding part enabled you to go faster. For a while. Then, we hit other bottlenecks.

This is not the only place where this happened. Take Lisp machines versus early Unix boxes. Unix machines were a lot simpler and cheaper, in exchange for somewhat reduced capabilities. Since you didn't have a lot of RAM available, it made sense to first take your C program and compile it into a binary, and then run the binary separately, without the compiler continuing to sit in memory. Better yet, you didn't have to load every single detail for your program into memory at the same time! You could just fire up the compiler for one C file, load all the relevant headers, spit out the code, and then quit, starting over for the next C file with a different set of headers.

Very memory efficient. Great tradeoff.

Obviously, there are some cases where you cannot do this. For example, you don't want the compilation step after every time you modify your simple shell script. Instead, you can just keep something much simpler in memory: an interpreter that goes line by line, evaluating your script. Obviously, it's much slower, but you don't have to ship around and launch a compiler that is both memory-hungry, slow to run and expensive (consider that open source compilers weren't even a thing many decades ago).

The alternative that already existed back then was Lisp machines. Instead of dealing with blobs of machine code, spat out by compilers that stop existing after finishing up their output, they let you interact with your code as you're writing it, make changes on the fly while the system is running, and have good introspection abilities into the data being stored in the system. Imagine the knowledge that the C compiler gathers while reading the header files in your code, not being thrown away after a couple of seconds, but being available for both you & the program itself, all the way while it's running.

Of course, all this is expensive. You're essentially keeping both the compiler and your program in memory. To do this, you need both more RAM (to store them) and a dedicated CPU architecture, so that high-level code runs reasonably quickly, without spending a ton of time optimizing binary output & dealing with how to map this back to higher level concepts. Once you do it though, you get a very pleasant development environment that people generally liked. A lot. Assuming they got access to these expensive pieces of equipment, typically at universities.

(This is probably the right time to bring up the NASA space probe that they ended up debugging via a Lisp REPL 150 million miles away. Try doing that in C.)

Time has passed. We now have orders of magnitude more RAM than Lisp machines ever had. (The computer I'm writing this on has about 8,000 times more than the default spec for a pricey mid-80s Symbolics 3640.) Meanwhile, the number of lines of code humans can write in an hour has stayed relatively stable.

You would think that this tradeoff got reconsidered, giving us more capable development environments than Lisp machines have ever been. After all, with all this RAM, keeping a compiler in memory at all times, even as part of the already shipped system, shouldn't take much effort. It is surely just a tiny, tiny fraction of the memory used for other purposes. Given the advantages of being able to debug and modify the system or the fly, this is a very tiny price to pay.

In fact, this is not what happened. We still do not have systems that are easy to debug by default. Compiling something from source, in order to be able to modify it, is still oddly much more complicated than just running it. Meanwhile, compilation time is still a thing mostly spent on parsing the same header files over and over, and then throwing away the results.

There are exceptions to this that also help highlight how weird this is. Take clangd for example. In order to give users a more pleasant IDE experience with autocomplete and smart code navigation, it does the same compilation steps as the compilers already are, using the same libraries (LLVM in this case). However, unlike the compilers, it is a long running process.

Of course, it's sometimes nontrivial to hack clangd into your build system. You typically need to capture the exact same commands that were used to run the compiler, feed it to clangd, and deal with all the discrepancies that arise regardless.

Instead of it and the compiler being just literally the same process.

The story then continues when you actually want to debug something. You tell the compiler to add debug symbols to your binaries. Then you use gdb to load up the symbols and connect to the executable. (Assuming, of course, that you could get your build system to run the compiler with the right flags, to perhaps disable optimization to the point where everything is basically unrecognizable.)

This, of course, makes sense; after all you can't just leave all the debug symbols around. That would make things use a lot of memory and leave everything unoptimized!

Because you do not have a C compiler built into your binary.

If you had a C compiler built into your binary, you could just tell it to recompile the one function that you're interested in with all the debug symbols in it. While the thing is actually running.

And while building a C compiler into your binary used to sound like complete madness 30 years ago, you can probably fit one these days in less RAM than it takes to to show a news article with three images in a browser tab.

Meanwhile, so-called "scripting languages" are doing something silly from the other direction. When they started out, packing an actual, fully-featured compiler that would JIT compile all the code was not yet an option. So we continued expecting them to be quick to iterate on, but slow. Assuming that "quick to iterate on" and "fast" is not two things that you could have at the same time.

This was, yet again, true in the 80s. No longer true though in the early 2020s.

Apart from Common Lisp (being the mandatory counterexample), there is Javascript, too, if you're looking for something more mainstream. The language itself is definitely not designed for speed, and yet modern browsers contain a JIT compiler that make it impressively fast. (To which we add some additional layers of frameworks that make it both slow to compile and a terrible user experience, but this is not the fault of the underlying architecture.)

Some things that used to be hard are not hard anymore. Maybe, after all, it's time to reconsider our once necessary but fairly outdated workarounds. We can now afford not having a meaningful difference between "your development environment" and "your finished product". Sometimes, shipping the entire thing is just easier.

(you can look at the entire cool Lisp Machine brochure on archive.org)