If you want high-volume, cheap manufacturing, you may want to use progressive stamping, where a flat sheet of metal is shaped and cut, step by step, by hard metal dies driven by a hydraulic press. If you can design your part such that it can be made by simply pressing and cutting a flat sheet of metal into shape with this simple process, it can be made absurdly cheaply and efficiently.

If you need something with far greater complexity than a simple flat sheet can provide, then there are a variety of other options – casting, lathes, CNC milling, 3D printing, and more. There also are many very special-purpose machines for making things like bolts, screws, wires, pipes, and more in bulk. A 3D printer may be able to make a part with an incredibly complex and intricate geometry, but it will generally be very slow, with limited choice of material, and often producing parts full of defects, voids, and weak spots. A lathe simply spins an object while moving a cutting head – great for making objects with a degree of rotational symmetry, though creative machinists have spent centuries finding ways to push this machine to do a surprising range of things.

And while you can hand a 3D printer any arbitrary 3D model to slice and print, there are quite a few things that you can’t print in practice. Objects with floating parts or overhangs can sometimes be salvaged with support structures or by reorienting the part, but for the most part are beyond the scope of what a 3D printer can do. Material choice is usually pretty limited, and most printers don’t support more than one material at a time.

We can look to sci-fi, there have for decades been dreams of atomic replicators and molecular assembly machines; effectively 3D printers assembling objects atom-by-atom. An alternate route was originally proposed by Robert Heinlein in his story Waldo (1942), where a scientist builds controllable mechanical arms, which then build smaller arms, which build even smaller arms, and so on until they reach down to the molecular scale. However, the machines from Waldo (often referred to as “waldoes”) run into problems; among other issues, physical processes work differently across scales and a design that works well at one scale may work very poorly at a different scale.

Atomic replicators, even if we could build them, still run into many of the same problems as 3D printers. Simply because you can specify every atom in an object with an atomic number and X,Y,Z coordinates does not mean that you can actually place atoms in that arrangement. You must factor in how the atoms interact with each other and the tool itself, you must factor in how the atoms move around once they’ve been placed, you must factor in whether or not a molecule is stable enough to hold itself together when it’s only been half-built. Any manufacturing process, no matter how exotic, has the constraint that the object it’s building must not only be stable as a final product, but also must be stable at every step throughout the entire process of forming it.

An atomic assembly process would also undoubtedly have limits in terms of what materials and temperature ranges it can work in. Whatever processes produced and shaped the parts of this machine can also unshape it, and may be outside the domain of what it can actually assemble. Can your extremely sensitive atomic assembly machine safely handle highly corrosive fluorine? Can it handle radiation? Can it print dry ice at -110 degrees? Can it print at 1500 degrees? Can it print a glass of water? Can it print a cell without the cell wall bursting like a bubble when only half-built?

I don’t think it’s very ridiculous to say that your 3D atomic assembly machine would not be able to print a nuclear fusion reactor, hundred-million-degree plasma and all. You could perhaps print the fusion reactor and pump in the plasma afterward, but plasma no matter how hot is just a cloud of atoms at specific positions with specific (albeit very high) velocities.

Of course, all of this completely sidesteps the question of how you would encode the design for atomically-precise objects. Whatever you manufacture would require either an atomically-precise hard drive many times larger than the object you’re attempting to print, or would require a substantial amount of data compression. Either way, some amount of computation will be required to read this data, decompress it if necessary, compute positions of atoms, plan and schedule the arrangement of atoms and assembly arms (or whatever mechanism actually does the assembly), and to put each atom in place. This computation will almost certainly have a thermodynamic cost, technically speaking the movement of the atoms would itself count as a type of computation whose thermodynamic footprint would need to be factored in, and so depending on the approach this kind of molecular assembly could also produce enormous amounts of waste heat and consume enormous amounts of energy, which would need to be managed (likely by means of even more energy) to prevent melting the object or the tool itself.

In short – there is no true “universal machine tool”, and even the craziest sci-fi dreams fall short.

Software Machine Tools

In computing, we can perhaps tell ourselves otherwise. Python is Turing-complete after all. In theory, we can build anything with it.

Matrix multiplication is close enough to Turing-complete, you can emulate any kind of if/else logic in it, and you can emulate any matrix math with enough if/else logic too. It might not be pretty, but it’s doable. A big enough matrix can emulate any amount of memory, and with enough layers you can emulate any arbitrarily long computation, and the inverses are equally true. There’s really no reason why, at least in principle, you couldn’t perfectly recreate all of Claude with conventional programming constructs – if/else, for/while, data structures, algorithms, arithmetic, strings, lookup tables, and so on. Doing arithmetic with actual arithmetic instructions would probably be more efficient, more understandable, and more correct than whatever crazy matrix math is emulating it.

The biggest barriers to building this is not the theoretical capabilities of the language, but rather a combination of the sheer volume of code that would need to be written and the lack of human understanding of how to write it. The whole reason why we use neural networks to solve these problems is simply that we have no idea how to build things like image classification and language models by more conventional means, and so we turn to a tool that can generate a program from a large enough volume of data.

We can consider the expressivity of a language as a matter of how many keystrokes, or perhaps how long of a source file, it takes to accomplish a task. There’s of course a difference between these two metrics – autocomplete can remove some repetitive structure in code, and any code will require much more iteration, modification, and debugging than simply writing out every line of code. Languages may vary in how expressive they are – small amounts of code may unfold into large amounts of machine code based on what language constructs exist. Languages may vary as well in how debuggable and understandable they are, though this is also a product of how the language is used. Manually writing out a table of neural network weights and running the matrix math will be more or less just as incomprehensible in C, Python, Rust, Javascript, Assembly, or APL.

There is a recreational programming activity called “code golf”, where programmers attempt to solve problems in the minimum number of characters possible. There are even dedicated languages for code golf, which do fun things like define an empty source file to print “hello world”. At the opposite extreme, there is a notion of a “Turing Tarpit”; a system which is technically Turing-complete, but that is hopelessly impractical and inefficient to use. Many cellular automata can be argued to fall into this category, for example.

Expressivity is also a matter of the skill of the user. Writing was invented many thousands of years ago – any words that can be dreamt up can be written down. Theoretically, the solutions to all the world’s problems are mere words away, if only they can be elucidated. The solutions to all the problems faced in the year 1,000,000 AD could have been pressed into clay tablets 5,000 years ago, if only someone had thought to do so. The tightest bottleneck of all, by a profoundly enormous margin, is not in the pen or the paper or the ink, or the stylus or the clay or the hand, but rather in the brain behind it all.

Humans engineer things based on what we understand. People have been trying to solve image classification problems for decades, and there is a vast array of “feature engineering” methods for hand-writing algorithms to recognizing shapes, objects, faces, and gestures. Then in 2011 a convolutional network called AlexNet was trained on a couple GPUs and began thoroughly obliterating all previous methods in benchmarks. There is some family of tricks that the neural network has learned to do that is more powerful than all the techniques we spent decades working on, and after 15 years we still don’t know what they all are. Of course, there are also many problems which AI tools have not yet delivered despite stubborn and persistent effort – the elusive “AGI” being the most obvious one.

The fact that a programming language is “Turing-complete” and “universal” means that in theory it can be used to build anything, but there is absolutely no obligation for this to have any truth in practice. If we want to solve problems that we do not know how to engineer with conventional programming constructs, it may take many orders of magnitude fewer keystrokes and decades of research to simply assemble a training set and pump it into a neural network than to implement everything by scratch.

Thus I propose the concept of a Software Machine Tool. There are more ways to build software than conventional programming languages, and neural networks are a clear example. The asymptotic capabilities may all be universal as we approach infinite effort and skill, but different tools can have vastly different capabilities in practice. No one has time to write a trillion lines of code to reimplement Claude or Sora, and training a neural network to correctly and efficiently replicate a browser or terminal or operating system is similarly insane.

The implication, of course, is that there are presumably many more that we have not yet discovered. Consider the immense value that programming languages have provided for the world in the nearly 70 years since FORTRAN was invented. Consider the immense impact being generated by AI now. Consider what would happen if we found a third class of software machine tools orthogonal in capabilities to either of these, and what that may unlock.

To some extent, the current wave of AI coding agents is arguably a third category of software machine tool, albeit one with aspects of the other two. Exactly what can be efficiently built is still very heavily constrained by the limitations of conventional programming languages, and the training set of code that has been written with them. There are undoubtedly many world-changing algorithms that have yet to be discovered that have not been added to this training set, and cannot be synthesized on-demand by an LLM. Asking an LLM to write out a table of embeddings with strange properties may be off the table, even if that table would be a relatively small amount of code. On the other hand, the sheer volume of code that can be produced, the iteration speed, and the expressiveness of natural language is a difference in quantity that can perhaps create at least a slight difference in kind.

There are other options as well. I actually was not someone who was terribly surprised by AI coding tools when they first appeared, as I had been keeping an eye on code synthesis research which had been going on for many years before. The format and scale was unexpected; I expected something smaller scale and more constraint-based; “generate code that provably satisfies this strange and difficult set of properties”. Done well, this kind of approach is still likely very viable and would be able to generate some very strange things that may be well outside the domain of LLM training sets. My guess is it’s more limited by the imagination of what kinds of useful problems can be productively framed in such a way.

Superoptimizers are another interesting approach; rather than writing tens of millions of lines of hand-crafted compiler optimization and analysis passes, simply write a relatively small tool for verifying program equivalence (in general undecidable, but there are decidable subsets of this that can cover the majority of real-world cases pretty easily, as well as ways of covering all cases with conservative-but-safe assumptions), and a search algorithm, such as MCMC or a genetic algorithm. This lets you frame code optimization as a matter of “find the fastest possible sequence of instructions I can pump into the CPU that is provably equivalent to this code.” The main thing that has held this back historically is that it’s “too slow” to be practical. However, it’s very amenable to caching (optimize your code overnight, pull from the cache during the day, only recent changes run slow), and the standard for “too much compute” from a few years ago is basically “less compute than Claude spends generating a single token”.

Superoptimizers also are notorious for generating code that is very fast; they can exploit tricks that no human would ever have imagined, that no human would have programmed into a compiler, but they also can generate code that no human can easily understand. Imagine something like the notorious fast inverse square root algorithm, but for every line of code.

If we’re considering general code synthesis, we could even generalize this. Perhaps you need code that has a particular set of properties, properties so difficult or unusual that you don’t even know how to write the code. As long as you have a tool that can verify whether or not a function has these properties, and perhaps define a gradient or error function to help navigate the search, a superoptimizer should be able to help. You wouldn’t be limited to just matrix multiplication like machine learning, you could have code with control flow, data structures, loops, etc.

There’s another very interesting option. Rather than focusing as much on the specific operations a function performs, why not instead put effort into the particular subspace of input/output pairs and how we map meaning onto them? This is fundamentally what neural networks operate on with embeddings of various kinds, and to a weaker extent it’s what we do with things like C-style enums and simple formats like ASCII. Another notable example to illustrate my broader point would be linear codes in error correction; if you simply choose the correct table of magic constants with special properties, you can solve difficult error-correction problems with very simple and efficient algorithms.

There are perhaps a lot more problems than we realize that can be addressed this way, but the question of how you generate the table of magic numbers, as well as the corresponding function that operates over them, is a difficult problem. We have some methods of addressing this with machine learning, but these neural networks are often very large and expensive, not the kind of thing that you can casually operate on in your code. Linear codes are extremely tiny and cheap. There’s a very large amount of unmapped territory in the middle.

More generally, there’s a sense in which RAM isn’t a very natural form of memory, at least compared to plain bit vectors inside of a circuit. If we look to more emerging and exotic forms of computing – neural networks, but also quantum and reversible computing – these often require memory to be a small vector of bits or variables that we apply gates to individually. We take RAM for granted in conventional computing, but it actually requires very large amounts of routing and addressing circuitry to implement. AI algorithms evidently have some way of representing hierarchically structured information even without pointers and random access as we would recognize it, instead just embedded in the matrix operations. If you just have a big vector of a few hundred or thousand bytes sitting in a CPU register, and you can apply some large, complex operations to it, what kinds of data structures and representations can you embed in that? And what kinds of unconventional properties might you be able to get out of it?

Code Analysis

While not strictly something for constructing code, there’s certainly a very large amount of room for better static analysis in programming languages. Modern optimizing compilers are full of extensive static analysis, and a great deal of this is directly relevant to the same kinds of properties that matter for the correctness of your code, there’s just no language-level interface for interacting with this at all, and the tools the compiler offers often are not really designed for casual use. Even something as stupidly basic as asking “what assembly does this function compile to?” is hard enough that there’s a popular web tool designed to do a task that should really be a standard CLI command in every serious compiler.

Debugging ultimately is the process of determining the difference between the code that you think that you wrote and the code that you actually wrote. This will only get dramatically more important as more code is written by AI. It’s hard to improve things that you can’t measure, and it’s not always easy to be aware that they even exist if you’ve never even seen them. Visibility into code today is pretty poor in general.

Further, a tremendous amount of innovation simply comes from viewing old problems from new perspectives. I’ve written before about how much geometric structure there is underneath the symbolic surface of code; if we can bring in tools to make this more obvious, and help people better understand this structure in their code, it may inspire thinking of new degrees of freedom to poke at. That may expand what people write with conventional code, but text is often a poor format for strictly geometric structure. There’s nothing stopping you from writing down a table of coordinates and shapes, but translating back and forth between shapes and coordinates in your head is surely difficult and error-prone. Many simple transformations would require precisely shifting things around, requiring rewriting all of the coordinates and constraints. It’s easier to just handle this with a graphical tool and automate it all.

I’ve been hard at work the past couple months on my chip startup, but some momentum is starting to build and interesting work is getting done.

Subscribe now

Share

However, when we dig a little closer and start asking some very basic physics-oriented questions, this narrative falls apart completely.

The vast majority of technology today is based on manipulating atoms, electrons, and photons. We get a fraction of our energy from splitting atoms with neutrons, though far less than we reasonably should for a technology we’ve used since the 1940s, and on very rare occasions we’ll use neutrons for imaging and some other niche uses. Even more rarely we will make use of exotic particles like muons and positrons, mostly for other types of imaging. Muons (heavy electrons) are sometimes used for imaging the 3D structure of rock formations or pyramids, though they are sufficiently difficult to produce that we rely almost entirely on the tiny quantity that the sun generates naturally rather than using more reliable engineered muon sources. Positrons (anti-electrons) are also occasionally used in medical scanners, and are mostly a byproduct of the decay of specific nuclear isotopes, one of the other applications of neutrons.

We may have discovered photons, six different types of quarks, eight different types of gluons, three types of electrons, three types of neutrinos, the W+, W-, and Z0 bosons, a wide variety of antiparticles, as well as the Higgs boson, but the vast majority of this is profoundly useless to us. Will discovering more particles make any difference? If we look at what makes atoms, electrons, and photons so practically useful, it’s that they are abundant and easy to build things with. Humans have been able to manipulate all of these things, at least indirectly, since the dawn of time. Everything else is difficult to interact with, and many of these things can only just barely be clumsily poked at with giga-electronvolt particle accelerators. We’ve understood quarks for 60 years, but there are still no “quark-gluon engineers” and it’s extremely unlikely that such a title will be meaningful even in another 600 years. Even if we can poke at such systems, it takes such vast amounts of energy to have such tiny and unpredictable effects that any useful engineering is unfathomable with any technology we can reasonably conceive of.

Muons were discovered in 1936, positrons in 1932, and neutrons in 1920. The fact that neutrons could trigger nuclear fission wasn’t discovered until 1938, a discovery which promptly kicked off the Manhattan Project. It would not at all be inaccurate to say that “discovering new particles” stopped being a driver of new technological progress after the 1930s, rather than the 70s or 80s.

We know that there are likely additional particles that we have yet to discover – dark matter, gravitons maybe, X17, sterile neutrinos, and perhaps something driving dark energy. However, the reason we haven’t discovered these is because they interact absurdly weakly with anything that we do know how to build – so weakly that for most of these we have never measured any obvious interaction with anything that we’ve ever built. We can only measure their very indirect effects by observing places where astronomical measurements don’t line up with our mathematical models, or spotting a few other very minor discrepancies at absurdly high energies. If your goal is to engineer useful things with these as your building blocks, this is the last property you’d ever want to have!

Our technology is overwhelmingly built with atoms, electrons, and photons. We have largely the same building blocks that our ancestors thousands of years ago had. Chemistry (or alchemy) is just mixing atoms in ways that rearranges their electrons to bond them together differently. “Discovering new particles” is a blatantly false description of what drove the past couple centuries of progress. Upon closer inspection, much of what drove material innovation was downstream from math – Newton building a much more powerful toolbox to model dynamical systems and geometry, followed by more detailed models of the tiniest components of the universe.

Humans have been aware of electricity in some form since we first glimpsed a thunderstorm or gotten a static shock. Occasional magnetic properties of meteorites were known in ancient times, and medieval monks like Petrus Peregrinus de Maricourt even were studying magnets intensely at least as far back as the 1200s. Benjamin Franklin famously experimented with electricity in the 1700s, but it wasn’t until the mathematical work of Faraday and Maxwell that electromagnetism really became something usable for serious engineering. Once a mathematical theory was in place however, it rapidly exploded into telegraphs, motors, radio, light bulbs, tesla coils, a plethora of vacuum tubes, cameras, the television, and more. It wasn’t until 1897, a couple decades into this technological revolution, that a particle called the electron was even confirmed to exist.

The existence of atoms and the theory of chemistry were increasingly fleshed out throughout the 1700s and 1800s, and many practical heuristics and principles were understood decades before any of their component particles were discovered. What enabled advanced chemistry wasn’t so much the discovery of protons and electrons, but the quantum theory that allowed us to accurately model their behavior. Even then, there is a lot of talk today about “quantum chemistry”; even though we have the equations to describe everything, we’re mostly confined to chemistry where interactions are simple and easy to compute. The vast majority of chemistry is much more complex, and the computations required to apply our equations to answer even the most basic of questions are so complex that people are concerned that we may only be able to answer them with quantum computers.

The ancient, classical world saw a substantial degree of technological progress too, and while today we may be hesitant to compare its magnitude to the industrial revolution, they did understand the principle of steam engines, were capable of manufacturing billions of standardized glass and clay pots, and even made use of water wheels, screw pumps, and complex bronze gearboxes. Vetruvius’ “De Architectura”, written in the first century BC, laments how underutilized water wheels were. This book appears to have had an impact, as archaeology suggests that water wheels became much more common in Rome over the following few centuries. We do not have any textual references to the extremely elaborate Antikythera mechanism, though surviving texts make casual references to enough bronze gearbox mechanisms like wagon odometers in the later Roman empire that, provided a time machine, the couple centuries after Christ may have looked to modern people as slightly steampunk. This is an impressive degree of technological progress to be made without discovering any new particles! Much of this progress is most likely the product of advances in geometry and mechanics, as well as the collection and centralization of knowledge from across the ancient world in vast libraries like the one in Alexandria.

Semiconductors and Condensed Matter Physics

Semiconductors are not exotic magic. Conductors are materials where the uppermost filled (valence) electron orbitals are very close or overlapping with additional, empty (conductance) orbitals where they can move more freely. Insulators generally have a very large “band gap” separating these orbitals, requiring a large amount of energy to be pumped into electrons to get them to jump from the valence to the conductance band. Other factors such as heat, light, and electrical fields can also affect the size of the band gap or how much energy electrons have available to try jumping it. If you simply ask “what happens if I have a material with a band gap large enough that it’s an insulator by default, but small enough that these other factors can temporarily change that?” the medium-sized-band-gap materials you end up with are semiconductors.

Semiconductors are also all around you. Most optical properties of objects are a combination of band-gap-like electron-photon interactions and scattering effects. What makes glass transparent is its band gap of 9eV; no photon of visible light (1.8-3.1eV) is capable of exciting its electrons to the conductance band, so they simply pass right through. Silicon has a band gap of 1.1eV; visible photons interact with it and can be absorbed and re-emitted making it opaque and shiny, while it is transparent to infrared light below 1.1eV. Band gaps within the visible range can create a variety of colors. Organic molecules often have more complicated electron structures, and so talking of band gaps and semiconductors there is more of an analogy, but it’s still just molecule-scale photon-electron interactions driving macroscopic coloring.

Band gaps and their more complicated organic cousins have been the underlying mechanism behind many manmade pigments for thousands of years. One such organic cousin is porphyrin, a core component of both hemoglobin and chlorophyll, and some of its behavior can be understood as a kind of semiconductor. It becomes chlorophyll when doped with magnesium, and heme when doped with iron.

Silicon isn’t some rare, special material. It’s just abundant, easy to work with, and has other reasonably nice properties, but there are many other semiconducting materials out there. I recall a conversation once with a semiconductor engineer who remarked that he considered the luckiest bit of any of this to be that silicon and silicon dioxide (glass) happen to bond together very nicely, making it very easy to engineer circuits with silicon while having good insulation between the wiring without huge engineering headaches from getting the different crystal lattices to stick to each other.

Semiconductors are not some exotic “top of the tech tree” material; they’re actually one of the first unexpected things you’d expect a civilization to discover once there is a basic understanding of quantum mechanics and conductivity, only to discover that it has actually been ubiquitous for eons. A wide variety of materials all around us have wonderful properties we never before dreamed of! We should actually expect many other bizarre materials with strange, emergent quantum properties to be found further along the tech tree, should we go looking for them. Known examples include superconductors and scintillators. The field exploring emergent quantum properties of materials is Condensed Matter Physics (the name is misleading, and really just refers to the study of emergent properties in large numbers of atoms), and it is still one of the newer branches of physics and rarely wins serious attention or Nobel Prizes despite its practical value. It however also seems to have a relatively limited toolbox for understanding the materials it studies, very often resorting to describing emergent quasiparticles and reasoning about their behavior with little more than fluid dynamics analogies.

Bits Hiding in the Atoms

There certainly are many problems which boil down to producing large quantities of some physical thing – specific chemicals and materials, heat, light, energy, etc. – and accomplishing each of these requires some ability to harness certain degrees of freedom in physics. The vast majority of problems however, as well as the implementations of methods for producing these physical things, are better understood as composing a small variety of these interactions in very elaborate ways rather than the simple prodding of a large variety of orthogonal fundamental forces.

We can consider new particles and forces of nature to perhaps give us new degrees of freedom for interacting with the world, but there is almost never any genuine need to use anything other than atoms, electrons, and photons. On rare occasions we will work up the courage to make some cheap energy using neutrons, and on far rarer occasions we will do some exotic metrology with muons and positrons, which are really no more than fancy cousins of the electron with different masses and charges. Everything else is a billion-dollar science experiment.

The problem of arranging some very complicated system which can coordinate a small number of physics interactions is fundamentally a matter of building something with the right behavior, that performs a correct and specific function, and this is fundamentally a computational problem no matter how many layers of indirection and denial we pile on top.

The actual future of technological innovation is not in finding new degrees of freedom, but in finding more powerful and useful systems which we can build from these primitives. Atoms, electrons, and photons are already Turing-complete and can send signals back and forth at the speed of light. Beyond nuclear power and a few rare, niche applications of exotic particles, what else do we genuinely need? We don’t need to dig deeper and break things into smaller pieces – the path forward is to take these component pieces and build bigger and stranger things with them than we ever dreamed of before.

The Library of Babel is infinite, and though most of it is gibberish it still hides unfathomable quantities of valuable things. The number of possible 32-byte programs is greater than the number of atoms in the observable universe. The number of 1000-line-of-code programs is for most practical purposes infinite, and we should expect that there will still be a steady stream of breakthroughs from novel ~1000-loc programs, algorithms, and theorems creating breakthroughs a million years from now. The most revolutionary technologies that the future holds are to be found among the unknown unknowns. We only bring these things closer to the known by expanding our imagination, by finding new organizing principles, new mathematical tools, new patterns in the world, new ways of thinking, new possibilities. The fact that so much of our science fiction and futurist art is retrofuturist, or that the most inspiring vision of the future of computing people can point to is Snow Crash or The Sovereign Individual, with the past 30 years of surprises that the internet has wrought failing to inspire compelling alternatives, are strong signals that it is our imagination that is failing us, not our ingenuity.

We have some leads. Machine learning has made a substantial and growing impact over the past 15 years. Faced with many problems which we do not know how to engineer directly, we now have a tool which can synthesize solutions using methods that large teams of interpretability researchers are only beginning to scratch the surface of. This works as long as we can craft a sufficient dataset, a viable neural net architecture, and provide a sufficiently vast amount of raw compute. Many things that we did not previously know how to build can now be built – and it would be of far greater value if we better understood how these things worked, were able to engineer them with more control, and better understood their capabilities and limitations.

The array of equillibrium states of a physical system is NP-complete. Nondeterministic (NP) computations are very useful for many constraint and optimization problems which are ubiquitous across science, physics, and engineering, but they are also difficult and expensive to compute. They do however contain a great deal of structure, which for a wide array of problems makes them easier to solve in practice. Much of the field of cryptography is focused on building systems with these kinds of properties which are not accidentally easier to break than we intend them to be. However, if we carefully construct a physical system structured in the right way, and we allow it to settle to a stable point, we can expect its final configuration to encode a solution to our problem.

On one hand we can lament that this makes physical systems hard to predict and model, but on the other hand we can celebrate that the universe gives us a physical substrate for computing many very useful problems which would otherwise be prohibitively difficult to compute. Sometimes a problem is just too difficult though, and we may run out of energy or time before the solution we want is reached, that Avogadro’s number of atoms is an insufficient number of FLOPS, and a better understanding of how nondeterministic computation behaves would enable us to optimize this, as well as potentially better understand when a proper solution is and is not attainable.

This also includes the chemical systems inside of cells. A point I’ve been trying to make to biology people (and that I’d like to figure out how to express more clearly) is that many of the problems biology constantly needs to solve are NP-complete, and that this simple fact about equilibria implies that DNA can simply encode a shockingly tiny number genes defining a few constraints, and that allowing the resulting biochemicals to react and settle would be able to solve a wide range of arbitrary constraint or optimization problems defined by these chemical constraints. If natural selection is just fumbling around design space randomly, and these systems are so absurdly useful and simple, we should be absolutely astonished if biology isn’t making use of them for nearly everything. The difficulty of computing these problems however means that programmers and researchers often avoid them like the plague and neglect studying them – however if every single chemical reaction counts as a computation, and a single cell may include billions or trillions of reactions per second, then we should expect biology to have extremely easy access to a shockingly powerful and useful set of computational primitives. A single cell would have comparable computing power to an entire laptop.

There additionally are many very nice properties of how these constraint systems compose that suggest very compelling answers to difficult questions of how extremely complex systems can evolve incrementally! This may also make them easier to engineer, if we simply understand the principles and build the right tools. I fully expect that if we had a few more polymaths crossing these domains, and if we managed to get more people to develop tools for reasoning about Nondeterministic computation, that we’d build out a mathematical toolbox that would at a bare minimum do for biology what calculus did for physics. Even if you can’t measure every variable within a physical system, the principles behind calculus and classical mechanics are enough to place constraints on what is and isn’t possible that otherwise would not be obvious.

And of course, this is to say nothing of applications beyond biology; this nondeterministic computation arises in all physical systems, and we should expect that many physical systems which we struggle to understand today will yield to a better toolbox. And even then, this is merely a known unknown.

As we push harder toward the physical limits of computation, computer science will necessarily need to account for the physical embedding of computation in space and time, its use of energy, its reversibility, and more. Ideas from physics will increasingly make their way into computing. Likewise, ideas will work their way back, and it will increasingly be possible to describe properties of physical systems as decision functions within certain complexity classes, or perhaps some richer descendant of this classification. It’s worth remembering that Big-O notation is really just a kind of Taylor Series.

The past six months I’ve been living in San Francisco trying to get my startup off the ground. Something about this has diminished my ability to write, but I am working my way back into the habit of writing and have several more essays in the pipeline. I wish everyone a Happy New Year!

Subscribe now

Share

Google created search engines, where information could be retrieved via a collection of keywords. They “solved” “search”. It is however a serious mistake to take this too literally.

More recently, we have gained LLMs, which often can be rather useful as a search engine. The kinds of questions that can be asked, and the detail of the responses, is much greater. Nevertheless, while LLMs can answer more complex and nuanced questions than Google, I find that they generally seem to be comparable to someone with an encyclopedic memory but who doesn’t think deeply about what any of this knowledge means or how it may be connected.

Of course, “search” can be defined even more broadly and abstractly.

For example, if we are seeking knowledge and answers, a text query of any kind will often be a rather poor and limited tool for finding unknown unknowns. How do you write a query for something you don’t know you’re looking for? Random discovery is often a better tool for at least this problem even if it is not particularly efficient.

How does one approach random discovery? Wikipedia seems somewhat better at this than Google or LLMs, in that one can very easily wander through links and rather quickly discover unexpected knowledge only distantly related to their original subject of interest. There is also no real “algorithm” filtering this discovery based on your typical habits, only your natural curiosity and the proximity of unexpected details to the things you were looking for.

A conventional print encyclopedia may have its own advantages that have been lost in the digital age, in that its volumes or table of contents may be organized by category, forming an atlas for the high-level arrangement of knowledge. Wikipedia is a vast iceberg, its true depths are not obvious from the surface, and it’s hard to comprehend or anticipate what it does and does not contain. An encyclopedia, a shelf of books in comparison, may be organized by subject in a way that categorizes all contained knowledge and gives some impression of roughly how much is there. There may be many surprises on a small scale, but for the most part a reader can quickly grasp the range of knowledge represented and their own knowledge or ignorance of it.

Of course, a simple list of categories may not reveal much structure, and some graph-based atlas may be an even more fine-grained tool.

We must also keep in mind that the sum of human knowledge is not and never will be complete. There are things we don’t understand, questions we have not answered, and no amount of searching the archives will locate such answers. You may instead be forced to search the world, something that Google and LLMs are not well-optimized for.

Most people spend at least some part of their lives – the unlucky ones perhaps a very long time – lost and in search of meaning, purpose, or inspiration. A text query isn’t very effective at resolving this. Some combination of religious experiences and life struggles will often prove a much better tool, though even this is only so effective. Religions and the meaning they offer have been subject to change, adaptation, and innovation over time, and presumably further innovation in the frameworks we use to guide people toward meaning should be possible. I personally suspect however that the generator functions – the ideas that motivated and produced these religious texts – have largely been lost, and that much of the decline of religion is the result of no longer understanding why people for so long placed such immense value in these ideas and frameworks, or how why some died out and others successfully spread to billions.

Some questions are reducible to math and are computable. Some may be in principle computable, but require mathematics that has not been developed yet, perhaps only in some distant or indefinite future. Others may require knowledge of the world that may or may not be possible to gather, or facts of history that have been lost to time. Some questions – whether they can be formalized or not – may simply be intractable or incomputable.

Not everything that is computable is something we presently know how to compute, and not everything we know how to compute is something we know how to productively apply. There is an infinite tower of complexity classes called the Polynomial Hierarchy, of which nearly all algorithms humans write fall into just the bottom couple levels, overwhelmingly only the first. Presumably the higher levels are increasingly more powerful and theoretically more useful, but we today have very little idea of what any of it is good for. Even then, the Polynomial Hierarchy is only a small part of the wide array of known complexity classes, which presumably are only a tiny part of all possible classes.

Ultimately, ambition is a skill. The actual range of possible achievement is for all intents and purposes infinite – both unfathomably vast in scale and even more vast in variety. Finding things that are bold enough to be worth doing while still remaining achievable in the present day is not a trivial problem. It’s best not to harden your goals in life too early, as with more time and experience you can often see even greater and more worthy goals. Of course, we stand on the shoulders of giants – we are able to catch rockets out of the sky not because we have bigger brains than people of the past (brain sizes actually declined after the introduction of agriculture), but because we have the advantage of centuries and millennia of accumulated knowledge. Our map of what is possible and how to get there is ever-expanding.

Ultimately, we should recognize that the problems we are working with today will seem but mere toys to people centuries or millennia from now, who will work with vastly more general and powerful tools than anything we can imagine today. On one hand there’s a humility in that, in knowing that we can only achieve things that are in a certain sense rather small and nearby. On the other hand, it also tells us that beliefs that a given hard problem is “solved” should not be taken too seriously, as most important and general problems in fact have effectively infinite scope.

Human imagination is actually a very limited thing, much more limited than we often assume, and something we should aim to expand and improve. Telling ourselves that hard problems are “solved” when they are not only serves to halt imagination of anything greater.

The hard problems are never truly over.

This has been a more philosophical article today. I’m hoping to have some more technical articles out soon, including another Bzo Dev Log once I can get a few more things done.

Share

Subscribe now

The Cerebral Cortex in the brain is composed of hundreds of thousands of cortical columns. This is a form of general-purpose learning circuit, and there are minimal differences between the columns found in the visual cortex, the auditory cortex, the motor cortex, the prefrontal cortex, or any other part of the cortex. With very few and largely insignificant exceptions, the six layers and the circuit they form is universally applied to all functions of the cortex. If something like the grid cells that the brain uses as a coordinate system can arise in the motor cortex for example, that implies that every part of the cortex regardless of function has circuitry that is capable of computing coordinates under the right conditions.

The most biologically plausible and detailed model of how this circuit works is the one being developed by Jeff Hawkins and his team at Numenta. The model is far too complex and nuanced to describe in detail here, but one of the foundational ideas is that the cortex is fundamentally built around prediction. A basic observation that drives this is that surprising events, even as simple as the ground being unexpectedly uneven while walking, are often extremely noticeable. If something unexpected is so noticeable, that implies that the brain must be at least implicitly computing a range of what is expected.

However, if we start asking what information is required to generate certain kinds of predictions, this begins to imply many other forms of data that the cortex must be tracking. Say you have a mug. There is a handle, there is an embossed image on the side, there is a rough, unglazed ring around the bottom. If you place your fingers on the mug and feel around, your cortex must know where each finger is relative to the mug, as well as the mug’s position and orientation in order to predict where each particular feature is. If your knowledge of the mug is wrong, if something feels off or out of place, you’ll notice quickly.

The current iteration of Numenta’s theory suggests that the location and orientation of various object and sensor reference frames must be encoded in each cortical column, and specifically layer 6 seems by far to be the most plausible candidate for where this occurs. For every patch of your somatosensory cortex that receives tactile information from a patch of skin, there is a layer of neurons tracking the coordinates of that sensor, relative to the body and relative to objects in its vicinity. The same goes for other sensory domains as well, patches of the visual cortex presumably track the 3D location in space of the object they’re observing.

Column structure is very consistent across the cortex, but many are not connected to sensors. Yet these brain regions also include the same circuitry that others use to track location. While perhaps it is currently difficult to reason about how abstract thought may make use of reference frames and coordinate spaces, the extremely consistent structure of cortical columns implies that effectively all cortical functions make use of some notion of locality and space.

Even when faced by a seemingly alien form of computer such as the brain, we should assume that the ultimate limits are generally the same even if the building blocks are different. If our mathematics leads us to conclude that computing a particular algorithm has certain fundamental limits, or should scale in accordance with a particular function, we should expect this to hold regardless of if the computation is implemented with proteins, neurons, transistors, or something dug out of a UFO wreckage.

Nevertheless, we don’t seem to ask many such questions, often dismissing such systems as largely impossible to understand. The brain clearly is not operating at full capacity at all times, but it’s rare for anyone to ask a question like what the brain’s equivalents of, say, malloc() and free() are.

Locality in Conventional Computing

Any computation can be unrolled into a computation graph. Each instruction is a node, which receives data from previous nodes and performs a transformation on them to produce a new value, which can be used by other nodes downstream.

If a loop has a hard upper bound on the number of iterations, the unrolled loop forms a finite compute graph. If there is no upper bound, the loop forms an infinite graph. Unless it is a genuine infinite loop for all possible inputs, you can expect at least one, often multiple possible exit points that spill out into other parts of the program, or perhaps even to the program’s final endpoint.

All computation can be encoded as such a vast and potentially infinite graph. Encoding the graph explicitly in full detail is infeasible for nontrivial programs, but these graphs generally contain so much repetitive structure (for example, in loops) that they are highly compressible. Many constructs of programming languages, as well as the concrete computations that they compile down into, amount to methods of compressing and decompressing these graphs.

The linear sequence of instructions that a CPU performs is a serialization of this graph. Functions may consist of fragments of this graph, and the CPU’s instruction pointer encodes a coordinate for which part of which fragment of this graph the computer is currently executing. Of course, this graph is usually far too large to encode with a single 64-bit pointer, and so we employ call stacks and nested functions to extend this address space to arbitrarily large spaces, with code fragments embedded inside code fragments, embedded inside other code fragments.

If we have a for loop, this loop will generally include a variable that tracks which iteration we are currently executing. If we have nested for loops, our position in this computation graph will be two-dimensional, which makes sense as such nested loops are often used to iterate over things like two-dimensional arrays. Trees are effectively just a discrete form of hyperbolic space; stacks and similar data structures often serve as coordinates for computations embedded in hyperbolic and tree-like spaces.

All of this ultimately is a way of procedurally generating this vast compute graph, determining which instruction exists at each coordinate point, and which point to land on next to perform the next computation. Many of the instructions the CPU performs don’t even directly perform the real computations, but rather serve to navigate this scaffolding.

Even in parallel computing – if we wish to perform some computation for every element of an array, we’ll generally assign each thread a coordinate or range of coordinates to iterate over. CUDA, the software framework for computing on Nvidia GPUs, is built around generating large multidimensional arrays of threads.

Conventional computation is littered with tools whose sole purpose is to encode positions which track our location on computation graphs, however there are some places where their presence is less obvious. The primary exception can be found in constructs like while loops and recursive functions, which also happen to be the primary places where undecidable and otherwise ill-behaved, infinitely-looping program behavior often arises. This isn’t to say that such constructs always lack such coordinates; a while loop can certainly emulate a for loop with an iteration variable, it just doesn’t have to, or can encode another more exotic coordinate space. A recursive function which performs a binary search encodes a range of possible locations for the result it’s searching for, which it progressively narrows with each iteration in a way that is guaranteed to terminate quickly.

Such language constructs are in a sense an admission that the various coordinate systems we have built many of our computing tools are not an exhaustive list and that there is sometimes a need to break free from them and build something bespoke or exotic. We can certainly fail in this of course, and build something which simply wanders aimlessly through some vast abstract space, with no obvious force driving it toward any exit point in any reasonable amount of time.

Decideability and Geometry

A powerful tool often used in static code analysis is the abstract domain, a mathematical object used to place bounds on the space of possible values that a variable or collection of variables can encode. Abstract domains are based on lattices, which as I’ve discussed before are very closely related to topological spaces. Fundamentally, our best tools for modeling and bounding the behavior of complex programs are geometric in nature.

One of the most universally used tools for understanding the behavior of algorithms and data structures is Big-O notation, where we represent our computation with a polynomial and then by convention drop all constants and everything but the most significant term. This practice of dropping terms has motivations, but is merely a convention and if we choose to ignore it we are left reasoning about computations in terms of polynomial expressions, which closely resemble Taylor Series and similar sums which are incredibly commonly used in physics. The presence of Taylor Series also implies that we can perform calculus on the properties of our compute graphs, which further implies the potential for geometric tools.

Physics, and its vast and chaotic swarms of particles, has absolutely no obligation whatsoever to be legible to humans, and yet once again our best tools for understanding it are consistently geometric in nature. The invention of calculus revolutionized physics, allowing us to better understand complex curves geometries, and revealing their presence not only in the static shapes of objects but in their complex and chaotic dynamics.

I’ll make a conjecture here – if all of our tools for making computation predictable and well-behaved can be interpreted as operating on geometric coordinate spaces, if our static analysis tools for automating the understanding of programs are based on geometric tools, if our most powerful tools for understanding even the most chaotic systems we can understand in the world around us are consistently geometric in nature, if the most obvious ways in which programs end up losing decideability properties involve removing obvious coordinate spaces, and if even billions of years of evolution seems to have converged on computation that tracks coordinate spaces for everything, then this suggests that there may in fact be a deep connection between what is and isn’t decideable about a dynamic program or other complex system and the extent to which its behavior can be mapped to some geometric structure.

Subscribe now

Share

Business investment is always risky, but tech VC is notorious for its extreme tolerance of risk, far beyond that of most other types of investors. Very often this is simply accepted as the price paid for innovation, without any deeper thought. Often the exact same people who insist that extremely high risk is purely a matter of innovation will, in a different conversation, preach about how every field outside of computing had stagnated since the 70s, an era when Silicon Valley VC spray-and-pray strategies would have been seen as insane.

Founding a business is fundamentally an assertion of contrarian truth about the world, that there is some opportunity that you can take advantage of that everyone else has missed. There is always a serious chance that you’re wrong about this, or at the very least wrong about how to reach this opportunity. There is always risk. However, this axiom of risk asserts that risk must exist, but it tells us absolutely nothing about how much risk – is there a 10% chance of failure? 50%? 90%? 99%? Fundamentally, the risk involved is the inverse of how much certainty there is.

If you decide to open a new Starbucks, there is a tremendous amount of data about the last 10,000 Starbucks locations, and so deciding the right location to maximize profit is a science more than it is an art. People have been digging up rocks from the Earth for a very long time, the risks of opening a mine can be greatly mitigated by some surveying. If you decide to start a software business, there’s far less risk if you’re building some dime-a-dozen SaaS product for which a standard playbook has been built. The risks are far greater if you’re venturing off into something far more experimental.

There is risk of opening a new Starbucks location and there is risk in starting a wild new startup. This is not a fundamental difference in kind, only in magnitude.

Certainly, some of the risk of tech investment is probably downstream from the fact that software businesses often produce very high returns; it’s far more common to 100x an investment in a software company than it is in a company with physical products or other conventional constraints. Investors may be engaging in a form of risk homeostasis.

Tech bubbles pretty universally fail to live up to their hype and expectations. Each new technology will absolutely reshape the world in unimaginable ways, if you buy into the hype.

I recall watching the Internet of Things bubble in 2017, when the near future was apparently full of smart toasters and smart juice presses that would collect so much data that we’d invent a dozen new SI prefixes to describe how many bytes we’d need to store it all. Buzzwords like “Brontobytes” and “Geopbytes” were thrown around, describing the scale of datacenters supposedly a couple years away. It turns out that very few of these smart devices provided any meaningful value and the data they collected was mostly worthless while creating a myriad new security concerns. This somehow dovetailed into the 2017/2018 crypto bubble, and soon everyone was raving about how your toaster would be uploading the data it gathered from you onto a public blockchain.

There is a certain mythology to how technology is understood which shapes how people imagine the future and where they invest their time and money into building it. This mythology may be widely believed but can easily turn out to be wrong. I remember when the 2017/2018 crypto bubble was flooded with millions of people who asserted with absolute certainty that Bitcoin was a slow and primitive prototype, and by some immutable laws of technology that it would soon be slain by a new and more innovative cryptocurrency. This never happened, and the contrarian Bitcoin maximalist narrative that Bitcoin would become a store-of-value reserve currency, trusted by nation states, where transaction speed didn’t matter much seems to be far more plausible today.

On the other hand, I also recall a great deal of fuss about cryptocurrencies like EOS and Solana, which abandoned the core ideology of decentralization that motivated other crypto projects in favor of maximizing performance. Scaling blockchains while maintaining all the properties people wanted never succeeded. The wild optimism about using arcane mathematics and protocols to reshape the world that I saw in 2017 seems long dead, and the Nasdanq meme gambling thesis seems to have won out against all odds. Solana is now extremely popular as a host of meme coins, presumably because decentralization is of zero importance if you’re simply running a casino.

Facebook has repeatedly chosen to make gigantic bets on these tech bubbles. After its ambitions to create a cryptocurrency faced political challenges, it renamed its company to Meta, hired game development legend John Carmack, and dumped $40 billion into building “the Metaverse”. Carmack eventually quit and the company faced enormous difficulties getting its own employees to even enjoy using the technology they were building. Apple also chased this trend, building their own AR/VR headset. My perspective was that while it may perhaps be a success eventually there is currently no real killer app for AR/VR, and that escapism into gaming is a dystopian vision that normies won’t adopt as easily as tech people expect. I debated many friends at the time, who insisted that Apple was a magical company whose products could never fail, that the billions being spent were a guarantee of success, and that we’d all be wearing their VR headsets in a matter of months. I recall a friend who would respond to every criticism with “Siri, show me that girl naked”, and how all progress in the internet was driven by porn. These friends have since all long forgotten that they ever made such claims and have since started AI companies.

As for the present AI bubble, I consider it a mixed bag, and a long discussion for another time.

Venturing into the Fog

Computing is a very strange, new, and unprecedented field. We are venturing into the fog, and the tools we’ve built for understanding the world over thousands of years are not very enlightening. The family of visions of the future of computing that you will find in San Francisco and its derivative tech hubs, that guide us in building the future, that tell us where the fruit worth picking can be found, are perhaps only 5% correct. This implies enormous potential in building alternative tech hubs with wildly different visions of the future of computing, provided some contrarian truths can be found and deeply mined. Perhaps they will prove equally blind and misguided, but getting even a different 5% correct could be profoundly valuable.

Given a mere 32 bytes of data, there are more possible configurations than there are atoms in the observable universe. Moore’s Law could have ended decades ago and we’d still be able to mine the Software Library of Babel for new algorithms and ideas until the end of time. Even then, we still wouldn’t have made the slightest dent, we would have simply exhausted all of the resources provided by the entire universe first, which is unfathomably tiny in comparison.

We are in the very, very, very early days of computing, and there is a strange future that stretches out far ahead of us, vastly longer and vastly stranger than any of us can possibly imagine.

There are many path dependencies in the modern computing industry. There are many decisions made decades ago that have heavily shaped modern computing, where a different outcome could easily have led us down a wildly divergent path.

The modern range of popular operating systems were designed decades before the internet, on the assumption that the user was an expert who wasn’t stupid enough to run untrusted software on such expensive business equipment. The failure of operating systems to provide acceptable sandboxing of applications littered the early internet with security landmines. Browsers on the other hand provide sandboxing by default, and web apps began growing in popularity as soon as Google made Javascript run fast with V8. If the operating systems of the 90s and 00s had better security models, the domination of browsers as a de facto operating system may not have occurred.

If CPUs had supported efficient context switches directly between userspace programs, the performance bottlenecks that pushed the 1990’s growth in driver code into the operating system may never have happened. In 1995, Linux was a mere 15,000 lines of assembly, and began growing rapidly once it bolted the bloated USB drivers into the kernel. We could easily have taken an alternate path where operating systems stayed small, and where hobbyists or startups could compete with Microsoft.

If digital payments had been more quickly adopted and trusted, it’s possible that advertising wouldn’t be such a common monetization model for software. The extreme focus on attention maximization has undoubtedly shaped much of modern computing, and many of the widespread complaints about the internet and its impact are likely downstream from this. Our interactions with computers favor quantity and volume over the quality of interactions, whereas if people had to more often open their wallets to use software we may have instead seen a bias toward quality over quantity.

Complexity Theory has a number of very misleading theorems and conjectures, where “real world problems” frequently end up landing in weird corner cases where harsh laws turn out to be less harsh. There certainly exist programs whose behavior is undecideable, but there are also infinitely many programs which are perfectly decideable, and these are the building blocks that humans prefer to build things from. There are many problems which are very useful to be able to solve which happen to lie in complexity classes like NP and are therefore exponentially difficult – but O(1.0001^N) is no less of an exponential than O(2^N), and real-world problems frequently lie closer to the former than the latter. We have likely stunted progress substantially by spending decades wrongfully teaching millions of CS students that good static analysis is mathematically impossible and that broad classes of useful algorithms are “intractable”.

If we take such harsh laws at face value and consider information-theoretic limits on Kolmogorov complexity, we would be forced to conclude that the entire field of data compression should be laughably impossible, something that is empirically very untrue.

Where else may the future take us? Where can we look for new threads to pull, new orchards to search for unexpected fruit?

Computing is a rather odd field, though I expect that over the next century we will increasingly realize that an enormous amount of geometric structure can be found from the right perspectives. Lattices, one of the fundamental tools used in the static analysis of code, are almost exactly mathematically identical to topological spaces. A great number of difficult problems for reasoning about the properties of code boil down to reasoning about the shapes of the trajectories that programs take.

In many ways, computing is vastly simpler than physics. We’ve engineered computers to be understood, but the universe has no obligation to be legible to us. Nevertheless, much of the progress of the past few centuries of physics has come directly from applying geometry to better understand physics. The ancient Greek thesis that geometry is the foundation of all of mathematics will likely prove to have a great deal of truth to it, and I expect any disbelief that computation and logic may have sewn will gradually fade in time.

It’s notable that we already describe the complexity of algorithms with polynomial expressions. The convention with Big-O notation is to drop all constants and all but the most significant terms, but if choose to ignore this convention we are left describing our algorithms with mathematical tools that often closely resemble the Taylor Series approximations that are widespread in physics. We could even very easily employ tools like calculus for studying these algorithms.

There are also other very interesting ideas I’ve been thinking about lately that seem to imply a connection between decideability and geometry, though I’ll save that for a future article.

Human engineering is constrained by that which we understand. The word “ape” refer to both the branch of primates humans descend from, but also can refer to a clumsy mimic or mime. Humans are fundamentally a species which acts through mimicry, by learning to replicate patterns in the world, and then extrapolate and compose them to reach goals, even if we march past countless simpler solutions in the process.

Much of the value of machine learning is that it can synthesize algorithms for many kinds of difficult problems without the biases and blind spots of human engineering, or at least very different biases. The incomprehensibility of the weights inside machine learning models are a glimpse of algorithms that humans may not fully comprehend and master for a very long time. You may stare into these arrays of numbers like ancient people stared into the chaos of ocean waves and storm clouds, seeing powerful, unknowable gods in what their descendants would describe with a few Navier-Stokes equations.

There is still abundant chaos, but we’ve at least made progress, and we can even harness this understanding to build fantastical things like flying machines.

Complexity Theory describes a vast menagerie of classes of computation. The P class contains the vast majority of algorithms that humans use, and even then we mostly confine ourselves to a very tiny, quasi-linear subset of it. While NP problems often contain an exponential factor that can limit scaling, it contains many useful problems which are vastly tamer than people expect. Furthermore, P and NP are only a tiny part of a much larger family of classes called the Polynomial Hierarchy (PH). Most classes here are generalizations of NP which tend to be exponential-time problems (which may be tamer than they look) with an additional polynomial slowdown. All of PH is contained within PSPACE, the class of problems which can be solved with polynomial space and at most exponential time.

There are also many other problems which technically don’t fall into PH such as counting problems and interactive proof systems, which are not always infeasible to solve either. This generally is the result of PH being defined in a very constrained way to limit their scope and make mathematical progress easier, and there are many kinds of algorithms which break these constraints without necessarily becoming impractically hard; they just play by slightly different rules.

Optimally solving chess is technically in EXPTIME, which is more difficult than PSPACE and by extension all of PH. Exhausting the entire chess search space is pretty infeasible, but approximating this well enough to outperform human chess grandmasters was done in 1997.

Yet the vast majority of the tools we’ve chosen to use remain in the very tiny quasi-linear subset of P. We’ve mapped out a vast wilderness, but we’ve so far chosen not to explore much of it.

We’ve seen Moore’s Law drive over half a century of exponential growth in computing power. Even if it ends soon and computing power stagnates, the range of things we could do with the hardware that we have is far greater than anything we can possibly imagine.

Subscribe now

Share

Nanometer Numbers

The cutting edge of transistors are “2 nanometer”. Intel has abandoned nanometers as a metric and have begun using terms like 20A, 18A, and 14A, measuring in angstroms (1/10 of a nanometer, or the approximate width of a typical atom). However, these nanometer numbers are entirely fake today. This used to be an objective measure of the width of the gate on a planar transistor, but planar transistors stopped working 15 years ago. Their current substitute, FinFET transistors, have no equivalent feature to measure. The physical feature these numbers once measured no longer exists, but some naming scheme is still needed.

The advertised transistor density of modern 2nm nodes is somewhere around 200-250 million transistors per square mm. There are one trillion square nanometers in a square millimeter, and dividing this out this gives us a footprint of 4000-5000 square nanometers per transistor, or roughly a square 60nm on each side. The gate pitch, or minimum distance between transistors, is generally in the range of 30-40nm, and the 60x60nm footprint is a result of geometric constraints on how densely irregular circuits can be packed, leaving unavoidable empty space.

The extremely tiny features on a chip are produced via photolithography – a light is flashed through a photomask to project an image onto a wafer coated in a light-sensitive photoresist, which hardens or liquifies depending on its exposure to light. The tinier the image can be made while retaining focus and detail, the smaller components can be made.

The hype over the past few years around EUV (Extreme UltraViolet) lithography is mostly in its extremely fine resolution. Previous generations of lithography equipment relied on wavelengths of ultraviolet light measured in hundreds of nanometers, which puts limits on just how small of patterns can be created on a chip. This was cheated somewhat using multipatterning – if your “pixels” are 200nm across, you can shift the image by 100nm and project a new image, giving you some extra resolution in how the two images overlap. You can even do this multiple times, and employ a number of other tricks to cheat out much finer details than should be theoretically possible, at the expense of much more complicated and defect-prone manufacturing. EUV uses light with a wavelength of only 13.5nm, which lands in the overlap between “extreme UV” and “soft X-rays”. Being technically a form of X-ray (prone to passing through things), optical manipulation of this light is difficult – magnification requires atomically-precise curved mirrors relying on the Bragg effect to diffract the X-rays backward, as traditional materials are very ineffective at conventionally reflecting X-rays. X-ray lenses are effectively off the table.

EUV gives us “pixels” of about 13.5nm to build our chip from. ASML is now rolling out its “High-NA EUV” machines, which exploit some optical tricks to get this resolution down to about 6 or 7nm.

These “pixels” are still dozens of atoms across. We could theoretically scale down further – a 6x6x6nm cube has about a quarter million atoms. I suspect that if someone were simply willing to bite the bullet and attempt a 1:1 scale between wafers and photomasks, all this complicated optics could be eliminated and soft X-rays with 1-2nm wavelengths or less could be used. A typical medical X-ray tube is orders of magnitude more energy efficient at turning electricity into photons than the method of lasering molten tin droplets that’s currently used to produce EUV light. An X-ray tube is a simple vacuum tube with an electron beam between a cathode and anode, and the specific wavelength produced is a function of the atomic number of whatever material that the anode is made from.

The next problem however is not the wavelength of the photons, but rather the chemistry of the photoresist. Conventional photoresists are based on polymers – long chains of molecules that link or unlink when struck with photons. Linking them together forms a spaghetti-like knot on top of the silicon. However, the links of these chains are not infinitely small – they are made from atoms just like everything else, and they only work when forming long enough chains. Conventional photoresists stop working below 7-10nm, and most unconventional resists stop working below 5. I’m not aware of any clear plan for scaling past this fast-approaching point.

Transistor Geometry

In a transistor, the voltage of the gate lying on top of the channel controls the conductivity of the channel beneath it, either creating an insulator or “depletion region”, or leaving the silicon naturally conductive. Of course, the effect is localized, and the further away from the gate the effect weaker it is. About 15 years ago, the gates got small enough that the insulating region no longer could reach very deep into the silicon. This shallow depletion region is hardly a barrier to electrons at all – the electrons began to merely travel underneath.

Planar transistors, which had served as the workhorse of Moore’s Law for nearly 50 years, had to be abandoned in favor of FinFET transistors – rather than the gate lying flat across the channel, the channel is now a thin, vertical fin, and the gate wraps around three of the four sides. This increases the surface area between the gate and the channel, and means that this shallow depletion region needs only to reach toward the center of this thin fin to block off the flow of electrons.

FinFETs created a dramatic increase in manufacturing complexity, and forced several big players like Global Foundries to abandon chasing Moore’s Law. They have been the new workhorse, but will not be nearly as long-lived as planar transistors. Instead, after a mere 15 years they have already reached their limits and are being phased out in favor of GAAFETs – Gate-All-Around Field Effect Transistors. Intel calls them RibbonFETs. Rather than a vertical fin with a gate wire on three sides, GAAFET has the gate wrap around all four sides of the channel, and then stacks several channels on top of each other for extra reliability.

GAAFET is a dramatic increase in complexity on top of FinFETs – the layering is particularly hard, especially as there must be a thin insulating layer between the channel and the gate, and this layer must now wrap around all sides of a stack of channels. This alone multiplies the number of manufacturing steps, and thus multiplies the number of opportunities for defects, which eat into the bottom line of running a chip fab.

Even GAAFET is only expected to last a couple generations before it too runs out of steam. The end of the road seems like it may be CFET. Transistors naturally come in two flavors – PMOS and NMOS, and the CMOS logic that serves as the foundation of nearly all digital logic involves using the two together. CFET is simply GAAFET, but stacking the NMOS transistors on top of the PMOS transistors without much attempt to scale them.

Other Limitations

AMD launched the first 1 GHz processor in 2000, and Intel was soon boasting that they’d have 10 GHz processors by 2010. It is now 2025, and the CPU in my laptop maxes out with a mere 4 GHz boost speed. Where’s my other 6 GHz? Where’s all the further gains Intel was supposed to deliver in the past 15 years?

What had enabled this clock speed growth for decades was Dennard scaling, the observation that as transistors are scaled down the power density of the silicon (watts / sq mm) should stay the same. However, leakage currents and other factors do not scale with size. At large scales these are negligible, but on small scales they are not. Dennard scaling died around 2006, and clock speeds have largely stagnated ever since. The rising power density of the silicon has several other effects – rising power consumption, more extreme cooling systems, and an increasing need to disable large portions of the chip at a time to reduce power consumption.

Currently, one of the big technologies that Intel is betting hard on is backside power delivery. There are two types of wiring on a chip – power and logic. The logic wiring tends to consist of many small, thin wires that switch quickly and have low capacitance. The power wiring consists of a smaller number of large, thick wires with a consistent voltage and high capacitance meant to absorb the combined voltage swings of all the circuits attached. For a variety of engineering reasons it is not ideal to have these two near each other, and so Intel has begun fabricating chips with the bulk of the power wiring engraved on the back, with millions of tiny wires passing through the silicon.

This is a very difficult technology – a challenge that TSMC is not nearly as daring in bringing to market. If Intel can solve their presently high defect rates, corporate warfare, and stave off bankruptcy and buyouts, this technology may serve as a valuable advantage. On the other hand, this is an optimization that could have been introduced decades ago. Why now? Because it is a high-risk and relatively low-reward, one-time optimization. You cannot repeat this multiple times for greater gains. The gains, while meaningful, are certainly not enormous. This is not the kind of optimization that you make when the future looks optimistic – this is the kind of thing you do when you’re running out of ideas and have to start scraping the bottom of the barrel.

The trend toward chiplets and die stacking over the past decade, while not completely new, is also perhaps another modest optimization that’s becoming increasingly attractive as alternatives dry up.

Photomask prices have begun to skyrocket in cost. Photomasks serve as a fixed cost for the production of chips – you need to pay for a set of masks whether you intend to manufacture a thousand chips or a billion. Masks used to cost a few hundred thousand dollars, but with 28nm hit $1M, at 7nm hit $10M, and are somewhere around $40M at 3nm.

If you’re manufacturing 10,000 chips at 3nm, the masks alone will add $4,000 to the price of each chip, which may otherwise be <$100. If you’re at Nvidia’s scale and are manufacturing hundreds of millions of chips, then masks may only add a few dimes to the price of each chip. This essentially kills small players at advanced nodes, and perhaps risks undermining foundry business models like those of TSMC, which assume a large number of small and medium-sized customers. With rising photomask prices, all but TSMC’s biggest customers may simply be unable to afford to make their chips at these advanced nodes. As fabs continue to get more expensive to build, at some point it may no longer be possible to find any way to fund building them – or worse, investments are raised and a factory is built on the false assumption that customers will materialize as they always do, but this time with unrealistically deep pockets.

If you’re paying $40M or more for masks alone (not counting all the silicon, manufacturing, packaging, testing, etc.), perhaps a $150M EUV machine doesn’t look so pricey anymore. Or perhaps the machines start looking a bit overengineered and overpriced and you start reading up on X-ray physics and photolithography.

Future

Gordon Moore always emphasized that his “law” was fundamentally rooted in economics, not physics. How small or fast or efficient a transistor can be made in a lab is of absolutely no relevance if they can’t be mass-manufactured at a price anyone is willing to pay. The death of Moore’s Law has been predicted for many decades, and progress has continued to march on far longer than anyone expected. It outlived even its namesake.

Nevertheless, nothing here looks good. This is not some single technical obstacle, but many hard technical and financial walls fast approaching all at once. If there is any way forward, it will require a serious departure from the past few decades of work. If it is not competitive on day one with manufacturing processes that have been optimized and perfected over more than six decades, it stands no chance in the mass market.

Invention often favors the easiest solutions to find over those that are the most efficient. Estimates for the size of the codebase of ASML’s lithography machines border on the billions of lines of code. This seems to be less a result of necessity, and more of a culture of copy-pasting code out of paranoia and the extreme pressure to ship on a tight schedule. There are tricky things that such a machine does, but a billion lines of code is pretty implausible for a company of this scale to even write without heavy automation or copy and paste. Some of this is essential complexity, the rest is likely one of the world’s largest mountains of technical debt.

Once it is no longer feasible to simply build a bigger fab, the only way forward is to go back and simplify things. If costs can be reduced substantially, then what would have been a half-trillion-dollar next-gen fab could be a bit more affordable, or a current-gen fab could be built on a much smaller budget. It may take a lot longer than two years to simplify things enough though.

This isn’t even particularly unreasonable – there are already multiple examples of people doing lithography in their garage that’s on par with the state of the art from the early 1990s. Once a viable path has been found, speedrunning it is often far easier and cheaper than the original stumbling in the dark. Technology also improves over time in many dimensions, and what may have been an absurdly difficult technical problem 10 or 20 or 30 years ago may be easily solved with a more recent innovation. If a literal teenager can build a 1990-level chip fab in his parents’ garage today, then what can be achieved with some startup capital?

By far the largest factor driving high costs in semiconductors is the demand for extremely low defect rates. Chips are often assumed to be near-perfect. Highly parallel processors and memory chips can often tolerate higher defect rates through redundancy. Highly parallel chips designed from the ground up for extremely high defect tolerance could perhaps be manufactured with a vastly lower-quality, and by extension vastly cheaper chip fab. Memory chips have even been using complex 3D structures for decades that logic chips shy away from – high defect tolerance may not only lower costs but may also open the door for extremely advantageous manufacturing techniques that a defect-sensitive fab like TSMC may be too afraid to touch.

This is an industry where dozens of equipment suppliers must heavily coordinate to build tools for three major companies. This is an industry that likely should have embraced vertical integration at least a decade ago. Technological progress very rarely follows a smooth curve, and often follows more of an uneven stair-stepped trajectory. Moore’s Law is perhaps a self-fulfilling prophecy, and perhaps served as a powerful solution to the difficult problem of coordinating such a large and complex supply chain to innovate in sync. However, it’s very unlikely that this was the most efficient possible path, free of any wasteful diversions. I wouldn’t be surprised if huge efforts for short-lived technologies like strained silicon could have been completely skipped.

Used Car Markets

Another possibility that has long been on my personal list of “future articles to write” is that the future of computing may look more like used cars. If there is little meaningful difference between a chip manufactured in 2035 and a chip from 2065, then buying a still-functional 30-year-old computer may be a much better deal than it is today. If there is less of a need to buy a new computer every few years, then investing a larger amount upfront may make sense – buying a $10,000 computer rather than a $1,000 computer, and just keeping it for much longer or reselling it later for an upgraded model.

Most cars aren’t purchased new, but are glorified hand-me-downs. People with deep pockets (or perhaps poor financial sense) purchase new cars for tens or hundreds of thousands of dollars, and several years later sell them at an enormous discount and go buy a new one. This makes a valuable asset that would otherwise be prohibitively expensive affordable to a much larger portion of the population.

As of late, I’ve been busy working on startup stuff with The American Compute Corporation. My priorities and ambitions have shifted somewhat and I’m not as stubborn about bootstrapping as I used to be. I’m currently raising some pre-seed funds, aiming to build high-performance, high-efficiency, general-purpose CPUs designed from first principles. I’m also increasingly convinced that, with the right compromises and specialization, a startup-scale fab may be viable and is at least well-worth the experiment.

A new chip architecture is also as much a software problem as it is a hardware problem. The Bzo compiler project went on the backburner for a while, but I’ve picked it up again recently and will hopefully have a dev log with new things to share soon. I have big ideas for the software for these chips. There’s an immense opportunity to rethink many different aspects of computing.

We’re entering the post-Moore era, I’m busy designing chips (and maybe a fab) for this new world. I’d be happy to talk to investors.

Subscribe now

Share

Each successive question narrows the range of possibilities. This is also a matter of information gain, which operates on a logarithmic scale. Uncorrelated questions can narrow possibilities exponentially, though can sometimes be hard to find. An idealistic case might be a binary question where half the solutions answer yes and the other half answer no. Assuming a sequence of such ideal questions, the game “20 Questions” should be able to narrow down to one of a million possibilities.

There are many different systems – information, design, art, engineering, software, literature, and so on – where possibility spaces are exponential in size. These kinds of questions can be asked a tremendous number of times while still leaving a tremendous number of options available.

32 bytes of information has more configurations than there are atoms in the universe. Every additional byte added multiplies that number by a factor of 256. This sentence, encoded in ASCII, takes 48 bytes. The numbers here get unfathomably vast extremely quickly, calling such numbers “astronomically large” is giving way too much credit to astronomy.

Questions are also constraints. A problem will impose constraints on what constitutes a valid solution. This limits possibilities in the same way that these questions do. Adding more degrees of freedom grows the state space exponentially, and adding constraints shrinks it exponentially.

Tendency Toward Complexity

There is a tendency to engineer very complicated solutions to problems. Simplicity is usually preferable, but complexity often is unavoidable and creeps in anyway.

If we place an upper bound on the complexity of a solution, it limits the total space of solutions. If there are a trillion solutions, but we ask a mere 40 binary questions, then we’ve narrowed the space down by about a factor of about a trillion and it’s likely that only a single solution satisfies all of our constraints. In practice, this tends to be more probabilistic – as mentioned, perfectly binary constraints/questions are an ideal case, and avoiding correlations between constraints can be difficult, and some constraints are a bit more lopsided where answering one way narrows things down by much more than half. Narrowing down the space like this, there may be many more than one left, or it may be none at all, depending on the exact interactions and correlations of the constraints. But it should on average be about one.

This explains complexity – as we apply more constraints to a finite possibility space, solutions get exponentially less abundant until none remain at all. If we add more complexity, we can buy some possibilities back.

Of course, we also need our constraints to lead us to solutions well. We can simply assert a list of constraints, and there may truly be solutions that satisfy them all, but finding these solutions can nonetheless be like finding needles in a haystack.

Finding solutions that satisfy a single constraint can often be easy, and constraints with a similar structure can often be compatible with each other in a way that makes finding solutions easy. In general however, finding solutions that satisfy many constraints can often be almost as hard as randomly searching for them, and when you’ve narrowed possibilities by a gigantic amount, this can make finding solutions similarly difficult. One simplified way of looking at this is through the perspective of Abstract Domains and Galois Connections, though that’s a complicated subject for another time.

Finite, Real-World Cases

Sometimes you have a problem that is less of a design problem and more of a search problem, in the sense that the options you’re sorting through already exist in the real world. There are undoubtedly some magnificent lost ancient texts preserved in the Herculaneum scrolls, but we can also be certain that there will be no shortage of significant works that are lost for good. We could perhaps place a few simple constraints, and by a numbers game – there are likely a couple thousand scrolls total in the ruins – find many different ancient texts that fit a short list of random constraints. However, if we get too specific, if we add too many constraints, the odds that a particular thing that we hope to find there quickly becomes very unlikely.

There are also background and sampling biases - in this case, the ancient library was owned by an Epicurean philosopher, and many of the texts that have already been read are on specifically this branch of philosophy. Texts on unrelated subjects almost certainly exist in the library, but may be in the minory. Good initial, near-binary questions for narrowing down texts in the library may be less like “are you looking for philosophy, science, or history?“ and more like variants of “what branch of Epicurean philosophy are you looking for?”

32 bytes, a mere 256 bits, has more configurations than atoms in the universe. Anything made from atoms will necessarily exist in a quantity far fewer than this. If you have a couple hundred binary constraints, you likely could search the entire universe for the item you seek and come up empty-handed.

Shaping Rocks and People

Rocks of different shapes may be suited for different uses as tools. While rocks may be extremely abundant on Earth, they are in finite supply, and of course vastly smaller than the number of atoms on Earth, which itself is vastly smaller than the number of atoms in the universe. A few dozen binary constraints on the shape of a rock that decide its use as a tool could easily exhaust the entire Earth’s supply. If you are looking only at rocks within walking distance and close to the surface, expect to be able to place fewer constraints. And if your constraints are tight enough, good luck searching all that area for just the right rock, and good luck finding a new one should you ever need one again. Early humans who wanted better tools could thus not rely on simply seeking out a rock of the right shape, they were forced to shape the rocks themselves.

If anything, the deeply human trait of “tool use” has much less to do with applying nature as we find it to solve our problems, and much more to do with engineering and shaping what nature provides us to better solve our problems. Primates are occasionally seen using sticks and stones as tools, but shaping the sticks and stones to create better tools is certainly a much more meaningful step that is less often observed.

There are many different changes that can be made to an object. The range of things that object has the potential to be transformed into with the right sequence of modifications is vastly larger than the single object it is at the present moment, and this buys a far greater constraint budget.

8 billion people may sound like a lot, but it only takes a mere 33 binary questions to narrow a search to a single person. If you have a few constraints that are uncommon, if you’re looking for someone who’s a one-in-a-hundred person in a few different ways, it might only take four or five such constraints before not a single person is left assuming the constraints aren’t correlated.

Dating apps that have framed modern dating as merely filtering for a perfect person are probably less effective than old-fashioned approaches where two people in the same town simply paired up and made things work. Big cities might provide the illusion of high optionality, but certainly cannot provide exponentially higher optionality. Despite the vast majority of human history lacking birth control, dense urban centers historically have almost always had much lower fertility than more rural areas.

Computational sorting may improve options and reduce incompatibilities, but there’s a very hard limit on how picky people can be, and unless your standards are particularly low you can expect it to be difficult to find someone without any need for adaptation. People also change over time. Eventually if your standards are high enough, your only option is to find someone close enough and make things work. It’s probably best to find someone who’s as close as you can to what you’re looking for and fix whatever incompatibilities that remain.

There is also great interest today in reshoring a lot of manufacturing. This fundamentally requires creating many new industries and businesses that require skillsets that are currently not in great abundance. Not even an AI-accelerated job board website with a billion dollars in VC funds would make a dent in this problem - a company that requires very niche skills may quickly exhaust the world’s entire supply of experts. At some point a training program is the only solution.

If you’re picky, you can only lean on the outside world to solve your problems for you for so long. Solving problems by shaping the world yourself is the most human way forward.

A lighter and less technical article today for Valentine’s Day I guess.

Subscribe now

Share

Human Software

Rituals and routines are human software, memes that spread because they are [perceived as] useful.

I generally think of the mind not as a single program running on some hardware, but rather as many little pieces of software. They are downloaded and synthesized via mimesis and observation, and they interact in various ways, sometimes positively and sometimes negatively. They compose surprisingly well and seem to rely a lot on semantic/associative links.

Clearly the brain stores a lot more knowledge than it can properly index. “List every fact you know” is a laughably impossible task, but clearly all that information is in there somewhere and retrievable somehow. Any fact that you have learned and have not yet forgotten can be summoned with the right stimulus. Sometimes things are very deeply buried. There are things deep in your memory that may be there for the rest of your life, but that you may never think of again for lack of the proper stimuli.

Replicating human intelligence very likely is not merely a matter of copying the software, but also copying the architecture by which this software can be composed, as well as the ability to rapidly acquire new software. Existing neural networks are far too data-intensive, requiring many thousands of examples to learn what humans can learn with only a couple examples.

Digital software allows us to automate many previously cognitive tasks. There is great merit to the computer analogy for the brain, though the brain is by all means a very alien form of computer. Nevertheless, the ultimate limits of computing will apply just the same to both, and the two should be expected to converge in many ways as pushed toward their limits, as they in many ways can be expected to have similar limits.

But human software can rely on a great deal of common sense, general knowledge, and likely a great deal of deeply buried cognitive infrastructure. While all running on very similar neural hardware, people of wildly different cultures can see the world very differently, and certainly people from merely a few centuries ago would struggle in the modern world, let alone prehistoric people. Even notions like progress and linear time are historically abnormal and relatively recent developments. Human and animal sacrifice was for a very long time a near-universal human practice, understood by all to be critically necessary in a way that modern people clearly fail to recognize.

Digital software may formally encode a great deal of what’s necessary to accomplish a task, but there’s also a great deal that it doesn’t encode. The entire problem of human-computer interaction is of course the interaction between digital software and human software, and the human side will continue to lack formal specification. Whatever drives a person to use a piece of software, whatever method they use to gather information from their mind and from the world and funnel it into the machine, is all human software that is completely independent of that which is stored in the machine. It is just as critical if not moreso, it cannot be abstracted away or removed, and it can vary enormously, even from person to person.

I remember a conversation with my mother, a nurse, about medical record-keeping software. There is standard, near-monopolized software, and there are major efforts to standardize things further, but to a large extent every hospital will use the same software in different and idiosyncratic ways.

As we better understand software, the brain, and the harmonized interaction between the two, we'll be able to push even greater limits of what each can do.

Computers as a Bicycle for the Mind

We’ve had writing for about 54 centuries, with proto-writing systems dating back at least 200.

We’ve had sculpture and other sophisticated visual arts (cave painting, etc.) for at least 430 centuries, with more primitive abstract art dating back perhaps a thousand centuries or more.

Modern computing is less than one century old. I don’t think we’ve achieved Steve Jobs’ goal of a “bicycle for the mind” – a tool for greatly augmenting human thought, analogous to how a bicycle dramatically increases the efficiency of human locomotion. As impactful as the smartphone and personal computer have been, the idea that we have it all figured out and that deeper and more valuable use of computers is not possible is an obvious failure of imagination.

Perhaps falling short of such an ambitious goal is a failure on Jobs’ part, but I expect it is more likely that this project is not merely a simple project that can be shipped in a few years but a great human project that will take centuries. He himself argued that computing a century from now would be vastly better than today while explaining the bicycle analogy.

To a large extent, human excellence is not a matter of nature but of environment and routine. Your habits and behaviors contribute enormously to shaping your mind and body. While perhaps raw genetically-influenced brainpower is a significant factor, often the most remarkable people have undergone great and unusual hardships, or come from strange and unique environments, or are simply very mentally ill. Great and influential people are often otherwise ordinary people who have some internal contradiction or conflict intensely motivated to change the world around them to resolve it.

There is only so much information about who someone is that can be encoded in and influenced by genes, much of the rest will be environment. Likewise, the behavior of a computer can vary extremely wildly based on how it is programmed. There are more tiny programs that fit into a mere 32 bytes than there are atoms in the universe, more than could ever be explored. There are short snippets of code that could in principle be typed into a computer that could achieve remarkable things that no civilization – human or alien or otherwise – will ever live long enough to see. The Library of Babel is vast.

A person who can train eight hours a day for years in a sport has a much better shot at being a world-class athlete than someone who spends their time differently. A bright person prone to deep thought may form a recognizably different style of intelligence from someone who reads voraciously while spending relatively little time reflecting deeply on or synthesizing the disparate things they have read. Someone who spends their time debating others will cultivate another very different skillset, as will someone who negotiates business deals all day.

Someone who spends their time thinking over things purely in their head may have a more entangled web of knowledge while someone who spends their time writing may perhaps become better at serializing their thoughts. Someone who does calculations by hand may gain a very different intuition for numbers than someone who types their calculations into a computer. If we bring a wider range of software into people’s lives, the way our tools for thought shape how we think may continue to shape new styles of thought that we have yet to see.

I would not be surprised if the Greek’s approaches to axiomatic mathematics and formal logic were downstream from their early adoption of alphabetic writing, which reduced writing systems which were previously extremely complex and logographic down to a relatively small set of general symbols. More ancient mathematics and reasoning was often more intuitive, arithmetic algorithms taught through repetition but lacking any notion of formal proof. Students would often be told a procedure, told to go through hundreds of examples pressing numbers into clay or writing on papyrus, and then eventually just intuit the patterns and understanding as to why the magical procedure worked.

Humans may have common interfaces such as language and culture, but this hides an enormous amount of underlying diversity and complexity. Different people, even with a tremendous amount of shared understanding, can nonetheless have wildly different ways of thinking. There are billions of bits of information flashing around the human brain every second, and many tens or hundreds of trillions of synapses. Meanwhile, human speech maxes out at only a few dozen bits per second. There is certainly much more data embedded in body language – every part of the cortex talks to and influences every other, and the brain does a great deal of thinking through action – but this is not well-understood or very consciously acted out, and mapping billions of neuron spikes to a couple hundred muscles still is necessarily an extremely lossy process.

Historical Human Augmentation

Early Greek mathematics often was more appropriation than invention, with many Greeks sailing to far-off lands in search of ancient geometric wisdom. They would then return to Greece and share what they had learned. As mathematics became more axiomatic with Euclid, the ability to rigorously build upon existing knowledge in new directions was made easier, and complicated concepts that could not be easily intuited from a few examples became much more viable. What had previously been superhuman mathematical reasoning suddenly became relatively common.

And this is far from the only example of information technology expanding human abilities, even in ancient times. Before the obvious example of literacy, preliterate oral cultures have made near-universal use of song and poetry – the additional regular, repetitive structure makes memorization of large volumes of text much easier. At the very least, these additional constraints can aid in error correction, narrowing the options of which words could come next in case that one’s memory grows hazy.

Poetry allowed the pre-literate Greeks to memorize their immense epics like the Iliad and Odyssey, whose combined length is about half that of the Bible. Chapter 2 of the Iliad even contains the “ship list”, in which Homer autistically lists off detailed statistics for every ship in the entire Greek navy. This is sometimes argued to have functioned as a way for ancient bards to show off their impressive memorization skills even more.

Aboriginal Australians are known to have used songs for navigation, called Songlines. These were songs that contained sequences of directions and landmarks as directions to points of interest, often interwoven with mythology. These could also serve as a form of passport – travelers who could accurately sing native languages and follow the directions were better respected and faced fewer threats in uncommon territory than those who could not. Many Aboriginal languages augment this with absolute direction – you do not have a left foot and a right foot, but an east foot and a west foot, and if you turn 90 degrees you now have a north foot and a south foot. Being so deeply baked into language, one cannot communicate without learning an extremely accurate sense of direction.

Memorization augmentation is not limited to only rhyme and meter, of course. The Method of Loci, or memory palace, has been known for thousands of years. Imagining yourself wandering through a vast palace, and using imagined objects in each room as hints can shift semantic information to spatial memory, where it may face less congestion.

Furthermore, mathematical notation and the manipulation of written symbols developed for algebra and subsequent mathematics was an enormous advance over ancient mathematics – what would have previously required a lengthy paragraph analogizing things to lines, squares, and cubes of various interrelated constructions can instead be replaced by a concise collection of symbols. Transforming those symbols into related equations which may reveal different insights is often a matter of mechanistically following some simple rules and applying some basic pattern matching and reasoning. This is a powerful aid to mathematical thought and for abstracting to difficult problems whose geometric meaning may be difficult to immediately intuit. Difficult leaps in logic can sometimes be replaced by manipulating symbols with simple rules.

Future Human Augmentation

The cerebral cortex is made up of modular brain circuits called cortical columns. Regardless of if one looks at the visual cortex (vision), auditory cortex (hearing), prefrontal cortex (abstract thought and planning), motor cortex (muscle movement), gustatory cortex (taste), or any other cortical region, and regardless of if one looks at humans, rats, cats, dogs, raccoons, or most other mammals, the structure of cortical columns remains shockingly consistent with the same 6-12 layers (depending on how one decides to count them), with consistent cell types and connected to each other in a consistent circuit. There is some variation – aquatic mammals often only have 3 layers and the human visual cortex divides layer IV into a couple extra sublayers – but these are rare exceptions.

This strongly implies that despite there being many patches of the brain with accepted functions, there is in fact very little specialization in the cortex – what makes the visual cortex is not that millions of years of evolution have pre-trained or optimized it for vision, but simply that it gets information from the retina. Experiments have been performed swapping the optical and auditory nerves in fetal ferrets, sending visual information to their auditory cortex and auditory information to their visual cortex, and they learn to see and hear just the same.

The visual cortex that evolved for hunting and navigation can be repurposed for drawing art, for reading, or for operating a computer or smartphone. No natural selection is required at all to rewire it. The human brain is incredibly general-purpose, and will certainly be capable of learning no shortage of future inventions and augmentations. As I have discussed many times before, sensory substitution and sensory augmentation are surprisingly underexplored technologies for their potential – the brain will simply learn to recognize patterns in whatever data you give it, no matter the sensory domain. Writing and data visualization are extremely successful examples of this. There are also many niche examples, such as blind people teaching themselves echolocation.

It is sometimes said that you remember 10% of what you read, 30% of what you hear, 70% of what you do, 90% of what you teach. Other numbers and orderings are thrown around, but the general principle – that passive absorption of information engages memory less than active engagement – is fairly consistent. One observation here is that more active engagement here would also involve a larger number of brain regions.

Whatever it is that you are seeing, hearing, feeling, or thinking, your brain needs some way to encode and represent it. If it lacks the neural coding to currently do so, it will repurpose existing neurons to learn a representation for this new information. Engage more with an idea and your brain will form a deeper representation of it. If that representation is stored entirely in your visual cortex because you only engage by sight, it must compete for space with everything else you engage with visually. If you instead begin engaging other senses, then we can argue from first principles that this model will be spread across other brain regions, potentially regions that are much less congested.

It is also worth noting that literacy has the effect of shifting a very wide range of functions over to the left hemisphere of the brain where the language circuit is, and reorganizing the visual cortex substantially to optimize the recognition of lines and corners. Another piece of personal speculation – since the right hemisphere often takes on more social reasoning roles, the religious tendency to anthropomorphize forces of nature may in fact move related reasoning to the less-congested right hemisphere, which may have enabled more complex reasoning on these subjects. It’s worth noting that one of the greatest cultural influences on early Christianity was Hellenistic Judaism, led by thinkers such as Philo who argued that Jewish theology, if read allegorically, had come to very similar and compatible conclusions on many subjects to the Greeks.

The human cerebral cortex has somewhere on the order of 20 billion neurons and 10-100 trillion synapses. Neurons fire at around 500 Hz and activity is generally rather sparse – only a couple percent of neurons may be firing at a time, though the exact amount can vary substantially over time. Assuming 2% sparsity, we can estimate that the cortex performs somewhere around 1-10 quadrillion operations per second. Each synapse may only store a few bits in terms of its weight, though there is also evidence suggesting that neuronal pattern recognition is timing-sensitive, and that synapses physically migrate up and down dendrites to adjust the timing, suggesting even more degrees of freedom (this may also increase computational power). Nevertheless, the synapse count is high enough that the brain’s storage capacity is well into the tens if not hundreds of terabytes.

This is a very large amount of computational power, and the large numbers involved are perhaps an indicator that in many domains we may not actually be pushing the computational or storage limits of the brain very hard. After all, as previously discussed ancient people found many ways of pushing limits well beyond what we would today expect to be possible. With future techniques, greater assistance from technology, and deeper neuroscience and psychology understanding, it may be possible to push the limits well beyond what we currently understand.

The vast majority of software we run on our computers is laughably poorly written and inefficient. Even code that makes attempts at optimization may often be implemented in languages that are fundamentally sloppy or wasteful with resources. A difference in quantity often results in a difference in kind – while greater computing power enables existing applications to be scaled further, it also enables new applications which were not possible before. Additionally, the most optimized version of a problem is almost never the first to be found, and so something that is just barely possible with a new generation of computers may, in time, be optimized to run on much weaker machines. A modern GPU may be comparable in performance to a supercomputer from 15-20 years ago, but the GPU is certainly far more useful than the supercomputer. There has been more time to figure out what exactly can be done with that much compute, there has been more effort into optimization to make costly tasks more tractable, and new ideas on what to use computers for in general will have been developed. The timescales may be longer, but there is no reason why this cannot apply to the brain just as well. With developments such as writing and advanced mathematics, it largely has occurred.

My personal critique of the Mentats in Dune – humans trained in mental computation to replace banished comptuers – is that the feats of such Mentats are often unimpressive compared to what may actually be possible based on the numbers. The people of Dune apparently understood intelligence well enough to replicate it in a machine, try the technology out for thousands of years, turn against it and banish the technology, and then spend ten thousand years using this deep understanding of the mind to optimize human thought, and still don’t seem capable of operating on more than a few thousand numbers at a time.

Regardless of the imagination of 1960s science fiction, I expect that the next few centuries of human-computer interaction will enable us to not only automate many of the things we use our minds for today as is often assumed, but to further push the limits of what humans can do. I do not however expect any unified such effort, but rather immense fragmentation – humans doing fantastical yet diverse things by pushing their minds very far in very different directions. Perhaps we will be most constrained by our need to interact and interface with each other, requiring that people’s minds don’t become too distant from each other.

We may see mathematics Mentats, engineering Mentats, historical Mentats, even Creative Mentats, and perhaps a wide range of other things well beyond our current conception. This will not only require more deeply understanding the mind and how to push its limits, but also understanding the types of neural computations required to accomplish these tasks, how to shape these through the diverse array of tools humans have historically augmented their minds with, and what new forms computers now enable for both offloading work and for shaping thought.

The early adopters of proper Mentat software may find themselves seemingly superhuman by historical standards. Intellectuals and creatives with seemingly impossible powers, in much the same way that memorizing the entirety of the Iliad may seem superhuman today, but was completely achievable with Bronze age cognitive technology.

And to answer the obligatory question these days - while “AI” technology is neat, I think it’s fundamentally very different from what I’m talking about here. There’s perhaps a role for AI methods to be applied, and there likely are many cases where the ultimate limits of the brain and the limits of other extreme computing systems share some resemblance, but I expect the tech tree for deepening human-computer interaction to be a very different tech tree entirely, and that there are no simple answers to this problem no matter how much AI is involved.

Subscribe now

Share

For example, Alan Turing’s original proof on undecidability shows that if we had a computer that could, for any given program and any input to that program, produce a perfectly accurate prediction of whether that program loops forever or if it eventually halts, that this machine would create a mathematical contradiction. Therefore, it cannot exist. This is often assumed to mean that answering such a problem is entirely impossible, but there is no reason at all why this would apply any less to the human brain than it would a computer algorithm. With that in mind, ask yourself, does the following loop halt?

while(true){
// twiddle our thumbs
}

This isn’t a trick question or anything. To anyone with a basic understanding of code, this is an obvious example of an infinite loop. Has your brain just violated mathematical law? Have you just proven that the human brain is not a computer but instead something magical? Nope, absolutely nothing of the sort, you’ve simply shown that the common narrative around what undecidability means isn’t very accurate.

If you dig into the details of Turing’s paper, you find at its core a diagonalization argument, which is just a method for showing that one set must be bigger than another. His argument really just boils down to a proof that the set of all possible program states which encode proofs of properties of other program states is smaller than the set of all possible states.

You actually can attempt to build a program that tries to answer undecidable problems. One trivial method would be to simply run the input program until it halts, and then report it as having halted. Another method might be to track all of the states that an input program can reach, and to look for loops or repeats. Plenty of other creative tricks can be devised as well, but inspecting the failure cases is interesting. Ignoring the fact that many of these algorithms could easily be hopelessly intractible to actually run because of how vast even very small program state spaces are, what we find is that any program we think of must either sometimes give a wrong answer, or the loop-detecting program itself will ironically get into an infinite loop.

We can in fact build a correct, approximate undecidability solver if rather than insisting that our loop-detecting program always return YES or NO, it can also sometimes return IDK. Technically speaking, returning “I don’t know” for every possible input is correct, albeit not very useful. Reducing the number of IDK cases to zero is impossible, but correctly answering in at least a subset of cases is certainly not.

And of course, halting is only the classic example of an undecidable problem - really detecting any nontrivial property of a program’s behavior is undecidable, per Rice’s Theorem.

Humans generally design software that is meant to be understood. If our programs truly contain a lot of properties that are impossible to verify, that’s often a good indicator that there is a bug in the code somewhere.

“Here Be Dragons” Analysis

There’s a method I like to apply to studying difficult problems which I call “here be dragons” analysis. The basic idea is to divide things up into a number of distinct cases which you may or may not be able to say useful things about. If you can show that many of these cases are well-behaved, you can show that the badly behaved ones must have or lack certain properties related to how you divided things up.

Let’s say we know something about our program, namely that it fits into a finite amount of memory. Finite memory implies a finite statespace, and therefore at least in theory the entire state space can be enumerated (even if this is too costly in practice). Therefore, this situation is actually decidable. If we have a powerful enough computer and enough time, we can trace out any path through the state space and look for cycles and guarantee when a program does or does not loop forever. Therefore, the truly undecidable problems cannot be programs that fit neatly into a finite amount of memory. They must in fact grow forever!

Then we can look at the unbounded cases. If we have a program that simply expands in a simple way, we might be able to figure out some clever analysis that determines the rule that it follows and show that it grows without bound. This is another case where the program fails to halt, albeit one where the program doesn’t so much get caught in a loop, but where it simply grows forever.

Then there’s another case - one where the program jumps around through some statespace without much of a pattern, but gradually expanding to larger and larger memory requirements without an obvious limit. If we cannot devise some analysis to detect a pattern here, then how can we tell if the program is spiraling off into chaos or if it eventually repeats without simply waiting a potentially infinite period of time for it to retrace its steps?

This unbounded chaotic expansion case is therefore the one case where we cannot show that the program eventually halts, and therefore every program which is truly undecidable must behave like this - jumping around without an obvious pattern while slowly (or rapidly) expanding its memory footprint. Granted, as mentioned before, there are a great number of cases that may be theoretically decidable which in practice are too expensive to verify.

One particular implication of this, albeit one that may not be appreciated by mathematics enthusiasts, is that in many respects the famous Collatz Conjecture is a good example of a problem which seems to fall into this potentially-undecidable case. Its inputs jump around chaotically, sometimes shooting up very high before dropping back down. Despite being a shockingly simple problem, it stubbornly refuses to yield to any kind of mathematical analysis.

In fact, at one point John Conway even proposed FRACTRAN - an esoteric programming language designed around a generalization of the Collatz Conjecture that is Turing-complete. So we know that it’s not hard to generalize Collatz into something that is provably undecidable in at least some cases.

Hierarchy Theorems

Beyond simply doing some trickery with diagonalization, Turing’s original proof of the undecidability of the halting problem relied on another interesting detail; self-referentiality. The specific example case that was shown to be undecidable is when a halt-detection program is asked to answer questions about a slightly modified version of itself, augmented to return the opposite of its. The specific way in which this is done results in a variation of the liar’s paradox, where the program is caught lying no matter what it answers.

Pinocchio Paradox : Pinocchio proclaims “my nose grows now”

The consequence of this is that many provably undecidable problems boil down to computers attempting to prove things about themselves. These are not the only cases which are undecidable, but just some of the simpler cases to prove.

However, we can actually get some valuable intuition for what’s going on here by looking at the Time Hierarchy Theorems, which are in effect a finite version of Turing’s proof.

The THTs look at what happens when we restrict computers to be able to run only for a finite amount of time. The analogue of the halting problem here is asking whether or not a program finishes running before a certain time limit. If we have any question we wish to ask about the behavior of a program beyond simply what it outputs when we run it, then doing so requires adding some amount of additional work that must be performed. This work is not free, and increases the runtime of our program. As a result, many possible programs that could run on a time-limited computer and finish close to the limit will, given some additional overhead, fail to finish before the limit. If we wish to answer our question for every possible program that can be run on our time-limited computer, we cannot do so unless we are given a computer with an even longer time limit, enough to account for the full time limit of the first computer plus the overhead for our specific question about it.

Put in simple terms - to be able to answer questions about every possible way in which a computer could behave, we need a computer that is fundamentally even more powerful.

The way I like to intuitively frame this is that the Time Hierarchy Theorems are the actual rule about how undecidability works, and that Turing simply proved a few decades earlier that the THTs don’t stop applying even when we have an infinitely powerful computer. Answering questions about an infinitely powerful computer, where programs could run for an infinite amount of time, requires more than an infinite amount of computing power, which is why the diagonalization argument involves both countable and uncountable infinities.

Implications

So if undecidability in fact is not actually so bad, then perhaps we should stop using mathematical impossibility as an excuse for why we don’t have better development tooling.

The truth is that actually a great deal of sophisticated static analysis is already done inside the guts of many modern optimizing compilers in order to determine which optimizations are and are not safe to apply. Those decisions require some sophisticated understanding of how code behaves. The methods outlined in this article are very general-purpose, but not very practical or efficient, whereas plenty of static analysis work has been done to find methods that are more useful, at least for the kinds of programs humans tend to write.

The problem is that this is largely designed to be some invisible black box applied during compilation, largely invisible to the programming language or the human. Perhaps that’s something we should be rethinking.

This has been a relatively short article tonight, on a subject that I’ve referenced a number of times but had been meaning to get around to addressing directly. I’m trying to write a bit more often, and honestly I have a backlog of interesting articles that I’ve been meaning to write for a while that I should just get out.

I have also been busy with other work, and that usually takes priority.

Subscribe now

Share

Directly computing quantum mechanical problems is generally exponentially hard, and even the cheaper approximate algorithms often still have nasty time complexities, such as O(n^7) or O(n^11). These are much better than exponential time, but still very impractical even at fairly small scales and may not be sufficiently accurate on account of being approximations. This is not merely a problem of “not enough compute” – these problems scale fast enough that Moore’s Law couldn’t compete even if it were plausible that it could last another few decades without major hurdles. The compute requirements are so high that some Kardashev 12 civilization that can tile the observable universe with thermodynamically-optimal super-GPUs would still find a great deal of hilariously tiny problems that they would struggle to simulate.

However, we are in the very early days of computing. Modern computing as a field has existed for only a little less than a century, and we can be guaranteed that an enormous range of groundbreaking discoveries in computing will remain undiscovered for many centuries or longer.

The symbolic approach to physics is interesting, in that it has yielded centuries of enormous progress, and there are no obvious signs that it is slowing down. The Hamiltonian mechanics formulation of physics has been around for nearly two centuries, and has in recent decades been augmented by significant progress in the field of symplectic geometry, a branch of geometry that studies spaces where transforms preserve the area of shapes. The primary problem with symbolic physics is simply that it was built for a world where compute was expensive.

So allow me to speculate where things may go in the future and where the next orchards of fruit may lie based on some hints we can find today if we know where to look.

Computer Algebra Systems and Scaling Symbolic Physics

Sussman’s classic programming textbook “Structure and Interpretation of Computer Programs” (SICP) is very famous, but far less well-known is the sequel “Structure and Interpretation of Classical Mechanics”, a physics textbook where the equations are replaced by Scheme code. The book also argues that physics traditionally doesn’t “typecheck” and that translating seemingly unambiguous equations into computer code is actually not a trivial task because of the amount of approximations and contextual information that is often left out.

The book centers around a Computer Algebra System that Sussman built himself, which can take physics equations written in Scheme and apply a variety of transformations on them that are useful for physics work. It can of course handle some very complex symbol manipulation simply because of how fast computers are.

It seems to me as though there is a great extent to which computers could automate at least some amount of symbolic physics work in a way that symbolic problems that would be impractically large for humans could be done by machines. Furthermore, we can start asking questions about the nature of this automation. Writing a mass of symbols on a page is one thing, but once it’s represented in a computer you have a more explicitly structured object – a tree or graph or something else of this sort. If the system of equations is viewed as a tree or graph, what might its structure imply? At the very least are there interesting implications for visualization that might not be obvious from historical approaches?

In general, what I expect we will find in the coming decades, if people choose to investigate this further as opposed to shutting their brains off and letting their AI algos go brrr, is that there is a middle ground between symbolic and numerical physics. Or perhaps simply an extrapolation of symbolic physics – rather than focusing on small, simple structures that humans can parse out manually, what kinds of dramatically more complex structures can we extract and operate on if we can throw a few teraflops or petaflops of compute at it?

Certainly, there seems to be some evidence that “AI” bears fruit when applied to physics, but if anything this is evidence for my point – there is structure in physics that computers can extract that is simply too complex for humans to do manually. However, the matrix multiplication that is the foundation of modern machine learning algorithms may be very versatile but it is often not the most efficient method of computing things. With a bit more cleverness, we likely will in time find more conventional mathematics and algorithms that can answer many of the problems that machine learning is starting to answer, but more efficiently and while yielding deeper understanding.

The math behind quantum mechanics is just linear algebra with very large matrices. These matrices also have additional constraints, generally that they are square and that they only encode rotations. Other more specific constraints may exist in more specific scenarios.

At least in the case of quantum computing, which granted is constrained to be easier for humans to reason about than mere random assemblies of particles and forces, the matrices that encode various quantum logic gates are often very sparse, and very symmetric. The operations on them generally include matrix multiplication and kronecker products, both of which lend themselves well to being reduced to simple operations on block matrices, or essentially matrices of matrices.

There are plenty of techniques that can make use of block structures in matrices. As someone who used to be very interested in game development, quadtrees look like a very applicable tool for mapping the various regularities in quantum computing matrices, both for tracking large zero blocks in matrices, as well as tracking when two blocks of a matrix may be identical to or simple transforms of each other, which may yield itself to symbolic simplifications of these giant matrix operations.

To a large extent, this kind of approach would simply be transforming operations on gigantic arrays of numbers into more complex but ultimately cheaper operations on trees that store compressed representations of these arrays. If handed a truly random problem perhaps this may not be much of an advantage, but if there is really so much structure in physics that we can spend centuries making progress through manual symbolic manipulation, then there likely is a great deal of structure that can be exploited in such problems.

Additionally, we should look to tools like SAT solvers and Abstract Interpreters.

SAT

SAT, or the Boolean Satisfiability Problem, as I have discussed many times before, is the classical example of an NP-complete problem. It is in the NP complexity class, which makes it in the worst case a fairly difficult problem, but it is also expressive enough to be able to emulate any other problem in NP, which covers a truly gigantic range of common problems. An algorithm that can efficiently solve SAT problems is in a sense a general-purpose problem-solving algorithm.

Such algorithms have been developed. While SAT was once dismissed as exponentially hard and fundamentally intractable, methods like DPLL and CDCL have shown that reality is far more complex. There are fairly small problems with only a few hundred variables that seem to be intractable, and others with millions of variables which should be unimaginably impossible and yet can be solved in under an hour by a single CPU core. Asking the complexity of a SAT problem is like asking the complexity of an interpreter – it depends entirely on the problem you give it, which can vary extremely wildly. You can hand an interpreter a tiny, infinitely-looping program which takes an infinite time to run, you can hand it a gigantic but well-behaved program that runs fairly quickly, or you can give it anything from a gigantic zoo of other program behaviors. SAT is no different, and real-world problems are very often surprisingly tractable.

In the case of SAT, the original DPLL algorithm worked okay for a long time, but it wasn’t until the development of CDCL in the late 90s that things really kicked off, and today there is a whole zoo of different heuristics and other features bolted onto SAT solvers to assist in solving problems. Much like the aforementioned quantum problems, we can view DPLL as operating on a tree that compresses an impractically large flat array of values for cheaper computation, but the more advanced techniques that developed later and built on top of DPLL were eventually what made these solvers dramatically more practical and useful.

There is actually a surprising amount in common between SAT and quantum mechanics – the former can be viewed as equivalent to simple operations on exponentially-large arrays of boolean possibility values, while the latter is simple operations on exponentially-large arrays of complex probability amplitudes. There certainly are differences, but the general organization of the arrays and the operations on each value are shockingly similar. SAT solving algorithms increasingly pay attention not only to the tree structure that DPLL generates, but also the graph structure of the SAT problem encoded in the interactions of constraints, which is something that should be just as applicable to quantum circuits.

I wouldn’t be surprised if deeper research into this in the coming decades can start to transfer ideas back and forth, and many of these quantum problems become much more tractable with classical methods. And by the time that happens, perhaps we’ll have a few more decades of SAT solver breakthroughs to apply as well.

Abstract Interpretation

Abstract Interpretation is another very interesting case that approaches even less tractable problems. Predicting the behavior of a program statically – as in, without running it – is generally considered undecidable, effectively mathematically impossible. However, this is really only the case sometimes, and there are a great deal of programs in practice that are relatively well-behaved and can be reasoned about. Turing’s original proof about the undecidability of the Halting Problem is, contrary to popular belief, not at all a proof that the halting problem can never be solved, but merely that the total number of states that a computer can be in is greater than the number of states that just happen to encode proofs about whether or not other program states eventually lead to an infinite loop.

Abstract Interpreters approach this problem by attempting to track abstract statespaces of a program. They check whether or not there is some possible input to a program that can result in the program reaching a known bad state, such as a crash. However, rather than running every possible input of the program (which may be practically infinite for many programs), they instead assign each variable some kind of lower-resolution model, such as an interval that encodes known largest possible and lowest possible values. If the program adds two numbers, their minimum and maximum values can be used to easily compute the minimum and maximum of their sum. Other “abstract domains” beyond simple interval arithmetic can track other approximations of state spaces, with the main constraint being that the set of states tracked by the domain should always be a superset of the actual statespace of a variable – if it says that X is between 0 and 10, it should be impossible for X to equal 11 or -23 or 8015, but it is not strictly necessary that X must be capable of equaling 8. There are additionally methods for sound approximations of loops, control flow, memory, and more.

Abstract Interpreters get around the problem of undecidability by occasionally answering “I don’t know” rather than exclusively “yes”or “no”. Their models of variables are sound approximations, but they are still merely approximations. They will occasionally insist that the program may have a bug even when it doesn’t, but they won’t tell you it’s safe when it’s not.

The reason why Abstract Interpreters are interesting for physics is that, like SAT solvers, they provide a library of practical computational methods for attacking incredibly general and practically useful problems that many would dismiss as fundamentally unsolvable. Furthermore, their focus on intervals and other structures seems very useful for physics, and not terribly far off from tools that are already occasionally used. Interval arithmetic is occasionally used in physics, and tools in chaos theory such as Lyapunov exponents share a lot in common with them, but Abstract Interpretation also contains many ideas for taking these methods and doing far more complex things with them that help to give good answers in cases where naive interval arithmetic may simply explode and insist that the result is “somewhere” between negative and positive infinity.

I expect that it will not be trivial to translate these ideas into physics, and frankly there should be a much greater effort to translate these ideas into practical software development tools rather than mere academic projects or niche industrial tools for verifying that planes won’t fall out of the sky.

Overall though, computing is still in its infancy, and efforts to adapt the past few centuries of physics work to the capabilities of modern computers are still very primitive and surface-level. Furthermore, it’s worth remembering that much of physics work historically has not been in applying new mathematics to physics problems, but rather using physics problems to inspire the development of new mathematical tools, which can then be applied elsewhere. We should not be surprised if the deeper investigation of computational physics also leads to serious breakthroughs in the broader field of computer science as well.

This is a relatively short article I threw together in a few hours late at night, with a bunch of rough ideas I’ve had on my mind for a while. I hope you found it interesting, and maybe a few readers might be inspired to start investigating this themselves, it certainly would be helpful and I expect there’s a lot of valuable work waiting to be done.

Subscribe now

Share

So I have here a much shorter article on some odd properties it has that seem to get less attention and some speculation on how they may be explanatory in biology.

Kolmogorov Complexity

Kolmogorov Complexity is ultimately a measure of how compressible information is. The KC of a given string of bits is equal to the shortest possible computer program whose output is that bit sequence.

There are several factors that complicate things here. For one, the precise complexity of a given string is extremely dependent on the language used to write the program, and thus this measure of complexity is always relative to a specified language.

There is a type of recreational programming called “code golf” that illustrates this dependency well, where people compete to write the shortest program to solve a particular problem. Some languages are more expressive than others, some are better tuned to specific problems than others. Various “code golf languages” have been developed, which often turn popular code golf challenges into single-character operations. There are many such languages where a “hello world” program is simply an empty source file, thus having zero length.

There are also information-theoretic limits. N bits of information can only encode a maximum of 2^N possible strings, and no amount of fancy language trickery changes this. As a result, if you pick completely random strings of N bits, only a very tiny proportion of them can possibly be encoded with very short programs, some may turn out longer than N bits because of overhead of the language (a program that simply prints a very long string needs some way to encode the print statement itself, or at least to differentiate between a string to be printed and a string literal to be used as an input for further computation), and on average the KC of such strings must always be N or greater.

Then there is the problem of the generality of the language itself. For maximally optimal data compression, a Turing-complete language is theoretically desirable. However, this is also where things get tricky. Determining whether a Turing-complete program has a particular kind of behavior, even as simple as determining if it ever stops running, is in general undecidable. Searching over the space of undecidable programs to find one that is in some sense optimal makes the problem harder, and therefore computing the true Kolmogorov Complexity of a string given a Turing-complete language is not merely undecidable, but actually turns out to be unimaginably worse - fundamentally incomputable.

In general practice however, understanding the behavior of programs is not nearly as bad as it is in theory, mostly because only a subset of programs are actually impossible to understand, and humans tend to build things that have convenient properties and are meant to be understood. If your program has behavior that is actually impossible to verify, that’s a very good indicator that you have a bug somewhere and it is doing something it is not meant to do.

However, in the case of KC we don’t care about engineering a program to be understood, we care about minimizing code size at absolutely all costs, and therefore all the bizarre, random, badly-behaved programs that no human in their right mind would ever write remain as options, unable to be dismissed. Kolmogorov complexity in its true form is not something that can be measured or studied directly, but instead is always a mere approximation of some deeper, inaccessible form of complexity.

One way around this is that we can give up on Turing-completeness. This means there will be certain types of structure in strings that will now be inexpressible, but we may at least be able to constrain our search to programs that we can make certain practical guarantees about. This sacrifices optimality but makes the problem much more tractable. If we assume that most of our data probably has a fair bit of random noise anyway, for example in the case of compressing images or other media files that sample the real world, such bizarre structures may be exceedingly rare and well worth compromising on.

A problem I haven’t mentioned yet is that, if we wish to compress a string, we must have some algorithm for converting the string into a program that actually generates it. Constrained languages can make this problem far more tractable than it would be in practice.

Technically speaking, all the classical methods of encoding compressed data - Huffman coding and the like - are languages, just highly constrained ones that are often very far from being Turing-complete. Methods like run-length encoding which encode simple looping constructs, and dictionary coders which can vaguely resemble function calls if you squint hard enough, are perhaps more obvious examples of this.

The Poorly Behaved Stuff

For many problems that have interested humans for the past century or so of computing, compression based on relatively well-behaved languages has been a common workhorse. But Kolmogorov Complexity promises that even greater treasures lie in the less well-behaved languages. Perhaps the problems we have chosen to focus on so far haven’t found a need for them, but as far as human endeavors go a century is a blink of an eye. The deep, distant future of computing will contain great wonders that the most observant of us can only see the faintest hints of today.

If we consider the inverse of compression and ask about “inflation algorithms”, it’s pretty easy to write very short programs that produce very long, complicated strings. Running certain cellular automata for very long, but finite periods of time, for example. An unimaginably more extreme example may simply be evaluating the Ackermann function on inputs greater than four.

In much the same way that a gigantic tree can grow from a very tiny seed, a very large, complex structure can unfold from a very small and simple seed program.

Kolmogorov Complexity is in a certain sense not only a measure of compressibility, but a measure of the amount of structure in a problem.

Assuming P!=NP, then the solutions to most optimization and constraint problems are fundamentally exponentially difficult to compute. However, these problems are in a certain sense highly optimal with respect to problem definitions that are exponentially simpler than the process required to resolve them.

This is a place where Kolmogorov Complexity and other forms of computational complexity diverge somewhat; KC doesn’t care about how long it takes to compute something, only what gets computed. Such solutions are effectively both highly random and highly structured simultaneously.

Computing even a locally optimal solution to many of these hard problems can require only a fairly small amount of code - just the parameters to define the function to optimize over, and some function that navigates the solution space and converges on a solution eventually.

If your problem is continuous, simple gradient descent can work pretty well. For discrete problems, something as simple and stupid as a brute force iteration over the state space until a valid solution is found is very simple and technically works, though you may be waiting a long time. There are a number of much more sophisticated methods that can search spaces much more intelligently that don’t obviously involve any gradients. Of course, none of these methods - gradient descent included - make any real guarantees about how long it takes to converge onto a solution. There can be problems that are well-behaved and converge quickly, and there can be others that take an extraordinary long time.

A Simple Example

Say we define some space that has a number of local maxima. If we throw a simple gradient ascent algorithm at it, then just about any initial condition - all zeros, something random, whatever is most convenient to define - it will guide us to one of these peaks.

However, perhaps this is a problem where it is easy to accidentally end up in a very suboptimal local maxima. We don’t want to just climb to the top of just any old hill in this space, we want the tallest, or at least one of the tallest.

Around each peak there is some space of points that will naturally guide a gradient-following algorithm to that peak. We can slice the space into these subspaces. Some of these spaces may be bigger than others, some may have very simple shapes and others much more complex ones. If we wish to use our gradient-following algorithm to locate a very specific peak, we only need to have the algorithm start somewhere in the space around that peak.

More precision in the location requires more information. Every additional bit allows us to narrow things to a window half the size. Perhaps the rest of the coordinates are all zeros, perhaps something random, but to get more specific, all else being equal, necessarily requires more bits.

However, overall, unless there is a gigantic range in the sizes of these separate regions and the one we aim to hit happens to be exceptionally small, the total amount of information required to specify which peak we aim to reach has far more to do with the total number of local maxima than it does the actual scale of the solution space. Once we have defined the structure of the space, the amount of information required to specify which optimal solution we wish to land on may be very tiny, perhaps only a small handful of bits.

Granted, with how long it may take to converge to a solution - perhaps we have a very large and complex space and gradient descent takes a very long and winding path to the top - then we may want more precision in our starting point to start a bit closer to the final solution. Kolmogorov Complexity at the end of the day is simply a theoretical ideal, with an infinite difficulty curve, and practical concerns always win in practice.

Some Wild Speculation on Biology

A chemical system will naturally trend toward equilibrium, and biological systems are ultimately just a digital language (DNA) that specifies and controls a very complex chemical system. Many of the problems that biological systems require to survive and compete for resources take the form of optimization or constraint-solving problems, and like the code golf languages that encode popular challenges with single characters (or even none at all), the chemical language underlying biology has optimization as its most primitive and fundamental tools.

Of course, biology is not completely without its concerns about the “computational” efficiency of reaching equilibria - it does care how long and how much energy it takes to find optimal solutions. It spends no shortage of resources catalyzing reactions with enzymes, accelerating protein folding with chaperones, and specifying solutions fairly directly with body plans and the like. But the point is that, given the tools biology has to work with, it appears that a tremendous amount of optimization should come for free as well.

This suggests that, while some things are relatively hardcoded, it is very plausible that a large amount of biology may simply be a set of evolved specifications of constraints, evolved specifications of tools for solving them, and a chemical system that naturally finds a very efficient way to solve the given constraints with the given tools. Of course, some problems may be particularly open-ended, with gigantic search spaces (optimal body plans for example), and in such cases it would be much faster to hardcode at least part of the solution to speed things up rather than waiting for some pink or green goo to search through absolutely every possibility.

The human body is incredibly complex, but is specified by only approximately 750 MB of DNA. Even then, that DNA also contains many very repetitive segments, and may be well over 90% “junk” DNA, with questionable contributions. There are only about 20,000 genes. Such enormous complexity will occasionally baffle people, with how much seems to be packed in there. The number of different behaviors that people claim are hardcoded into our biology and our brains probably exceeds the number of actual nervous system genes by a very wide margin, perhaps even eclipsing the number of total genes altogether. It seems to produce a very large number of diverse behaviors from a surprisingly small amount of code.

Biology, viewed as a chemical computer, certainly does have many recognizable parts. DNA transcription closely resembles function calls, and there are a variety of proteins that act as repressors and promoters that allow genes to be switched on and off by other genes and other chemical signals. On the other hand, biology is in many respects a very alien form of computing, one unconstrained by human understanding and that has been around far longer than us. We should be very surprised if it is not exploiting very strange phenomena that require a much deeper understanding of computing to appreciate. Why would it not be exploiting very strange properties of Kolmogorov Complexity to squeeze more expressive power from its code? And by extension, a small mutation in DNA may as a result explode into a much more impactful change in function than it would in familiar, human-written code, while also being almost guaranteed to land in some kind of local optima, once the chemistry plays out.

Inputs from the environment would also count as part of the specification, and stochastic properties of the system would need to be factored in as well. In KC terms, this is like having a large portion of the input program be missing, absent from the DNA, and simply gathered as an input from the environment. The DNA is not the sum total of all the data required to construct this specific creature, but only the part that is not collected from the environment.

This also reminds me a bit of Richard Dawkins’ concept of the extended phenotype.

If you read books from Dawkins’ crowd of evolutionary biologists, a common discussion is around how game theory equilibria may influence gene distributions in a population. There is a great deal of discussion around how natural selection is capable, after enough generations, of converging onto optimal solutions to tricky game theory problems. However, computing Nash Equilibria of finite games is NP-complete, and therefore is within the computational power of a sufficiently expressive chemical system.

There are occasionally signs that evolution seems to work much faster than these models of natural selection suggest. However, from the perspective I’ve outlined here, all of this computation spread across countless generations could, with a small handful of genes, simply describe the problem and allow the chemistry within a cell to solve it automatically. Natural selection would still play a very important role in learning which problems are most productive to solve, but a great deal of the difficulty would be offloaded onto the chemistry within an organism.

It’s also worth pointing out that constraint systems, which are expressive enough to describe any NP problem, have the property that additional constraint that do not add new degrees of freedom only narrow possibilities. If you have some genes that describe only some of the constraints necessary to solve such a problem, you may still converge on a valid solution by luck. You’ll at least end up with something that is locally optimal. Additional constraints, additional genes, simply narrow exploration of the search space to improve the reliability of landing somewhere good.

So not only should such a chemical constraint-solving system be extremely natural to express given the tools biology has, and not only does it explain why it can generate such a wide variety of effective behaviors well-adapted to its environment (input) from a relatively small amount of code, and not only should it be able to solve problems within a single generation that would otherwise take many generations to converge on purely by natural selection, but it also should in theory be very easy to learn all of this bit-by-bit, with something that resembles a fairly straightforward, if somewhat stochastic gradient.

So it appears that every cell in your body is likely a constraint-solving algorithm searching for the most optimal path forward with the tools and resources that natural selection has found it, and the exponential costs of this optimization problem are dealt with by each of the billions of chemical reactions per second per cell second contributing a calculation, occasionally with a hint from a chaperone or enzyme.

All of this is of course my own speculation, I’m no biologist, but it lines up with a lot of observations and explains quite a lot of things.

Subscribe now

Share

Let’s say we want to build software that augments the human ability to understand complex things.

If we want to understand a problem like "understandability", we can take the simple, brute force approach of "just use an LLM bro", or we alternatively can try to actually develop a theory of how understanding itself works, which would likely give us more powerful and less obvious tools, allowing us to do an even better job. If we set our standards at simply having an LLM explain something to us in plain English, we might miss some deeper and more powerful possibilities.

Human imagination is far more limited than we often pretend it is. The world outside your skull is infinitely more complex than the model of it inside your skull. If you aren’t mining the world for newer, deeper inspiration, you will fail to imagine cars, planes, and rockets as opposed to simply faster horses.

If people are going to worry about LLMs replacing humans, then perhaps our response should be to start thinking more deeply about how to push humans to their limits.

If we make the classical comparison and claim that the human brain is a type of computer, one natural direction to go in is asking about the software it runs. There certainly are some substantial differences between ordinary code and things like culture and memes, but there are also substantial differences between a C program and the weights of a deep neural network, and we really don’t seem to have a difference accepting that they’re both software. There are different, alien modes of software out there that have yet to be discovered, computer science is less than a century old and the Library of Babel of code is unfathomably vast, even if we limit ourselves to fairly short program lengths.

Neural software seems to rely a lot more on mimicry - the way humans and animals download and share it is by observing others and copying their behaviors. Humans have the additional method of language, where we can associate abstract symbols and sounds with objects, actions, and other things in the world. We can describe causality chains through stories and explanations, and we can transfer knowledge to others, including abstract and internal states that would otherwise not be apparent purely from the visible behavior alone.

The format of “human software” also changes substantially over time. One notable example is that pre-literate societies often placed an enormous focus on memorization. Music and poetry were often far larger parts of daily life, as a bit of rhythm and rhyme introduce a lot of repetitive and redundant structure that make things easier to memorize perfectly.

It’s worth remembering that Homer’s Iliad and Odyssey were composed in the 700s BC, during the Greek Dark Age, some three or four centuries before Greece regained writing. These are absolutely gargantuan texts (the two combined are almost half the length of the Bible), and were passed down entirely by memory for centuries. They also include a number of details about the Trojan War and the Bronze Age Collapse, albeit heavily fictionalized.

The Iliad also notoriously includes the Catalogue of Ships, a chapter which autistically lists off a ton of statistics about the entire Trojan-War-era Greek Navy, and perhaps was partially a way of allowing bards to show off their impressive memorization abilities. The Method of Loci, a memorization technique based on constructing a mental space to navigate full of meaningful reminders, also traces back to ancient times.

Things that were completely mundane and ordinary to such pre-literate societies may seem like unbelievable superhuman mental feats today. It’s worth considering what other feats the brain is capable of if pushed in the right direction.

There are a variety of numbers that people throw around about the computing power of the human brain, though they often heavily simplify the neuroscience and most probably underestimate the brain by a fair bit. Nevertheless, the numbers are usually enormous - usually peta- or exa-scale compute, and tens or hundreds of petabytes of storage. Given the "human software” angle we’ve been taking, there are many different ways of writing programs to get the same job done, but some are vastly more efficient than others. Is the code we’re running on our brains closer to heavily-optimized “Neural C”, or is it a really poor Python implementation? Various divisions of cognition (rational versus intuitive thinking, for example) may be analogous to wildly different programming paradigms, though perhaps we should be skeptical that these are all that is possible. As we understand the brain better, we can likely push things in a more optimized direction.

So if we want to push humans closer to their true limits, perhaps as a way of maintaining relevance against AI for longer, perhaps as a way of better understanding our own minds, perhaps as a way of exploring what brains are truly and unexpectedly capable of, perhaps in pursuit of a form of “cognitive innovation”, we need some place to start. A good place to start may be in finding some cognitive thing that we have some basic intuitive sense of, and digging a bit more deeply into it, with the eventual goal being to build tools to augment and accelerate it.

Back to understandability. This is a fairly abstract problem, but I’d like to highlight a few observations about it:

• Understanding is rooted in familiarity. Humans have certain motifs they follow for building stuff, and the inverse of that (pattern matching) is a key part of how we interpret new things.

• More components, especially redundant components, can often increase the "surface area" of something. If you can see the same thing from multiple perspectives, the odds that you'll recognize a familiar fragment to latch onto to help you "get it" improve. Understanding of one component can often narrow possibilities enough to help understand connected components.

• This works in the opposite direction too; heavily optimized code (superoptimized code especially) often will prioritize unfamiliar shortcuts over staying within a more limited toolset of familiar patterns. Human familiarity is a constraint that limits options, and if you optimize for something unrelated, you'll end up with something difficult to understand.

• If you can see a large number of moving parts and the order behind the chaos (visualization often does this), that often can help too, though some perspectives will make relevant order more or less legible.

• Many things like code and theorems have large equivalence classes; there are a large number of ways to rephrase them that are all equivalent, but where the precise details of how they work may be very different. There are dozens of known ways of proving the Pythagorean theorem, there are dozens of ways of reframing Euclid's Parallel Postulate. There is nothing special about these particular theorems that give them so many equivalent perspectives; they are simply problems that have gotten a lot of attention for a very long time, and so many alternative perspectives on them have been found. If we threw enough mathematicians at the problem, we could in time have a hundred very different proofs of a much harder problem such as Fermat's Last Theorem, each of which may provide a completely different perspective on why it is true that may highlight new connections in mathematics, and may make more or less sense to different people.

Of course, we also want to expand human familiarity. What is and isn't a familiar technique may vary substantially from person to person, especially when dealing with more niche methods. There also needs to be effort put into not merely reframing things into familiar terms, but in making the unfamiliar familiar whenever it makes sense to.

Astrolabe and The American Compute Company

I’ve been thinking for many years at this point about how to improve the efficiency of modern CPUs. The way we design modern CPUs generally says a lot more about historical accidents, half-century-old technical constraints, and primitive early human concepts of computing than it does anything more fundamental or physically constrained. Modern CPUs spend enormous amounts of resources attempting to extract miniscule amounts of parallelism from sequential chains of instructions and waste huge amounts of energy throwing very crude heuristics at managing the physical layout of data.

A few months ago I got around to incorporating this project as The American Compute Company. Designing a completely new ISA from scratch with a number of unique ideas around data layout and semi-static scheduling is an ambitious problem. This necessarily throws away a great deal of software compatibility, and in a way that efficient emulation may be tricky. I’ve seen other companies build very impressive hardware and then fail because of poor programmability and a lack of software support, and I am not interested in making the same mistake.

My approach in developing this chip is to put a large focus on software. I need usable and useful software to ship with the hardware on Day 1. It makes sense to build the software and hardware in tandem. The final hardware may be very alien compared to traditional CPUs, but it’s necessary to also make sure it does not become so alien as to be unusable. Good and innovative development tooling can help make up for some of this, but software should absolutely not be an afterthought. Furthermore, with the aim of pushing the efficiency of hardware, it makes sense to have real software to optimize the hardware around to make sure that features in the hardware, especially experimental features, properly reflect behavior of actual code.

This software-heavy focus also has some other advantages. It lets me try out some experimental ideas (I have a number of ideas around development tooling), and some of this code can also be run on existing hardware in the meantime and can be sold to bring in some revenue to fund chip development.

A few days ago I released Astrolabe, an interactive codebase visualization and mapping tool that I built. It’s fairly experimental at the moment and there are plenty of additional features and polish I’d like to add in the future to make it more useful, but at the moment it’s fun to mess around with, generates some eye-catching visualizations, and get some basic intuition for big codebases.

In the age of AI, we should be aiming to build tools that advance human intuition and understanding of complex systems.

It’s currently fairly language agnostic and isn’t extremely sensitive to code structure beyond the identifiers it parses out, but it supports a decent number of languages. I have some ideas on text embeddings and approximate parsing methods I’d like to try out in future versions to hopefully make it a bit more useful. It supports a couple dozen common languages, so it should work with anything from Haskell to Javascript to C to Verilog.

Astrolabe is currently priced at $25 and can be found on the American Compute Company website for those who want to support the project and mess around with something a bit more fun than piping grep and awk together to locate stuff. I have some other software projects in the pipeline as well, which will be announced as I make progress on them.

It’s been a few months since the last article, I’ve spent a lot of time coding lately, though I’d like to get back to writing frequently and have a number of vert intersting articles in the works.

Subscribe now

Share

There are a great many instincts and behaviors that are claimed to be “programmed” or “hardwired” into the brain. In fact, seeing as the human genome only contains a mere 20,000 genes, and only a small fraction of those are dedicated to the nervous system, the number of behaviors that people insist are so deeply baked in probably outnumbers the number of genes that could conceivably do it.

The much more likely explanation is that a great many of these instincts and behaviors are either emergent or are actually learned, but on some very deep and subconscious level.

Human engineering is generally built around a kind of explicit engineering - we pick some set of desired features and behaviors of the end product, we design individual mechanisms for each one, and we link them together. If there is a more abstract and less obvious way in which we could produce this same functionality through a more emergent system, this is beyond the scope of human understanding.

Human engineering is fundamentally limited by our understanding. Engineering is a form of mimicry, we observe some phenomena in the natural world, we learn how to replicate it, and we then extrapolate the consequences to apply it as a tool. This often involves a great deal of tinkering, we cannot simply sit in our armchairs and project forward all the infinite implications of a new technique with perfect accuracy, we must instead confront the chaos of the world by tinkering and experimenting. Nature is far more creative than we are, and will throw curveballs at us that contradict our intuition. Any decision tree of how to interact with a sufficiently complex tool will inevitably result in an exponential explosion in the number of branches, and exploring them all one by one in our minds is far too costly a task to do in any reasonable time without missing the vast majority of interesting things.

The human mind is a library, and when we build things we pull ideas from it. If that library is missing a technique no matter how simple or powerful, humans will overlook it until it is discovered through tinkering, understood, and added to the corpus of human knowledge. And there always will be a tremendous amount that is missing, the human mind may have a tremendous capacity, but it is nothing compared to the Library of Babel.

The endless tinkering of the blind watchmaker of evolution meanwhile is untethered to such limitations, and is thus free to master far more bizarre modes of engineering that neither it nor humans can comprehend. The forces that designed the human brain are free from the constraints by which we work. It may perhaps innovate more slowly, but it quite reliably produces things that we would otherwise have no idea how to replicate without our own mimicry work.

Mimicry as an Emergent Behavior

Many human and animal behaviors that are assumed to be innate and “hardwired” are more likely to be either learned or are emergent. One I’d like to pay particular attention to is mimicry, and I’d like to propose an emergent mechanism.

Navigating the world is fundamentally difficult, and I think the extent to which it is difficult is greatly underappreciated solely because we have collected such a tremendous corpus of knowledge with which to navigate it. Life today is in most ways much easier than it was a few centuries ago, let alone a few thousand or longer.

If we assume that the brain, by observing the world around it, attempts to learn to make predictions about it, and if we assume a basic ability to perform pathfinding and path integration, I argue that these basic components alone are sufficient for mimicry to emerge.

Much of the cortex appears to consist of circuitry that evolved originally for navigating physical spaces that was then adapted to navigate conceptual spaces. If we have the ability to understand our current position, understand the position of our desired destination, and have some map of the obstacles along our way to getting there, then some simple pathfinding algorithms can guide us there.

The brain navigates with a combination of grid cells, place cells, and head direction cells; grid cells which encode our coordinates using a hierarchy of hexagonal grids, place cells which activate when we are near known landmarks, and head direction cells which encode the direction in which we are facing. It really doesn’t take much to combine these basic tools and get a pathfinding algorithm that approximately resembles something like A*.

Granted, A* works best if the landscape is relatively sparse of major obstacles, and it doesn’t work at all if the destination itself isn’t known. Navigating 3-dimensional euclidean space is relatively tractable, but navigating the much higher-dimensional, hyperbolic spaces in which conceptual decision trees and complex interactions in the world are encoded is a vastly more difficult problem. In terms of complexity, the majority of problems there turn out to be in the NP or NP-Hard complexity classes. Such problems are exponentially difficult if you do not know the solution, but are comparatively trivial as soon as you do.

If you don’t know how to get somewhere, it will be exceptionally difficult to find. If someone gives you the directions, it becomes easy.

Suppose you observe someone perform actions A, B, and C and get result D. If you had no understanding of how to accomplish D before, you now have some awareness of the mechanisms by which D can occur, and are free to repeat the formula yourself. Even if you are missing some important details, understanding part of the process can guide your exploration of these vast search trees and help you find a working solution vastly more quickly than some kind of blind brute force would likely permit.

This mechanism requires no dedicated, hardwired instinct for mimicry, but rather emerges from simply watching the world, learning to predict it, and then attempting to navigate to desired goals. It also fits extremely well with complexity theory, in that the existence of a known solution to a problem instantly makes otherwise intractable problems exponentially simpler. In fact, even partial solutions can narrow the search space dramatically; transmission can be lossy, the environment can be noisy, and exponential improvements in tractability can still occur.

Mimicry may be less of a preprogrammed instinct and more a phenomena that emerges from the fundamental laws governing the ways in which information about even partial solutions make complex problems exponentially more tractable.

Memetic and Genetic Lineages

The human genome is 3 billion base pairs, each of which is equivalent to two bits of information. This in total is 750 MB of data, though the vast majority is “junk DNA”, which does not appear to directly store useful genes, though may serve other purposes. Even if such junk DNA is much more functional than we expect, 750MB is the hard limit on information storage.

The human brain however contains many tens to hundreds of trillions of synapses, equivalent to many terabytes of storage. Even if we insist that only a tiny percentage of this can be information capable of being propagated across generations, and even if we assume that the vast majority of animals have brains far smaller than ours, it still does not take much for the information being propagated memetically to greatly exceed that being propagated genetically.

It’s not uncommon for people to observe pets raised around other animals taking on traits from other species. I recently had a conversation with someone who remarked about their cat having the habit of “barking” on account of spending so much time around their pet dogs. It’s not uncommon for people to share photos and videos online of pets with stunningly human-like behaviors. It may be that all animals are in fact mimicry machines.

A great deal of an animal’s extended phenotype may in fact not be genetic at all, perhaps much less than expected. Evolution may even optimize animal body plans over time to better conform to the kinds of common behaviors that they are learning and transmitting down generations. I also would not be surprised if this works in the opposite direction as well; that these changes made to animal body plans make such common behaviors more convenient and therefore make them more likely to be tried again even if the memetic lineage is broken.

Thank you for reading today’s article. I’ve been very busy lately but am hoping to publish a bit more regularly here.

For those who have not heard, some friends and I have launched a magazine for optimistic, realistic science fiction called Possibilia. Our goal is to provide alternatives to the overwhelming dystopian doom-and-gloom that takes up so many conversations about what the future will look like. It’s currently hosted on substack as well, and we are hoping to launch a print magazine later this year.

We just published our second story last week, with a nonfiction companion piece for it dropping tomorrow (Friday, April 5th). We’re currently aiming for a weekly publishing schedule over there, though with a mix of content, not just stories and companion pieces.

Possibilia Magazine

Possibilia is a literary magazine for optimistic, realistic science fiction. We're working on bringing you positive visions of the future, in both digital and print magazine formats - through short stories, nonfiction companion pieces, and illustration.

I’m hoping to publish more content here too of course, so feel free to share and subscribe for more.

Share

Subscribe now

Leave a comment

I’d like to highlight a few physical constraints that apply to computation. Modern microprocessors - CPUs especially - are hilariously wasteful when it comes to how they use transistors, and as a result contain many other features that limit performance long before many of these limits become relevant to their full extent. As Moore’s Law grinds to a halt and further performance gains become dependent on squeezing out these inefficiencies, these constraints will become increasingly more relevant and more visible.

Distance is Latency

To state perhaps the most obvious, the speed of light is finite. It may seem very fast, but is much slower when you're operating at gigahertz speeds. Sending signals a few millimeters across a chip can easily take several clock cycles and sending signals across a CPU motherboard to RAM can easily take hundreds. Modern CPUs attempt to execute many instructions at a time, even on a single core, and so waiting on a piece of data that's inconveniently far away will regularly waste thousands of instructions worth of work, forcing the CPU to stall and patiently twiddle its thumbs.

How far data must travel and how many possible locations it can be tansported to also impacts power consumption. In some cases such as fiber optic communication this may be negligible, but optical communication within a microprocessor has extremely limited practicality and passing electrons through wires with capacitance and resistance depedendent on their size is the regular way in which on-chip communication is done. If a piece of data is being routed between locations, and there are two possible destinations, some amount of computation must also be performed to send it along the correct path, and larger numbers of possible destinations increases this cost further. Extremely flexible circuits that can shuffle data to an extremely wide variety of locations will generally be much less efficient than circuits with more limited and hard-coded paths.

Volume and Entropy

Also obvious, storing information takes up space. This goes for memory cells, but also goes for logic. A circuit, to perform some computation, must store some number of inputs, some number of outputs, and the states of some number of intermediate gates and wires that usually add up to something much larger than the inputs and outputs combined. Plug your inputs into a circuit and the gates and wires collectively navigate a large state space, gradually walking their way toward a stable equillibrium state from which the output can be easily derived. All of this entropy must be embedded in physical space.

To combine this point with the previous one, this happens to mean that there is a physical limit to how much memory and compute can be accessed within a certain amount of time; only so many memory cells and circuits that can be reached by light or electrons within a certain time limit. If you wish to push the physical limits of computing, you may have to consider tradeoffs between memory and compute resources and the time it takes to evaluate. Accessing more memory or greater compute resources necessarily requires more circuit volume, and will necessarily take greater amounts of time to propagate signals to every destination they must go to to compute a solution.

Having a greater deal of certainty about what you're looking for can narrow this; this is what caches do. By relocating frequently-used data to be physically closer, or on a larger scale physically sorting data based on the expected likelihood it will be used, average latency and travel distances can be greatly reduced. How well this works is of course highly dependent on how good you are at judging future usage patterns, which can be extremely dependent on the program. A program that pulls random items from a large hash table is going to have very different limits from a program that relies heavily on a few small data structures or consistently takes well-worn and predictable walks through its data.

Caches are also fundamentally driven by heuristics, and what heuristic is best is often dependent on the particular application. More complex heuristics also of course require more compute and memory, and therefore are working against you when pushing limits. Modern CPU caches try to aim for heuristics with the widest applicability, but certainly would be inferior to something that at least has the option to include a bit more information that may be provided by the compiler or programmer. Tracking usage of individual cache lines like modern caches do creates overhead in the form of additional entropy that must be stored, lowering memory density. Locating memory based on its "address" rather than its physical location also creates additional indirection; the cache must not only account for every possible arrangement of bits we wish to store in it, but also must simultaneously handle every possible physical arrangement of that data that the cache supports, and all the entropy required to encode that.

All in all, the physical layout of data in space matters tremendously.

Geometry of Parallelism

A small and unimpressive circuit that is sufficiently flexible can perform computations as elaborate as a much larger circuit, simply given more time. Given N bits that exist over T time steps, there are 2^(N*T) possible states. Doubling the core count expands the state space just as much as running the same cores for twice as long. Encoding a larger circuit to be emulated requires additional entropy in the form of CPU instructions and instruction memory, which must also be stored and regularly fetched.

However, suppose we have two tasks we can run in parallel. If task A takes N time steps to evaluate and task B takes M time steps, running the two tasks on a single core should ideally take N+M time steps (ignoring aforementioned factors like memory latency). If we have two cores, we know that these both take up physical space and cannot occupy the same location. Sending a message from one to the other takes time, dependent on how far they are apart. If we decide to send task B to another core, the round trip of communicating with that core should be less than N time steps or else it will take longer to send B off to run in parallel and wait for the result to come back than it would be to simply evaluate A and B sequentially.

If we have many different tasks with complex dependencies, perhaps we have some flexibility in latency with regards to N and M; how quickly these two specific tasks are evaluated may depend less than the sum total time it takes to evaluate some thousand other tasks as well. On the other hand, it's also perfectly possible that there are bottlenecks in the dataflow of our program and some critical path between N and M does matter. Different programs behave differently.

Programs also often consist of many different abstractions, with big functions calling smaller functions, and those functions calling yet smaller functions. Any of these functions, all of them, or perhaps none of them, may contain some opportunities for parallelism. There is potential for parallelism at all scales of a program; zooming into functions and the functions they call, and the functions those functions call, etc. leads to you to smaller and smaller tasks and sub-tasks that may be shuffled to different cores or execution units. Parallelism fundamentally has a fractal structure, and how much parallelism is available at each scale impacts how far tasks can be sent across a chip or a data center before latency costs defeat any benefits.

Memory locality also matters; it may in fact be much better to send a task to another core if that core is located closer to relevant memory. How much this matters depends on the volume of memory that must be moved around and how far it must be moved.

Curvature of Computation

Not all spaces are flat, some spaces have curvature. The three main types of curvature are positive (elliptic/spherical) curvature, zero (euclidean/flat) curvature, and negative (hyperbolic) curvature. One important property of these spaces is that the volume within a certain radius is dependent on their curvature. In flat, euclidean space, the volume of a circle or sphere is dependent on the radius to the power of the number of dimensions; r^2 for 2D, r^3 for 3D, etc.

Pick a point on the surface of a sphere however and ask how much surface area is within a certain distance from that point. It increases up until the distance is equal to a quarter of the sphere's circumference. Past that point, its rate of growth slows until the distance is equal to half the sphere's circumference, in which case every point on the sphere is within the distance. Hyperbolic space, even stranger than the other two, sees volume increase exponentially as the radius is increased.

Why this matters is that it affects the relationship between volume and distance on a chip. I don't expect we'll be getting real spherical or hyperbolic computers any time soon, but many computations we perform on the chips that we do and will have are in a certain sense curved. A tree data structure is in many respects a discrete form of a hyperbolic space, with an exponentially large number of nodes accessible within a given number of hops. While it might be convenient to think of a tree as always requiring logarithmic time to locate a node, a large enough tree will become bottlenecked on the physical scaling limits of memory in euclidean space, and eventually converge to the same scaling rates; a processor with 2D memory forced to store a large enough tree will eventually converge to ~O(√n) time to locate nodes.

Many other algorithms and data structures scale in similar ways, and the dynamics by which the amount of computation and intermediate data grows and shrinks throughout evaluation is a complex geometric object that must be embedded in and confined to the limits of the lattice of memory cells and logic circuits that make up a computer.

Square-Cube Law, Heat, and Bandwidth

Take any object and scale it up. Make it twice as tall, twice as long, twice as wide. Its volume has increased by a factor of 2^3, or 8. Its surface area however has increased by only a factor of 2^2, or 4. The ratio of volume to surface area has doubled, and this has a profound impact on many properties and processes involving this object.

The amount of heat that can be stored within an object is proportional to its volume, but the rate at which it can expel this heat to the outside world is proportional to its surface area. If you have a 3D circuit and decide to scale it up, you eventually hit limits where the chip must be limited in how much work it can do (and therefore how much heat it can generate), or face building up heat too fast and melting.

Heat is of course just waste entropy, and the rate at which we can move useful entropy (information) in and out of the processor is bound by the same constraints. A sphere of computronium has bandwidth with the outside world that is ultimately bounded by its surface area. This decides how much information and heat can go in and can go out.

Manufacturing 3D circuits is difficult, error-prone, and expensive. 3D memory has been around for a while now, and the beginning of 3D stacked logic will likely be reaching consumer devices by the end of this decade. However, chips already generate large amounts of heat and are finding it increasingly difficult to operate without producing too much. Some efficiency gains can be made from the reduced distances between gates possible by moving from 2D to 3D, but this is a small reduction compared to the large increase in heat from more densely packing gates. While memory circuits can keep heat output low thanks to how infrequently most memory is accessed, logic circuits will likely be forced to stay close to 2D to keep the scaling of computing and cooling as closely coupled as possible.

Conclusion

While perhaps our usual chips are not well-suited for these concerns with computing, the future will likely rely more heavily on tiled architectures, such as Adapteva’s Epiphany architecture, or Cerebras’ gargantuan wafer-scale chips. These offer massive core counts and massive amounts of performance. They also offer very precise control over the physical layout of data and compute across the chip, offering huge efficiency gains over the standard approaches of large, purely heuristic-driven caches.

The tricky part is that I’m not sure we’re ready to program these things yet, at least not for anything beyond simple operations on large, flat arrays. As I’ve described above, there’s a lot going on and a lot of complex factors to account for.

Of course, we’ll never be ready for a change like this. Once a good, general-compute-oriented tiled architecture makes its way onto the market and gains enough users, new branches of computer science and code optimization will flourish, and likely many decades of experimental work will follow. These architectures will likely age like wine, getting more capable as people gain a much deeper understanding of how to properly optimize them.

Beyond that, it’s hard to say what computing will look like far into the future. While tiled architectures seem to offer a close-to-optimal approach for linking cores together, there’s a great deal about how individual CPU cores are currently designed that I doubt is actually very optimal, and is more of a bunch of fairly contrived and rigid ideas that people have locked themselves into.

The computers of the distant future may look very different from those of today, but we at least have some guidelines from physics to sketch out very rough ideas about them.

Share Bzogramming

Share

Subscribe now

Physics can be reframed in different ways. Another framing is Lagrangian mechanics, which looks at both the kinetic and potential energy of a system and asks how to convert that potential energy into kinetic energy in the minimal amount of effort. There is also Hamiltonian mechanics, which looks at the sum of kinetic and potential energy and analyzes the consequences of the total energy being conserved. Hamiltonian mechanics in particular lends itself easily to extremely rich mathematical tools such as linear algebra and group theory, as any physical system can be framed as rotations of a high-dimensional sphere (conservation laws are encoded as a constant radius) and is the foundation of a great deal of modern quantum mechanics.

It's worth asking if a similar analogy can be made in computing - are we currently stuck on "Newtonian computing", with future computing paradigms on the horizon as future developments? What might these things look like? Where should we look? What might we expect to come out of them?

We probably shouldn’t stick to this analogy too strictly - though perhaps a creative person could come up with some fairly literal translation of Lagrangian mechanics or something that might actually be useful. That’s not where I would start though, these examples from physics are just an analogy.

The Idea

Specifically what we’d likely be looking for is an alternative way of viewing computation. Currently much of computing is based on a simple gate-based model; you have some collection of bits, you pass some subset of them in as inputs to a gate, and this performs an operation on the input to generate a collection of new output bits. These outputs can be added to the pile of bits, or perhaps if you have limited memory can overwrite existing bits.

Whatever alternative frameworks we develop would still need to encode all of this, but might highlight some new relationship or property that wasn’t obvious from simply staring at a sea of gates.

Reversible Computing

Reversible computing may be one interesting perspective that may be deeper than we currently realize. This is a research area that already exists, though has a fairly small number of active researchers. It is concerned with the properties of computation that preserves information. All gates have an identical number of inputs and outputs, and all are bijective; every output has exactly one corresponding input. If you want to encode something like an AND gate, where three of the four possible inputs all map to the exact same output, additional “ancillary bits” are required to make these outputs different.

This is effectively conservation of information; while these ancillary bits may appear to store unnecessary junk information, they happen to encode just the right information required to run the circuit in reverse. Running circuits in reverse is in general NP-complete, and often can take exponential time. This is why breaking cryptography is hard, among many other things. What makes it hard is that there is a tremendous amount of guesswork in reversing a circuit; if you’re only told that an AND gate outputs a 0, you have no way of knowing if the inputs were 00, 01, or 10. A guess-and-check approach is fundamentally required, and while some amount of inference can be used to narrow the range of guesses, multiple unknowns in a large circuit tends to result in an exponential explosion of possible states to check.

Reversible computing preserves all of this information; if you have the ancillary bits, no guesswork is required. This is not magic, nor is it some threat to cryptography; the person who computed the cryptographic signature or whatever it is you’re trying to break would be the only person with those ancillary bits, and handing them over for an attacker to use is an obviously stupid idea.

Reversible computing will likely become more important in the future anyway. There are fundamental thermodynamic limits on the energy efficiency of computing (Landauer’s Principle), and we are only about 4-7 orders of magnitude away from hitting them (depending on ambient temperature). If we wish to throw away or erase information, this fundamentally takes the form of generating waste heat, which corresponds to a certain amount of energy spent. The only way within known physics of circumventing this is to build physically reversible computers that never generate waste heat because they never throw any information away1. This will require changing many deep and fundamental ideas around both software and hardware, and will undoubtedly require us to abandon CMOS transistors in favor of some fundamentally different technology.

If quantum computing ever pans out (I’m doubtful), it too is reversible, and will force reversible computing principles on any engineers who decide to use it.

With all this said, there is something very interesting in this idea of conservation of information that echoes conservation laws in physics; in fact, the extreme dependence on Hamiltonian mechanics found in quantum math and the inherently reversible nature of quantum computing are deeply related. It is likely that, with the right approaches, much of the rich mathematical toolbox that modern physics uses in Hamiltonian mechanics could likely be imported into studying computing, provided we choose to conserve information.

Constraint Solvers and NP

As stated above, reversing circuits is a problem that is exponentially hard, and fundamentally boils down to a great deal of guesswork. It is also NP-complete; complete in the sense that this problem is expressive and powerful enough to be able to emulate any other problem in NP, which is a truly vast range of problems including both almost all the problems we solve with computers daily as well as many more that most programmers are often too afraid to approach.

I also mentioned that there are inferences that can be made to reduce the amount of guesswork. If you are given an AND gate and a XOR gate that both receive the same inputs, and the AND gate returns a 0 while the XOR gate returns a 1, you can infer that at least one of the inputs must be a 1, which narrows the range of possible inputs to the AND gate from 3 to 2. The input must either be 01 or 10, as if it were 00 the XOR gate would have returned a 0. This is a simple example, much more complex inferences are possible too.

Overall, this approach to computing can be seen as looking at the entire state space of a problem (or at least looking at a collection of smaller and more manageable subspaces) and crossing out possibilities that don’t make logical sense, which in turn often can propagate information and cross out other possibilities. The overall problem is actually fairly simple, and most of the complexity comes from trying to do it with tractable amounts of computing power and memory, as the state spaces are exponentially large.

These inferences can dramatically reduce the amount of work required to solve these types of problems, though exactly how much work can be saved can be very difficult to answer. These systems can easily play by strange rules and approaches to the P-vs-NP and related problems have really only demonstrated that we will likely need to develop fundamentally new mathematical tools to see serious theoretical progress on the big, open questions.

While these problems can be very tricky to work with in theory in in the general case (they can be a bit easier to reason about if you focus on special cases), in practice they seem much more promising.

While there can be many situations where there are exponential explosions in difficulty, the past few decades of research into SAT solvers and similar algorithms have shown that there are a shockingly large number of cases - especially when dealing with "real world problems" where things turn out to be much more manageable. There are many very useful situations where the power of logical inference overwhelms exponentially exploding guesswork.

I would not be surprised at all if there are insights we can gain from studying these types of problems more carefully. If you encode a polynomial-time problem as a SAT instance or similar NP-complete problem, you tend to do so by building a circuit of gates using Tseytin transforms, and the way these gates are wired together creates a kind of information cascade where there is never any ambiguity or guesswork as long as gates are evaluated in the right order. Work on SAT solvers seems to suggest that there are more complex cases that involve only modest amounts of ambiguity that is easily resolved, and it’s worth asking if there are new types of algorithms that we could build that exploit this.

I personally find that a large chunk of modern “AI” hype is actually just using good-old-fashioned search/optimization algorithms, perhaps with a neural network bolted on. The notion that hard search algorithms can be accelerated substantially with the addition of a learning algorithm is nothing new, and caused a revolution in SAT solving before I was even born. The AI people are late to the party.

I also suspect (motivated by some personal research and projects of mine) that current research into backtracking algorithms (like the ones that SAT solvers use) is merely the tip of the iceberg, and that a tremendous amount of new innovation could be made with more serious effort into this space. I expect that such algorithms would likely prove much faster than these neural networks, though might be slightly more difficult to build than simply shutting your brain off and letting the GPUs go brrr.

Been a bit busy lately, though I’ve been thinking over some plans on where to take this blog in the future. I have some projects I’m working on that I’ll likely share here in the coming months. I’d also like to do an in-depth series on NP-complete problems and continue my neuroscience series, though we’ll see when I get around to that.

I should probably do a Bzo devlog soon.

Share Bzogramming

Subscribe now

My response is usually along the lines of “Oh neat, it’s another chatbot with some new quirks and a couple new gimmicks bolted on.” Such statements do not make me popular.

AI demos are like monster movies. Really, modern AI hype in general is a big monster movie. You never get to look at the whole monster in plain daylight, you only ever see brief glimpses of tiny parts of it in poor lighting. Everything else is implied, your imagination filling in the rest. These films would be vastly more tame if it wasn’t for the fact that the monster in your imagination is always far scarier than any silicone prop could ever be.

The big difference between AI and a monster movie is that in a monster movie, the people making the film know that the monster isn’t real and aren’t trying to warn the public about the immense danger posed by the thing they caught on film. This is less of the case in AI. Machine learning models are described and treated as black boxes, and little effort is spent trying to understand what it is they're doing. The AI is a neural network, so no one knows how it works, and so people proceed to let their imagination run wild with what it could be doing and avoid learning anything about what the network is actually doing that might kill the magic.

There certainly is some effort going into fields like Mechanistic Interpretability to approach this problem, but progress is fairly slow and the problems I've heard people trying to tackle in that space sound less like pragmatic day-to-day tools and more like extremely ambitious pipe dreams that try to solve everything all at once. There's likely a lot of low-hanging fruit that's being ignored here, and I suspect that there is also an immense financial incentive to not invest in these tools for the sake of maintaining the magic and continuing to hype up investors and frighten the general public.

None of these AI tools have taken over the world yet, and I have yet to see one that has really lived up to the hype. I remember talking with tech friends during the early days of GPT-4 where they all were claiming that this new high-quality natural language interface would suddenly make it extremely easy for anyone to get an AI to do whatever they pleased. Google would soon be bankrupt because of how much better GPT-4 is at answering questions. A week later the conversation had shifted to the difficulty of getting good answers and how much of a skilled profession prompt engineering will soon be, and a few of them are now building AI search startups and complaining frequently about how extremely difficult it is to keep the AI from hallucinating fake answers to your questions.

These AI models would appear far more mundane if we actually had good tools for peering into these neural networks. If every new AI model is “going to be AGI this time, trust me guys!” and none of them is yet, maybe our ideas about how this progression toward better and better AI isn’t very accurate, and maybe peeking under the hood might inform us as to why we haven’t gotten there yet. It’s worth remembering that many of these LLMs are consuming many millions of times more text than any human could consume in a lifetime and yet they still aren’t obviously AGI. Something is very obviously wrong regardless of how people try to rationalize it.

Even if we build tools that aren’t sufficient to give us a perfect picture, we could probably still piece together a great deal of what is going on. The ancient Egyptians had a pretty decent idea of what most organs in the body do (with perhaps some misunderstandings about the heart and brain) without any of the modern scientific knowledge or tools we have today.

Frankly, this pessimistic approach to the interpretabilty of neural networks really isn’t even that unique in computer science. A very similar thing is going on with popular misunderstandings of undecideability. Turing’s original proof stated that the set of program states that encode proofs about the properties of other program states is a subset of all possible program states, and that therefore there exist some states that a computer can be in for which no corresponding proof exists. The conclusion is that a program that can correctly answer questions about properties of general code with 100% accuracy cannot exist, and that a program that can answer such questions perfectly can be used to create logical paradoxes and contradictions.

This is often misinterpreted as meaning that such questions are completely unanswerable in all cases, which is false. Look at these two pieces of code here. Can you answer whether each one will halt? If so, is your brain actually solving a logically impossible problem, or is it merely that this problem is not as impossible as it is often claimed?

i = 99;
while(i > 0){
    println(i, " bottles of beer on the wall");
    println(i, " bottles of beer");
    println("take one down, pass it around");
    i--;
    println(i, " bottles of beer on the wall");
}

// underage version
i = 99;
while(i > 0){
    println(i, " bottles of beer on the wall");
    println(i, " bottles of beer");
    println("take one down, put it back up");
    println(i, " bottles of beer on the wall");
}

The reality is that even if there are some programs that cannot be reasoned about, there still are many programs that can, and that problems that are actually useful to humans are often easy to model with fairly well-behaved programs. The harder ones often just spit out near-random noise, and are usually not worth paying attention to unless your name is Stephen Wolfram. You can build tools that can look for certain types of common structure that is easy to reason about - Abstract Interpretation is one such tool. Not all valid programs will have that structure, but most that you care about will! If you have a program that proves hard to analyze, that might be a better indication that you have a bug than it is that you’re doing something tricky.

If static analysis tools got as much attention as other development tools do, if we had as many hobbyist programmers building abstract interpreters as we do building their own languages, and if some of them actually gained adoption the way that hobby languages sometimes do, we’d likely be living in a completely unrecognizeable world with an extremely different outlook on what is and isn’t possible and feasible in software development. Very likely a much more desirable world than ours.

Perhaps machine learning is in some ways a trickier problem. These models are not directly designed by humans and likely use many weird tricks that we don’t yet understand, let alone know how to build tools to thoroughly reason about for us. In other ways though, it’s provably easier; these models are finite, they have a finite running time, there are no infinite loops or unbounded tapes that introduce impossible problems into the math. There are some interesting ideas around viewing AI models geometrically that I suspect will prove to have a lot of merit, though perhaps will need some time to develop. At the very least, it’s something that is fairly general-purpose and well-connected to already rich and mature mathematical tools.

I’m really not afraid of some scary AI monster. Maybe there will be things worth worrying about someday, but it seems much more likely to me that everyone is just lost and scared and seeing things. If we open the black box, if we turn on the lights, if we take off this monster’s mask, I think we’ll find it’s a lot less scary than we imagined.

I’m trying to put out more regular articles now, even if they’re a bit shorter. I have another article mostly finished that should hopefully be out this weekend, and a couple more in the pipeline for next week. Thanks for reading, and stay tuned!

Share Bzogramming

Subscribe now

Ubiquitous Computing was a philosophy of computer interface design that imagined computers fading into the background, becoming almost invisible. As Weiser saw it, people don’t set out to interact with a computer, people set out to solve a problem. The computer is merely a means to an end, and much of computer design just gets in the way. Weiser also often made analogies to written language - we barely notice the thousands of objects all around us covered in written text, nor do we think much about how we interact with these things, we do so intuitively. The goal was to design computing systems that would behave similarly.

Weiser’s vision was that this would be accomplished by having a spectrum of different kinds of computers - “computing by the inch, the foot, and the yard” as he put it. More specifically, this involved computers called tabs, pads, and boards.

Tabs were imagined like digital post-it notes, and could function as ID badges for employees or objects, PDAs, common functional items such as calculators, or controllers for appliances such as smart thermostats. There was also an idea of tabs being embedded into a wide array of household items, perhaps with beepers in order to help locate lost items.

Pads were imagined like a sheet of scrap paper, or a magazine or book. They were meant to serve as devices for individual work, somewhat like tablets of today.

Boards were imagined like chalkboards or whiteboards, meant to fascilitate group work and interactions.

Another core idea was that these devices would all talk to each other across the internet, and that devices would largely understand who you are (perhaps by scanning your ID badge tab), and that any device would be able to grant you access to all your files based on your identity. You wouldn’t have much of a personal collection of tabs, pads, and boards, but rather you could walk up to any such device and start using it as though it were your own.

This of course would be an incredible security nightmare, but these were idealistic people in idealistic times. Some attention seems to have been given to security with talk of encrypting files over the internet, though it’s hard to imagine how this would have really made much of a difference without making the rather poor assumption that every device is completely trustworthy.

Ubiquitous computing was also made explicitly opposed to two popular ideas in computing at the time - Virtual Reality and Personal Computing.

VR was opposed on the grounds that it attempted to have the computer replace the outside world entirely rather than augmenting people’s interactions with the world. Spending all day in a digital fantasy world doesn’t put food on the table. I think this is a fair critique of VR and likely a major contributor to why it still to this day has yet to really take off.

Personal computing was seen as an arcane and unintuitive form of computing, compared to how “scribes once had to know as much about making ink or baking clay as they did writing.” Weiser saw it as merely a primitive transitional step, one that isolated people in their own little worlds, and that no one in the future would use for the same reason that no one has a single, extremely important book they carry around everywhere for all their paper-based needs.

The Limits of Simplicity

Every object you interact with has a sort of language embedded into it. You can enumerate some set of possible interactions, which can be thought of as symbols, and sequences of interactions or symbols behave like statements or conversations which result in particular outcomes. At least in the abstract, this applies just as well to interacting with humans and computers as it does with, say, a hammer, or a pen, or a piece of clay. Our minds are filled with many bits of knowledge of the form “do W, X, and Y, and then Z will happen” that guide our interactions with the objects around us. The space of all of these objects can be imagined as akin to a finite state automata, a big flow chart, or a descision tree, which our minds navigate as a series of actions meant to arrive at a planned outcome.

I suspect that the human brain models its interactions with tools in much the same way it models conversations, through this kind of linguistic model. The low-level circuitry in the brain’s cortical columns show no apparent difference between humans and nonhumans or linguistic and non-linguistic brain regions. It’s the same general-purpose thinking algorithm, just applied to a new problem.

If we assume a practically finite number of potential interactions with an object, and perhaps assume some locality and other factors that make some interactions more convenient than others, we can imagine some enumerable space of interactions. We can augment it with a cost model - longer sequences of actions tend to be more expensive, complex, and/or time consuming to perform than shorter sequences. There are also information-theoretic constraints. If there are two possible actions we can take with a given tool, and we want a million distinct outcomes, then it is not possible to create a language where there are not at least some outcomes which require log2(1,000,000) or ~20 actions to reach. In practice, many sequences of actions may produce identical or useless results, and so accounting for this redundancy will often increase the number of actions required to reach the average desired outcome.

A little redundancy can of course be useful, as having multiple ways to accomplish a task can enable personal style and flexibility in learning, as well as creating flexibility for forgiving the occasional mistake.

Perhaps some cases involve continuous systems where the state space may be technically infinite, though in practice a combination of the finite resolution of the human mind, the tendency of continuous systems to map miniscule changes in input to miniscule changes in output, and the fact that the exceptions to these cases can be modelled as merely having a random element beyond human precision to control, all together make such technicalities a non-issue.

Using this information-theoretic constraint, we can see that with any given tool the complexity of the average interaction or task is directly limited by the number of distinct outcomes that are possible. We can fiddle with the design of our tools to shift what is and isn’t convenient to accomplish, squeezing out redundancies and inefficiencies where they exist, but there is always a fundamental limit on how expressive we can really be for a given amount of brevity.

The Case of General-Purpose Tools

If we have a constrained problem with a manageable and finite number of possible outcomes, defining a simple and efficient interface isn’t too hard. However, as we increase ambition and utility, interactions get far more complex.

Computers are general-purpose machines, that’s their whole point. There is a vast and effectively infinite range of behaviors and functions we could achieve with a given computer, and to start pruning these options in any meaningful way would eliminate much of the point of the computer.

If we wanted to create a “language” to optimize our interactions with computers, to reduce the average number of actions required to get a computer to perform a task, where do we even start?

There is a “sport” among programmers called code golf, in which the goal is to write a specific program using the absolute least amount of characters possible, no matter how unreadable or bizarre the output. Seeing as some languages tend to be more verbose than others, there has tended to be a metagame to it around language choice. Eventually, “code golf languages” were developed primarily for the sake of exploiting this. Such languages tend to do funny things such as defining single-character functions for solving common code golf challenges, or defining an empty source file to be equivalent to a hello world program, making the most common code golf challenge require literally zero characters (something a little hard to beat). Other design choices are often made around minimizing the file size of other common code golf challenges with zero regard made for readability. In a narrow sense, these languages are theoretically highly optimized and concise, but in practice they are optimized for a game with little practical value.

As mentioned before, a little redundancy can often be a good thing.

If the point of a computer is to be general-purpose, to be capable of a very wide variety of things, to be capable of nearly anything, then there really is no one central goal to optimize around. The most fair thing to do would be to pick some basis that is not too inconvenient to implement many common things with while optimizing for something practical and fundamental such as energy efficiency.

Instead, the real game is not in finding the one weird trick that enables us to speak single sentences into our computers and have the precise, often complex outcome we want to land at our feet with close to zero effort. This limits detail in specification, and often leaves us with undesireable results on any underspecified dimension.

This line of reasoning can of course also be used to pour cold water on the many pipe dreams around things like declarative programming and prompt-based AI, but for now I’ll leave that as an exercise for the reader to contemplate.

So the game of finding the one weird trick that makes all our interactions with computers extremely terse and efficient is largely a fool’s errand if we take it too seriously - undoubtedly there are useful innovations that can and have been found, but it will never give us the kind of magic that so many dream of. Rather, the real game is the one we’ve been playing all along - infrastructure and customization. Tooling. Personalization.

Everyone has a different set of tasks they need to use their computer for, and just as people organize and customize their house and possessions to help them better accomplish their work and chores, they do the same with their computers. The way to build something that is both general-purpose and that is convenient to use is to build something that conforms to whatever the particular user needs to use it for.

The software you have locally installed on your machine doesn’t just reduce how much you need to download on the fly, but also reflects the kinds of problems you spend your time working on. Any convenience that comes from using something already on your machine rather than something that must be installed before use is something that optimizes for your use and makes you more productive. The machine conforms to your personal needs, and thus becomes more convenient for you to use than some standardized, one-size-fits-all solution. It may be less convenient for someone else to use, but who cares? It’s yours.

If you’re building software, you’re doing so with a vast stack of development tools and code libraries that make things more convenient with the assumption that they represent a solution to a problem that you’re likely to reuse. If you have a package manager or other cache of libraries that you specifically have downloaded for your coding needs, you’ve optimized this system around your personal convenience. Someone else will likely have a very different personal code collection.

The way people use internet browsers, with tens or sometimes hundreds of open tabs reflects a similar kind of use. If your browser crashes and you lose all those tabs, it may be awfully inconvenient to reconstruct all you had built for yourself for the problems you were working on. You had a mental map of what kinds of tabs you had open and how to get to them all, though some of that knowledge was probably still stored by the browser. Much of that map becomes second-nature and intuitive as you build and maintain it, at least for you.

Perhaps there’s some inconvenience from browsers not being designed with the goal of juggling so many tabs, but that’s a different problem that’s solvable with better browser design.

If you have a general-purpose tool, where little can be made convenient by default because of the infinite breadth of potential functionality, the way you make it convenient to use is to build structures within it that compress the paths you most frequently take so that it is convenient for your personal purposes.

Back to Ubiquitous Computing

There are some interesting and nice ideas to be found in Mark Weiser’s work. His ideas have been influential in some corners of design around IoT and cloud computing. I wouldn’t be surprised if there are a few remaining gems in his work that have yet to be given wide adoption in the right context.

I imagine however that the balancing act needed to maintain computers with the level of personalization necessary to be meaningfully and maximally useful while also opposing ideas from Personal Computing would have proven difficult if Weiser would have lived long enough to continue his work that far. The analogies with paper and similar items only can go so far - paper is useful for writing and drawing, but its uses are also vastly more limited than computers. If it were not, we would just keep using paper for everything.

Effortless simplicity of the kind that Weiser dreamed of is incompatible with generality. You must pick one or the other, and the only obvious compromise is with a system that gradually conforms to personal needs, which brings us right back to personal computing. Perhaps there were some ideas around using ID badge tabs and personal file access that could have made that work, but isn’t that just personal computing with extra steps?

Sometimes people get too attached to an idea because of how nice it seems that they become blind to its flaws. Oftentimes grand, simplifying ideas don’t actually pan out that well - the world is far too complex and any structure requires assumptions to be made, assumptions that will not hold across all situations in the world. Any broad vision that “everything must be done in this way” is going to eventually run into friction and contradictions in some cases.

Excessive standardization in the face of a very complex problem is doomed to fail.

If there were a constrained, limited range of usecases that covered 80% of what people use their computers for - social media, Youtube/Netflix, word processing, spreadsheets, maybe a few others - perhaps some approximation of this vision could be achieved while still letting the hardcore users use their laptops, desktops, and supercomputers for the rest. Perhaps that would also be limiting what computers could evolve into though, limiting our imagination of what computers could be to mere office supplies, crystalizing things too early.

For those who do not follow my Twitter, I recently announced that a few friends and I are launching Possibilia Magazine, a literary magazine for optimistic, realistic, near-term science fiction. The overall goal is to offer brighter alternatives to the abundant, bleak depictions of the future that are so often presented in media, while also being able to showcase esoteric and new technologies and ideas that we feel get far too little attention. If you’re interested, you can join the mailing list here to be alerted of updates. We also have opportunities for writers and artists for those interested.

This is a crazy idea I had in the summer of last year that has unexpectedly snowballed into something that’s been gaining a lot of traction and getting a lot of amazing people involved. I honestly never expected things to move this quickly or get this much attention, and the result is shaping into something far more ambitious than what I originally dreamt up last year.

It’s also been keeping me a bit busy lately, which explains why my substack posts have been so sporadic. I do have a few articles in the pipeline though, and I’m still spending a significant amount of time on coding projects, so you can expect more frequent content here in the near future.

Share

Subscribe now

There’s been some hype over the past week or so around a supposed room-temperature, ambient-temperature superconductor. There are plenty of reasons to be skeptical of it, but if it’s real it’ll be a very big deal. So far though, there have been some independent replications that have shown that it may at least be an exceptionally strongly diamagnetic material, so at the very least the material has some interesting and potentially useful properties even if it isn't a full superconductor.

[Note: as I was editing this, an independent paper backing up its superconducting properties has dropped.]

One interesting property of the claimed material however is that it’s actually fairly simple - just lead, phosphate, and doped with a bit of copper. Compare that to the common approach to superconducting materials, which often are full of exotic and expensive elements. We will very likely know whether or not it’s actually legit within a week or two. There’s already a decent number of people trying to replicate it independently, including a few of private individuals. If we ever get a room-temperature, ambient-pressure superconductor that sees widespread adoption, it will have to be something simple and overlooked like this lead compound; anything too exotic will simply be impossible to produce at scale.

There’s certainly a lot of potential such a material could have, and I’ve seen a number of people claim that it could bring about “another industrial revolution.” That’s a bit of an exaggeration in my opinion, but not too big of one. Much cheaper MRIs would certainly be nice to have, and there's a good chance that the diamagnetic properties that it seems much more likely to have may at least be useful for building maglev trains. With that said, the idea that we *need* an innovation like this before we can start doing big things again sounds a lot like a coping mechanism to me.

America needs to start building stuff again. Stuff made from atoms, not just bits. Software certainly has its place, but it shouldn’t take priority over absolutely everything else to the extent that it has. "Tech companies" used to refer to all kinds of technologies, whereas today the term seems to primarily refer to software companies. Innovation outside of computing seems to have slowed to a crawl over the past few decades. We need to build rockets, factories, robots, planes, trains, automobiles, nuclear reactors, ships, tractors, machine tools, houses, skyscrapers, titanium foundries, and more. Much more. We need to push innovation and boundaries again that involve things other than merely moving electrons around.

The one person who of course can be pointed to who is doing something about this, cliche as he is, is Elon Musk. While there certainly things he does very well with his businesses, I’m inclined to say that any business or engineering brilliance isn't actually the biggest factor to his success.

I know a number of people who entered university intent on getting a degree in a field like biochemistry or geology, and left with a CS degree instead. I’ve heard biologists lament how bad their software tools are, because any biologist who can write even mediocre Javascript or Python code leaves to get a much higher-paying job as a developer, leaving the biology software to be written by the people left over. I’ve personally come across archaeology databases implemented as complicated and ugly SQL databases not because that was the best tool for the job (far from it), but because it was the only tool that the elderly archaeologist who decided a database was needed could figure out.

I’m fairly convinced that the real secret to Elon’s success is not his brilliance in engineering or business - though that likely is a partial factor1 - but rather that software (and perhaps also finance) has overwhelmingly brain-drained every other industry. The people who used to build rockets and airplanes, optimize industrial processes, design complex machines and appliances, and more, are currently all writing code instead. If you simply transfer the top software engineers into other fields, or at least make rockets and electric cars sexy enough to convince young people to pursue those fields instead so they can work at your company, you quickly gain a gigantic talent advantage over everyone else and can rapidly make progress that otherwise wouldn't have materialized for decades if ever at all.

At the end of the day, materials science matters a lot. New materials give us new building blocks to work with and opens up new technologies that weren’t available before. The space of possible materials is vast, but also seemingly rather underexplored. It somehow took until the 1980s for anyone to get the idea to make an alloy by mixing five or more metals in equal quantities, and only in the past decade has this really gotten serious academic attention, in the form of High-Entropy Alloy research. This sounds like the first thing a child would suggest to do upon learning what metal alloys are (just mix them all together!), and it’s now a promising and fairly new field for finding new materials with unique properties. The fact that this wasn’t uncovered through basic experimentation a very long time ago is something that I find shocking.

A lot of the intelligent effort into materials science today seems to go into things like quantum chemistry simulations, I would assume because all the smartest people even adjacent to the field write code all day and when all you have is a hammer everything looks like a nail. This is somewhat promising, but from listening to enough lectures and talks and reading up on the math involved, it seems to me like a poor approach. All the algorithms are at best something absurd like O(n^11), and at worst exponential time. I certainly will be the first person to say that exponential problems are often much less scary than they're made out to be, and that there are a plethora of tactics for making them more tractable, but nevertheless there are some cases where it's just not practical. Maybe I haven't dug enough into the field myself yet, but from what I've seen it seems like there must be better approaches.

There’s certainly value in computational tools; you can visualize and measure properties that would otherwise be invisible in the physical system. Depending on how you architect the system, you might even be able to rewind, fast-forward, or fork and iterate upon the system in ways that just wouldn't be possible in reality. However, you'll always be resorting to approximations - often approximations of approximations of approximations of theories of physics we know aren't actually completely correct. You also have to deal with the fact that reality will always be able to compute precise results faster than our computers ever will. If you lower the resolution of your simulation with weak enough approximations, sometimes you can get something that can run pretty fast, but you'll be losing a lot of detail, and when it comes to material science that detail is often what matters most, hence why all the computational approaches are trying to face these exponential complexity walls.

There is the argument that we can utilize advanced models to better understand and study the principles behind these systems, that if we can build a good enough model in a computer we can use such models to get a deeper understanding of how these systems work to accelerate innovations in general. This has some merit to it, but any system that is fundamentally irreducible below exponential time (as is the case with most quantum systems) will necessarily be impossible to reduce to the kinds of simple explanations we often want. We can simplify some parts, but this will never give us the full picture, and simple trial and error and real-world experimentation will likely always play an important and irreplaceable role.

I can certainly imagine a hypothetical business that simply pays a bunch of people to mix random metals and minerals together all day and test their properties. If done on a large enough scale, it could be very lucrative at creating novel and valuable materials in ways that a company with access to a giant supercomputer and the best quantum chemistry algorithms would likely struggle with. If there is any part of such a business that could benefit from computing, I expect the greatest value would actually be in leveraging some basic Monte-Carlo-style algorithms and statistical tools for more efficiently navigating chemical possibility spaces toward interesting materials.

There certainly may be flaws in such an idea, but we at least need more people thinking about such issues and we should be questioning if the current approaches are actually ideal. Many of the existing areas of material science seem to love focusing on exotic crystals made with rare earth metals and other exotic things that would never work at scale. Either that, or organic chemistry labs that try and almost always fail to overcome biology's 4-billion-year headstart and copy biology with little to no understanding of how such biological systems actually work. A lab focused on high-throughput experimentation would likely be one biased toward materials that are cheap and easy to produce, which is not a bad bias to have!

Tech Turmoil

This brain-drain situation with software is something that I expect isn’t going to last forever. In fact, I expect things will start to seriously change by the end of the decade.

While perhaps many tech companies are happy to tell you that business is going great, their actions seem to tell a different story. Facebook/Meta, which has all its eggs in just one basket with 98% of its revenue coming solely from advertising, has been seemingly desperate to jump ship to something else for a few years. First they tried to start their own cryptocurrency with Libra/Diem, and more recently they burned $40 billion trying to build a “metaverse” that not even their own employees wanted anything to do with, while much of the public considers the technology an abomination.

Twitter/X is now trying to diversify away from ads and pivot toward more of a subscription model while its new owner is heralding it as a future WeChat-style superapp and financial giant.

Google/Alphabet is meanwhile commiting fraud, taking actions to fight ad blockers, and taking other unpopular tactics such as threatening to delete old gmail accounts and then walking back after public outcry and claiming that old Youtube videos won’t be deleted - yet. Meanwhile with the most recent quarter being an exception, Youtube (which has never been profitable to begin with) has been seeing multiple consecutive quarters of declining ad revenue. Keep in mind that out of the plethora of new video streaming services that have popped up in the past few years, not a single one of these new players has done anything but hemmorage money, and that’s without having to maintain what is likely a multi-exabyte video library that never shrinks.

While perhaps it’s difficult to say for certain that advertizing isn’t as sustainable of a business model as it seems, and that a twenty-year adtech bubble may be about to burst, following the old idiom that "actions speak louder than words" provides a strong hint that things aren’t as good as they appear.

I’ve personally begun backing up terabytes of Youtube videos as a side project.

It’s not just an adtech crisis that could cause a problem in the next few years. With Moore’s Law rapidly grinding to a halt, companies will no longer get exponentially growing subsidies from hardware improvements. If throwing 3x more compute at a problem can get your company 1.5x more revenue, growing this way is really only sustainable if you can somehow double your compute/$ in the time it takes to build out that compute. This works today, but will very soon stop working. I expect AI companies to be hit especially hard by this.

This is also true to some extent for storage costs, of course. Most of our storage technologies are starting to run out of steam; the NAND flash in your SSD scales the same as your CPU and will level off around the same time, RAM stopped scaling a decade ago and the only viable technology to replace it failed in the market, and mechanical harddrives are closing in on theoretical limits of how much data can be stored per square inch of magnetic material. The only storage technology that isn't yet close to its theoretical limits would be tape drives, but unless companies decide to tell their customers to deal with 4-hour waiting times to access data not in a cache, or perhaps switch to a crazy business model such as renting out tape drives, the exponentially growing storage capacities of datacenters will soon become prohibitively expensive to maintain.

Chip companies also are going to have a hard time with Moore's Law slowing down, though I expect that to be a slower process. The past few decades has created an environment where real innovation has been discouraged by the market; any time you spend making a CPU with a more efficient architecture is time that you're not spending porting last year's architecture to run on smaller transistors. Throw in compilers that up until recently struggled to translate code to unusual architectures, as well as the fact that our modern notion of "large volumes of open source software freely available on the internet" is a relatively recent phenomena, and you have a strong incentive to stick to the status quo as opposed to innovate. Remove these factors and suddenly innovation into exotic architectures is incentivized again, and backwards-compatibility matters a lot less. We're alrady seeing this to a large extent with GPUs and AI accelerators, but I expect it will come for CPUs soon. There are ways of making CPUs orders of magnitude more efficient, even with tiny engineering teams.

Someone soon is going to build a CPU dramatically more powerful and more efficient than those that Intel, AMD or ARM produce. The biggest opportunities for this will involve exotic things like tiled architectures, which are still general-purpose but will necessitate rewriting a lot of code to optimally use the physical layout of data and compute on the chip. There are also other technical reasons why tiled architectures may completely change the way that context switches work, which could eliminate the factors that have caused operating systems to accumulate millions of lines of driver code, and making it possible again for even hobbyists to build new and competitive operating systems from scratch again. As these incompatible architectures grow in popularity, it's possible that many companies with large codebases designed around obsolete hardware will need to be at the very least heavily adapted, turning many large codebases from massive assets to massive liabilities.

Software certainly isn't going to collapse entirely, but I expect that the next decade or two will involve a lot of turmoil, downscaling, bursting bubbles, major bankruptcies, and new players. Many business models and even entire products and services that seem viable today may turn out to be unsustainable long-term, especially the companies that provide massive, resource-intensive services for free. Companies will likely have to get a lot more lean, and the exceptionally high wages for relatively little work will likely go pretty quick.

Return to Atoms

Earlier, I called the excessive focus on superconductors a coping mechanism. My reasoning behind this statement is that there are many promising technologies, including many at the intersection of computing and other forms of engineering, that may be very valuable. The real bottleneck preventing innovation outside of computing likely has very little to do with a lack of breakthroughs, but a lack of sufficient talent.

I’ll list a couple examples of technologies I expect have a lot of potential, though I expect there’s a lot more out there than I know of.

Hyperspectral Imaging

One technology I expect is going to play a major role in the future is hyperspectral imaging. The basic premise is that, by passing light through a prism or similar optical system, you can separate out the individual component frequencies that make up that light and get significantly higher color resolution. Rather than simply having red, green, and blue color channels, you instead distinguish between hundreds of different, narrower color channels.

You are effectively getting a full spectrogram for every pixel, which communicates a tremendous amount of information about the materials and chemistry going on in that pixel. Two different materials may look like identical shades of red to the human eye, but a hyperspectral camera may inform you that they are reflecting or emitting slightly different frequencies of red light, with these spectral acting like fingerprints that can identify these materials.

If there is any technology that could funnel some of the past few decades of innovation in bits into innovation in atoms again, this is something I’d consider a good candidate.

A lot of the existing work that is going into hyperspectral cameras often involves attaching them to drones. Flying the drone over a field of crops can tell you detailed information about the health of each plant (e.g., “these plants here have spots in this narrow frequency band that the human eye can’t tell apart from any other green light, that suggests they have a magnesium deficiency”), and flying them over earth can tell you of the minerals in the ground, enabling easier surveying for mining.

The technology is also used in the food industry to rapidly discern healthy from unhealthy produce; there may be color differences indistinguishable to the human eye that can indicate mold or insect infestations, allowing for machines to automatically filter out bad produce with high accuracy.

Right now the biggest things that seem to be holding the technology back are software and cost. Software tools for analysis and compression of hyperspectral images are widely considered to be poor in quality, and it’s likely that some small teams of clever software engineers could push the field much further along.

In terms of cost, hyperspectral cameras are often tens of thousands of dollars. I have no idea why they are much more expensive than a phone camera; they’re little more than an ordinary camera behind a glass prism and some other simple optics. My best guess is that the market right now is too small to support companies without making large sums of money from individual sales. With a good marketing campaign to make people aware that the technology exists and what it’s capable of, I could definitely see a “Raspberry Pi of hyperspectral cameras” company being enormous and sparking a lot of innovation.

Geometric Folding Algorithms

The mathematics behind origami and similar folding-based techniques has seen a lot of attention and innovation in the past few decades. Algorithms for designing and engineering complex machines based on such approaches have been developed, and these mechanisms seem to have a lot of advantages.

This is a space that sees a lot of research attention, and occasionally some attention from pop science, but hasn’t quite broken out into many applications yet.

Auto Industry Business Models

This is less of a material technology and more of a social technology, a matter of the people and ideas that surround and support a technology as opposed the physical gadget itself.

Software today seems to largely circle around a small range of business strategies that work well for internet and app companies, but work less well for others. There are a lot of VCs who are very familiar with such business strategies and likely more money available for your company if you use them compared to if you don’t, but that doesn’t mean that they’re a good fit everywhere.

For example, the strategy of “acquire lots of customers at a loss and figure out profitability later“ can easily fail with physical products, especially if your “daily active users” only make one purchase every few years.

I find that the early car industry seemed full of creative business models and business tactics that seem well outside of the toolbox of modern entrepreneurs. After all, cars are a very difficult sell when road networks are minor and poorly maintained, streets are full of horseshoe nails that pop your tires, and where everything is built around the assumption that no one can travel very far in a day, and so there is no reason to travel far often.

Many early auto companies were vertically integrated to a degree not seen today. Valvoline did (and still does) everything from producing the various fluids that go into cars to running the service centers that actually put them in your car. Michelin famously created the entire modern restaurant industry just to sell tires, and even then that’s just the very tip of the icerberg when it comes to their tactics. Entire books have been written about their tactics, which involved everything from printing educational magazines to teach people how to properly install tires to funding and marketing pro-natalism organizations to prevent a long-term decline in the French economy.

If we want innovation outside of software again, we’re going to need to relearn how to run successful businesses that aren’t merely software companies, and that is a much deeper and more complex problem than most people realize.

Share

Subscribe now

Hey, it’s been a while since the last article. This was my primary income source for a few months between jobs, but early this year I started a new job doing some really exciting work. It’s not quite ready so I’ll refrain from making major announcements here yet, but I’m excited and things are going well.

The work I’m doing there is also generating some very fascinating new ideas that are likely to make it into some really great articles here.

That did start to take up an increasing amount of my time, and eventually the weekly publishing schedule here became unsustainable. I broke out of my regular rhythm of staying up later and later every Sunday night / Monday morning. I’ve had a number of articles I’ve been working on since then, but nothing I’ve gotten around to publishing.

I definitely plan on writing more here soon, though perhaps at a slower or less regular rate.

This certainly won’t be an exhaustive list, mostly just elaborations on ideas adjacent to last week’s article on graphical programming. I won’t be covering AI tools here for example, that will be a separate article in the future.

Some Legitimate Wins for Graphical Programming

Graphical programming has seen some success in education. Part of that benefit seems to be from reducing the intimidation factor that text-based programming seems to have on non-programmers. I don’t find this advantage particularly interesting from a technical perspective, as it’s more of a cultural issue.

A more interesting case that graphical programming sometimes can handle well, especially as an educational tool, is short feedback loops and interactivity. If the visual aspect of the program is primarily focused more on the output rather than the code itself, that can be more useful.

One reader who reached out mentioned their fondness for the Logo language, though granted - Logo is a text-based language focused on generating graphics rather than a purely graphical language.

Some game engines have also been using graphical scripting and shader languages. These also seem to be reasonably successful, though note that they are a much more constrained situation than normal programming languages.

Making your programming language a bunch of boxes and lines works better for simple shader functions meant to be written primarily by artists rather than a full programming language. The limited size prevents such graphical code from becoming an incomprehensible hairball.

Interactivity

Logo and perhaps some similar languages clearly get the interactivity part correct - getting more feedback from programs is definitely something that deserves a lot more engineering attention.

Some interactivity already exists in programming languages, and has for a long time. REPLs have been a common feature in many functional languages for decades, though unfortunately they don’t scale well to large programs.

There has been some discussion in the past few years, popularized by people like Bret Victor, of more advanced alternatives to REPLs. Victor’s specific idea is print out the contents of variables while code is being written in a special interactive window.

This definitely seems like it would be useful for some simple applications. Bret’s demo here shows it detecting a lot of low-hanging fruit while he writes his demo code.

One critique I’ll give of this idea is that it doesn’t clearly scale well beyond printing numbers and small strings, and the mechanisms Victor shows for dealing with loops probably won’t scale well past a few iterations.

Granted, similar ideas to this are used pretty regularly in notebook systems like Mathematica and Jupyter. Those systems are clearly much more successful and have been around much longer than Victor’s version - the meaningful difference is that notebooks only print the data you specifically request to be printed, and offer proper visualization tools for many common types of data (though probably are trickier to use if you have your own custom data structures). Victor’s version, which tries to print everything, is trying to cover many hard cases that notebooks choose not to cover, and doesn’t have many clear ideas about how to scale to the needs of these harder cases.

Victor also has some other ideas with flashier visualizations and interactive tools, though I’m not convinced any of them really generalize well. Maybe you can have a nice tool for visualizing circuit properties or animating simple videos like he shows in that demo. I haven’t seen any ideas from him however that seem all that useful for more general code.

Galois Emulation?

All of these interactive emulation systems share a common limitation; while they’re great for rapidly debugging the behavior of a program on typical inputs, many bugs are the result of rare inputs that break code in unexpected ways. These rare inputs can be exceptionally difficult to find - mathematically speaking, doing so is NP-complete (or even undecideable when loops are involved).

For reference, NP-complete means it’s at least as hard as breaking cryptography in the worst case. In fact, if you write code designed to break when the correct key is passed into a cryptographic function and then ask when it breaks, by definition you have a debugging problem that is as hard as breaking that cryptographic algorithm.

If there are loops involved, especially particularly tricky ones, then it’s of course much harder than cryptography and may in fact be impossible to solve.

While this all may seem like very good reasons why such limitations must exist in our tools, the reality is of course that NP problems are some of the most bizarre things in modern mathematics, with behavior in practice that is extremely different from their behavior in theory.

Most code is meant to actually be understood by humans, and that property results alone in a lot of seemingly insurmountable problems actually being easy in practice with the right approach.

I actually just wrote a SAT solver a couple weeks ago, SAT solvers being one of the kinds of tools available for solving these kinds of hard problems. It’s not even a very efficient solver (there’s no unit propagation or CDCL, though I have some plans for more novel optimizations), and yet it’s able to solve many 64-bit random SAT problems within a few seconds while running on a single core on my laptop.

Put another way, it’s sifting through a search space with 1.84*10^19 values and reliably finding a needle-in-a-haystack solution in only a few billion (10^9) CPU instructions. This is somewhere on the order of 10 billion times faster than brute force, while still technically being a “random guessing” algorithm. Making SAT solvers faster is just about learning about the problem as you go to make smarter guesses.

I’m specifically running this on random SAT instances with a clause count chosen specifically to maximize difficulty. Random SAT tends to be much more difficult than many “real-world” SAT problems.

SAT solvers (and other exponential-time algorithms in general) are one of those fun cases where you can make claims like “this optimization made my code a billion times faster” and not be exaggerating.

In practice, SAT solvers can be a bit too precise for many common code correctness problems. Converting code into something they can reason about usually encoding every individual logic gate and wire necessary to implement the function in circuitry, and they focus on every minute, low-level detail.

SMT solvers are another tool available, which basically amount to a toolbox of higher-level logic tools that can more efficiently solve many problems, with the option to fall back on a standard SAT solver in the case they aren’t sufficient.

Another tool, particularly well-suited for looping code, is Abstract Interpretation.

In both SMT solvers and Abstract Interpreters, there is a common tool used to approximate large possibility spaces called a Galois Connection. Galois connections can be thought of as a tool for translating between a high-resolution and a low-resolution model of a problem, or even between different low-resolution models that invest their resolution differently. They are an extremely powerful tool for making many hard problems tractable.

They also however are based on a type of mathematical structure called a lattice, which as I’ve pointed out before are actually mathematically almost identical to topological spaces, strongly implying there might be a way of meaningfully visualizing them.

So imagine if instead of the Bret Victor approach of visualizing a single set of inputs flowing through your function, you could visualize a representation of all possible paths at once? This seems like it should be possible, and most of these Galois connections are not as complex as they may sound from my description here.

I’m not quite sure what this looks like yet, but the math at least looks promising.

I do plan to write an article on Galois connections at some point - luckily some of my work on Bzo right now involves writing my own SMT solver, and while it’s still pretty early in development I think I’ll have a lot of interesting stuff to say in a couple months.

Searching “How to write an SMT solver” tends to give few helpful results beyond “you should never write your own SMT solver, it’s too hard, just use Z3”, so I figure I might be able to write a few authoritative articles on the subject.

Non-Textual Programming Paradigms

I’ve written before about non-textual programming, mostly in the context of exploring approaches to programming that are not well-suited for a text-based medium in the first place.

If you want to edit an image, you open it up in a program like photoshop, not a text editor. Not that it’s not possible to edit images in a text editor, but images are very poorly-suited for human-legible text representations.

There are a lot of cool ideas in theoretical neuroscience, with plenty of unique and desirable properties that effectively come for free with the paradigm. It would be great to be able to engineer such systems, but that's a paradigm where the primitives are basically overlapping spheres in 1000+ dimensional sparse bitvector spaces. It's hard to imagine how to make that legible in a text format, but it's at least easier to imagine a more graphical approach being viable, albeit with the need for some creative ideas first.

Machine learning is arguably in a similar situation, where most of what actually defines an ML model is a giant collection of data files and only a tiny amount of text-based code. Under the hood, neural networks are incredibly geometric systems. In the long term, engineering with them (especially if we want to do so in a way that makes safety guarantees about their behavior) might require some more geometric interfaces.

Foveal Optimization

I often find that my most interesting ideas in my articles are not always explained as well as I’d like, and I usually find much better ways of explaining things in the days after I publish an article.

In my last article, I tried to discuss an idea about how our modern understanding of software interfaces seems deeply ignorant of how the human visual system works and how to interface with it effectively, as well as that things like syntax highlighting and code indentation are in fact a few of the kinds of things we should be doing more of that have somehow made their way into common practice. I had a friend reach out with some questions, not fully getting my point.

I’ll end tonight’s article with a quote from the conversation that seemed to clarify it pretty well for him:

Maybe another way of putting my point about graphical programming is that part of the problem is that you're trying to optimize for how quickly the eye can access the information it needs. The only part of your eye that can read text on your screen (assuming you don't have a gigantic font) is the fovea, which only can look at a tiny portion of it at a time. So the real problem isn't just dumping a bunch of text to the screen, but making the data legible to the low-resolution peripheral vision to minimize the amount of saccades to locate important information.

If you could blur the screen so that the text is all unreadable and still be able to quickly and reliably guess what's a loop, what's an if/else or switch statement, where each block of code starts/ends, where the loop header/branch condition is, etc., you're going to have much more readable code than if you don't have that.

That's what indentation and syntax highlighting are ACTUALLY doing.

And part of my point is that we could probably push things much further in that direction, using other tricks like leading lines to make it even more efficient to navigate code, as well as finding ways to recognize structure on even larger and more zoomed-out scales.

I definitely appreciate feedback on my articles, though don’t always get much, so getting multiple people reaching out after this last one with lengthy responses really caught me off guard. Thanks. I think I’ll remove the paid-subscribers-only requirement for the comments on my articles. That means more feedback for me, and makes it easier for me to clarify anything I don’t fully get across the first time.

Thank you for reading!

Subscribe now

Share Bzogramming

Leave a comment

Usually it is claimed or at least implied that this will make code much easier to understand, perhaps to amateurs and non-programmers who might be intimidated by text-based coding, but anyone who tries to use these inevitably just develops a giant, messy hairball of a program that is the exact opposite of “easier to understand”.

Of course, one often-toted “definition of stupidity” is “doing the same thing repeatedly and expecting different results”. This, and similar ideas, have been tried many times, and have yet to catch on as any serious, common engineering tool.

Are there some that have found niche success? Sure. But if they were really all that revolutionary or superior to text-based programming they would have become dominant a long time ago - LabVIEW is only a year younger than C++.

Composition, Programming the Visual Cortex

There is a certain allure to the idea of graphical programming. At the very least, there seems to be an intuitive sense among people that it’s better in some way, even if it’s difficult to elucidate why.

Contrary to how many programmers often seem to think of it, the human visual system is not merely a camera with a grid of pixels. The majority of the eye’s “pixels” are focused in a tiny, very-high-resolution region called the fovea at the center of the visual field. This is sorrounded by low-resolution peripheral vision filling the rest of the visual field.

Human vision works very differently from how, say, deep neural networks accomplish similar tasks. The visual cortex invests most of its resources into categorizing little pieces of images at a time, and uses the peripheral vision as a mix of low-resolution categorization and as a tool for guiding saccades, which are tiny, rapid eye movements to scan the fovea to different points of interest.

The effect is that vision is not about big grids of pixels, symbols, and details, but rather many little grids arranged in space, read as they are demanded.

This actually makes human vision an incredibly interactive system - the eye focuses on what it needs to, and can locate and scan different parts of objects based on what is currently relevant or reduces the most ambiguity.

There are many ways to convey meaning in the composition or arrangement of details across space within an image. The visual arts have been exploring these ideas for a very long time. Things like the “rule of thirds”, various methods of arranging shapes, textures, empty space, and colors, and the introduction of leading lines have been well-understood for centuries.

I find leading lines particularly interesting, as the eye will often latch onto and follow them, settling wherever they lead - putting the most important details of an image in locations where many lines or curves converge is very effective in conveying meaning and making an image easy to intuitively understand.

I like to think of it as the visual system having a certain kind of algorithm for navigating scenes, and tools like composition are essentially a visual tool for “programming” it to produce meaning.

These are powerful, thoroughly-tested tools for conveying meaning to the human visual system. Unfortunately, modern programming makes extremely little use of them.

Indentation

We should keep in mind that text-based programming already has at least successfully conveys a little meaning spatially however - that being indentation. Granted, it’s really only used to denote scope and nesting, but I find it shocking how nearly every language makes use of it, often in almost exactly the same ways, while programming language theory seems to have paid it little to no deep attention.

If anything, indentation is an anomaly - something based in rules completely foreign to formal programming language theory, but that persists more stubbornly than almost any other feature because it just works so extremely well. It’s worth taking a deeper look at this anomaly and considering if there’s more to learn.

Syntax highlighting is another slightly interesting thing, leaning slightly on some color theory. It’s definitely an improvement over monochrome, probably by letting the low-resolution peripheral vision more easily narrow down on points of interest for upcoming saccades. I’m not convinced we’re anywhere close to using it optimally though.

I think that a few esoteric languages like AsciiDots and Befunge present an interesting idea, giving syntactic meaning to long chains of line symbols, turning code into a form of ASCII art filled with meaningful leading lines.

Do I plan to copy this idea for Bzo and make the language an ASCII art path-based language? Not exactly - AsciiDots suffers from the same hairball problem that other graphical languages have, but with an even more tedious UI than drag-n-drop. However, reasonable ways of introducing leading lines into the syntax to guide the eye to important details is something I at least will be very seriously considering once backend stuff is done and syntax is a more immediate concern.

A Mountain Through a Microscope

One other factor that I think people at least subconsciously miss that I suspect might partially drive this allure for graphical programming is the lack of any ability to “zoom out” of a codebase.

The human eye can view anything from the not-quite-microscopic if we look down at something closeby up to cosmic scales if we simply look upward to the sky.

If we want to view a mountain and get a good idea of its overall structure, the most natural way to do so is to look at it from a distance.

A remarkably stupid idea would be to walk around the mountain inspecting individual rocks with microscopes and magnifying glasses, inspecting rocks one at a time, and trying to guess at and gradually map the overall structure of the mountain that way.

Yet, when we are viewing code in a text editor, the individual characters of text correspond to individual bytes of memory. Very little exists that can meaningfully zoom out on a whole codebase at a time.

Sure, some IDEs have “minimap” features, but those tend to be limited to individual files rather than focusing on whole codebases. Congrats, you’re looking at a few boulders at a time now instead of individual rocks.

A Little Demo

I have a tool I’ve been building a little bit on the side for a couple weeks. Right now it just loads files and displays a couple visualizations of the first 256kB or so, the main visualization being a hilbert curve. Each pixel corresponds to an individual byte, colored based on the byte value and some derived properties of it. The initial goal is to have something useful for visually inspecting the binary structure of arbitrary files with (eventually) many more options than a normal text/hex editor.

It’s not a terribly impressive tool yet - I’m busy with many other things right now - but I think even in the current primitive state it gets my point across.

Hilbert curves are a pretty nifty tool, being able to map a 1D space to a 2D space, turning long, pixel-thin lines into easily-recognized blocks. This is great for visualizing large amounts of data in the form of a simple, linear sequence and getting an intuitive sense of contiguous chunks.

Files are just linear sequences of bytes, and code really isn’t any different aside from often consisting of several files.

Here is a PDF file drawn with my tool - the green spots seem to be PDF script text. The black spots are all blocks of zeroed bytes, I’m not sure why they’re there - perhaps suboptimal packing code? The teal background seems to be all the compressed data that makes up the PDF, and has some interesting structure of its own. Some regions appear to just be high-frequency noise and others, such as the lower-right quadrant, seem to have more low-frequency, longer-range structure. My guess right now is that the more structured teal regions are compressed images.

If you know what to look for, just staring at this image for long enough tells almost as much of a story as the actual PDF when properly read.

With some more work, this could probably be a fairly useful tool. Bigram plots can be really useful for categorizing data based on byte values. The tool already computes bigram-bitvector embeddings for 256-byte segments of the file, and I’ve started work on a k-means classifier, which should allow for the computer to automatically categorize distinct parts of the file that “look similar”.

I had a side project a couple years back trying to do this with large codebases - I had a similar hilbert curve plot of the entire Linux codebase, visualizing a whole gigabyte of code in one image. I’d have to do some digging through some old hard drives sometime to find that project though.

The nice thing about doing it with code rather than raw binary data is that there’s a lot of easily parsed structure in text-based code - identifiers, keywords, comments, indentation, parentheses/brackets/braces, operators, literals, and so on. A much wider and richer set of tools for deciding exactly how to color the big array of pixels should be possible that are more specific and useful to understanding code than relatively crude tools like plotting raw byte value, or k-means clustering.

If you want to zoom out on your code, you don’t need to limit yourself to just one method of doing so; much like the interactive nature of the human visual system, an interactive approach to zooming in and out and adjusting the color-coding of the map would undoubtedly be very valuable.

And of course, all of this says nothing of the many other visualization methods beyond hilbert curves.

Overall, I don’t think what is needed is any more drag-n-drop lines-n-boxes graphical programming languages. Text is a pretty robust medium and it’s far from its actual limits if we’re willing to be a little creative (and maybe take some cues from ASCII art).

Could someone actually make a graphical programming language that works? Sure. I’ve had a few rough ideas of my own in the past along those lines. For any chance of success though, something actually unique needs to be done.

The real value however is not in replacing text as a programming medium, but rather augmenting it with more visual tools to produce understandings of code that would be impractical to gain from merely reading the code.

Thanks for reading my article today. I might have some big announcements soon, so keep an eye out. If any readers are in Austin for SXSW this week, maybe I’ll see you around town - I don’t have tickets for anything, but I do have friends who know about some of the after-parties and will likely be at the FUTO event on Friday.

Subscribe now

Share