Overambitious Standards
There's incredible value in keeping things simple
I was recently alerted to the extent to which PDFs support Dynamic Content; the file format that was originally meant to be a digital form of printed documents also features support for embedded Javascript that can be used to build things such as playable videogames, embedded in the document.
Now granted, there’s a tremendous amount of hacks required to get things to work. In the video above, the speaker only built a simple breakout game, and even that required multiple tricks using pdfs in a way that they really weren’t intended for.
From a certain perspective, there’s something cool about this; it’s always fascinating to see people take a set of tools and use them to accomplish something that wasn’t believed possible with those tools.
On the other hand, I’d like to highlight a more serious problem with this, especially in the cases where what starts out as a crazy hack becomes standard practice.
It’s worth remembering that the web was originally intended to be nothing more than a platform for hyperlinked text files, perhaps with some images and tables. Basically, wikipedia.
Today, it is a full-blown application platform. How did it get this way?
The basic process is something like this:
Some set of features is added to the web
People begin experimenting with it, pushing the limits of what it can do
Someone finds some new use case for the web using the new features
It becomes obvious that the existing features are limiting, and newer features are required to unlock the full potential of the new use case
Repeat
What was originally meant to be a proto-wikipedia soon gained interactivity, which led to much more complex platforms being built on top of it. Eventually digital marketplaces, games, and social media emerged.
A serious problem with standards is that they require backwards-compatibility; once a feature is in there, it’s very hard to remove or change it. If some feature makes some set of assumptions, and then some new feature comes along that breaks those assumptions, the resulting complexity increase can be enormous.
It is not possible to design a system that does everything. When designing something, it’s always necessary to limit the scope. In order to build anything, assumptions must be made about how it will be used.
The more assumptions you make about how something will be used, the easier it is to build. You have fewer edge cases to handle. From a mathematical perspective, you can imagine looking for some random program in the space of all possible programs; if you know that a program only has a couple simple requirements, you have a very large selection of possible programs to choose from that satisfy those constraints. A wide variety gives you the freedom to choose which program you want based on other, more programmer-friendly factors, like how pretty and elegant the code is.
On the other hand, if you have a longer list of requirements, it gets increasingly hard to find any program that satisfies them all, and when you can, the few you’re left with might be among the ugliest and slowest programs you’ve ever seen.
HTML was originally just a simple format for text layout. Now it’s used to build complex GUIs. Changing the GUI involves modifying a text file, and then reparsing it. This isn’t a feature that’s necessary to make GUIs work; it’s a hack. A hack to turn a system that was never designed to be used for GUIs in the first place, into something that can, and has since become standard practice.
This introduces a tremendous amount of complexity into any system that touches it. This is a solution that any sane engineer would never consider adding if building such a system from the ground up.
Today, the web is an incredibly complex beast. If we were to do a clean-slate redesign, it would likely involve scrapping most of HTML, CSS and Javascript, and building everything based on WASM and a simple, low-level rendering API. Such a solution would be extremely general-purpose and make very few assumptions about how it will be used. Some built-in WASM libraries may be useful for adding a compatibility layer between things, though that of course adds assumptions on what kinds of features need such a standard compatibility layer.
That’s perhaps the direction that the web is moving in, though the tremendous baggage we have is unlikely to be deprecated anytime soon.
The insane complexity of the tower of duct tape we’re left dealing with is a huge drain on the industry. New Javascript frameworks are built on an extremely frequent basis to keep development tractable. That’s not something you see much of in other parts of the software industry, and that’s not just because the web is more popular.
The closest that we really see is probably the use of game engines in the gaming industry, but the complexity there is due to the fact that game development is actually just really hard. When playing an online game, game developers may build very complex systems to try to hide the high latency between player connections. Most websites, even those built by gigantic tech companies like Google and Facebook, often handle latency by just running really slow so that the network latency doesn’t seem out of place. This isn’t necessarily because web developers are lazy or bad engineers, but rather that the web gives them an excessive amount of complexity to deal with, and making huge compromises elsewhere is the only way to keep development feasible.
So what’s the solution to all of this?
Keep things simple.
There’s great virtue in being humble. If you’re building something, keep your ambitions reasonable. Changing your mind on what the goals are can be okay, but it’s important to ask if the fundamental assumptions have changed. If they have, big changes are going to be necessary to make things work. If you’re constrained by a userbase that requires backwards-compatibility, you should not underestimate the insane complexity burden you’ll be putting not only on yourself, but all the developers who work with the things you build.
At some point, it’s not worth trying to greatly expand the scope. If you build a standard for one thing, and people want to use it for something it was never designed for, it’s probably a better idea to just build a new standard just for them than to try to maintain some Frankenstein’s-monster-software-project that tries to do everything.