A Quarter Century of Web Coding
Tracing the Fractal of Complexity
Early 2023
The year 2023 marks two special occasions for me: I have now been using computers for 35 years and worked professionally with programming for 25 of them. A quarter century - the mind boggles.
What follows is a retrospect of the evolution of programming for the web, and some attempt to come to terms with why it's turned into the obscenity it is today.
The Pimply-faced Intern meets VRML
It started innocently enough. Immediately after my first encounter with the Internet in 1995, I wanted to work with "web design". Back then that was synonymous with writing HTML, doing a bit of routine sysadmin tasks, cropping and editing scanned photos and whipping up the odd page logo in a suitable paint program. VGA was still very much a thing, so web safe colors were important, and you couldn't even take for granted that all browsers supported tables. It was a strange and wonderful time with a wide range of end user hardware, meaning the net was still mostly about text (Usenet, IRC, mail, FTP, Gopher) while forward-looking companies tried pushing things like the 3D "markup" language VRML, used in the Metaverse precursor Worlds Chat.
A couple of years later, I landed my first real tech gig. I was mostly a digital errand boy, performing simpler HTML touch-ups, basic sysadmin tasks and repetitive, manual cleanup of data files. The company, like so many others of that era, was a Perl shop that mostly churned out simple CGI scripts interfacing with flat file data. Netscape 4 was not just the dominant browser, it was the browser, against which all contenders were measured. An internship followed by a summer of employment ended as quickly as it started, but I was hooked. I wanted to work with web production or, I felt, my life would never be complete.
Bobby Tables, Junior Developer
Luckily, finding another web gig wasn't that hard. During the early days of the dotcom bubble you could literally be hired off the street if you so much as breathed about knowing something about computers. Perl had now been replaced with PHP and flat files with MySQL, and the job was truly what you'd call full stack development. All of us did everything, from compiling Apache, PHP and MySQL on freshly installed Slackware boxes, to designing databases, coding PHP and churning out HTML. Deployment was simply a matter of uploading the files from your local machine to the production server.
Testing? Maybe banging the keyboard a bit to see if your forms validated reasonably, but not much more than that. Truth be told, I couldn't program to save my life. I - well, we - committed many grave and terrible sins. SQL injection mitigation? Please, this was the time of the honest web. Who would even try something so sinister?
The daily work was mostly routine churn: HTML forms submitted to a PHP backend, saved in a MySQL database. These were simpler sites for simpler times: a single person could easily write, understand and manage the code, database and daily running of what was at this time considered a state of the art web shop. JavaScript was maybe used for mouseover effects or scattered lines of glue logic - after all, the biggest competitor to Netscape 4 was, at this point, Netscape 3. But there were whispers of CSS - mostly used to create hover effects on links - and one of my coworkers was a dedicated designer, doing nothing but page mock-ups in Photoshop and HTML all day.
Alas, what I've decided to call "the company culture" wasn't conducive to living a sane and prosperous life, so I quit. Finding another job was easy. Venture capital was everywhere and despite my tender age, I now had work experience.
Boom and Bust
Enter the Really Big Dotcom Bubble Company, which was little more than a digital ad agency with a tech shop attached. The place was brimming with creative directors, art directors, illustrators, copywriters and 3D artists. There were parties left and right, free beer and snacks every Friday and of course an overall laissez-faire corporate culture. Naturally, no dotcom venture was complete without a visionary CEO lauded by both politicians, the press and large corporate investors. Ours was, I hasten to add, a friendly and personable one.
Yes - I've worked in advertising. What's worse, I thoroughly enjoyed it: it was a glorious bubble to live in. Stonk could only go up, companies could only grow and the future couldn't be brighter.
The tech stack, as it's called nowadays, was all Microsoft, and I was introduced to enterprise-y stuff like COM, SQL Server, IIS and ASP. The frontend was all about DHTML, a fancy abbreviation for somewhat fancier mouseover effects than had previously been attempted. Internet Explorer slowly started to take over as the default browser, leaving Netscape behind - but a plethora of user agents still had to be accounted for. Most of "creative" had a background in print, which meant that customers were shown and thus expected pixel perfect designs, making life terribly annoying in more ways than one. Go on, try it yourself: Build an exact replica of a Photoshop sketch. Using tables and frames. In Netscape 4. Oh and, yeeaaahhhh, If you could go ahead and make sure it works in Netscape 3, too, that'd be great. Thaaaanks.
To accomplish this, we used a lot of if statements and even more 1x1 pixel transparent GIF:s. And no, nobody cared one jot about accessibility. We did care about bandwidth, though: To ascertain minimum file size, That 1x1 pixel GIF had once been generated by a Perl script. We named it "s.gif" to ensure no unnecessary markup was sent across slow dialup connections.
For bigger site launches, more developers than one or two were involved. Frontend development was mostly about DHTML, CSS and glue logic in ASP, which handed things off to COM objects and databases built and designed by a backend team. Even so, multinational sites for large, multinational companies rarely involved more than a handful of actual developers. But if the code was less complex than today, the tooling and project management was almost nonexistent. Staging servers? Version management? Maybe, on flagship projects for big clients. Time estimates were typically made once, at the start of a project, and then the developers were left to their own devices. Daily status meetings? Code reviews? Some kind of methodical testing? Surely you jest!
Frames not working? Try some Frameworks!
Eventually, the bubble burst and I was laid off. Thankfully, I quickly got a new job - the one I'm still at.
This was during the Cambrian explosion of web development. Suddenly AJAX was everywhere. Frustration with busywork and browser vendors gave birth to frameworks like Ruby on Rails and cross browser libraries like jQuery. Internet Explorer was the biggest browser, everyone hated it and Microsoft refused to update it. I specifically remember visiting the Fronteers conference in 2009, where some poor guy from Redmond spent the better part of his talk apologizing for them having caused so much suffering and the remaining part promising a better future.
And indeed, something significant did happen - even if Microsoft was no longer as big a player as they might have wanted. Web development suddenly gained momentum and started accelerating into the juggernaut of complexity it is today. CSS and ECMAScript were maturing faster than browser vendors cared to work. Browser incompatibilities led to tons of boilerplate code, long-winded hunts for intricate bugs and, in general, massive headaches. Libraries like jQuery and Underscore saved us from the hassle of rolling our own, but the real epiphany was frameworks like Backbone and full on GUI toolkits like ExtJS. Single page apps were expected, asynchronous JSON shuffling the norm and people had started using new buzzwords like "Mobile First".
What do you mean, "arrow functions"?
For a time, it really looked as if we'd caught a break: When Google entered the scene with Chrome, the ecosystem started to converge and some of us almost thought we'd be able to ditch our third party helper libraries and focus on clean sites with standards compliant vanilla JS and CSS. Unfortunately for us, the rest of the world was less eager to upgrade their browsers.
Weary from battling a hydra of legacy Internet Explorer versions while a veritable parade of HTML, CSS and JavaScript upgrades passed us by, web developers at some point instead decided we never wanted to write a single line of boiler plate code ever again. Hence, we loaded our foot guns and downloaded so much open source and third party code we had to use package managers, transpilers and build systems to deliver our sites. Heck, we could even write our backend code in JavaScript now, if we wanted to!
The side effects of Side Effects
I both pity and admire beginner web coders of today. Unlike me, they've not been able to accumulate gradual knowledge of HTTP, HTML, REST, JavaScript, the DOM, CSS, AJAX, JSON, asynchronous execution and event driven object oriented programming over a period of decades. They haven't walked the long path from CGI scripts to modern server side tomfoolery via PHP, ASP and various MVC frameworks. They're just brutally thrust into a complex world of Gulp, Grunt, TypeScript, React Hooks and MobX-State-Tree and it's assumed they somehow already know about all that other stuff. Computer Science has moved into the frontend in earnest and yet it still seems as if many view "web development" as "making homepages", and that it's something you can learn over a period of weeks, not years.
In retrospect, web development has always been a bit of a struggle against the powers that be. Some things that were pretty bad for quite a long time have gotten better. But, on the whole, I dare say it's much worse now than when I started. Much like how Commodore 64 programmers could keep a map of the entire computer in their head, a moderately competent developer could churn out an acceptable web site in a matter of weeks, understand every single aspect of it and get paid in the process. If I, a quarter century ago, had possessed the experience and knowledge I do now, the simplicity of those early web pages would've felt surreal. And yet, we apparently provided a service that was of some value to some people. A digital commodity, nothing more, nothing less. Actually useful software.
Intentionally Accidental
This, then, begs the question of why web development is so complex now. I've discussed this at great length in many other texts, but the gist of it is that up to a certain point, complexity is not only convenient but also extremely profitable. It really is faster to write a Windows GUI application in Visual Basic 6 than in C. More people can become productive faster with Excel and Word than with bc and TEX.
Like a fractal, this abstraction and complexity repeats, from the lines of code in our editors, through our hardware, all through the corporate org chart, out into society and back into our tech stack again. We have abstracted away lots of things from our daily lives and replaced them with whatever complexity seemed convenient at that particular point: Rocks with hydrogen bombs. Outhouses with plumbing. Firewood with nuclear energy. Domestic manufacturing with globalization. Some good, some bad.
For quite some time, this complexity has served us well. It has made life copiously more endurable and rewarded us with great material wealth. Small wonder, then, that we want to squeeze every last drop from it. Sure, your electric toothbrush really shouldn't require firmware upgrades, but there's a buck to be made in all kinds of planned obsolescence. Web sites don't need tens of megabytes of ads and tracking scripts to function, but someone is still paying for those sweet, sweet data points at which to target ads. It's most certainly a strange idea to run a text editor inside a bundled web browser - but that way it's easier to build for more platforms and you can even hire simpletons like me to write much of the code for it. And since more software is produced than ever, more bad software is also produced than ever before.
I guess the web can take another one for the team
Of course, systemic complexity cannot increase indefinitely. There is a breaking point, even if we're never sure when it'll occur or what it'll look like. Since complexity is so complex, its breakdown can be complex as well: A virus from China leads to congestion in the global supply chain, which means there's a shortage of shipping containers, which means truckers can't make deliveries, which means potatoes from the local farmer can't be hauled to your neighborhood grocery store - and even that is a gross oversimplification. Since certain types of complexity also acts like a gatekeeping mechanism, it leads to worse working- and middle class prospects, which leads to a rush for higher education, spurring fierce competition between people with more or less useful degrees, leading to the creation of tech jobs with little or no connection to the actual production of tech.
Regardless of whether the purpose is to enable ever more contrived ways of rent extraction, adhering to increased compliance regulation or satisfying a growing number of externalities in an ever-expanding corporate organization, societal complexity is reflected in code complexity. No matter if you prefer Burnham's Managerial Revolution, Turchin's Elite Overproduction or Graeber's Bullshit Jobs, they're all variations on this same basic theme.
Despite having more experts, administrators and governing bodies than ever, we're currently dealing with severe infrastructure failures, soaring inflation, a growing energy crisis and dwindling faith in foundational institutions such as elections and due process. In the tech biz, recent mass layoffs are echoing the dotcom bubble described above. Those of us still employed are either busy putting software into door bells and stovetops, or even trying to fix fundamental societal problems with code. True to form, our solution to increasing complexity is simply adding more of the same.
Are we at a crucial tipping point? It sure feels like it, but I certainly don't hope so. While I yearn for simpler software and less complex code, a systemic breakdown is rarely a walk in the park. Considering what might otherwise come to pass, I'd much prefer spending 25 more years sitting on my butt in a climate controlled office, together with my coworkers, writing the best code I can muster, and get paid doing so.