About Software Complexity...

Complexity is the worst enemy of a software developer. It makes organizations slow, businesses unscalable, and developers unhappy.

When starting a project from scratch, it feels clean, it's easy to reason about, and easy to extend with new features. That's a good feeling; I bet you've been in that situation and have thought:

"If only the big old boring project I need to maintain at work was like that..."

And wasn't it like that when it started? It's very likely that it was, then what happened? What made the project a big messy pile of code? Is my new clean and shiny project also destined to be like that if it grows?

Let's analyze the main sources of complexity in software:

Lack of skill/motivation:

Sometimes, developers just want to get paid. They want to solve the problem right in front of them as quickly as possible and collect a paycheck at the end of the month. That's fine, but that mentality naturally pushes developers to ship lazy and sub-optimal solutions as long as they solve the immediate problem. This adds up over time, stacking patches together making a code base exponentially harder to maintain as it grows.

RDD (Resume Driven Development):

Other developers just want to try out the new stuff. They are too passionate not to try out every new piece of technology that comes out, or maybe they just want a better resume in order to switch jobs. Simple problems are simple, but RDD is about making them complex by designing a system for 100x the expected scale, introducing every existing OOP pattern just for the sake of trying it out, or rewriting chunks of an app in the new framework just because it's new.

Evolving Business:

Here things get tricky. Businesses evolve, and the code in charge of providing features A, B, and C now needs to provide feature D, but it looks like feature D does not quite match with some assumptions made when shipping features A, B, and C. If there's time, a refactor for cleanly holding feature D can be done, but maybe there's not enough time, or maybe D is crazy different. This source of complexity tends to be unavoidable, but it can be greatly mitigated with good management, business vision, and reasonable deadlines.

The problem is just complex:

Maybe you are building the compiler for the next C++ replacement, or maybe you need to serve millions of requests per second with single-digit milliseconds latency. Some problems are just complex. This might be the only pleasurable source of complexity to have.

Visualizing a code base entropy

Imagine a complex code base that you've worked on. Try to imagine it as a whole, with all the files, all the folders, all the modules, all the packages...

What do you see?

I welcome you to

3D force-directed graph

A 3D force-directed graph is a way of representing a graph in a three-dimensional space where nodes are placed based on some attraction/repulsion forces:

• If a node is connected to another one by an edge, an attraction force will pull them together.


• If two nodes are not connected, a repulsion force will try to take them apart.

In

Otherwise, code bases that are messier, with no clear separation of modules, will tend to have all the nodes grouped together with no clear sense of clustering: