The big rewrite

I see it all the time and it goes like this:

Our system sucks. Let's rewrite it! Let's do it right this time!

With a fresh team, new technology and no legacy code, the team is left alone to hack away at the brand new version of the system, full of energy, enthusiasm and commitment. The organization gives their blessing with a perfect environment, great tools, great setting and complete trust.

As the story goes, each programmer now happily start coding their piece of the system. Relaxed, in the zone, that perfect place where code just seems to flows from the keyboard into the system. Perfect code -- your code. Earphones blazing away your favorite music, shielding you from any disturbance. Life is good.

This goes on for several months, maybe a year, but eventually the system is released. With a huff and a puff, the thing eventually is up and running! And it works! Rejoice, let's make some money! After all, that is why we say that software is valuable for our organization and that is the only reason we are getting paid to produce it. Working software is an asset. Management is happy, customers are happy and the team feels good. A job well done. Now they can relax.

During this honey moon, a few (well, quite a lot actually) bugs are found and fixed. Several important changes and new features are implemented. Things starts to heat up and it seems like this flow of new changes never ends. It is a steady stream of new features, mixed up with bugs. The team is running just a head of the tide, constantly adding new stuff to the system.

A bad smell

After a while, development seems to drop somewhat. The stream of changes is slowly catching up to the team. What used to take only a couple of hours to fix now takes several days, and for some reason a lot of time is spent fixing bugs which seems to pop up all the time. The team squash a bug, only to have it pop up in another, unrelated, part of the system. Even worse, the fix itself introduces another issue which must be fixed. Testing also takes longer and longer, since everything is done manually and every change results in a complete re-testing of the whole system. The testing guys are slowly drowning in tests, sometimes allowing things to slip just to make the next release on time. Meanwhile, the code grows more and more, attracting more bugs, more complexity and more todos.

Huh, what is going on here?

Baffled, management does not understand why things is going so slow all of a sudden. The same stuff the team did just a a few months ago now requires a huge effort to implement. Problem is, the world is still turning, competition is fierce, customers demanding and those features must be released now!

It is obvious that the team cannot handle this on their own anymore, so what is a manager to do? Step in and help the team of course! First they must know what is going on, so they add more meetings and daily status updates. "Where are we and much time left do we have?" "More detailed specifications!" "Why is this taking so long?" Lets apply some healthy pressure by assigning selected parts of the system to specific members of the team. That should make it clear who is responsible if anything goes wrong! Overtime is encouraged and rewarded. The message is crystal clear: failures are not accepted and no missteps allowed.

Morale starts dropping, overtime is the norm and there are signs of early burn outs. People are on their edge all the time, and some have problems controlling their frustrations. Team members blame each other and the lead programmer is often seen working late, fixing other team members code. No-one is willing to take any risks and creativity is stifled. Some have learned that to avoid the heat it is best to not do anything at all while still manage to look busy.

After a while, some team members quit and eventually the manager gets removed (or promoted) and new, fresh and eager people arrives to replace the old guys. After some time, it is clear to them what must be done about the system and they convince the organization:

Damn, this system sucks! Let's rebuild it!

And so on...

What happened?
It was clearly not an overnight thing. Instead, slowly, like a creeping bad feeling which got stronger and stronger and eventually the whole thing just fell apart. I am sure this happens all over in all types of organizations, and I am sure you have seen it yourself. Maybe not as extreme as this, but bad enough.

In reality, the whole project was pretty much doomed from the beginning. All the signs were there, but no one was there to watch for them. I can not really blame management for what happened, it is not their job to be masters of systems development after all. The fault here is the developers. The organization put their trust in the programmers and they failed. I see this all the time: individual programmers working on their own, creating code that may work for the moment, but over time degrades into an unmanageable big ball of mud.

Then the blaming starts: it is either the customer or the organization (or both) which forced the code into this situation. Well, I don't believe that. Not for a second. It is our responsibility, our duty, to act as professionals and to do everything in our power to not let this happen. Yet it does. All the time.

Programmers should be working together
The first step is to drop the age old pretense that hacking is an art done in isolation, in a hacking comfort zone. That mentality has to go. Next, we must start working together, not just sitting in the same room, but truly working together. Think of this as being part of a band if you want to. Everyone needs to be perfectly in sync with each other for the music to work. It is the same with developing software.

Creating truly great, sustainable software of todays standards requires that we work as professionals and that means working in teams. In my world that boils down to embracing agile values, and to teach our peers and managers how and why. It is not enough that we should blindly follow these values and practices. It is a good start, but the goal is to eventually create something new and unique with them. To bend the rules, to adapt and to make them our own.

In following posts, I will try to explain these values and how they can change the way we create valuable, working software.