Sunday, February 17, 2013

Using Whys to Understand Good Behavior

When a problem occurs, we usually know what happened and how it happened, but it can be difficult to know why it happened.

Lets say a bug is found in your system and you are able to reproduce it (what), isolate and study the code containing the badness (how) and produce a patch for it. Bug fixed, problem solved, right? Not really, because we don't know why it happened, so we didn't really learned anything from this. The point is to do it better next time and avoid the problem in the future.

This is where root cause analysis, RCA, can help. In short, with RCA you work your way backwards from the problem statement and ask "why" enough times until you end up with something that seems to be the real cause. In this case, it might be that the database abstraction they've been using is too complex and they should use a more straight forward approach.

This might result in a discussion about values like simplicity and concepts like meaningful abstractions, which in turn can be a new point on the definition of done list: "Is the code simple enough?"

Turning RCA around


I was thinking about using RCA not to solve problems, but to analyse good behavior and find reasons for that behavior. This could be used as a way for the team to "self-learn to self-organize" without having to force feed them theories about team dynamics. My idea goes like this:

First brainstorm quickly with your team around "What characterizes a great team?". You might end up with stuff like regular deliveries, stable velocity, shared goals, conflict management etc. Don't go too deep here. Stop when you have a handfull of phrases like that up on the whiteboard. This sets the context for the next step

Now run a RCA starting with the statement "This team produces great results" and ask why? several times!

This may go completely out the window if the team is not confident or mature enough, but I'd like to think that something like the following should be the result of this experiment:

"This team produces great results!"

  1. Why? Everyone focuses solely on team success and are able to put their own personal agendas aside
  2. Why? They call out each others bad behaviors when someone breaks the rules and decisions
  3. Why? Everyone commits to and understands the goals and are confident in the teams decisions
  4. Why? All opinions and ideas have been vented and argued so no-one feels left out
  5. Why? They are mining for potential conflicts and are not afraid to passionately question ideas even though it might be uncomfortable
  6. Why? They use unfiltered, honest dialogue to reach the truth rather than as a way to win arguments or push ideas for political or personal reasons
  7. Why? Team members trust each other knowing they can be open with their fears, mistakes and shortcomings without having to risk any kind of retaliation

The major breakthrough in this exercise is when (or if) the team reaches the root cause "trust" and it dawns on the them that trust is, without doubt, the most important characteristic of a great team and the first solid stone to success.

Sunday, July 8, 2012

Overtime is Waste of Time


Working overtime produces more results faster, right?

I'll just lay it down for you:

Developers working overtime does not produce better software. The occasional crunch day is fine, but as soon as overtime turns into the accepted norm, things will go downhill: bugs, crappy design, complexity, no testing etc. And that's only mentioning the technical problems.

A lesson for managers and developers (myself included)

Developing software is problem solving. Solving problems requires a focused and creative mind. A tired brain loses it's ability to concentrate. It's like playing tennis with a broken racket: pointless. Quit, go home and come back fresh in the morning.

Having said that, I know that being stuck on a tricky problem and suddenly having to quit is extremely annoying, especially when you think the solution is just around the corner. My natural instinct is to stay put, work on it some more, "try a few more things" and then leave. It never works. You're not going to solve it that way. Just bite the bullet and go home, unwind from the problem and jump on it in the morning. My experience is that you will solve it almost instantly once you're back the next day. Or (as it often happens) your mind suddenly realizes there is a solid way to sidestep the issue completely.

A true story

Let me tell you about one place where overtime was the accepted norm. Even though the devs worked overtime, nothing of value was finished. Instead everything just took sooo muuuch looonger, like they had stretched the normal eight hours so they would last the entire day. Obviously, their brains were overworked, tired and completely void of creativity. As you'd suspect, it was a slumbering mess of zombie activity.

How did that happen? For starters, those who dared to leave the office at 17 got the evil eye from the boss and a snide remark. Leaving on time got you branded more or less like a traitor. The last person leaving the office in the evening was, without exception, the dev boss. I'm certain that was just to make a point of it. I remember one occasion he was wearing a t-shirt with "22 is the new 17" printed on the front. I kid you not.

Accepting overtime, on the other hand, was rewarded with fist bumps, ata boys and other signs of approval. Simply put, you had to work overtime to get accepted by the chain of command. So naturally, the programmers did that. Eventually, it fell apart. Development practically stopped due to bugs and sloppiness. People got fed up and quit. In the end I believe only the boss stayed and I bet he's still there, hacking away in the night...



Sunday, October 23, 2011

Team Entropy


Code that is left alone will eventually start to break down. This is called code entropy  and is like this evil force of nature which can only be kept at bay by constant code refactoring. 
Each time you add code, entropy increases making the code a little bit more complex and a little bit harder to maintain.  In short, technical debt is slowly increased little by litte.

But what about team entropy?

Like code decay, team entropy is an invisible force constantly working on you to let things slide, beckoning you to return to the way things were before. At the core, this is one of our most basic instincts: avoid work by taking the easy path, not necessarily the smartest path. 
"I should go to the gym, but...". 
"We really need to do something about this, but ..."
All teams are influenced by it and many succumb to it. When energy is high, this is not a big deal, but eventually energy is lost. After numerous sprints and releases, things starts to get cosy and routine. Complacency sets in. The result is that continuous improvement suffers and the team eventually falls back to a pre-agile state. But the funny thing is that the team still believes they are agile, even though they are only going through the motions.

Here are some indicators that a team is affected by entropy or complacency.

  • Retrospectives becomes a comfortable routine
  • Longer iterations are introduced
  • Re-planning during iterations are accepted
  • Stand-up meetings gets longer and does not create energy for the team
  • Less pair programming and/or pairs rarely switches
  • The whole team is getting more quiet
  • Agreed-upon practices are skipped in silent agreement
  • Workarounds are accepted instead of fearless problem solving
  • Bugs are starting to appear again
Team entropy is hard to fight or even notice once it has started to set in. They way to fight it to be vigilant (for lack of a better word). You need to be on your toes. Constantly challenge yourself and your team to improve and question the way you work.

How to fight it?

Retrospectives! Treat them seriously and never accept routine when it comes to retrospectives!
Change the way you do them, make them shorter or longer, try a new agenda, hold them in a new place, use different tools, try another time. In short, surprise your team!
For inspirations there are a few books which can help you with this. I recommend that you add Agile Retrospectives and Collaboration Explained to your reading list.

Monday, September 12, 2011

Don't Protect Your Team!

I was involved in a project which had not gone as smooth as we'd hoped. By the end of the project we had a chat about tying up the pieces, so I asked the project manager how she felt we could do better next time. I explained that looking back at the project to see what we can do better next time is a way to avoid repeating the same mistakes again.

One problem with this project was that the team had to work extra to correct and implement things too late in the project time frame. The reason was poor planning, which in turn was a natural consequence of the team not having enough information available. It's worth noting that there had been enough time for them to do everything, but they thought they were finished and had started to work on other things.

Why did that happen? Well, it turned out the developers had no contact with the customer and was left in a vacuum once the initial sale was completed. Promises where made and then shifted over to the team to "make it happen". The customer had absolutely no idea who was on the team, and the team had no idea who the customer was (apart from the company name). Not a single time during the project did any of the team members talk to the customer.

I asked why the developers were kept out out the loop like that. The response I got from the project manager was that "shielding the team from the customer is a way to save them from having the customer calling all the time and disrupting their work". In other words, don't give them your phone number.

It turned out the project manager had been in a situation once where one customer had been a real pain, constantly disturbing the team. This can happen in a maintenance/support situation, but it was not an issue in this case. I said that I thought the developers were professional enough to deal with such issues if they popped up and that is was no big deal.

I honestly don't believe this was the real cause for blocking the teams communication. I'd rather think the project manager didn't want to give up control. Loosing control can be frightening, and the only way to deal with that is to have a bit of confidence in the developers ability to do their job. Or it may  be that the project manager was so used to doing things that way that it didn't even occur to her that letting go of the reins might be a valid approach.

In the end, the project manager dismissed this idea and said her approach was preferred: is is better to shield them from their customer and instead channel all communication through her. (Side note: the project manager had been absent for most of the project, increasing communication problems even further. From that point of view she did trust the team... to fend for themselves.)

Here is what I think. Allowing the team to interact with their users during development makes it much more likely that problems are dealt with and helps make adjustments quickly and timely. It also increases trust between the two parts which allows for more ideas and better creativity, not to mention improving chances of doing business with the customer again.

Another important factor is that the team, through direct communication, feels more responsible and committed to the tasks: "I'm doing this because I want to make a great job delivering as I've promised." as opposed to "The project manager wants me to do this, because she says so." What do you think is more likely to produce great results?

Monday, August 1, 2011

Teams, Architects and Decisions

I know lots of great programmers. Some calls themselves architects. This text is not, in any way, intended to lessen what they do. Also, I want to make it clear that I don't support cowboy coding or poor design in any way. Agile, done well, holds quality and correctness in high regard. So do I.

Let's talk about Software Architects. The title itself doesn't really mean anything, it's just a label and everyone has their own definition of it. Also it seems everyone and their uncle is calling themselves architects these days. I suppose that is natural, because the title has a nice ring to it and it tends to come with benefits such as better pay and authority.

Unfortunately, many architects are just average programmers who never should have been given that particular rod of power in the first place. This has far reaching consequences, not only for poor developers having to work with the mess, but the entire organization and its customers. It's also worth mentioning that architecture, as a term, associates development with construction work and that has nothing to do with creating software.

All this dilutes whatever meaning the title once had. I kinda think that is good, because in the end we are just programmers with different amounts of experiences and skills. Some more than others. So should a highly skilled programmer be called an architect? I don't know, and it's not very important. What is important, on the other hand, is making sure your teams have enough competent programmers who understands how to structure the system in such a way that it ultimately makes your customers happy. I don't know if that is good architecture or not, nor do I care. What I do know is that we are in the business of deliver working software, not architectures.

Creating software requires all manners of technical skills: analysis, test, design etc. That's what all developers have to deal with on a day to day basis. From the top of my head, I cannot think of a single thing an architect should do that an experienced programmer should not (I leave customer/requirements out of this, those tasks are handled by the Product Owner).

What is your definition of an architect as compared to a developer? How can you single out the architecture from the system?

My definition

Not surprisingly, I think of a software architect as a great developer. A great programmer should be humble, patient, eager to explain and use her experience to coach and mentor her team mates and not wield her knowledge as some kind of power. Having a least one experienced programmer sharing her knowledge with the team is very important, so I think the definition of an architect as an experienced programmer with coaching skills is spot on.

I recommend Martin Fowlers take on this topic. In short, he says that architecture is "the shared understanding of the system" which makes the architecture a social construct. Further, the role of the architect is "to mentor the development team, to raise their level so that they can take on more complex issues.".

I agree with him. It makes the architect a servant leader and not a decision maker. Again, a great developer.

Teams and leaders

The reason for having a team in the first place is that it should be able to perform better than just the sum of its parts. That is obvious, but you cannot simply throw some people in a room and expect them to work as an effective, coherent team from day one. All teams have to go through several stages to start performing better than its parts. Unfortunately, you cannot skip any of these stages.

Getting a team to the performing stage takes effort and courage from the organization. Not surprisingly, this is where it fails. The team does something wrong and/or conflicts arises so the organization feels the need to "step in" and do something about it. Take charge. Usually this is handled by putting a powerful leader into place to make sure these things does not happen. This person is called their team leader, project manager, architect or something like that. She is put there to control the team and make sure it doesn't fail.

Putting a leader with decisive authority in the team is the classic way of dealing with problems. What happens is that the developers becomes dependent on a single person for direction and decisions, so they never really learn to trust their own skills. In the end, this effectively blocks the team from reaching the performing stage and never learns to perform effectively. They end up being mediocre at best. How ironic is that?

A better approach is having a coach on the team. A coach is a type of leader without decisive power. Her primary goal is not to control, but rather to support the team being their "servant leader", helping them grow their skills and teach them to use their abilities. A coach guides the team through the decision making process, helping them come up with new ideas and thinking in different directions.

Creating a powerful team costs a bit in the beginning, but you make up for it in the end. It's like technical debt, but for teams.

Who should decide then?

I like the Scrum definition of a team:
"There are no titles for Development Team members other than Developer. Regardless of the work being performed by the person, there are no exceptions to this rule". 
A team should consist of cross-functional developers with equal authority, able to deliver the next product increment (it's the same in XP as well).

There is always someone with more experience, but that does not mean they should make the decisions. A team working to reach a convergent decision does a much better job with that compared to a single person on his own. So there is no need for any single person (architect or team leader) with decisive powers. In the end, decisions are made and owned by the team.

Summary
  1. Self-organization and collaboration is the best way to create software
  2. This requires every developer on the team to be an equal when it comes to decision making
  3. A person with decisive technical power is contradictory to that

A few questions with answers

How can you treat everybody 'equal' if one has just one year experience and one has 20 years experience?

The team should be able to handle this. It means they should self-organize to create their own decision making structure. The team will eventually learn to know each other and start trusting each individuals area of expertise. Naturally, the input from a senior will carry a bit more weight in the eyes of the other team members, and that is as it should be. It does not mean he can override their decisions though.

What is the problem if somebody is in charge of some specific topic and lead the team?

That is not a problem as long as the team is making the decisions, not that single person. Obviously the teams database admin will have much more to say about databases than the other guys, but he are not allowed to do as he pleases.

We are looking to hire an architect. He should have these skills: testing, design, deployment, support, business knowledge, clean code, gui, networking, databases, legacy…

That does not look like the skill set of a single individual, but rather the skill set of a cross-functional team of developers. When complexity increases you don't go look for one single person to deal with it, you start looking for teams of people collaborating better to solve the problems.

Sunday, May 29, 2011

Software Development is Not Construction

When talking about software development, it is customary to compare it with construction. We put the case like this:

"First you must decide what to build. Talk with the client and gather their requirements. After that, create an architecture and a design which acts as the blueprint or specification of the system to be. For this phase, you have an architect. Then you start constructing the system. The programmers then build it. Once they are done, the system is complete".

This construction metaphor sounds great, but there is a huge problem: It is wrong, completely wrong. It gives the impression that software should be created like we build houses. Nothing could be further from the truth. In fact, I see nothing in the above comparison that resembles programming. I have never seen houses created with a text editor, nor have I seen software created by digging holes in the ground and filling them with concrete and iron bars. They are simply two different things, with nothing in common. It makes no sense whatsoever to keep using this comparison.

The construction metaphor is the main cause for lots of problems we have in the software industry today. It has permeated how we think and how we work. Clients believe it, organisations believe it and many programmers believe it. It is the reason why consultants are allowed to waste many years on a client project before it fails. Even the words we use for writing software echos this sentiment -- architecture, specification, construction, building, phase, completion. A carpenter builds, a programmer writes. We are not in the business of building software, we write software. 

A Problem with Houses

When construction begins, requirements are set in stone. In construction, clients are forced to make expensive decisions early on when they have little knowledge what they really want. Only when the building is completed and you have lived in it for real can you appreciate (or not) your early design decisions. This is the reality of construction; you are constrained by the laws of physics, regulations, material etc and there is not much to do about it. Or at least it is very costly to fix. The upside with houses is that they are generally easy to understand and decisions are mostly correct.

The difference between houses and software is that software is, well, soft-ware. Software is free from physical constraints, which means we can create software in whatever form we like. The options are endless and that makes it much harder than deciding than how many rooms a house should have. 

With software, we have a unique opportunity to write just a small part of the system, evaluate it and adjust the next step based on that. That is just not possible in construction.

So why do we insist on intentionally constraining software development to the same limitations that apply to construction? It makes no sense to me. After all, programs are plain text. Change it when it turns out to be wrong. Evolve it. There is no physical concrete-and-iron structure holding us back.

In short, software allows us to delay expensive decisions until we have as much information as possible to make them as good as possible. That amazing thing is what we are giving up when subscribing to the construction metaphor. The loss is obvious, but what do we gain from that and what do our clients gain from that? Nothing, except it fits our mental model of how physical things are constructed.

Software Evolves

Software is never finished. It evolves (or dies). Pick any program you use today and consider how it has evolved over time. Your Windows system, your browser or your favorite IDE. They are hardly the same today as they were when they first released. The code, design and everything else has changed over time. Software is never completed, releases are simply arbitrary points in time when someone decided that now is a good time to make it available.

This is even more obvious with agile software which are released weekly or even daily. Why make such a big fuss over the first release as compared to the second or the 23rd? Why spend so much time with architecture and architects when the only thing we can know for certain is that it will change or, more probably, be replaced when we learn more about how to use it? Especially when software can easily be changed and houses cannot.

Sunday, May 22, 2011

What Makes a Project Successful?

The other day I had the opportunity of introducing agile to a client. I usually do that by walking through how Scrum works; backlog, sprints, sprint planning and spending time talking about organizations and how they deal with change. 

That approach usually works well, but this time I decided to use another approach. I simply posed this question to the team:

What makes a software project successful?

After a short, but great discussion they came up with the following answer:
  • Specification - So we know what to build.
  • Estimation and milestones - How long will it take.
  • Project manager - Keeps project on target.
  • Resource utilization - So people have things to do all the time.
  • Testing - So we know it works.
Quite possible it was the first time they had ever talked about it, so I'd say this was a new experience for them. 

To make it a bit more interesting, I had prepared a list of my own what I think is most important in a project:

Sunday, April 17, 2011

Keeping Management at Bay

We estimated the effort to 300 development hours, but when discussed with management, they think we have grossly overestimated. How can we keep them out of our development process?
I'm sure you, as a programmer, are familiar with the sting of anger felt when someone "corrects" your estimates. What's the feeling? Anger, frustration, humiliation? Probably a little bit of all. It comes from the fact that they are not trusting your work as a professional.

It's important to realize they aren't doing this because of spite. They simply don't understand development and the complexity of it. Or for that matter, it may be that they have bad experiences with some programmers or have been taught to distrust estimates in general. Either way, the result is a nasty conflict.

The solution is not to keep them out. Input from stakeholders is crucial for a successful project! The problem is they are overstepping boundaries and the way to handle it is to agree on a contract consisting of two simple rules:

§ 1 Developers should have the right to estimate their own work.
§ 2 Stakeholders should have the right to prioritize among that work.

Estimation and prioritization are two forces that work very well together, once both parties accepts their responsibilities and rights. So instead of wasting time arguing, just agree upon this, respect that both parties will do their work and let the forces of collaboration do its thing.

Monday, January 31, 2011

Stacking or Slicing?

How do you design a new system? What order do you develop things?

One common approach is using separate layers to define what the system should do. Let's try it!

A user interface layer at the top. Then the server side with a business model layer coordinated by a controller layer. At the bottom is the data access layer and the actual database itself. This "separation of concerns" makes it possible for each technical expert to work on their specific layer.

Next, we continue by fleshing out each layer. For each layer, a solution is designed according to customers demands. This means we plan for all known requirements and maybe a few extra, because, as we all know, the customer may change her mind in the future.

Great! Next up is building it. With the design completed, experts run off to implement their specific layer. They know their TDD and write automated tests all the time to ensure each layer works as expected.

Once all are completed, we just have to put them together like a stack of pancakes and deliver the whole system. Sirup's optional!

Can you spot any problems with this approach?


Monday, January 10, 2011

The Power of Low-Tech

This industry is obsessed with tools. Planning or tracking, the first impulse for any business is to reach for a software tool to handle their new agile project. While I can understand their reasoning, I think it's wrong to believe that a software tool is going to help achieve any agile goals.

I sometimes get asked to assess the software tools market and make recommendations on an agile tools mix for the best way forward. This implies that software tools are required for successful agile transformation. That is not true. 

If your team is using Scrum, Kanban or XP correctly ("by the book"), you don't need to use any software tools. It's rather the opposite: Start out as simple as possible using nothing but pen, paper and some whiteboards. 

This may sound a bit odd to some of you, so let me explain the reasoning below.