The Analysis Paralysis anti-pattern
Let’s say you’re the CTO of a healthcare company. You are in charge of digital transformation, and you have asked Cathy, your solution architect, to create a system architecture to automate the entire business process. She has enthusiastically started work and occasionally shares new architecture designs with you. They all look great and very detailed.
After two months of progress, you decide to sit in on the weekly meeting with the architect, the lead dev, and the team. Cathy is at the whiteboard pointing out details in an elaborate UML class diagram containing over a hundred individual classes. You notice the version number, which reads ‘0.68’.
“And these classes here represent our query engine and Object-Relational Mapper, which loads business data into these Data Transfer Objects here, that reside in the data layer. With this Command Pattern, we can run arbitrary business queries on our data.”
Bob, the lead developer, interrupts her.
“Sorry Cathy, but I think I see a bug. What if our clients want to group project expenses by accounting category, and then sort the list by revenue earned per region? The command pattern can’t handle that.”
Cathy looks at her design and is quiet for a moment.
“Darn, you’re right. Okay, I know how to fix that. I’ll make some changes, and we can start generating the code as soon as I’m done with this class design.”
And with that, the meeting breaks up. You’re a bit embarrassed that some of the technical jargon went over your head. But all the optics look good: Bob and Cathy have a great chemistry together, Cathy ran the meeting efficiently, the team is enthusiastic and participating, and the architecture looks very thorough. You decide that everything is under control.
I’m going to make three predictions about the project:
- You will not hit the delivery deadline
- Progress will be much slower than you’d expect based on team size
- The execution will be erratic, with the project stopping and restarting many times
Welcome to the Analysis Paralysis anti-pattern.
What are the symptoms of Analysis Paralysis?
When a patient walks into a hospital with a fever, aching joints, and a red skin rash, a good doctor will immediately recognize these symptoms and realize that a chickenpox infection is likely.
Being a CTO is no different. Technical organizations can also get sick from afflictions like anti-patterns. Each antipattern comes with its own unique list of symptoms. A good CTO will recognize these symptoms, quickly identify the anti-pattern, and start working on a cure.
Here are all the symptoms from the tech meeting that point to Analysis Paralysis:
First of all, Cathy was working on a UML Class Diagram. This is an unusual choice because it’s an extremely detailed diagram that leaves developers no room for any creativity or initiative. It can make sense to use these diagrams in small parts of the architecture, but Cathy is using a class diagram to describe everything.
And remember the version number of the architecture. It was ‘0.68’, suggesting that Cathy has revised her architecture 68 times without reaching ‘1.0’ yet. She also mentions generating code, which means she intends to automatically generate code from her class diagram. As a consequence, the dev team has to wait until she’s finished before they can do anything.
The architecture contains a business query engine. But instead of representing it as a black box, Cathy’s design apparently shows the internal workings of the engine, because there is enough detail visible for Bob to infer a corner case that the query engine can’t handle.
With this level of detail, the architecture no longer makes sense to the domain experts. This is a problem because a rule of thumb of good design is that an architecture should always be understandable by the domain experts. Remember how Cathy pointed out the ‘Command’ pattern in her architecture? This is not relevant to the domain experts and suggests that the design has gotten way too technical.
And finally, notice that Bob can force an architecture redesign with a hypothetical query. There is no business representative in the meeting, so the team actually has no evidence that anyone would ever use this query in the product. The team is designing an architecture based on hypothetical information.
All of these symptoms point to a team that is stuck in the analysis phase of the project. They are trying to create the perfect design, and in the absence of actual business feedback the team is making up their own hypothetical business scenarios. The architecture is never finished, and the dev team will not start until the design is perfect.
The four root causes of Analysis Paralysis
Like all antipatterns, you can’t just simply walk up to your team and demand they work differently from now on.behaviorviour of the team arises from the company culture, and so the first thing you need to do is identify how the culture is affecting the team, if you want to encourage any kind of lasting change.
So let’s take a closer look at the root causes of Analysis Paralysis. Why would anyone get stuck in the analysis phase?
Analysis Paralysis often happens in companies that adopt the ‘Waterfall’ model. In this model, everything is planned upfront. The team only starts writing actual code when the highly-detailed technical design document is completed. Everybody assumes there will be no deviations from the design during coding, and the final product will exactly match the design specifications.
Waterfall can be used successfully, but only if all requirements and the scope are fixed, the product itself is firm and stable, and the technology is completely understood. This rarely happens in technical projects. If any uncertainties arise in either requirements, scope, functionality, or technology, the development will need to be halted, the design updated, and new code generated from the updated design. This is what causes the stop-restart cycle I predicted earlier.
Another cause is weak role boundaries. Cathy could have simply created a high-level UML component diagram, with all major architecture components represented as black boxes, and have Bob work out the inner details of each component. Instead, she is micromanaging both the high-level architecture and the low-level code implementation This is counterproductive because Bob, as lead developer, should be creating the low-level implementation. With Cathy doing all the work, Bob has basically nothing to do.
Why is Cathy doing this? Maybe she doesn’t trust that Bob will do a good job. Or she has only recently been promoted into the architect role and is trying to continue to ‘program’, using designs instead of code. Or she is simply bad at delegating. Whatever the reason, she is doing Bob’s job and slowing everyone down.
By creating overly complex designs, Cathy is demonstrating a poor grasp of the company’s vision and goals. In the meeting Bob talked her into making the design even more complex, based entirely on a hypothetical situation. If she had a solid grasp of the company’s vision and goals, she would know exactly which use cases make sense and which do not. She would also know at which point the architecture design is complete – when it clearly implements these vision and goals, and not neccesarily when it describes their low-level implementation in full detail
And the final root cause is fear.
Waiting for a finished design before starting work is typical behavior in a culture of fear. You can never be blamed if you stick to a design that someone else created.
Of course, for this to work you’re going to need the design to cover every eventuality. There is absolutely no room for experimentation of any kind because that will re-expose you to potential blame. So if anything unexpected happens, the project will need to be halted while the architect revises the design
Does your organization have bullying executives who blame, demean, and belittle others? This is a fertile ground for the Analysis Paralysis antipattern. Before you know it, everybody will be hiding behind a design that someone else created.
How to fix Analysis Paralysis
It’s not hard to fix Analysis Paralysis, but it requires some bravery from your organization to step out of its comfort zone.
The first thing you need to do is tackle the culture of fear. Executives who rant, criticize, and blame will create a toxic and fearful work environment, and you should have a zero-tolerance policy for this kind of behavior.
A culture free of blame is a very interesting work environment. In the absence of blame, everybody is free to try out experiments, move outside of their comfort zone, and be secure in the knowledge that mistakes will be treated as learning opportunities. This will stop people from hiding behind a design.
The second thing you need to focus on is the project methodology. The ‘Waterfall’ method has been thoroughly discredited in IT, It has been superceded by agile methodologies like Scrum or Kanban.
In an agile methodology, the team works autonomously on small subsets of functionality called ‘sprints’ and there are specialized roles for controlling scope and making sure the team can work unimpeded. Agile is very efficient and it can easily handle unexpected changes in scope, requirements, functionality, or technology. These are the very things that Waterfall assumes will remain constant during the project.
Analysis Paralysis cannot happen in an agile project, because the architecture is created organically during the implementation phase of the project and the dev team handles all low-level design at the start of each sprint.
Another thing you can do is make sure there is a clear and open communication channel between the architect and the customer. The architect should basically have the customer on speed-dial. Even better, embed the customer within the team. Have him or her sit in on design meetings and offer context on functionality. An architect in daily contact with the customer will know exactly what is expected, and will not go overboard creating designs that try to handle every hypothetical situation.
And finally, you should plan for outcome. Instead of creating complex designs that try to predict what the product will look like in advance, take a step back and try to find out what the customer really wants. Is your product their goal, or are they trying to achieve a specific goal with the product?
Instead of spending all of your time creating the perfect product design in advance, how about sitting down with your customers and co-create a new design that helps them achieve their goals instead? You may be surprised where that path will lead you.
Would you like to know more? I’ve created a series of blog posts that look at each anti-pattern in detail. Each post is loosely inspired by actual events that happened during my 20-year IT career.
Take a look: