product
When reading this Kill Your To-Do List blog post, I thought that managing personal to-do list can be similar to product backlog management. Not in the part that you should totally kill your product backlog, but in the “one thing at a time” part. This is by no means a new thought. E.g. Mary Poppendieck was heard saying that product backlog should be eliminated.
So, as in to-do lists, there’s no point in nurturing and moving product backlog items back and forth. In this case, a lot depends on what is your definition and understanding of product backlog. I’d say backlog is an environment in which a product grows and exists but not a set of components that should be implemented for sure. Looking further into, what kind of environment is it? Backlog might be a repository of all the slightest shades of ideas that have a very vague chance to be implemented. A chaotic heap. This heap can hardly be called a “backlog” but rather a by-product of brainstorming. OR a product backlog might represent a careful selection of user stories that will be implemented for sure.
Both of these options, as often is the case, have the optimal state somewhere in between. In our production workflow, we’re now using several buffers – the first layer is raw Requests and Ideas (coming from our HelpDesk, or from the PO, or from the team). The second layer is Product Backlog with User Stories (Requests and Ideas are now groomed and converted to User Stories by Product Owner who decides that some time these will be implemented for sure – based on how many people requested this feature, based on current product development strategy etc.). The third layer is Planned state for User Stories in Kanban board.
Here’s a quick graphical representation of this product backlog upward funnel:
That’s how User Stories lifecycle looks on Kanban board, from Planned state on:
Backlog is not seen on Kanban board. When done with their current user stories, developers pull new user stories from Planned state.
Previously, when we worked with iterations before switching to Kanban, there was no buffer between inception and implementation – so time-boxed releases and iterations have been planned directly from the backlog. With Kanban, the buffering is done by moving stories to Planned state and prioritizing them.
I’d say that planning with iterations provides less flexibility than planning with Kanban. As you drop user stories to time-boxed iterations, you commit to implementing all of them within a given period of time. Kanban is way more flexible since user stories can be pulled one-by-one from Planned state and implemented with no time restrictions. I can’t resist citing an analogy here: it’s the same as moving with the smallest possible steps to posture yourself before hitting the ball in tennis. With large steps, you do not have flexibility. With small steps – you’re very agile and flexible to position yourself for the perfect shot.
So, the buffered Planned state in Kanban is like this breaking down into small steps, instead of taking one giant leap and committing to the whole pack of user stories in iteration.
That’s the way it goes for us. You’re better off moving by small steps, taking it one-by-one (this brings us back to the inspiration blog post reference in the beginning :) than with giant leaps.
January 13/14 is the Old New Year holiday. Seems like today is the latest appropriate time to look back and recall the most interesting blog posts by TargetProcess in 2009 :)
Based on visitors count, the posts are ranked as follows (descending order):
1. Lean and Kanban Software Development Digest: In May 2009, this digest came along right on time as Kanban adoption started to grow. We’ve been sifting through the Lean/Kanban buzz and considering if Kanban might be a good tool for our development process, so this post has the most valuable findings we’ve made and shared with agile community.
2. Refactoring vs Rewrite: This post is a real train of thought of a Product Owner trying to make a decision on how to proceed with product development — rewrite or refactor. Can well be used in textbooks for software product management :)
3. Mind Maps: Scrum, Extreme Programming, Lean: Another by-product of our research on agile development processes. The specific value of mind maps is that they help grasp complex things with visual representations.
4. Tale: Deadline and Technical Debt: Once upon a time… Who could ever expect that the fundamental principles of product management can be outlined in a fairy tale ? :) There we go: smart Arthur, the cunning king, quest for princess — the metaphorical expression of the danger of technical debt in software development.
5. 5 Wrong Reasons to Apply Kanban. For some reason (no pun intended), 5 wrong reasons ranked higher than 5 right reasons. Maybe it’s just human psychology — to go from “what’s wrong” instead of ”what’s right” …
6. How We Use Kanban Board. The Real Example: Once we figured that Kanban process is just the right thing for us and put it in action, we shared this experience with our blog readers.
7. 5 Right Reasons to Apply Kanban: There they are :)
8. Zero Defects? Are You Kidding Me? : Can this juicy frog be sure that it swallowed the very last bug? This post is a warning against the so-called “zero defects mentality” in software product management.
9. Simple Rules, Complex Systems and Software Development: Complex systems function at their best when guided by simple rules. Look at ants, birds, space rockets and … software development.
10. BDD and User Story Specification: Examples — This post includes some real user story specs in BDD for TargetProcess product. Enjoy and use.
These are the TOP 10 posts in 2009 from TargetProcess agile blog (click here for more)
Happy OLD NEW YEAR! :)
Code base of a large project is getting worse over time. I hope there are lucky exceptions, but in general it is true for most projects. The reasons are quite obvious:
- More and more features. It leads to increased complexity.
- Shortcuts and hacks to support “We need this fancy search till August. Period!” features
- Developers rotation. New developers don’t know all the fundamental decisions and ideas behind the architecture. Knowledge gets lost with transition inevitably.
- Development team growth. More people – less communication. Less communication – bad decisions. It leads to code duplication, hacks to make something work without deep understanding of the underlying conditions etc.
Suddenly you can’t add new features easily, you can’t make significant changes easily, you have tons of technical debts and development team is close to bankruptcy. You want to change that and have just two options: refactoring or rewriting everything from scratch.
Refactoring and Punctuated Equilibrium
Punctuated Equilibrium is a theory in biology, but it is applied to other complex adaptive systems like societies as well. It states that the system has almost no changes in significant period of time and then changes very rapidly. Then it acquires the equilibrium state again. Sounds familiar? Sure, refactoring is almost the same.
Any change in the system introduces chaos in the short term. The goal of refactoring is to eliminate chaos, but often you increase it initially. When you are working on changes you make system less stable, but as the change is completed, the system can be more ordered. That is not always true for software development. If you use branches, this makes changes safer. Still significant changes increase the risk of new bugs.
The real danger of refactoring are local optimums. With a full rewrite you may have a better architecture than with refactoring. Solution? If you have a vision of final architecture, use it and try to make a path from current architecture to the new architecture. In general simplicity should emerge from refactoring.
Rewrite and Chaos
When you rewrite from scratch, you add such a large portion of chaos that it is hard to predict the final result. You have a new singularity that will explode to the new product universe. But are you certain that it will be better than the previous universe? How many the ‘same bugs’ will you fix in the new product version?
However, rewrite may look faster. I mean you may release a new version faster with rewrite, but most likely with more bugs and less stable.
I think we may expect something like that:
The green line shows how chaos changes with refactoring. After each refactoring there is a small increase of chaos, but then the system becomes stable and chaos decreases. You see that the final release is quite late, but keep in mind that there have been many releases before, so customers benefit earlier.
Black line is how chaos changes with a full rewrite. We have the old system during rewrite, so chaos is constant. After the public release chaos increases significantly. Quite many new (and old) bugs and quirks are expected, so stabilization period is longer. But the release itself is faster. The reason is that there is no burden behind. For example, there is no need to support all the places when you do a change, while in refactoring it is required to keep system working and stable all the time, and it demands additional effort.
Adaptation vs. Revolution
There is another angle to this problem. Refactoring is adaptation, while full rewrite is revolution. Again, revolution is a chaotic beast. You may slowly adapt the product for new external conditions or make one revolutionary rewrite.
So, Rewrite or Refactor?
I do think there is no universal correct answer to this question. If time to market is paramount, if you feel that you’d lose business if a new version will not be published in 6 months, you may try a full rewrite. But beware side effects! Quality may drop significantly and long stabilization period may hurt existing customers.
In most cases refactoring is preferable. Slow pace, high quality, constant improvements, happy customers. I like it more. But rewrite is so magnetic…
Everything has its life-cycle. Even stars. Even The Universe. Everything.
It is quite sad for intelligent creatures, but it is just a fact of life. I want to draw some quite obvious parallels to reveal the real danger of Technical Debt for a software product.
Humans
Let’s take a human. When we are young, we are overflowed with energy. Do you know how a 7 year-old boy spends his day? I have a son, so I’ll tell you. He wakes up at 7 am (sometimes before 6 am!) and tries to play some quite silent games (it is too early to make much noise). He can restrain himself till about 8 am, then he starts to jump, does somersets, some boxing etc… By 8.30 we have a real “tornado” in our house. It may take him an hour to exhaust and have a breakfast. Such periods of activity can be easily repeated 2-3 times a day. And he literally turns off around 9 pm – sleep is when he recharges his batteries.
An old man is quite the opposite. He saves every movement, everything is hard and slow. He makes many mistakes and health maybe getting worth. Learning new things is also getting harder at times. Yes, wisdom and tremendous life experience is there, but very often there is no energy to move forward.
Software products
Software products have exactly the same behavior. Young product is energetic. It has an excited development team behind. It moves forward quickly and feels warm sun on the top. It learns lightning fast and grows on new knowledge. Everything is smooth and shiny. Time to market is everything and tradeoffs are so common! Let’s fix it fast and mark with //TODO: refactor this comment. Let’s hack this framework here to release another cool new feature this week. Let’s re-invent the wheel and write some custom javascript code. Gosh! We don’t have time to learn this new cool javascript framework! These are quite common patterns for young and energetic products.
But what’s next? Years later progress stops. Product becomes old with all the bad and good side-effects. Suddenly it is much harder to add new features. Ages of development make the product more complex. Millions of technical debts rotten its body like cancer . There is no clear architecture anymore and there are so many patches that one small change can produce a totally unpredictable impact and bring along new bugs in unexpected areas. Yes, development team has wisdom. But often no courage and energy to revive the product.
Technical Debt Is a Cancer
How long does this cycle last? It really depends. We all visit doctors. Most of us want to live longer and healthier. Early cancer detection gives good chances to fight and win the disease. We should do exactly the same with software. Here are typical symptoms:
- Velocity has dropped significantly over the last several iterations/releases
- A bug fix triggers one or several new bugs too often
- Nobody knows ideas behind the original software architecture
- Team spends more time fixing bugs than developing/improving features
- Automatic tests are red 80% of the time (if it is 100% — the product is most likely in coma)
If you see at least 2 such symptoms, you’ve just discovered a product cancer — Technical Debt. Technical debt is a true killer when you have deadline (time to market). If you have the symptoms, you should fight the disease right now. You may think that it is OK to wait several months, add some more “cool and highly requested” features and then get back to the real problems. It is a wrong decision, believe me. I used to make it and I used to fail with it. It bogs you down. You lose focus and make stupid mistakes. It leads to fear. And fear is a bad ally. You go from one extreme to another to only increase entropy, nothing else.
If you miss the deadline, all the possible actions will not help. Cancer will win. And then you will have just two options: re-write the product completely from scratch or start a new product.
If you see these symptoms, you should stop and think about the attitude of your development team. If you’ve survived over several years, priorities should be changed. Reset your development team and use chemotherapy.
- Focus on quality. Fix the roots of the problems.
- Teach the original architecture to all, pair program, communicate.
- Introduce “No new code without tests” rule.
- Fight fear. Let the knowledge spread. Knowledge eliminates fear.
- Put the most experienced people on fundamental problems solving.
You have to fight the cancer to bring energy back, to bring courage back, to live and produce a great software product. Otherwise it will be as dead as Lotus Notes.