Here is the snapshot of our real TargetProcess production Kanban Board taken today (November 18). You can see what we are working on right now, what is in progress, what is in testing and what will be released soon.
Legend:
Estimation takes time. Even if you do planning poker and use story points, it still takes time. What do you do to improve estimation accuracy? You gather some data, analyze the data and discuss the results. You are spending time on all that. But are you sure that estimates really provide any value? Most likely not. It is a waste. You’d better spend this time doing something important for the product.
If you don’t have estimates, you can speak without fear and explain things clearly. You can enumerate problems and explain how they were resolved. You can show that you work really hard and take all the necessary actions to release this story as soon as possible with great quality.
If you have estimates, be ready to hear something like “you screwed up, maaan! You had to deliver this feature till the end of the month! What the f%$k is going on?” as an argument. The team put themselves on a weak side immediately and have to care about deadlines more, not about quality or a better solution. Is it something you really need?
You are not relying on people’s optimism (which is built in). People are optimists (almost all of them) and inevitably give optimistic estimates. You can use complex formulas or very simple rules to have better estimates, but is it really worth it? You can spend many hours defining correct formula for exactly this development team. People will not trust you, since you (rightfully) don’t believe in their estimates. Numbers will look “made up” and still you will have incorrect estimates in the end.
In some teams Estimate is equal to Deadline. That’s bad. What do you do to meet the estimate? Compromise quality? Tolerate average architectural solution? Abandon polishing? All that leads to poor solutions that end users will not like. It is better (almost always) to spend more time, than planned, and release something really usable and useful.
Imagine, you have a planning poker session and estimate all stories inside an epic. You sum up all the stories and define it as a 1000 pts epic. You just stick a “fear” label on the epic. People do fear large problems. A 1000pt problem is huge and psychologically it is harder to decide “let’s start it right now”. Product Owner will have a temptation to implement 100 smaller user stories instead of this huge epic. However, it may be a bad decision. This epic may be really important, the morst important thing for the product.
If you don’t estimate it and are not completely sure about its size, you have a better chance to actually start doing it.
Humans receive 95% of information through visual perception. We spend countless hours staring into laptops reading, analyzing, interpreting and feeling information flows. I think visualization is something we lack in many disciplines and project management is not a lucky exception.
Interestingly, latest trends in agile project management do care about visualization. Kanban success heavily depends on great visualization. Everybody loves Task Boards and Burn Down charts in Scrum. As you can see, there is some progress, but I think it is not systematic, just a side effect of other activities. It would be fascinating to create an orthogonal movement in agile project management community that focuses on visualization, I call it Visual Project Management.
There is only one really important rule about visualization:
Visualization should reveal problems, states and trends
Sure, we can add more rules, but they’d be secondary. Imagine you can see a Backlog and diagnose a disease right away:
Imagine you see a Board, and at the same instant you grasp all the bottlenecks:
I can’t provide quick visual solutions right away, but I think this approach will change the way teams make decisions and improve their development process.
I wonder why we still have no good visualization tools for project management. I think the answer is that project managers are not strong in design, and designers are not strong in project management. Separation of responsibilities leads to basic, trivial decisions. PM should learn design and data visualization techniques to really invent new visualization. Designer should learn PM domain to invent new visualization. The ideal mix is a team where PM knows design and Designer knows project management.
Let’s review some examples to get a basic feeling of how visualization re-define things.
Information presentation really affects the ways we run projects. Here are two screenshots that contain exactly the same information. You can easily say which one is better.
This is a simple list of user stories taken from TargetProcess (I am working at TargetProcess, so forgive me biased screenshots). It contains quite many data about user stories like state, assigned people, effort, etc. However, this data is hidden. To make some decisions, you need to dig and put some effort into it. It means you have a higher cognitive load. It means you’ll miss some details and sometimes make a wrong decision.
Next screenshot is a simple Kanban Board. While it is not the best example of Kanban Board, still it is good enough to reveal the difference. You see the same user stories on this screen. However, you can quickly identify there’re too many user stories in In Progress state, but it is not critical, since there are no holes in development flow. You can somehow feel that most likely the team is doing pretty good. In some cases you can’t even quickly explain why you think so, you’ll need to analyze your own feelings to provide logical answers.
But the truth is that you have arrived to some conclusion with enough accuracy, really quickly. That is why visualization is so important in any discipline, including project management.
I believe we can have something much better than a simple Kanban Board. Kanban Board visualizes development flow, but misses other important things like people load, local problems, overall progress for all projects etc.
What happens if we have several teams and want to see their work on Kanban Board? Definitely, the board above will not show that. So far, I haven’t seen good solutions to such problems. Software products are not good in presenting large amount of data in an interesting, meaningful manner. This should be changed.
Gantt Chart is a great tool as well. It is heavily used in traditional project management, but often with poor results.
Most of the project charts look the same and make the same mistakes: analytically thin, bureaucratic grid prison, not annotated, little quantitative data.
E. Tufte
This Gantt Chart is quite good and useful:
This Gantt Chart is bad and useless:
Stop. Think a little bit. Can you define the difference? Can you say why I’ve made such a conclusion?
First chart shows quite large project phases. Second chart shows very granular tasks. Gantt Chart is not a good tool to handle granularity. It works best to visualize long project phases, like releases or iterations. It just plain fails to visualize 500+ tasks in a project.
Gantt Chart has a large white space, it lacks information density. Thus it should be used carefully to visualize important information, not everything you have.
I think the frustration with Gantt charts arises more because of tool issues. People’s contexts of use may require information that is obscured by a Gantt chart. But Gantt charts are the dominant (sometimes only) visualization in many tools, and it’s difficult to impossible to extract and present the data in other forms.
Brian de Alwis
These were just two examples of visualization in project management. I believe we can be more creative to provide better tools and concepts, invent new ways to present project information and improve transparency. We definitely can do better. And we should.
I will post more about visual project management. This topic is really intriguing and promising to me.
Many interesting posts and discussions about Scrum and Kanban published last weeks. Someone even called this a war :) I don’t think it is a war, but some posts indeed may drive such feeling.
I think Ken is wrong. His arguments against Lean and Kanban quite ridiculous. Sure there is no intention to treat software development as a factory and apply lean manufacturing principles blindly. Of course there are people who will try (or already tried) that and fail. But vast majority of lean community in software development do understand the difference and working on concrete practices. Lean philosophy is great, but tools and practices can’t be taken from manufacturing directly.
Pull system is more complex than Push? C’mon!
David has wise arguments and perfect position. I think Kanban will be more popular than Scrum in long-term perspective, but it will take time. Visualization is a key thing to manage complexity, and software development is a complex system.
Cumulative Flow diagram is a very good starting point for stop-the-line or retrospective meeting. Here is a real example from TargetProcess development.
You can see in the chart that we had a bottleneck in the beginning of December. It was caused by a quite complex user story. In theory a single user story should not affect cycle time significantly and should not create bottlenecks, but if you violate some rules it might be the case.
The user story was about replacing jQuery javascript framework with ExtJS framework. It took 1 month to implement by 1 developer. During this month we’ve made several releases and all went smooth. Then this user story was verified by testers and all existing acceptance tests were passed. So we’ve made a decision to merge this story to the main code line.
Unfortunately, after the merge quite many bugs were found in the build during smoke testing. These bugs took more than a week to fix, and during this time we were unable to release anything, since the merge was already done and the rollback was quite complex as well.
The lesson learned is to put more effort into testing of user stories that are more complex in nature. This particular story affected many places in the application and usual smoke testing was not enough. So we are going to introduce a new class of service (let’s call it “technically complex story” so far :) which would mean more in-depth testing and verification before the merge.
In general, Cumulative Flow chart is a valuable tool to analyze historical data, but for emergency bottleneck identification Kanban Board is even better. If you have limits on Kanban Board, you see problems immediately.
You Kanbanized your development process and setup a perfect flow. You see the flow and feel its power. You even measure cycle time and want to reduce it. But it is not as easy as expected. Some problems in your development process are hard to find and are revealed on team level. Here I’ll describe an interesting technique that may help you.
The idea is quite simple. Take a single user story and visualize its states transition from start to finish, thus you see the whole flow of a single story. Draw all available states on Y-axis and days on X-axis. As a result, you will have a chart like that (click to enlarge).
It is a real user story from TargetProcess project. It was quite complex and its cycle time was 19 days. Let me provide some comments.
User story has been in Planned state during just 1 day, then developers took it and moved to In Progress state. So far so good. 4 days of development and user story is moved to Coded state, but it sits there for 2 days. Why? Why testers did not take it immediately? Clearly, we found 2 days of delay and the reason of the delay should be investigated, since it increases story cycle time.
Then testers took the user story and tested it for 2 days. Then the story was moved to In Progress state again and developers worked on it for 2 more days (including Saturday). Why? It is a half of the initial development time. It is a clear sign of a waste, but we need more details. It appeared, that testers found 11 bugs. Half of the bugs were caused by unclear user story specification, other are subject to investigate. We should apply root-cause analysis here to find the real problems.
OK. Then testers have verified the user story for 2 days and moved it to Ready to Merge state. It was merged quite fast (in 1 day), but still there may be a waste to eliminate. Finally, the user story have lived in Merged state for 5 days and was released Jan-29.
In total (and as a minimum!) we discovered about 9 days of waste. It means we can reduce cycle time significantly by eliminating this wait time (waste). If you check several user stories using this technique, I believe you will find comparable results for simple and complex stories.
Here is another example for a quite simple user story. Still we clearly see wait times in our flow! We can reduce cycle time to 2 days instead of 6 by just eliminating waits in the flow. But it is not easy ;)
This chart is a really powerful tool. It is easy to create and you could use it to find waste in the development flow.
January 13/14 is the Old New Year holiday. Seems like today is the latest appropriate time to look back and recall the most interesting blog posts by TargetProcess in 2009 :)
Based on visitors count, the posts are ranked as follows (descending order):
1. Lean and Kanban Software Development Digest: In May 2009, this digest came along right on time as Kanban adoption started to grow. We’ve been sifting through the Lean/Kanban buzz and considering if Kanban might be a good tool for our development process, so this post has the most valuable findings we’ve made and shared with agile community.
2. Refactoring vs Rewrite: This post is a real train of thought of a Product Owner trying to make a decision on how to proceed with product development — rewrite or refactor. Can well be used in textbooks for software product management :)
3. Mind Maps: Scrum, Extreme Programming, Lean: Another by-product of our research on agile development processes. The specific value of mind maps is that they help grasp complex things with visual representations.
4. Tale: Deadline and Technical Debt: Once upon a time… Who could ever expect that the fundamental principles of product management can be outlined in a fairy tale ? :) There we go: smart Arthur, the cunning king, quest for princess — the metaphorical expression of the danger of technical debt in software development.
5. 5 Wrong Reasons to Apply Kanban. For some reason (no pun intended), 5 wrong reasons ranked higher than 5 right reasons. Maybe it’s just human psychology — to go from “what’s wrong” instead of ”what’s right” …
6. How We Use Kanban Board. The Real Example: Once we figured that Kanban process is just the right thing for us and put it in action, we shared this experience with our blog readers.
7. 5 Right Reasons to Apply Kanban: There they are :)
8. Zero Defects? Are You Kidding Me? : Can this juicy frog be sure that it swallowed the very last bug? This post is a warning against the so-called “zero defects mentality” in software product management.
9. Simple Rules, Complex Systems and Software Development: Complex systems function at their best when guided by simple rules. Look at ants, birds, space rockets and … software development.
10. BDD and User Story Specification: Examples — This post includes some real user story specs in BDD for TargetProcess product. Enjoy and use.
These are the TOP 10 posts in 2009 from TargetProcess agile blog (click here for more)
Happy OLD NEW YEAR! :)
Kanban looks so simple. In theory. Map your flow, set limits, draw some lines and stick stories on the wall. There you go. Is that it? Obviously, no. One of the hardest things is to change your behavior, shift from “push mindset” to “pull mindset”. And that may be tricky.
You have Kanban board and there are no free slots in “In Progress” column. It means new stories should not be started — you’ve reached the limit. Developer comes and says “I know that there are no free slots, but I want to start this new small user story, since all the people are already working on the items and my help is not required”. You might think “Hmm, indeed we are moving forward with a good pace, this small story should not take much time…” and you say “OK, John, take it. I don’t think this would be a problem to exceed the limit right now.”
Two hours later Pete comes to you with a similar question: “I just completed my user story, testers will take it soon and I want to start something new.” You say “Well, we just took one story above the limit…”. Pete replies “Please, boss, I don’t want to sit and do nothing, I want to add some value, I am so much into work today!” Can you say no? I bet in most cases you say something like “OK, let’s do it!” to not discourage your teammate.
What happens then? Testers find some bugs in just implemented user stories, but there are no free developer to fix them. John and Pete are working on new stories. They may switch to bugs, but they’d lose focus and you have 2 stories in progress that can’t be passed to other developers easily (it will take time to make the transition and it looks like waste, isn’t it?). As a result, you deliver some stories later. Yes, later!
I understand that it is very hard initially to say something like “No, you’d better go and read something interesting about new technology. We can’t start new story right now, because if testers find bugs in some running stories, we should fix them ASAP, so we need a free developer”. Moreover, you should train your team to not ask such questions. WIP limit is a rule that should be followed. You may change it, but not arbitrarily. This change should have something behind (better productivity, more developers, etc). And this change should be discussed in retrospective meeting or during a Kaizen event.
I should say it is really, really hard to accept the fact, that sometimes developers and testers should do nothing (I mean do not start new stories when they are free). But it’s absolutely required to do so, otherwise pull system will not work.
It is always interesting to look behind the scene. In previous post I described our development process, now I will show how we use Kanban Board in TargetProcess. Maybe it will give you some new ideas to try.
The most interesting part of every Kanban Board is its flow. So here is it:
Planned: contains all stories and bugs that are ready for development. It means they have full details/specification, so developers may start working on any item from this list anytime. Items are prioritized in this columns, first item should be taken first. Only product owner can put stories and bugs into Planned state from Backlog.
WIP Bugs: this is quite a special column. When testers find bug in just coded user stories, these bugs are drag-and-dropped in WIP bugs column. It means developers should take them ASAP to make this story Done-Done ASAP. Sometimes we put important bugs from production here as well. This state was introduced just several weeks ago. Works fine.
In Progress: shows stories and bugs that are in development now. When development pair takes a new story, the story moves here.
Coded: shows that story is implemented by developers. It is a queue for testers in fact.
Testing: shows stories that are in testing now.
Ready to Merge: we implement each story into a separate branch. If user story is in this state, it means it is implemented and tested, so may be merged into master. Every day someone merges all the items from this column.
Merged: this column contains all the stories and bugs that will be included into the next public build/release. We normally do weekly releases.
Done: shows several items included into the latest build.
We have a large TV in the office that displays Kanban Board all the time. Daily meetings are near the TV, so everybody can see it and understand what we are talking about.
