Edge of Chaos | Agile Development Blog

Cumulative Flow diagram is a very good starting point for stop-the-line or retrospective meeting. Here is a real example from TargetProcess development.

You can see in the chart that we had a bottleneck in the beginning of December. It was caused by a quite complex user story. In theory a single user story should not affect cycle time significantly and should not create bottlenecks, but if you violate some rules it might be the case.

The user story was about replacing jQuery javascript framework with ExtJS framework. It took 1 month to implement by 1 developer. During this month we’ve made several releases and all went smooth. Then this user story was verified by testers and all existing acceptance tests were passed. So we’ve made a decision to merge this story to the main code line.

Unfortunately, after the merge quite many bugs were found in the build during smoke testing. These bugs took more than a week to fix, and during this time we were unable to release anything, since the merge was already done and the rollback was quite complex as well.

The lesson learned is to put more effort into testing of user stories that are more complex in nature. This particular story affected many places in the application and usual smoke testing was not enough. So we are going to introduce a new class of service (let’s call it  “technically complex story” so far which would mean more in-depth testing and verification before the merge.

In general, Cumulative Flow chart is a valuable tool to analyze historical data, but for emergency bottleneck identification Kanban Board is even better. If you have limits on Kanban Board, you see problems immediately.

You Kanbanized your development process and setup a perfect flow. You see the flow and feel its power. You even measure cycle time and want to reduce it. But it is not as easy as expected. Some problems in your development process are hard to find and are revealed on team level. Here I’ll describe an interesting technique that may help you.

The idea is quite simple. Take a single user story and visualize its states transition from start to finish, thus you see the whole flow of a single story. Draw all available states on Y-axis and days on X-axis. As a result, you will have a chart like that (click to enlarge).

It is a real user story from TargetProcess project. It was quite complex and its cycle time was 19 days. Let me provide some comments.

User story has been in Planned state during just 1 day, then developers took it and moved to In Progress state. So far so good. 4 days of development and user story is moved to Coded state, but it sits there for 2 days. Why? Why testers did not take it immediately? Clearly, we found 2 days of delay and the reason of the delay should be investigated, since it increases story cycle time.

Then testers took the user story and tested it for 2 days. Then the  story was moved to In Progress state again and developers worked on it for 2 more days (including Saturday). Why? It is a half of the initial development time. It is a clear sign of a waste, but we need more details. It appeared, that testers found 11 bugs. Half of the bugs were caused by unclear user story specification, other are subject to investigate. We should apply root-cause analysis here to find the real problems.

OK. Then testers have verified the user story for 2 days and moved it to Ready to Merge state. It was merged quite fast (in 1 day), but still there may be a waste to eliminate. Finally, the user story have lived in Merged state for 5 days and was released Jan-29.

In total (and as a minimum!) we discovered about 9 days of waste. It means we can reduce cycle time significantly by eliminating this wait time (waste). If you check several user stories using this technique, I believe you will find comparable results for simple and complex stories.

Here is another example for a quite simple user story. Still we clearly see wait times in our flow! We can reduce cycle time to 2 days instead of 6 by just eliminating waits in the flow. But it is not easy

This chart is a really powerful tool. It is easy to create and you could use it to find waste in the development flow.

Individual estimates accuracy is another metric requested by several TargetProcess users. First of all it may be useful if and only if:

• Task estimated by one developer only (if whole team estimates tasks you can’t measure individual accuracy, only whole team accuracy).
• You are using time tracking to track all spent time for all tasks

For example, Ted subscribed to Quick Search user story. He broke down this user story to tasks, estimated each task and got 50 hrs of effort for user story. Then he implemented the user story and spent 60 hrs. It means his estimate accuracy for this task is 50 / 60 = 0.83 or 83%. In the end of iteration we can calculate all Ted’s assignments and spent time and define estimate accuracy for next iteration, let’s say it is 0.7.

OK, but how we can use this metric? Well, if Ted subscribed to 60 hrs work it means he will spend 85 hours and for 2 weeks iteration it means at least 5 hrs overtime. Ted should take this information into consideration and remove some tasks from his ToDo. This works if Ted estimate accuracy remains the same, but is that always the case? In reality Ted’s accuracy may vary from 0.5 to 0.9 and exactly for next iteration it may be 0.9 and in this case Ted will be able to do all committed work.

As you see estimate accuracy coefficient should be used as a recommendation only, but if Ted insists on his commitment Project manager/Scrum master should agree and let Ted do the job. Again we have similar problem as with individual velocity when Scrum master may demotivate developers with numbers in hands.

I see value in individual estimates accuracy, but I see some danger as well. This metric should be used by developers, not by Project managers/Scrum masters.

I think we should care about team estimates and team velocity, not about individual estimates and individual velocity. Agile development is all about jelled teams.

We are receiving more and more requests to add individual velocity measurement in TargetProcess. People want to see this information and use for iterations planning. But is it required to measure individual velocity? What caveats such calculation has? It may be not clear initially, but there are some problems with individual velocity for sure.

What is an individual velocity? It is a measure of how much effort a person may complete in a defined time frame. In agile development time frame is an iteration. If Ted completed several tasks with 40 hrs of effort in total and Jerry completed tasks with 25 hrs effort only, we should say that Ted has better velocity in last iteration. Does it mean that Ted is a better/faster developer? Not at all. There are hundreds of reasons why Jerry completed less. He helped other developers with tasks and mentored them; he got serious headache during two days in a row and did almost zero progress; he had two birthday parties last week and his performance was no good next day after each party; one of his tasks was underestimated due to unexpected performance problem with third-party component. We can bring many more reasons on the table. Performance in last iteration says nothing.

OK, but what about average velocity? Surprisingly, Jerry’s average velocity is 54 hrs per iteration. Gosh! What happened with Jerry last two weeks? Will his average velocity help us to make correct iteration plan? If we sum up individual velocities all developers will it help us to create a better iteration plan? No, since we already have Iteration Velocity metric and it will be exactly the same. Why should we care about individual velocity in this case? To make better assignments for each person? But we still don’t know how much effort Jerry will complete next iteration. The good guess is from 25 hrs to 60 hrs. Is it helpful? Maybe, a bit. Is it worth the darkside of the individual velocity measurement? And sure there are some problems with it.

Individual velocity measurement has wrong focus on individual performance. The focus should be on team performance, individual performance is not important. If Jerry knows that his velocity is measuring, he maybe will not help other team members a lot. He will focus on his performance as an individual developer. The worst thing company can do is to bind bonuses to individual performance. This nips teams in the bud. The right solution is to measure team performance, but in agile development we already have team performance metric, it is a well-known iteration velocity - very good, very simple and very helpful metric that enough for iteration planning. Individual velocity will not bring additional benefits there.

Individual velocity measurement forces work assignment, while in agile teams it is all about work commitments. Team should do a commitment to complete as many user stories as they feel can be completed during next iteration. On iteration planning meeting Jerry may feel bad about his poor performance in last iteration (yes, developers know their performance without measurements) and will commit to 70 hrs work items in total and complete them all with courage. If you measure individual velocity you tend to assign stories based on numbers you have in hands. “Hey, Jerry, are you kidding? You did only 25 hrs last iteration, how are you going to do 70 hrs?” It is good if Jerry has courage to answer something like “Just believe me, I will do that” and project manager/Scrum master agrees and allows Jerry to make such commitment. But Jerry might think “oh, he doesn’t believe me… Well, let it be, I will take 50 hrs, it is close to my average velocity”. Individual velocity may demotivate people, and many managers having it in hands will use it incorrectly. It is very common to revert back to muscle memory of waterfall days and make assignments instead of commitments (especially when your project is in troubles).

I don’t see much value in individual velocity, but I do see many problems it may cause (or hide, or support).