Edge of Chaos | Agile Development Blog

Many interesting posts and discussions about Scrum and Kanban published last weeks. Someone even called this a war I don’t think it is a war, but some posts indeed may drive such feeling.

• Ken Schwaber bashed Kanban. “I was told that Kanban is frequently used when an organization cannot readily adopt Scrum. Many of Scrum most difficult aspects are then sidestepped. Managers are still in charge of telling people what to do. People can be interrupted at any time. People are still work in functional silos, preserving the jobs of functional managers. People are not allowed to work in containers, sharing skills and knowledge to bring complexity into solutions – instead they are worked on a pull (more sophisticated than push) production line.”
• Karl Scotland discusses Scrum and Kanban difference as Intentional vs. Implementational approaches. Interesting perspective in fact. Karl thinks that Kanban can be used with Scrum to reveal even more problems in development process.
• David J. Anderson posted the most rational article about Scrum and Kanban difference. I like it a lot. Some phrases are true gems: “Kanban is not a project management or software development lifecycle method. It is an approach to change management - a framework for catalyzing change in an organization.” and “Kanban uses a WIP limit as a change agent and Scrum uses commitments. This is a fundamental difference in approach.” David also posted some reflections later with interesting thoughts about anarchy and science.
• A year ago Tobias Mayer didn’t believe that it is possible to use Kanban at all. “I fundamentally disbelieve that there is any such thing as a “value stream” when you are working in a complex environment, in a creative process, building new products or generating new ideas.”

I think Ken is wrong. His arguments against Lean and Kanban quite ridiculous. Sure there is no intention to treat software development as a factory and apply lean manufacturing principles blindly. Of course there are people who will try (or already tried) that and fail. But vast majority of lean community in software development do understand the difference and working on concrete practices. Lean philosophy is great, but tools and practices can’t be taken from manufacturing directly.

Pull system is more complex than Push? C’mon!

David has wise arguments and perfect position. I think Kanban will be more popular than Scrum in long-term perspective, but it will take time. Visualization is a key thing to manage complexity, and software development is a complex system.

Cumulative Flow diagram is a very good starting point for stop-the-line or retrospective meeting. Here is a real example from TargetProcess development.

You can see in the chart that we had a bottleneck in the beginning of December. It was caused by a quite complex user story. In theory a single user story should not affect cycle time significantly and should not create bottlenecks, but if you violate some rules it might be the case.

The user story was about replacing jQuery javascript framework with ExtJS framework. It took 1 month to implement by 1 developer. During this month we’ve made several releases and all went smooth. Then this user story was verified by testers and all existing acceptance tests were passed. So we’ve made a decision to merge this story to the main code line.

Unfortunately, after the merge quite many bugs were found in the build during smoke testing. These bugs took more than a week to fix, and during this time we were unable to release anything, since the merge was already done and the rollback was quite complex as well.

The lesson learned is to put more effort into testing of user stories that are more complex in nature. This particular story affected many places in the application and usual smoke testing was not enough. So we are going to introduce a new class of service (let’s call it  “technically complex story” so far which would mean more in-depth testing and verification before the merge.

In general, Cumulative Flow chart is a valuable tool to analyze historical data, but for emergency bottleneck identification Kanban Board is even better. If you have limits on Kanban Board, you see problems immediately.

You Kanbanized your development process and setup a perfect flow. You see the flow and feel its power. You even measure cycle time and want to reduce it. But it is not as easy as expected. Some problems in your development process are hard to find and are revealed on team level. Here I’ll describe an interesting technique that may help you.

The idea is quite simple. Take a single user story and visualize its states transition from start to finish, thus you see the whole flow of a single story. Draw all available states on Y-axis and days on X-axis. As a result, you will have a chart like that (click to enlarge).

It is a real user story from TargetProcess project. It was quite complex and its cycle time was 19 days. Let me provide some comments.

User story has been in Planned state during just 1 day, then developers took it and moved to In Progress state. So far so good. 4 days of development and user story is moved to Coded state, but it sits there for 2 days. Why? Why testers did not take it immediately? Clearly, we found 2 days of delay and the reason of the delay should be investigated, since it increases story cycle time.

Then testers took the user story and tested it for 2 days. Then the  story was moved to In Progress state again and developers worked on it for 2 more days (including Saturday). Why? It is a half of the initial development time. It is a clear sign of a waste, but we need more details. It appeared, that testers found 11 bugs. Half of the bugs were caused by unclear user story specification, other are subject to investigate. We should apply root-cause analysis here to find the real problems.

OK. Then testers have verified the user story for 2 days and moved it to Ready to Merge state. It was merged quite fast (in 1 day), but still there may be a waste to eliminate. Finally, the user story have lived in Merged state for 5 days and was released Jan-29.

In total (and as a minimum!) we discovered about 9 days of waste. It means we can reduce cycle time significantly by eliminating this wait time (waste). If you check several user stories using this technique, I believe you will find comparable results for simple and complex stories.

Here is another example for a quite simple user story. Still we clearly see wait times in our flow! We can reduce cycle time to 2 days instead of 6 by just eliminating waits in the flow. But it is not easy

This chart is a really powerful tool. It is easy to create and you could use it to find waste in the development flow.

January 13/14 is the Old New Year holiday. Seems like today is the latest appropriate time to look back  and recall the most interesting blog posts by TargetProcess in 2009

Based on visitors count, the posts are ranked as follows (descending order):

1. Lean and Kanban Software Development Digest: In May 2009, this digest came along right on time as Kanban adoption started to grow. We’ve been sifting through the Lean/Kanban buzz and considering if Kanban might be a good tool for our development process, so this post has the most valuable findings we’ve made and shared with agile community.

2. Refactoring vs Rewrite: This post is a real train of thought of a Product Owner trying  to make a decision on how to proceed with product development — rewrite or refactor. Can well be used in textbooks for software product management

3. Mind Maps: Scrum, Extreme Programming, Lean: Another by-product of our research on agile development processes. The specific value of mind maps is that they help grasp complex things with visual representations.

4. Tale: Deadline and Technical Debt: Once upon a time…  Who could ever expect that the fundamental principles of product management can be outlined in a fairy tale ? There we go:  smart Arthur, the cunning king, quest for princess — the metaphorical expression of the danger of technical debt in software development.

5. 5 Wrong Reasons to Apply Kanban. For some reason (no pun intended), 5 wrong reasons ranked higher than 5 right reasons. Maybe it’s just human psychology — to go from “what’s wrong”  instead of  “what’s right” …

6. How We Use Kanban Board. The Real Example:  Once we figured that Kanban process is just the right thing for us and put it in action, we shared this  experience with our blog readers.

7. 5 Right Reasons to Apply Kanban: There they are

8. Zero Defects? Are You Kidding Me? : Can this juicy frog be sure that it swallowed the very last bug? This post is a warning against the so-called “zero defects mentality” in software product management.

9. Simple Rules, Complex Systems and Software Development: Complex systems function at their best when guided by simple rules. Look at ants, birds, space rockets and … software development.

10. BDD and User Story Specification: Examples — This post includes some real user story specs in BDD for TargetProcess product. Enjoy and use.

These are the TOP 10 posts  in 2009 from TargetProcess agile blog (click here for more)

Happy OLD NEW YEAR!

Kanban looks so simple. In theory. Map your flow, set limits, draw some lines and stick stories on the wall. There you go. Is that it? Obviously, no. One of the hardest things is to change your behavior, shift from “push mindset” to “pull mindset”. And that may be tricky.

You have Kanban board and there are no free slots in “In Progress” column. It means new stories should not be started — you’ve reached the limit. Developer comes and says “I know that there are no free slots, but I want to start this new small user story, since all the people are already working on the items and my help is not required”. You might think “Hmm, indeed we are moving forward with a good pace, this small story should not take much time…” and you say “OK, John, take it. I don’t think this would be a problem to exceed the limit right now.”

Two hours later Pete comes to you with a similar question: “I just completed my user story, testers will take it soon and I want to start something new.” You say “Well, we just took one story above the limit…”. Pete replies “Please, boss, I don’t want to sit and do nothing, I want to add some value, I am so much into work today!” Can you say no? I bet in most cases you say something like “OK, let’s do it!” to not discourage your teammate.

What happens then? Testers find some bugs in just implemented user stories, but there are no free developer to fix them. John and Pete are working on new stories. They may switch to bugs, but they’d lose focus and you have 2 stories in progress that can’t be passed to other developers easily (it will take time to make the transition and it looks like waste, isn’t it?). As a result, you deliver some stories later. Yes, later!

I understand that it is very hard initially to say something like “No, you’d better go and read something interesting about new technology. We can’t start new story right now, because if testers find bugs in some running stories, we should fix them ASAP, so we need a free developer”. Moreover, you should train your team to not ask such questions. WIP limit is a rule that should be followed. You may change it, but not arbitrarily. This change should have something behind (better productivity, more developers, etc). And this change should be discussed in retrospective meeting or during a Kaizen event.

I should say it is really, really hard to accept the fact, that sometimes developers and testers should do nothing (I mean do not start new stories when they are free). But it’s absolutely required to do so, otherwise pull system will not work.

It is always interesting to look behind the scene. In previous post I  described our development process, now I will show how we use Kanban Board in TargetProcess. Maybe it will give you some new ideas to try.

The most interesting part of every Kanban Board is its flow. So here is it:

Planned: contains all stories and bugs that are ready for development. It means they have full details/specification, so developers may start working on any item from this list anytime. Items are prioritized in this columns, first item should be taken first. Only product owner can put stories and bugs into Planned state from Backlog.

WIP Bugs: this is quite a special column. When testers find bug in just coded user stories, these bugs are drag-and-dropped in WIP bugs column. It means developers should take them ASAP to make this story Done-Done ASAP. Sometimes we put important bugs from production here as well. This state was introduced just several weeks ago. Works fine.

In Progress: shows stories and bugs that are in development now. When development pair takes a new story, the story moves here.

Coded: shows that story is implemented by developers. It is a queue for testers in fact.

Testing: shows stories that are in testing now.

Ready to Merge: we implement each story into a separate branch. If user story is in this state, it means it is implemented and tested, so may be merged into master. Every day someone merges all the items from this column.

Merged: this column contains all the stories and bugs that will be included into the next public build/release. We normally do weekly releases.

Done: shows several items included into the latest build.

We have a large TV in the office that displays Kanban Board all the time. Daily meetings are near the TV, so everybody can see it and understand what we are talking about.

Inspired by Kanban at Lonely Planet. I think it is a good idea to share software development processes from different companies. Maybe even it would be great to create a repository of the processes somewhere on a popular agile web site/blog.

Context

• Product development (quite large web application)
• Two development teams: Core (5 developers, 1 scrum master, 3 testers) and Integration (3 developers - one of them plays scrum master role, 1 tester)
• Quite mature product: 4 years
• Technology: C#, ASP.NET, ExtJS, NHibernate

Process

• Development team has all the power to try new practices, keep working practices and remove bad practices. There’s a little push from scrum master side.
• We do not use iterations.
• We do use releases, but often it is just a set of all the features implemented last month.
• We do not have iteration planning meeting, but discuss every user story JIT before implementation on a meeting with PO + Testers + Developers.
• We do not estimate stories and bugs, just a very brief estimate like (”oh, this will be a large story, let’s split it”).
• We split stories, but sometimes not as aggressively as required.
• We release new builds every week (sometimes 2 builds per week). The goal is to release every day, but we do not have a good automated functional tests coverage so far.
• Smoke testing of the build takes 3 hours (3 testers).
• We have a limit of 3 user stories or bugs in progress.
• We have retrospective meetings every 2 weeks.
• We sometimes have technical user stories (for example, remove jQuery from application), but that is rare.
• We use Kanban Board in TargetProcess heavily and have a large screen in development room.
• We have a flat hierarchy in the company. There are only two layers and no boundaries between the layers. No formalism at all. Also we do not have positions like Senior Developers, Junior Developers, etc. All people are equal (however, there are informal leaders).
• Average Cycle time for user story is 13 days and 5 days for bug.
• We track spent/remaining time on tasks and stories.

Development practices

• We develop each user story or bug in a separate branch. Master is always ready for release (after smoke testing). We use Git (switched from Subversion).
• Pair programming on all user stories and complex bug fixes. Sometimes people work alone on smaller/simpler problems.
• Developers switch pairs every day, even if user story is not completed yet. The goal is to a have fresh look, knowledge spreading and real collective code ownership.
• Before story implementation, pairs discuss possible solutions. It is quite formal.
• TDD. All the new code is covered by unit tests. Situation is worse on JavaScript side, we just started TDD adoption for UI.
• We use Cruise Control and have quite many different setups. Smoke builds, release builds, etc. Functional tests run in parallel and it takes 1 hour to run them all.
• Selenium. Automated functional tests are based on Selenium. We’ve created a framework in C# especially for our application to simplify tests creation. Tests coverage is not good so far, but we are working on it (there are 2 automation testers in our team).

ToDo

• We want to add performance tests into automated builds.
• We want to have 80% functional tests coverage to be able to release new builds every day
• We are incorporating User Experience into development process. This is our main goal for the next year. It includes Customers UX Community creation, interactive prototypes for all user stories, usability testing.

In previous post I mentioned 5 wrong reasons to apply Kanban. Tobias Mayer asked to blog about 5 right reasons Obviously there should be right reasons, otherwise Kanban adoption would fail everywhere. So there they go:

#1. Ability to release anytime

In Scrum or XP you can’t release in the middle of an iteration. In Kanban you may release anytime. When user story is ready, you may release it. Definitely it is a challenge to setup development process this way. It is required to have “branch-by-feature” source control policy, merge often, integrate and test often. But it gives you an ability to release often. That is something worth to fight for.

As a PO I like this ability very much. An important user story implemented? OK, let’s release it tomorrow in v.2.15.2. Our customers may benefit from this release as soon as possible. Lead time is faster — customers are happier.

One thing to mention — you need to have a good automated test suite, otherwise it will be problematic to make small releases with a good quality.

#2. Ability to change priorities on the fly

In Scrum you can’t add stories on the fly into sprint (usually). At least it is not an easy process and development team often resists to replacing a story from Sprint backlog. The story has been discussed, estimated etc. A new story may be discussed in a hurry, some details will be missed and as a result significant re-work will be required. So in general iteration or sprint should be frozen.

In Kanban if you have an urgent request to implement or a really important user story, you can just put it on top of the queue. It will be taken as soon as a free slot is available. It is simply a Product Owner’s dream

#3. No need in iterations

Why you need iterations? Initially iterations help to reveal real problems in the development process quickly. Further on they establish a project rhythm and rituals. However at some point in project I think you no longer need them.

My vision is that you need to iterate first, then flow. Our backlog is fuzzy now, plans change often. We learn how to do agile in general and iterations are not helpful there anymore. Without iterations there is no need in iteration planning and iteration demo meetings. They are waste. Instead we have more smaller just-in-time meetings with people in a feature-team before starting the development for each user story.  It works perfect.

Some people are missing the rhythm, but I think it is more to habit. Kanban establishes more complex rhythm and it may take time for development team to recognize it. Still stable rhythm may be the only reason to keep iterations in the late phase of product development, in my opinion.

#4. No need in estimates

Why you need estimates? In iterative development you need them to predict how many stories you may take in the next iteration. You may even predict a release date based on estimated backlog and iteration velocity. The other reason is that PO wants to know how big the user story is. If it is big enough, PO  may consider moving it to the next release. If it is small, PO may decide to put it into the next iteration.

Obviously, iterative development is hardly possible without estimates. But if you drop iterations, it is not a problem anymore. Can Product Owner live without estimates? Well, yes. In our company we don’t estimate user stories. Why not? Because I as a PO don’t need them and we don’t use iterations. All I may ask is a very rough estimate like  normal, large, very large.

In our situation (I stress that, in our situation), estimates are waste. We don’t spend time on estimates. We have a prioritized backlog and just take the most important user story and implement it. It may not work in some conditions, but we have quite a mature product and hundreds requests from customers. There is no clean field development, so normally we don’t have any defined release date. We release when it is ready (I understand that many companies can’t do that, but we have this favor).

#5. Perfect flow visualization

Kanban Board provides a very clear view on current work in progress. It visualizes flow and enables fast planning and tracking. It is really a great tool, we love it.

Kanban is becoming a popular agile tool. Indeed it is very good for software projects of certain types. However, there is a danger of false reasons behind Kanban adoption.

#1. User Stories Diversity

“Our stories vary in size a lot from 1 point to 40 points. Large stories just do not fit into an iteration”

It is very easy to say you can’t split user stories, thus iterations should be abandoned. An easy solution is not the best one. There is something important behind the fact you can’t split stories. Most likely you don’t know how to split stories right. It is quite hard from the beginning and demands creativity.

Moreover, according to Queueing Theory, it is better to have small stories with roughly equal size. Small batches with equal size improve flow (and you have a better and more predictive cycle time). So in Kanban you still have to split stories to smaller stories and try to make them equal size.

#2. Failed Iterations

“We can’t complete most stories in a single iteration”

There may be many, many reasons behind. Mini-waterfall approach, velocity greed, large stories, manual testing, poor stories description, etc. You may even have a wrong iteration length. For example, in large projects 1 week iteration may be an overhead with a huge transaction cost. So 1 month iteration may be much better in this case.

You need to analyze true reasons, the roots of the problem.

#3. Failed Retrospective Meetings

“Retrospective meetings are waste, they do not help in process improvement and we want to remove them”

You just don’t do these meetings right. One popular failure is no Action Items after the meeting. Without action items a Retrospective Meeting is just a kind of informal chat that you may have over a glass of beer. You meet, chat about your problems, and get back on the same track next day. Rule #1 is to collect and write the list of Action Items that you will try to accomplish during the next Sprint [or any other time box].

One more  common pitfall is to skip action items execution. You may collect them, but not really try. You may even try them, but abandon too quickly. Almost all new rules or practices put you off the comfort zone. It takes time to learn them, use them and like them (or dislike), but it should be an expert opinion, not just a gut feeling.

If you expect that you will replace failed retrospective meetings with a nice and simple “stop the line” rule, you’ll fail, since it is even harder than scheduled regular retrospectives and demands even higher level of self-discipline.

#4. Shared People / Functional Departments

“We have a single pool of developers and share them between projects. We can’t form stable project teams”

Do you really think that Kanban will solve your social and organizational problems? C’mon! It helps to visualize flow and find bottlenecks, but if you don’t have a cross-functional team — you have hard times. It is proven in so many sources and researches that cross-functional teams perform better, produce better results and better software.

If you are experiencing difficulties planning sprints with shared pool of developers, try to fix the root of the problem first - switch to cross-functional teams and eliminate multi-tasking.

#5. Simplicity

“Kanban is so simple! No plans, no estimations, no iterations, no overhead”

Indeed Kanban looks simple. But it provides nothing interesting by itself. To ensure a successful Kanban adoption, you need to apply Lean principles first. This new buzzword may sound like a silver bullet, but obviously it  is not. Hard work, discipline, target on perfection and constant improvements - all that is required to apply any agile methodology.

Don’t get me wrong. I am not saying that Kanban is bad. We use it within TargetProcess and I personally believe it works way better than iterative development (for us).  I just want to make a point that you’d hardly resolve all your underlying problems as you switch to Kanban.

P.S. I usually don’t read posts like “10 reasons to …” Can’t believe I wrote such post myself!
P.P.S. Read next post 5 right reasons to apply Kanban.

• What is Cadence? Great post explaining cadence using nice pictures. “The purpose of a cadence is to establish a reliable and dependable capability which demonstrates a predictable capacity. Cadence gives some confidence in the upcoming work when we are triggering rather than scheduling work.”
• Series of amazing Danilo Sato posts about Velocity anti-patterns: Done is not done, Making up points, Used as a performance measure.
• Alan Shalloway has interesting and thought provoking post “Is Part of the Agile Community Acting Like the Waterfall Community of Old?” “The pattern is unfortunate - it seems that there are those who don’t like discussing some thing have taken one of two approaches. If they can intimidate people who are using Scrum Alliance certifications for their livelihood, they do so. If they can’t, they attack the integrity of those people they don’t like (e.g., David and myself and others I haven’t mentioned). “