Edge of Chaos

Agile Development Blog

Scrum, Lean, Kanban, Visualization, User Experience, IT Lifestyle

kanban

Rolling Wave User Stories: How To Handle Specifications

There are two extreme views on requirements documentation (most use user stories to describe requirements these days):

  1. User story is just an invitation to communication. It should be very short.
  2. We should document requirements fully, with all known details.

Most likely, you accept option 1 and reject option 2. If you ask me, what option I prefer, I will answer “both”. Hold on your tomatoes and read on.

Let’s say, you start a project and have a backlog with 100 user stories. This is the stage where you should keep user stories descriptions short. Don’t waste time, many of these stories will never make it into production, they will die.

You pick a user story and ready to start development. At this moment you have a solution and this solution should be documented. Short description is not enough, since some details may be lost and implemented incorrectly, this will lead to re-work and time waste. At this stage you shouldn’t worry that detailed description of a user story is a waste, this story is going to development phase anyway.

Simply speaking, each user story has a life-cycle: Birth → Solution → Implementation → Release. Let’s review it:

User Story flow phases

Birth
Someone has an idea and user story is added into a backlog. Just a short description is fine in this case.

Solution
There are various methods to create a solution for the problem that user story describes. Maybe you have a special UX phase and special UX teams that brainstorm possible solutions. Maybe you discuss and invent solution in development team just before implementation. Maybe Product Owner creates a solution alone (worst case). Still somehow you have a solution.

Then you should document the solution with quite many details to make sure that solution will not be misinterpreted. Here you will argue that live communication is better than specification. I agree and will follow on below, but if you have a gap between Solution and Implementation phases (which is often the case), you have a good chance to forget many things about the solution. You will end up with sub-cycle: Forget → Remember → Lost Some Details → Re-Work. This sub-cycle generates nothing but waste.

Implementation
Implementation may take several hours, or may take several weeks (happens). If a solution was invented outside a development team, it’s extremely important to have a kick-start meeting, discuss the solution again and find all weaknesses in it. It may happen that solution is unideal and can be improved. Kick-start meeting brings everyone on the same page and every team member understands how it should work.

Right after the kick-start meeting user story should have as much details as possible, including positive and negative flows, texts and labels, final design, etc. Memory is a good thing, but it fails quite often. Don’t rely on your own memory and on memory of other people, put everything you know into text and images.

Release
When user story is done, there is not much value in documentation. If there are some minor discrepancies — that is not a problem. Sure, it can be used as a reference for technical writer, for example, so, depending on your context, you may want to have it corrected.

The main idea is that importance of documentation is low initially, highest in the implementation phase and falls down again after the release. Full documentation just in time is a good practice.

User Story phases

You may worry that it sounds heavy, but in real life a user story can pass all these phases in a single day. For such quick stories full documentation looks like an overhead, but usually it’s very short as well and may take 10 minutes to write.

I personally like when user story is fully documented (at the right time!) with all important cases, has final design attached, has all labels nailed and provides a good vision about the solution.

With many user stories you have rolling waves. Like that:

rolling waves user stories

Now fish out your tomatoes and throw as many as you want.

Announcing an Article: Estimates in Software Development. The New Frontiers.

There’s more and more buzz around estimates and #noestimates in software development. People like to write bold statements and go extreme about things in blogs. Usually, personal dialogues are much more balanced. Some hate estimates and believe it’s a useless activity. Some defend it with arguments of arguable truth.

I want to dig into intrinsic estimates complications, what people mean by “estimate” and what future directions we may attack → Read the article.

TargetProcess 2.24: Fast Search, Relations Network, new Visual Reports, 60 custom fields, History API

We are excited to announce a new major spring 2013 release!

TargetProcess 2.24.0 includes quite a few small and big improvements. Check what we’ve done to help you work better, faster, more comfortably.

10x faster search

We’re close to less than 1 sec search time, but this new search is not only fast. The search results look nicer now, and you can open them both in new tabs and as a pop-up.

Read more  about the new fast search in our Help portal.

Relations and Relations Network Diagram

Quite often one piece of work is “informally” tied to another piece of work. For example, the core team is working on an API, and the other teams are dependent on the core team’s progress. Keeping such informal dependencies in your head can be tiresome…

Starting from 2.24 you can track such informal dependencies right in TargetProcess as Relations!

Relations Network Diagram represents a network of related entities:

Relations can be created for any entity (i.e., user story, task, bug, etc.), and they can be of great help when planning or prioritizing. Be sure to check the article about Relations in TargetProcess Help Portal. It has more screens, and explains how Relations are helpful in planning and progress tracking.

New Visual Reports:Process Control Chart and Lead and Cycle Time Distribution Report

The Process Control Chart shows cycle time distribution for completed entities (user stories, features, bugs, tasks, requests) within a certain time frame. Check the screen below. If you see user stories between the warning limit and the control limit lines, this is suspicious. If a user story goes beyond the control limit line, this is really a bad thing, and you should investigate why it took so many days to complete. These limits are calculated statistically; they depend on the overall distribution of stories in the chart.

You can find a comprehensive description of the Process Control Chart here. It includes more screenshots and some examples on how the chart can be used.

Note the new powerful filters. You’ll see more of them later. They can now be used to filter out the entities in the Relations Network Diagram and in the new visual reports.

While the Process Control Chart quickly spots the entities that took too long to get done, the Lead and Cycle Time Distribution report helps you make realistic forecasts. You can filter out any entities, just as in the Process Control Chart:


Read more about Lead and Cycle Time Distribution Chart.

Improved Views

There’s a bunch of nice improvements to the views: quick convert (all the properties are kept intact for the converted entities), assign 2+ people, switch project quickly in the Info box for an assignment, auto-convert URLs to clickable links.

Besides, starting from TargetProcess 2.24 you can add up to 60 custom fields to any entity.

New History REST API

Check out the  new History API in TargetProcess REST API.
The new API will track state changes for Bugs, Feature, Impediments, Requests, Tasks and User Stories.
More info on the new History API in TargetProcess Help Portal

Flow. Discover Problems and Waste in Kanban – 2 Years Later

Almost 2 years ago I published the Flow. Discover Problems and Waste in Kanban post. The idea was quite simple: visualize the flow of a single user story or bug, and track their life cycle to Done:

You can spot such problems like delays and re-work very fast this way:

Now we’ve brought this idea to life. The Flow chart for every user story, bug and feature will be available shortly in TargetProcess v2.22.9.

The chart gives answers to a whole lot of hands-on questions:

  • For how many days has this User Story been in this particular state?
  • Were there any delays?
  • Was there any re-work?
  • Who was responsible for the User Story?
  • When were Bugs and Tasks added and closed?
  • How much time was spent each day?
  • Were there any impediments?

Let’s look at some examples. The user story on the chart below has been  in Open state for 25 days (it means, in the Backlog). Then it jumped right into In Progress state. Two developers (Alla and Vladimir) started working on it (so it was pair programming). They’d been working for 3 days and then the story was moved into Re-open state. This is quite surprising, most likely they had to switch to something else (no good). Then they got back and spent 15 days working on this user story. That’s way too long. Most likely there were switches as well,  so this should be investigated.

Starting from Oct-18 the progress was very good: development went smooth, tester found several bugs and they were fixed in 2-3 days. Finally, the user story was released to production with no more delays.

You immediately get a high-level overview: delays and up/down state transitions. It is a clear sign of some problems, the systematic ones or not known so far, but we already have some background info to start an investigation).

Let’s check another example. It looks like the user story on the chart below was taken to development right as it was added. That’s true in fact, since it was a customer request to which we reacted immediately. It was implemented in a single day, and there was a small delay before tester took it to the  testing phase. We found quite many bugs and fixed them in 2 days, everything is fine. But then the completed user story was hanging in Release Branch state for 11 days, and that’s no good.

We’re planning to extend this Flow Chart and put more information there (comments, attachments, etc.) The goal is to provide the complete production timeline uncovering hidden malignant patterns and problems. You should be able to get a high-level overview in an instant and dig into as many details as possible.

Our Development Process: 1.5 Years Later

1.5 years ago I wrote a post that described our development process. It is interesting to see what’s changed. 

Legend:

  1. Survived
  2. Removed
  3. Added

Context

  • Product development (quite large web application)
  • Two development teams: Core and Integration Several Mini-teams.
  • Quite mature product: 46 years
  • Technology: C#, ASP.NET We are moving to full Javascript front end, ExtJS jQuery, NHibernate, ESB

Process

  • Development team has all the power to try new practices, keep working practices and remove bad practices. There’s a little no push from Scrum Master side.
  • We form mini-teams for every Epic. A mini-team includes 2-3 developers, tester and designer (half-time). It is fully responsible for Epic implementation. Usually mini-teams live for several months.
  • We do not use iterations.
  • We do use releases, but often it is just a set of all the features implemented last month. We do not plan releases.
  • We do not have iteration planning meeting, but discuss every user story JIT before implementation on a meeting with PO + Testers + Developers.
  • We do not estimate stories and bugs, just a very brief estimate like (“oh, this will be a large story, let’s split it”).
  • We split stories, but sometimes not as aggressively as required. It is still our weak side. Sometimes user stories are too big.
  • We release new builds weekly (sometimes 2 builds per week). The goal is to release every day, but we do not have a good automated functional tests coverage so far. Automated tests have been improved significantly over the last 18 months. We’ve migrated to Jenkins recently and parallelize tests aggressively. Now all the tests run ~40 minutes and test runs on every push.
  • Smoke testing of the build still takes 3 hours (32 testers).
  • We have a limit of 3 user stories or bugs in progress.
  • We have retrospective meetings every 2 weeks. We run JIT meetings instead of retrospectives. We have an Issue Board, limited to 3 items. When the limit is hit, we run a meeting.
  • We sometimes have technical user stories (for example, remove jQuery from application), but that is rare.
  • We use Kanban Board in TargetProcess heavily and have a large screen in development room.
  • We have a flat hierarchy in the company. There are only two layers and no boundaries between the layers. No formalism at all. Also we do not have positions like Senior Developers, Junior Developers, etc. All people are equal (however, there are informal leaders).
  • We track spent/remaining time on tasks and stories.We don’t track time anymore

Development practices

  • We develop each user story or bug in a separate branch. Master is always ready for release (after smoke testing). We use Git.
  • Pair programming for all user stories and complex bug fixes. Sometimes people work alone on smaller/simpler problems. Pair programming is used for complex tasks only.
  • Developers switch pairs every day, even if user story is not completed yet. The goal is to a have fresh look, knowledge spreading and real collective code ownership. We eliminated pair swapping, it appeared to be distracting and annoying for people.
  • Before implementing a story, pairs discuss possible solutions. It is quite formal.
  • TDD. All the new code is covered by unit tests. Situation is worse on JavaScript side, we just started TDD adoption for UI. We have strong JavaScript testing suite now. Also we use BDD as much as possible.
  • We use Cruise Control and have quite many different setups. Smoke builds, release builds, etc. Functional tests run in parallel and it takes 1 hour to run them all. We use Jenkins now and it takes about 40 minutes to have a new build with all the tests completed.
  • Automated functional tests are based on Selenium. We’ve created a framework in C# especially for our application to make tests easier. Tests coverage is not good so far, but we are working on it (there are 2 automation testers in our team).

ToDo

  • We still want to add performance tests into automated builds. We have performance tests running already, but not as a part of new build so far.
  • We want to split user stories better. Much better.
  • We are incorporating User Experience into development process. This is our main goal for the next year. This  includes running a UX community for customers, interactive prototypes for all user stories, usability testing. We are making steady progress here and we have defined our UX process. However, UX injection takes more time than expected. As usual.

5 Reasons Why You Should Stop Estimating User Stories

1. You don’t waste time on estimation

Estimation takes time. Even if you do planning poker and use story points, it still takes time. What do you do to improve estimation accuracy? You gather some data, analyze the data and discuss the results. You are spending time on all that. But are you sure that estimates really provide any value? Most likely not. It is a waste. You’d better spend this time doing something important for the product.

2. You shouldn’t explain to higher managers why it took soooo loooooooong

If you don’t have estimates, you can speak without fear and explain things clearly. You can enumerate problems and explain how they were resolved. You can show that you work really hard and take all the necessary actions to release this story as soon as possible with great quality.

If you have estimates, be ready to hear something like “you screwed up, maaan! You had to deliver this feature till the end of the month! What the f%$k is going on?” as an argument. The team put themselves on a weak side immediately and have to care about deadlines more, not about quality or a better solution. Is it something you really need?

3. You don’t give promises that are hard to keep

You are not relying on people’s optimism (which is built in). People are optimists (almost all of them) and inevitably give optimistic estimates. You can use  complex formulas or very simple rules to have better estimates, but is it really worth it? You can spend many hours defining correct formula for exactly this development team. People will not trust you, since you (rightfully) don’t believe in their estimates. Numbers will look “made up” and still you will have incorrect estimates in the end.

4. You don’t put additional pressure on development team

In some teams Estimate is equal to Deadline. That’s bad. What do you do to meet the estimate? Compromise quality? Tolerate average architectural solution? Abandon polishing? All that leads to poor solutions that end users will not like. It is better (almost always) to spend more time, than planned, and release something really usable and useful.

5. You focus on really important things

Imagine, you have a planning poker session and estimate all stories inside an epic. You sum up all the stories and define it as a 1000 pts epic. You just stick a “fear” label on the epic. People do fear large problems. A 1000pt problem is huge and psychologically it is harder to decide “let’s start it right now”. Product Owner will have a temptation to implement 100 smaller user stories instead of this huge epic. However, it may be a bad decision. This epic may be really important, the morst important thing for the product.

If you don’t estimate it and are not completely sure about its size, you have a better chance to actually start doing it.

Not Just Another Kanban Board

Although he hadn’t heard of the TargetProcess contest beforehand, Robert just had to win the iPad for his autistic daughter. We never imagined that a simple contest on our Facebook page would bring such joy to someone — but we are delighted and touched that it did.

Robert J. Walker from Mindshare Technologies happened to be making some adjustments to their Kanban board, when a colleague of his suggested that he submit his board to the competition. ‘What competition?’ wondered Robert, but went ahead and submitted it anyway — and won. What of the iPad?

TargetProcess: How do you use the board?

Robert: At my company, we wanted to make use of a kanban board as a passive “information radiator,” something we could have up somewhere conspicuous, allowing anyone to walk by and see the status of the user stories and bugs we were working on. Our first pass on this was using a whiteboard and sticky notes. This fulfilled its purpose, but it meant that Jon Grover, our project manager, had to maintain the same items in both TargetProcess and on the whiteboard, which felt like a pretty significant waste of effort for someone who already had plenty to do. TargetProcess comes with a kanban page, of course, but its design is geared more for interactive work at a computer, and not so much as an information radiator.

robert-j-walker-and-the-winning-kanban

Because I found the project interesting, I spent some free time writing what eventually became our kanban board. It periodically queries TargetProcess for the data, so Jon doesn’t have to worry about keeping a separate board up-to-date. Assignable items are shown as little numbered “tickets,” blue for user stories and red for bugs. Effort for those items is shown as a colored “tail” on the item; the longer the tail, the more effort that item has. The boards columns represent phases in the development process, and the effort in each phase is subtotaled at the top. This view is great for seeing the big picture across the entire development department, especially for seeing where bottlenecks are occurring.

It can also be filtered down to a particular iteration or team. In this view, there are fewer items on the screen, so the board allows you to expand the tickets to display the description of each item as well to make better use of the available space. This is great for our individual team stand-up meetings. You can also click on a ticket to view more details, jump directly to that item in TargetProcess, or change its state. We have a new development area with a touch screen in the meeting room, and with a few tweaks I was able to make the board work very nicely with a touch interface.

TargetProcess: How you learned about the contest?

Robert: If I remember right, I had made a couple of changes to the kanban board and was showing them to Jon. He said, “You really should enter this into TargetProcess’s contest.” I didn’t know about the contest, or even that TargetProcess was on Facebook! He filled me in, and he and others on our dev team urged me to enter.

TargetProcess: How do you feel about winning?

Robert: I have a 5-year-old daughter with autism. She’s very bright and my wife has worked very hard with her to help her improve, but she still struggles with communication and social problems. The iPad has been in the news a lot lately as being a remarkable tool in autism therapy. When I heard that the prize for this contest was an iPad, I knew I had to try and win the contest for my little girl. I’m so excited to be able to give this to her and am really looking forward to seeing the things that she can do with it.

robert-j-walker-daughter

One last thing:
Robert does have one request though, one which we are more than happy to help with – April 2 is the fourth annual World Autism Awareness Day. In commemoration, prominent buildings all over the world will be lit in blue light and many people will wear blue to raise awareness of autism and the struggles of those who have it. You can learn more about the event at their web site: http://www.lightitupblue.org.

Once again, thanks to Robert for sending this in, congratulations on winning, and we would love to hear how the iPad works out for you!

Development Practice: The Ultimate Clean Up Day

cleanup_day

We are using Kanban for product development. It works great. As a Product Owner I can re-prioritize backlog anytime and put things into Planned state when I want to. It works. But. Every Product Owner is passionate about the product. He wants to improve so many things and do that RIGHT NOW. Prioritization is really hard and Planned state has a strong tendency for saturation and overflow. For example, our Planned state has limit of 20 items. In the worst case there were 50 items in it (most of them are bugs and small enhancements). Mea Culpa.

The obvious side effect is that most items are buried in Planned state for weeks.  A more important user story pops out and a small enhancement moves down. As a result, enhancements and bugs were never fixed/implemented. Planned column turned into Backlog column someday. That was a real problem. Yes, bugs were small and quite easy to fix, but more important problems constantly pushed them back.

We got tired of this and decided to try a new practice: The Ultimate Clean Up Day. The rules are quite simple:

  1. From the start of the workday everybody who can do programming (including CEO :) take any bug/small story from Planned state and fix it. Repeat till the end of the day.
  2. Someone suggested to draw a star on the whiteboard for each fixed bug and find out who is the best bug fixer.
  3. Testers verify fixed bugs as soon as possible and cooperate with developers to ensure the fix. Repeat till the end of the day.

We were afraid that the clean up day will take longer than 1 day due to re-opened bugs and so on, but this did not happen. It was a very focused activity resulting in 23 fixed bugs in a single day. It was fun and development team decided to do this monthly. Next clean up day will be in the middle of December and I will re-fresh my .NET skills again :)

Targetprocess

Agile Project Management Software

for Scrum or Kanban

Take a Tour!