Our Development Process: 50 Months of Evolution

Michael Dubakov, TargetProcess Founder
May, 2012

Our company is almost 6 years old now. It was founded on agile principles and grew up on them. We've used Extreme Programming from day 1, immixed some Scrum ideas later on and switched to Kanban in the end. Here below I've tried to review our process changes for the last 4 years.

Context

We create TargetProcess, quite large web application. Our development team is co-located.

2008200920112012
Company size (people) 15 22 30 40
Team structure One cross-functional development team Two development teams:
Core (5 Developers, Scrum Master, 3 Testers)
Integration (3 Developers, Team Lead, Tester)
Several Mini-teams. We form a mini-team for every Epic. It is fully responsible for Epic implementation. Several Mini-teams (quite clear backend/frontend separation)
Technology C#, ASP.NET, ASP.NET Ajax, NHibernate C#, ASP.NET, ExtJS, NHibernate C#, jQuery, NHibernate, NServiceBus C#, NHibernate, NServiceBus, 100% Javascript front-end, REST, Python

Observations

Single Team → Mini-teams. With company growth we've replaced a single large team with several mini-teams. A mini-team is fully responsible for Epic design and implementation and usually consists of the Product Owner, Designer, 1-3 Developers, 1-2 Testers and the Writer. A mini-team makes all technical decisions, brainstorms ideas and provides solutions. It is very autonomous and focused. Currently we have 6 mini-teams.

Technology. It is obvious that backend part is quite stable. It is still .NET stack. We've added ESB for plugins recently to improve scalability. We rely less and less on NHibernate and move to CQRS pattern. Front-end part is the most fluid. Current UI is an unpleasant mix of ASP.NET Ajax, ExtJS and our own javascript framework. It is quite hard to keep everything top-notch. Our current effort aims UI unification. We will get rid of old technologies and provide completely new UI soon. Here is our technology focus by year:

Process

Orange background - practice change caused significant problems.
Green background - practice change led to nice improvements.
Gray background - no changes in practice

2008200920112012
Iterations weekly none, we switched to Kanban
Tracking and reporting tools Task Board, Iteration burn down, release burn down Kanban Board, Cycle time Kanban Board, Cycle time, Builds board Kanban Board, Team Board, Cycle time, Builds board, live roadmap on the wall
Retrospectives every 2 weeks We run Just In Time meetings instead of periodic meetings. We have an Issue Board, limited to 3 items. When the limit is hit, we run a meeting. We have stop-the-line meetings with people related to the issue. They are quite rare now.
User Stories estimation Planning poker. Estimate user stories in points don't estimate
Time tracking We track spent and remaining time on tasks and user stories. We don't track time
Requirements discussion Iteration plannin meeting User Story kick-start meeting. Discuss every user story JIT with Product Owner + Testers + Developers + Designer.
WIP Limits Time-boxing We have a limit of 3 user stories or bugs in progress. Flexible limits, not defined clearly 2 user stories in WIP per developer (1 is recommended). In general a mini-team decides for itself.
Release planning Formal release planning meeting. Release has ~2 months duration We do not plan releases We create long-term roadmap that shows future vision on a very high level (Epics). The roadmap changes every 3 months.
User Stories splitting User Story should fit 1 week iteration. We split stories, but sometimes not as aggressively as required. It is still our weak side. Sometimes user stories are too big. Some improvements reached, but it's still a very problematic practice.
Definition of Done (DoD) for a User Story Actual spec
Set of passed test cases
Unit tests
Kick-start meeting
Actual spec
Set of passed test cases
End user documentation
Demo to the team
Kick-start meeting
Actual spec
Set of passed test cases
Set of automated test cases
End user documentation
Demo to the team
Kick-start meeting
Actual spec
Set of passed test cases
Set of automated test cases
Green build
Copywriting
End user documentation
Demo to the team
Performance tests
Meetings Release planning (team)
Iteration planning (team)
Iteration demo (team)
Retrospective (team)
Daily (team)
User Story kick-start (3-4 people)
User Story demo (4+ people)
Retrospective (team)
Daily
User Story kick-start
User Story demo
Stop-the-line (team)
Daily
User Story kick-start
User Story demo (before testing)
Stop-the-line (rare, related people)
Daily
Daily meetings Yes, at 10 am, 10 minutes on the average Yes, at 11 am, 15 minutes on the average
UX What? Wireframes Sketches (many ideas), Live prototypes, live usability tests on prototypes, design studio Sketches (many ideas), Live prototypes, Live usability tests on real system
Craftsmanship Whaaaat? Workshops Salary depends on learning
Mini-conferences
A paid major conference visit for every developer
Workshops
Friday shows
5 hours per week to personal education
Salary depends on learning
Mini-conferences
A paid major conference visit for every developer
5 hours per week to personal education or project

Observations

Iterations → Kanban. We've been running iterations for 3 years and then switched to Kanban. This shift was great overall. We aimed at releasing every user story and every bug-fix asap. We cancelled many meetings and replaced them by JIT meetings. Eventually we stopped to track time and estimate work. However, when you remove time-box constraint, you tend not to split the work. This way huge user stories and features try to slip into production. We fight that stubbornly with mixed success.

Points → No Estimates. This shift became possible with Kanban adoption. Clearly, you should estimate work in iterative development. Now we have only horizon-level estimate like "days", "weeks", "month", "months". Sometimes the Product Owner misses the estimates, but with the new Roadmap plans horizon-level estimates got surprisingly accurate.

Time tracking → No time tracking. We tracked spent time from day 1 even when there were just 4 co-founders on the startup. Time tracking was deep inside us and even the salary was based on the hours spent. It was quite a difficult decision to break this rule, but everything went surprisingly smooth. In fact there are almost no reasons to track time if you don't estimate work. Some people complained initially, but in 2-3 months everybody was happy with the decision.

Release Planning → None → Roadmaps. These transitions might seem a bit strange. Initially we did strict release planning, but with Kanban adoption we quit this practice. It appeared to us that it wasn't a very good idea. The problem was that we lost the focus. A release with a clear focus is a very good thing, it helps define what should be done and what shouldn't. Without the focus you do many small and large things not featuring a common goal. That blurs development effort and you release just a set of features. So we decided to use roadmaps instead. There can be 3-4 large Epics in in development. Each Epic has a clear goal and takes 3-12 months to complete. Roadmap just shows all Epic in progress and some future Epics.

User Stories split. The funny thing is that we still don't get it. It is really hard. Yes, we have reached enchancements, but still fail to split large stories consistently. It seems that splitting has some intrinsic complexity that is hard to grasp. Sure we've tried various approaches, but still don't have a feeling that we do it well.

Definition of Done. You see a clear trend — DoD becomes more complex and embraces more items. The process matures and we want to ensure that every User Story is ready for release. The recent additions are copywriting and performance tests. I believe they are really good. There are many facets in each User Story and all of them should be perfect: design, implementation, texts, and end-user documentation. You have to check everything, otherwise there is a chance you might miss something important.

Daily meeting. This practice survived 4 years without modifications. We tried to keep it short and concrete. We can easily run a meeting with 15 people in 15 minutes.

Meetings. The trend is to have less general meetings and more focused ones. The only periodical meeting we have right now is a daily meeting, the rest are on-demand. In Scrum you have quite many formal meetings, which may be good if you've just started agile adoption. With time they tend to deplete. Just-in-time meetings are great: Less people, better focus, better results.

User Experience. That was huge. 4 years ago we didn't even hear about UX. Now it is almost built-in into the company DNA. Many people in the company appreciate usability, design and product perception. It was quite a long process that started from the shift in a product vision. We educate people, attend conferences, run internal workshops, read books, as well as try various practices inside the company. It took us about 2 years to get first good results. Now we are working on a new major release and intermediate results are awesome.

Craftsmanship. Education and professional development were not considered as part of the company culture several years ago. Again, that was a huge shift in 2010. Now we have a learning spirit.

Here are the company focus changes by year:

Development practices

2008200920112012
Source control and branching Subversion. Single-branch. Git. We develop each user story in a separate branch. Master is always ready for release. Git. We are trying a single-branch development again to enable continuous delivery. It is impossible to do that with the feature-branches.
Pair programming Pair programming is required for all user stories. Developers switch pairs every day, even if the user story is not completed yet. The goal is to have a fresh look, knowledge sharing and real collective code ownership. Mandatory for all user stories and complex bug fixes. Sometimes people work alone on minor/simple problems. We eliminated pair swapping, as it appeared to be distracting and annoying to people. Pair programming is used for complex user stories only. Pair programming is completely optional. The mini-team decides for itself.
TDD/BDD TDD is a highly recommended practice. Unit Tests are must have for every user story. All the new code is covered by unit tests. The situation is worse on JavaScript side, we just started TDD adoption for UI. Early attempts to use BDD. We have strong JavaScript testing suite now based on QUnit. Also we use BDD whenever possible. Clear focus on BDD. We’ve customized NBehave for our needs and implemented VS add-in to write BDD scenarios.
Automated Functional Tests Some Selenium tests. Automated functional tests are based on Selenium. We’ve created a framework in C# to simplify tests development. Functional tests run in parallel on 20 virtual servers. Test coverage is much better. We are migrating to Selenium WebDriver.
Continuous Integration Very basic. Just compile source code and run unit tests in Cruise Control. We use Cruise Control and have quite many different setups. Smoke builds, release builds, etc. Functional tests run in parallel and it takes 1 hour to run them all. We use Jenkins now and it takes about 40 minutes to have a new build with all the tests completed Still use Jenkins. The goal is to have a Continuous Delivery process eventually.
Feature toggling None We apply feature toggling and heading to continuous delivery process.
Test Coverage don't measure We have 50% unit tests coverage and 30% functional tests coverage We have 70% unit tests coverage and 40% functional tests coverage

Observations

Single branch → Feature-branches → Single branch. A qurious loop, isn't it? Feature-branches were possible after Kanban adoption. We have utilized them for more than 2 years. Now aim at applying continuous delivery practice and that demands a single-branch strategy. It was hard transition, separate branches are much easier and safer to maintain. For several weeks trunk was red 80% of the time.

Pair Programming. This practice was changed from mandatory to optional. Required pair programming is an obtrusive thing. You have to communicate all day long and stay focused. Well, it sounds good, but in reality it sucks your energy out. Most people can work in this environment for 2-3 months, and after that it gets stressful. Now we use pair programming for very complex tasks and it is up t the mini-team to decide.

TDD → BDD. Test-Driven Development was adopted from the first line of the code. We used it extensively all the way through TargetProcess development. BDD brings more business oriented focus and it is easier to translate BDD scenarios into executable specification. The Product Owner writes most specifications in a Given/When/Then format now.

Some stats

Wrap-up

This review was fun to create. I dug into old blog posts, wiki pages and memos... It's been a long way indeed — many things have been changed and improved. Still our development process is far from perfect. That is OK, as we keep on learning and improving.

Questions?

Want to leave a comment or ask a question?



All articles by Michael Dubakov

2011, September The Future of Agile Software Development
2012, March Patterns for Information Visualization
2012, September Visual Encoding
2013, May Estimates in Software Development. New Frontiers
2013, October Visual Specifications



© 2013 TargetProcess, Inc. Agile project management software.

* Header illustration source