Edge of Chaos | Agile Development Blog

In Part I I’ve described 5 mistakes in agile adoption, this part has 5 more.

6. CST Knows Everything

Certified Scrum Trainer is not a God. Yes, he knows a lot and has a decent experience. There is a high chance that he may be a capable person to help with agile adoption, but only help. He may know nothing about your domain, about your unique situation and about root problems of your organization. If you rely solely on CST and delegate agile adoption to him, it will fail.

Everything should start from the culture and people. It means everything should start with you. You should share agile values and support CST with all the passion and force you have to make agile adoption a success.

7. Functional Departments Should Survive

You are starting a new agile project. Design manager delegated one designer to the team, but refused to re-allocate his desk. QA Manager delegated several testers, but refused to re-allocate them. Developers sit together, but all the other team members are separated by walls. Sounds familiar? Most likely there is no cross-functional team there, and agile adoption will suffer. 

There are absolutely no reasons to keep functional departments and functional teams. People should work together as a team whenever it’s possible. Definitely, when you have testers in US and developers in India, it is hardly possible. But it is just plain dumb to NOT have cross-functional team if everybody sits in the same building.

Cross-functional teams have so many positives, and no negatives. With cross-functional teams you improve communication, reduce functional competition, simplify problem solving, enable ad-hock process improvements and creativity. Again, there is NO reason to work with old traditional functional teams.

8. We Can Live Without Customer Feedback

Agile is mostly about fast feedback. Feedback from customers is the most important. If you build something with wrong methods, that’s bad. But if you build a wrong thing, that’s just terrible. Why? You will have to throw it away later. Regular (and fast!) feedback from customer is like a sailing direction in a bay with reefs. You constantly correct the course based on wind change and other factors. Customer is the most valuable team member and should be treated accordingly.

Extreme programming recommends to have customer on-site. While this is a great tip, it is rarely practical. Still, you have customer available remotely all the time to answer questions and communicate about a product. If you can’t have feedback in a reasonable amount of time, agile methods will not work. You may build a technically perfect product, but yield zero customer’s satisfaction in the end.

9. Self-organization is Easy

Scrum heavily advertises self-organization. Complexity theory has something to say about it as well. Self-organization is based on a set of (simple) rules, non-linearity and interactions between agents (in our case between people). As a Scrum Master you should just set some (widely known) rules, protect the team and expect the team to self-organize. Right? In reality it is not that simple and no, it will not work out. I’ve never seen self-organization working with just a set of rules.Self-organization in a software development team needs more, it needs leadership. I believe that self-organization can’t happen without leadership.

Pure self-organization assumes that a leader will emerge. That’s not happen frequently, in many cases team will stagnate and fluctuate around mediocrity without a leader. Leader sets a vision and pushes team to the right direction. Leader empowers confidence, passion and self-reflection. This leads to self-organization eventually. 

What happens when leader leaves the team? In most cases it falls back or degrades slowly. This is a clear sign that self-organization was not there. True self-organized team will keep its values and progress without a leader.

10. False Goal (e.g. Customers asked us to be agile)

If you have a customer who insists on agile process for his project, you should praise Heaven for this gift. Use this chance as a turning point for agile adoption.

Unfortunately, many companies just try to “emulate” agile adoption with a desire to get this contract. They send some people to CSM courses, purchase an agile project management tool and apply Scrum superficially. They do all that without deep goals and culture change. They do all that without passion. They do all that with a false goal. Almost inevitably there will be the following symptoms:

• Sprints fail. Always.
• No commitment. 
• Scrum Master assigns people to user stories.
• Testing phase in each sprint.
• etc.

The result of “false goal” agile adoption is failure and a long-term disappointment with agile software development.

I encourage you to share another mistakes in agile adoption!

Many interesting posts and discussions about Scrum and Kanban published last weeks. Someone even called this a war :) I don’t think it is a war, but some posts indeed may drive such feeling.

• Ken Schwaber bashed Kanban. “I was told that Kanban is frequently used when an organization cannot readily adopt Scrum. Many of Scrum most difficult aspects are then sidestepped. Managers are still in charge of telling people what to do. People can be interrupted at any time. People are still work in functional silos, preserving the jobs of functional managers. People are not allowed to work in containers, sharing skills and knowledge to bring complexity into solutions – instead they are worked on a pull (more sophisticated than push) production line.”
• Karl Scotland discusses Scrum and Kanban difference as Intentional vs. Implementational approaches. Interesting perspective in fact. Karl thinks that Kanban can be used with Scrum to reveal even more problems in development process.
• David J. Anderson posted the most rational article about Scrum and Kanban difference. I like it a lot. Some phrases are true gems: “Kanban is not a project management or software development lifecycle method. It is an approach to change management – a framework for catalyzing change in an organization.” and “Kanban uses a WIP limit as a change agent and Scrum uses commitments. This is a fundamental difference in approach.” David also posted some reflections later with interesting thoughts about anarchy and science.
• A year ago Tobias Mayer didn’t believe that it is possible to use Kanban at all. “I fundamentally disbelieve that there is any such thing as a “value stream” when you are working in a complex environment, in a creative process, building new products or generating new ideas.”

I think Ken is wrong. His arguments against Lean and Kanban quite ridiculous. Sure there is no intention to treat software development as a factory and apply lean manufacturing principles blindly. Of course there are people who will try (or already tried) that and fail. But vast majority of lean community in software development do understand the difference and working on concrete practices. Lean philosophy is great, but tools and practices can’t be taken from manufacturing directly.

Pull system is more complex than Push? C’mon!

David has wise arguments and perfect position. I think Kanban will be more popular than Scrum in long-term perspective, but it will take time. Visualization is a key thing to manage complexity, and software development is a complex system.

We live in the age of added value. It’s everywhere. Value-added services, value-added products, value-added goods etc. etc. etc.  Actually, so much value has been pumped up in our life, that it’s even strange that this value is not protruding from us like clothes from an overly packed vacation suitcase.

If we take a closer look at the back side of added value, a huge surprise is waiting there. The example I find most notorious is cell phones. What if I want a simplistic cell phone with NO Internet, no camera, even no voice mail, just live calls and SMS?  You’ll never ever find such a phone.

I bet that a phone manufacturer who stops the rush for more new features, would make a fortune in an instant selling the “new frugal” cell phones. In this case, the added value is content which comes from Internet, capabilities to exchange content. Why should someone want a phone without Internet?  My word, very soon we will see such phones on the market. The niche for them is already there. Here’s why:

More content and more various channels to produce and exchange content is now commonly presented as added-value. Hence, a communication device which happens to be a humble phone is supposed to deliver this value. But as we’re oversaturated with content, no buzz is a huge deal. The luxury of focusing on one thing at a time is something that only a few can afford. Besides, it’d be very frugal (frugal is actually the new buzzword :) to buy a phone for a reasonable price, and sell a used iPhone to some geek. Oh, pardon me. It’s all about iPads now :)

There’re plenty of such examples. Another one: the added value of having a car means lack of natural movement, the necessity to pay for gym etc. It actually brings along the whole array of more added value  goods and services that turn out to be not of added value but of less value, since you pay for what you could do naturally.

Take organic products. Now they’re added value. 100 years ago who could have thought that something natural adds value? Now it’s a rollback. What is simple and natural costs more, but has less value compared to the original added value concept for the matter, and the cycle goes on and on.

And we’re nailing it down to our favorite: project management tools. Versatility and too many features now have bounced back to the simplistic Kanban board.

It looks like it’s time to not only practice lean production, but to produce lean products. Gaining focus requires focused tools, one way or another.

When reading this Kill Your To-Do List blog post, I thought that managing personal to-do list can be similar to product backlog management. Not in the part that you should totally kill your product backlog, but in the “one thing at a time” part. This is by no means a new thought.  E.g.  Mary Poppendieck was heard saying that product backlog should be eliminated.

So, as in to-do lists, there’s no point in nurturing and moving product backlog items back and forth.  In this case, a lot depends on what is your definition and understanding of product backlog. I’d say backlog is an environment in which a product grows and exists but not a set of components that should be implemented for sure.  Looking further into, what kind of environment is it? Backlog might be a repository of all the slightest shades of ideas that have a very vague chance to be implemented. A chaotic heap. This heap can hardly be called a “backlog” but rather a by-product of brainstorming. OR a product backlog might represent a careful selection of user stories that will be implemented for sure.

Both of these options, as often is the case, have the optimal state somewhere in between. In our production workflow, we’re now using several buffers – the first layer is raw Requests and Ideas (coming from our HelpDesk, or from the PO, or from the team). The second layer is Product Backlog with User Stories (Requests and Ideas are now groomed and converted to User Stories by  Product Owner who decides that some time these will be implemented for sure – based on how many people requested this feature, based on current product development strategy etc.). The third layer is Planned state for User Stories in Kanban board.

Here’s a quick graphical representation of this product backlog upward funnel:

That’s how User Stories lifecycle looks on  Kanban board, from Planned state on:

Backlog is not seen on Kanban board.  When done with their current user stories, developers  pull new user stories from Planned state.

Previously, when we worked with iterations before switching to Kanban, there was no buffer between inception and implementation – so time-boxed releases and iterations have been planned directly from the backlog. With Kanban, the buffering  is done by moving stories to Planned state and prioritizing them.

I’d say that planning with iterations provides less flexibility than planning with Kanban. As you drop user stories to time-boxed iterations, you  commit to implementing all of them within a given period of time. Kanban is way more flexible since user stories can be pulled one-by-one from Planned state and implemented with no time restrictions. I can’t resist citing an analogy here: it’s the same as moving with the smallest possible steps to posture yourself before hitting the ball in tennis. With large steps, you do not have flexibility. With small steps – you’re very agile and flexible to position yourself for the perfect shot.

So, the buffered Planned state in Kanban is like this breaking down into small steps, instead of taking one giant leap and committing to the whole pack of user stories in iteration.

That’s the way it goes for us. You’re better off moving by small steps, taking it one-by-one (this brings us back to the inspiration blog post reference in the beginning :) than with giant leaps.

“You can’t apply Scrum without an external expert”
“You can’t apply Scrum without a Certified Scrum Master”
“You can’t apply Scrum without XYZ”

You can replace Scrum with any other buzzword. Is it really necessary to have an agile coach on board to join agile camp? Is it really required to send someone to CSM courses and delegate Scrum adoption to this brave knight in a shiny armor? I believe it is not the only way to go, and there are 2 reasons that support my vision:

1. I did not attend any courses, conferences and other events, but I did learn agile and became an agile expert.
2. We implemented agile process in our company without any external help.

To me it looked (and still looks) natural to improve development process internally. It was an ad-hoc agile adoption  in our company. There was no defined structure, but on-going learning. People have been reading books about agile development which I recommended. I’ve been making some presentations about agile history, main processes and so on. People have been reading articles and discussing new ideas. It worked out, finally. Even with such an unstructured approach you are able to set up an agile company. But are there better ways?

Agile Coaches

What’s wrong with external coach/trainer/consultant? Nothing, actually. You pay someone to teach you. You pay someone to inject agile culture into your company. You pay someone to get it faster.

The problem is that this process is so slooow. It is inevitably slow, because all the mindset changes are slow, and you can’t speed them up greatly. My estimate is about 1 year as a minimum (no proof, sorry, just personal experience). You can’t absorb all the agile spirit in a shorter period of time (maybe some geniuses can, but they don’t need coaches anyway). I do believe that a good agile coach can speed up agile adoption, but not as greatly as often advertised.

OK, so changes are slow. It is absolutely required to manage the change over a long period of time. You can’t setup something, run it for a couple of iterations and leave. There is a high chance that development team will degrade and slip back to old-and-oh-so-known practices quite fast. It leads to several possible conclusions:

1. If you hire agile coach, it is better to have about 1 year contract.
2. If you hire agile coach for just 3-6 months, setup internal learning/change process as fast as possible.

The main idea of agile adoption is a paradigm shift. People should change their habits, rituals, working patterns and activities. It is SO hard! Some people just can’t accept it and leave the company. The goal of agile adoption is to change mindset of as many people as possible and let rock-hard minority go.

So why I personally don’t like the idea to hire someone who will teach us agile? It immediately puts our intelligence to question. Are we really not able to learn ourselves? Can’t we read some books, discuss them and try, for example, Scrum in a single dev. team? If we want to hire someone, to me it looks like we’ve already given up and can’t do anything cool without external help. It looks like we got exhausted and now need a power recharge. It is similar to an external CEO hired in a desperate attempt to save company.

Don’t get me wrong. Agile coach can help greatly. But I don’t buy the paramount idea that development team can’t go agile without external help.

Problems and Solutions

I like challenges and like to solve problems without external help. What I like is a full team involvement and participation. It is so cool when people discuss problems and generate solutions. If I ever hire an external consultant, it will be a person who will teach how to solve problems. Problem solving is the MOST IMPORTANT skill (can I stress it more?) for any good developer/tester/designer/you-name-it.

Here I see a major flaw of Scrum as a methodology (so far?). It does provide a framework, but it says nothing about problem solving. OK, you should have a retrospective meeting every other week. You should identify problems, generate ideas and write action items. You should execute action items and retrospect in 2 weeks. Sounds good. But HOW to identify problems? HOW to generate ideas? Scrum says nothing about that, but I believe this should be the core of any agile process.

That is why I pay more and more attention to lean and other problem solving techniques like 5 Whys, TRIZ, etc.

Knowledge From Within

There are various ways to ignite the team and solve problems internally, but you can’t solve problems effectively without knowledge. Study Groups could be the most promising way to educate people. To be honest, we did not try it. As I mentioned, we did not have any structural approach so far, but intuitively came to something similar to Study Groups.

Study Group is a small group of people (4-6) that have cadence sessions e.g. weekly. There’s no distinct leader, they rotate with every session. People should be really interested in the domain. For example, if you have Study Group to learn TDD, all the members of the group should be passionate about TDD learning and actively participate in discussions.

Why I think Study Group should work? First, there’s no Boss inside. People can freely discuss various problems without pretending that they know everything. Second, there is a clear focus. You should not form a group with a goal to solve all the development problems, instead it is better to form groups to study UX, study TDD, study Scrum, study Lean. Once again, Study Group is not a problem solving technique, but it helps to educate people. And it is absolutely clear that educated people will generate better solutions faster.

Here is the nice summary about Study Group. And here is the list of other learning methods that every agile team should pay attention to.

There’s so much room for observations and analogies in the evolution of production trends. Analogies are not merely a candy for the brain. They bring along a deeper understanding of phenomena and ultimately are one of the greatest aides to align (or misalign) with mainstream.

If we look back, to the 18-19th century – mass production was a dream. The philosophy was: produce more.  Lavish architecture designs, garments, gardens – everything created by people was about going more massive and taking much space. Standards of innovation have been changing over the centuries  - what’s been innovative and massive, has been becoming obsolete. Minimalism is the new innovative. Is it because humans subconsciously feel they’ve taken too much terrain and sky on this planet for industrial experiments and now are trying to compensate for that by being minimalistic in everything? Or simply finding ways to fit in?

Hardware/software as well as visual designs and interfaces are meekly following the same trend. This just shows how subtly the “new innovative” standards are taking over. We remember huge PCs. Now we’ve got all kinds of minimalistic devices the names of which start with “i“. We remember waterfall. CMMI standards  with their tons of rigid rules, regulations, documented processes. Now we’re  going “lean” and “agile” – the same minimalistic tendency.

People have managed to stuff the overproduced artifacts not only all over the planet but all over themselves. Fatness is the problem. Again, what a shift in standards –  as late as in the beginning or even middle of the 20th century it was considered trendy to be fat. Well, not obese, but hearty fat. Now, we’re all about lean. Off with plump beauties. Straighten up now, lean is the philosophy of minimalism in production.

P.S. I truly believe that all the fat folks are hiding the “lean” insignia deeply inside them. It just takes some effort to peel off the layers :)

Recently I’ve come across a post on harmfulness of analogies with martial arts. Indeed, there’s a handful of posts comparing software development, or agile adoption, or product development  with  martial arts  - Aikido, Karate, Judo etc.

If you make a direct analogy of software product development and mastering some martial art, it would not be exactly accurate. I guess  people revert to the analogies as they try to project their copies as romantic martial arts disciples into their usual lives as software developers, managers etc.  Perhaps, they’re making the image of their lives more mission-filled this way since there’s not too much space for showing primeval qualities of warrior in software development. But the need to exercise this attitude is still there, as it turns out.

We don’t have beasts to fight, bloody battles and mortal combats. This wrap-up for courage, strong will, persistence in achieving goals and readiness to fight has remained in the past largely. With no wrap-up, people forget that there’s lots of space to exercise these qualities in their usual life.

Let’s go back to analogies. Roger Federer is an unbeatable specimen of mastered elegant performance to me. Look at the way he plays. No waste. He knows, what to do, he knows when to do what. It’s a perfect flow,  the model for effective lean production with no waste and –  the model for perfect warrior  in our modern life. Elegant, no blood on his hands, but he fights, has pitfalls on the way,  stands up, recovers, marches on and wins gracefully.

But he’s just a guy in red T-Shirt. As many software developers :)

Another point is that it’s much harder to fight, win and achieve goals when there’s no immediate physical danger involved as in tennis.  Soo much room for elegant warriors, isn’t it ?

The point I’m trying to make is – let people compare their lives and their jobs to whatever they want. If it inspires and motivates them, makes them feel good about their lives and their work – and makes them feel like warriors, achievers, believers, fighters, winners.

P.S.  February 23 is a perfect day for such a blog post :) Wish you well, guys!

Cumulative Flow diagram is a very good starting point for stop-the-line or retrospective meeting. Here is a real example from TargetProcess development.

You can see in the chart that we had a bottleneck in the beginning of December. It was caused by a quite complex user story. In theory a single user story should not affect cycle time significantly and should not create bottlenecks, but if you violate some rules it might be the case.

The user story was about replacing jQuery javascript framework with ExtJS framework. It took 1 month to implement by 1 developer. During this month we’ve made several releases and all went smooth. Then this user story was verified by testers and all existing acceptance tests were passed. So we’ve made a decision to merge this story to the main code line.

Unfortunately, after the merge quite many bugs were found in the build during smoke testing. These bugs took more than a week to fix, and during this time we were unable to release anything, since the merge was already done and the rollback was quite complex as well.

The lesson learned is to put more effort into testing of user stories that are more complex in nature. This particular story affected many places in the application and usual smoke testing was not enough. So we are going to introduce a new class of service (let’s call it  “technically complex story” so far :) which would mean more in-depth testing and verification before the merge.

In general, Cumulative Flow chart is a valuable tool to analyze historical data, but for emergency bottleneck identification Kanban Board is even better. If you have limits on Kanban Board, you see problems immediately.

I’m digging into Lean Manufacturing and it’s so interesting to learn manufacturing history and its trends. There are so many parallels between manufacturing and software development. Of course they are not direct, but a curious mind can draw them quite easily. Let’s take Cellular Manufacturing as an example.

Cellular Manufacturing is a workspace organization technique. The main principle is to organize production of a single product in a small dedicated unit (cell) that consists of several people and several workstations. The benefits are surprisingly huge:

“The result is very fast throughput. Communication is easy since every operator is close to the others. This improves quality and coordination. Proximity and a common mission enhance teamwork.”

It is a significant improvement over the functional space configuration, when you have large queues and long transition time.

Now, if we think about software development, it is becoming clear why distributed teams have problems and why functional departments will not survive in the near future.

Cellular manufacturing applied to software development directly leads to cross-functional teams that work on a single project. Cell is a single team that has its own inventory and cross-trained people. Expanding it further, it means that developers should be able to do testers’ job, and vice versa.

From lean perspective, managers job is to setup cells configuration efficiently. It is not required to track individual work, but to track cell work instead. It brings process optimization to a higher level and powers system thinking.

It is really amazing how much we can extract from lean manufacturing and adopt this knowledge to software development…

I got this insight on lean and agile techniques in military context when reading  “Ideas Made to Stick” book.  The workflow of the military was described there as an example of how important it is to make messages as concise as possible to accomplish tasks.

The evolution of the military strategy with dates and sources is not a subject of this blog :)
With no extra details, here’s the core of the point I’m trying to make:

First the Army’s approach was to make sure that every single action is planned down to smallest details. But, as they found out, “enemy bears no plans”. An unexpected move could destroy the whole set-up so  “all king’s horses and all king’s men” could not make the Army function again. Effectively, I mean.

So, they reverted to something very similar to agile technique of creating user stories – Commander’s Intent. Commander’s Intent is “the commander’s stated vision which defines the purpose of an operation, the end state with respect to the relationship among the force, the enemy and the terrain; it must enable subordinates to quickly grasp the successful end state and their part in achieving it”.

Do you see the resemblance? It’s a replica of lean production principles, only in military, not in civic, context. And it’s exactly like the agile principle of engaging people and encouraging their creativity to achieve one common goal.

I will not go into quoting sources on the web on Commanders’ Intent and military doctrines to find more analogies and similarities in military setup and lean/agile methodology.  I just outlined the idea.

There’s a misconception related to this topic: Some people think that agile works all OK for lazy folks. It’s all about freedom, about looseness, no obligations. Not at all. Agile is less conventional, it does not care about being in the office at 9 am sharp, or about wearing a tight business suit. But agile projects do have their Commanders’ Intent and they require genuine responsibility and engagement from the team – the soldiers.

Now, as  the Army follows agile principles, would you need any better proof on the effectiveness of agile and lean methodology?