How to deliver more value faster? I’m constantly thinking about speed in software development. This article is a result of my thoughts. I built a model that describes all obvious and complex dependencies between software development practices and briefly analysed this model. I hope you will find something new and enjoy the read.
“Every single CEO of any IT company wants to build software faster. Time is the most expensive and valuable resource. You can’t waste it on re-work, refactoring, meetings, physical activities. Right? It depends…”
Read the full article
A simple tale of how we created Targetprocess, the visual project management software.
It all started back in 2004. In those days only a few people in Belarus had heard about agile and almost no one actually used it for their work. Iterative development, what the heck is that? Pair programming? TDD? However, at one local software development company where Eugene and I worked, we had a chance to try Extreme Programming for one of our internal projects. As a side effect we needed a software tool to manage such a project. During that time there were several horrible free tools and a few paid ones that were ridiculously expensive. We thought to ourselves: “Why don’t we make our own XP project management tool?” as many programmers at that time must have. For us, this single question turned into the concept of Targetprocess (the very first name was SWIPE — Sprint Web Integrated Process Environment).
, IT lifestyle
, organizational culture
, product development
, tech leadership
I dedicated last 10 years of my life to a single product: Targetprocess. It’s an agile project management software. Company had its highs and lows, product evolved, goals changed. The hardest problem to me as a Product Owner was features prioritization.
We have thousands of ideas: small and large, good and great. Many of them came from customers, many of them were invented inside a company. I always struggled and rarely felt a deep satisfaction about features order in a backlog: “This feature has so many votes, maybe we should do it first?” and then “But we have a cool vision and people don’t see it so far, we should focus and deliver a set of features that support our vision”. These voices took endless battles inside my head and I was quite unhappy. I just couldn’t decide what argument is the best. Now I know the answer — both.
I think there are two major phases in product development: Innovate and Refine. And you, as a Product Owner, should use different mechanisms to prioritize features in these phases.
Innovate phase is fully dedicated to your vision. You do listen to customers and pay attention to their feedback, but in the end you invent a solution that supports your vision and solves many problems in a new, unexpected way that almost nobody can see so far. Let’s say, you want to mix two domains like Project Management and Information Visualization. Nobody asks for that. When you share your ideas you get positive feedback, but many people still ask about a faster horse. You ignore these requests and move forward with your vision.
In this phase you rely on your intuition more.
Refine phase is completely different. Now you silence your ideas and focus on customers requests. Customers are extremely good at small and moderate improvements. They spot all major problems and give your an honest feedback. Definitely, there is no need to implement every single idea — that is a direct road to hell. If you have a service like UserVoice where customers can vote for ideas, most likely you can safely implement top 5% of requests.
In this case you refine the product and polish it based on customers feedback.
It is clear that these phases demand different prioritization schemes. What work for Innovate phase will not work for Refine phase. That is why I had cognitive dissonance, and it took me 10 years to nail the problem and find a solution.
The solution is to mix these phases, like that
You switch focus every 3 months and it makes prioritization a breeze. Next 3 months we are going to innovate? OK, let’s take these two features that will deepen our vision and differentiate us from competitors even more. Refine phase? OK, let’s take these top ten requests from customers, find solutions, get feedback and implement these solutions.
There is no constant battle between innovation and refinement anymore. You cease to compare oranges with apples. You finally feels confident and calm. And everybody wins.
Existing project management tools have several serious flaws. They hide data inside and give the perception that you are in control. You are not. You don’t see the important things and have to dig through endless lists and reports in order to squeeze out the information you need. When you finally find something, it is not easy to change data you need. You have to navigate away, and visit several screens to achieve what you need. What a pain.
The real problem is that all existing project management tools are bad at information visualization.
Many think that information visualization consists of fancy infographics, dashboards, and reports. That is far from true. There are many definitions, but I like this one:
Information visualization utilizes computer graphics and interaction to assist humans in solving problems. [Purchase et al., 2008, p. 58]
In a nutshell, you should be able to extract the data you want, present it any way you want, and manipulate it right away any way you want.
Problem #1. Find
It is quite easy to add data into software: projects, tasks, people, plans. It is not as easy to get this data back. Quite often you have limited filtering options and can’t narrow down lists, for example, of user stories in order to see just user stories you want. So you have to dig through the list, change pages to find a single story, do something with it, and repeat.
What if you can extract anything without limitations? Zoom and focus on things? Find exactly what you need?
Problem #2. See
Even if you can extract something, you can’t always see information in a way you want. Most tools rely on lists heavily and provide little options to see things better. Lists, some predefined boards, some predefined reports — that is all.
What if you can switch between different representations in a single click? What if you can see information in a one dimensional list, a two dimensional board, or a timeline with a single click?
Problem #3. Change
Even if some tools do provide visualizations, they are static. You can see a report, but can’t do anything with the data. You can see a timeline, but rarely can change it right away.
What if you could see something important and change data right there?
If you take any project management tool, you’ll see that it doesn’t follow information visualization principles. Everything is fragmented. Lists are here, boards are there, timelines are rare and often are not interactive. There are many limitations with finding exact data you need. There are even more limitations with changing data.
Targetprocess 3 was built to solve these problems in the roots in project management domain.
Targetprocess 3 is a tool that has information visualization principles in its core. It removes many of the limitations that other tools have and gives you the freedom to handle data the way you want.
Solution #1. Find
In Targetprocess 3 you can find anything using powerful filters. There is almost zero limitations and you can create very clever filters. When you have no limitations, you can extract exactly what you want and narrow down the data to see.
Solution #2. See
In Targetprocess 3 you can select data and quickly switch between various views in a single click: Boards represent two-dimensional views, Lists show hierarchies and Timelines show progress of things.
You can create any board in a minute: Kanban Board, Task Board, Work by Person, Roadmap and many, many others. The Board UI handles huge data easily via collapsing, focusing and zooming.
Ben Shneiderman stated a Visualization mantra:
Overview first, zoom and filter, then details-on-demand.
Views in Targetprocess take this mantra seriously. You can see the whole picture on board, zoom and filter when you want, and dig into details when you need.
Sometimes you need to work with hierarchical data. For example, product backlog. Lists in Targetprocess 3 are really good at it. You can use filters and flexible settings to see exactly what you need on all levels, update anything in a snap.
Most tools can’t give you a sense of Time. Any data in Targetprocess can be seen on a Timeline. Progress tracking on all levels is a breeze. You can track releases and iterations, see people’s workload, spot long tasks and delays, discover unexpected patterns in your development flow. Moreover, you can create projects portfolios, product roadmaps, plan releases and iterations. Visually.
Solution #3. Change
When you see something, you often want to change that. Targetprocess 3 helps you add entities quickly, select any entities you want and manipulate them via batch drag and drop. Lists provide beautiful experience in adding and editing data. Timelines enable visual manipulations to create roadmaps, plan releases and iterations.
In fact, Targetprocess 3 is a domain-specific visualization software. It focuses on project management domain and provides not static reports, but interactive visualizations. And that makes a big difference. Now information is at your fingertips and you can really find it, see it and change it on the spot.
Targetprocess is free for teams of 5 people. Go see for yourself.
It is really, really hard to design getting started experience in a tool. We’ve had many iterations and improvements, but still are not happy with the current implementation. So we’ve started from scratch and want to improve the existing experience in Targetprocess.
It is always interesting to explore how other vendors solve similar problems. This post is a quick summary of getting started experience in 15 (mostly project management) tools.
Salesforce shows you a Getting Started page with several videos. This is it. Videos are quite lengthy and you have to spend about 30 minutes on watching them. It is interesting that such a sophisticated tool like Salesforce solves getting started problem with videos only.
Trello is a super-lightweight tool for project collaboration. It’s based on an idea to create many boards and manage them. First board is actually a getting started board where you will read about all the features. It cleverly has 3 columns: Basic, Intermediate and Advanced features. If you open all these cards and actually do what they say you have a good understanding of a tool and know how to use it. Great implementation.
Asana is similar to Trello, but much more complex (I personally don’t like it). Getting started experience is more detailed as well. First, you see a short video that actually explains a little. Video is on a separate page and it’s hard to miss it. Then you have a 3-steps wizard to define your identity, invite team members and create a team. It is quite handy and well designed. Finally, you dive into the tool. UI is loaded with many buttons and panels. The only help you have is “Videos” section at the bottom. Still Asana has a style and looks modern.
Plan.io meets you with a wizard that guides you through the system. I liked it initially, but in a minute I got frustrated with too many steps. Then it became really annoying. Huge texts in popovers are not welcome to read, so I skipped them and learnt a little. And “Something is still missing” message drove me nuts in the end.
Clarizen is a feature-rich project management tool. Initial wizard is OK, but I have had some “Hmm…” moments during project type selection. Then you see another wizard that shows you a 6-min video and obvious navigation panel. The tool looks heavy, with many controls and screens. It doesn’t feel hard to grasp though.
Basecamp is a famous light-weight coordination software. It’s very simple. First, you see a bright welcome message. There are similar bright messages in new areas you visit, which is good. Sample project describes real features in Basecamp, so you can explore UI and learn something about the tool at the same time. Interestingly, I like Trello’s implementation of this idea more. Not sure why.
Before you start, you have to lie about your phone number (I’ll never understand why this field is required). Then you see a several steps wizard. While some steps make sense, I don’t think that “Assign task via email” is so important to have here. It is not so clear how to complete this Welcome Quest, I had to think a bit to find out why some steps are still there. Also Skip link is hard to find. Maybe it is intentional, but such solution has its flaws. There are several areas on top and when you put the mouse over an area icon you see a popover with a video. Quite a clever idea, but the implementation is annoying. Maybe it is good to hide these videos at some point.
In this traditional project management tool you see just 2 help messages after login — and this is it. Minimal effort in getting started design. I am not convinced the tool is simple enough to handle the getting started experience that way.
You immediately see a news feed with product updates. Maybe it is better to show a feed with product usage tips. So there is no getting started design at all.
The welcome message you see is irrelevant and too lengthy. Similarly to Trello, there is a ToDo list that describes the tool itself. Empty areas helps you to make a correct action, like add a Task. System shows you good tips in the right moments. I’d say getting started in Redbooth is well-designed and usability is great.
TeamLab looks complex. Right after the registration you see a video and wait while your account is being created. Good use of waiting time! Then you start a project and see three(!) yellow messages right away: one about Android app, another about email activation and the last one is a getting started wizard. Too many for me. The wizard is very short with just four steps. The system gives prompts to some actions when you navigate to a new screen, which is very handy. Help is built-in, so you don’t navigate away when you want to learn something. Quite good.
This is an agile project management software. It meets you with a video (too small) and a project setup wizard. UI is not stylish for sure — too many colors, various icons and overall impression is not great. Large bright buttons with actions in empty areas are good though. The tool was easy to grasp, but I’m a domain expert :)
Jira’s getting started experience is similar to Salesforce — it is very basic. You have a dashboard component with Getting Started steps. Small wizards are nice. Service Desk area is different and shows you quite many popovers. I got lost with them, in fact.
This agile project management tool shows you a 6-step wizard that describes main UI areas. That is it.
Targetprocess is not an easy tool. It is really flexible and has a new UI-paradigm, so we wanted to show people how to use it properly. First, you select a process to start with. Then you see a 3-minute video that explains a concept and main ideas. Then you jump into a wizard. We put significant effort into the wizard and I should say it is the most clever implementation I’ve ever seen. But it seems the wizard is very long with too many actions and steps. We have stats here. Around 70% of people abandon the wizard after step 2. It means 70% of people learn nothing from it. And even the pulsating blue dot doesn’t help to grab their attention.
Let’s summarize the most popular approaches to getting started. Most show a video and provide easy access to more videos about a product (Asana, Yodiz, Targetprocess). Several tools use the tool itself to design ToDo with getting started actions (Trello, Basecamp, Asana, RedBooth). Some tools have UI wizards (TeamPulse, Targetprocess, Clarizen). RedBooth tries to give guidance to users in a good way, I think. Nothing novel, for sure.
I personally think that a long wizard doesn’t work and most people just skip it. Also it is required to unblock as many paths as possible in user interactions. If a tool demands something at some point of time, there should be an obvious way to fulfil that demand. For example, if it’s not possible to add a user story without a project, there should be a clear way to add a new project right from the “Add user story” screen.
Today I want to discuss a topic, that many of us thought about for sure. I want to talk about interest and fun at work.
An Interesting Task
What does “interesting” task mean? People are different and interesting tasks are different, but it is possible to define some common properties. Maybe you can provide many properties, but I think only two are particularly important.
An interesting task is usually challenging. It is close to the edge of your skills. You feel that you can crack it, but don’t know how exactly so far.
An interesting task should give you a new knowledge. If you solve the problem and learn nothing, it doesn’t feel very cool. When you solve the problem and learn some new concepts, then you feel much better in the end.
So interesting task is challenging and generates knowledge.
However, there is another thing — developer’s experience.
They almost always work on interesting tasks. Any task provides some challenge, any task generates new knowledge.
Initial enthusiasm is gone and now it is a serious job with less and less fun. If the fun level drops below a certain edge, you are in trouble. It means you learn nothing. If you learn nothing, you will become a “zombie developer” eventually. A zombie developer goes to work and solves similar problems again and again.
No real thinking, no real challenges, more brainless activities like meetings, small talks (not the language), status reporting, hacker news, Facebook, Twitter, chats, death. True zombies don’t like interesting tasks. They like familiar tasks that demand minimal effort to complete. They work on the lowest energetic level possible.
The day when you say to yourself “I didn’t do any interesting task for weeks” should rise a red flag in your head. Go and fix that. Talk to managers, change position, change a company. There should always be a healthy mix of interesting and familiar tasks. Fight for that.
After 10 years of professional programming they accumulated a good backpack of knowledge, experience and sarcasm. There are not so many tasks they find interesting anymore.
Some give up and just do their job. They say: “We are professionals and solve problems not for fun, but for the value to the people. That’s what we do”. They try to re-frame a problem and replace a real solution with psychological one. Sometimes that works, but often not so well. These people are unhappy. They do the job well, but you should not expect any enthusiasm, initiative and courage. Create one more 3-layers enterprise solution? OK. Build another one social network? OK. Zombie Developer Level 80.
Some try to find a way to the mastery, solving the same problems better and better. For example, writing compilers for 20 years. It is an honourable road only a few can ride. You should be very determined, patient and calm. These qualities are rare in the Western culture.
Some push forward to the limit to work on really challenging problems. They change jobs, change countries, change industries and technologies. They jump from problem to problem, developing astonishingly wide skill sets. Definitely, they don’t have depth in all the areas, but they do provide unusual point of views and can attack problems from surprising angles.
Any good company should provide enough interesting tasks. It is a serious mistake to believe that people will do everything with equal passion. They can, but only if the balance of interesting/boring tasks is good enough.
Most likely, every company should have people with all levels of experience. Junior developers will be happy to work on tasks that veterans treat as trivial. Experienced developers will be happy to work on tasks that veterans treat as OK. And you need veterans to storm really challenging problems (if you have any).
I personally thought that junior developers were not required in our company. I was wrong. It leads to a situation when veterans have to work on simple and trivial tasks more and more often. Fun level drops. Productivity drops. Danger to lose veterans increases.
Which distribution of junior, experienced and veteran developers a company should have? It depends on problems you have. There are 25 developers in our company: 19 experienced and 6 veterans. So our current distribution is 0-75-25%. I think a more healthy distribution would be 25-50-25%. What we are going to do is to hire 6 junior developers to have 20-60-20% distribution so far.
Here is the distribution for various types of companies. X – developers expertise, Y – % of developers in a company:
As you see, currently Targetprocess team is skewed into the “scientific projects” type, while it is certainly not scientific so far. We should move back into the product development distribution.
- Almost every task is interesting for junior developers.
- Experienced developers should maintain a healthy balance between interesting and OK tasks.
- Veterans have two viable roads: sharpen their limited set of skills to the mastery or attack new problems all the time and become a smart generalist.
- Company should care about “fun level” and have the experience distribution that fits existing problems.
Routine Pros and Passion Cons
Continuous Problem-Solving Is No Accident
Cognitive Endurance Basics for Software Developers
It’s excruciatingly boring to write specifications. It seems like it’s the most boring thing in the world that a product owner has to do. This could be the reason why most specifications are horrible and come across as the root cause of delays, reworks, and bugs.
This problem can be partly alleviated if a product owner is always available to talk to, but even that is not a cure-it-all remedy.
The agile movement has its peculiar point of view on specifications. The most extreme part of agilists express their feelings quite clearly:
F%$k the specifications!
There are two extreme views on requirements documentation (most use user stories to describe requirements these days):
- User story is just an invitation to communication. It should be very short.
- We should document requirements fully, with all known details.
Most likely, you accept option 1 and reject option 2. If you ask me, what option I prefer, I will answer “both”. Hold on your tomatoes and read on.
Let’s say, you start a project and have a backlog with 100 user stories. This is the stage where you should keep user stories descriptions short. Don’t waste time, many of these stories will never make it into production, they will die.
You pick a user story and ready to start development. At this moment you have a solution and this solution should be documented. Short description is not enough, since some details may be lost and implemented incorrectly, this will lead to re-work and time waste. At this stage you shouldn’t worry that detailed description of a user story is a waste, this story is going to development phase anyway.
Simply speaking, each user story has a life-cycle: Birth → Solution → Implementation → Release. Let’s review it:
Someone has an idea and user story is added into a backlog. Just a short description is fine in this case.
There are various methods to create a solution for the problem that user story describes. Maybe you have a special UX phase and special UX teams that brainstorm possible solutions. Maybe you discuss and invent solution in development team just before implementation. Maybe Product Owner creates a solution alone (worst case). Still somehow you have a solution.
Then you should document the solution with quite many details to make sure that solution will not be misinterpreted. Here you will argue that live communication is better than specification. I agree and will follow on below, but if you have a gap between Solution and Implementation phases (which is often the case), you have a good chance to forget many things about the solution. You will end up with sub-cycle: Forget → Remember → Lost Some Details → Re-Work. This sub-cycle generates nothing but waste.
Implementation may take several hours, or may take several weeks (happens). If a solution was invented outside a development team, it’s extremely important to have a kick-start meeting, discuss the solution again and find all weaknesses in it. It may happen that solution is unideal and can be improved. Kick-start meeting brings everyone on the same page and every team member understands how it should work.
Right after the kick-start meeting user story should have as much details as possible, including positive and negative flows, texts and labels, final design, etc. Memory is a good thing, but it fails quite often. Don’t rely on your own memory and on memory of other people, put everything you know into text and images.
When user story is done, there is not much value in documentation. If there are some minor discrepancies — that is not a problem. Sure, it can be used as a reference for technical writer, for example, so, depending on your context, you may want to have it corrected.
The main idea is that importance of documentation is low initially, highest in the implementation phase and falls down again after the release. Full documentation just in time is a good practice.
You may worry that it sounds heavy, but in real life a user story can pass all these phases in a single day. For such quick stories full documentation looks like an overhead, but usually it’s very short as well and may take 10 minutes to write.
I personally like when user story is fully documented (at the right time!) with all important cases, has final design attached, has all labels nailed and provides a good vision about the solution.
With many user stories you have rolling waves. Like that:
Now fish out your tomatoes and throw as many as you want.
How to use ideas from the great book by Daniel Kahneman Thinking, Fast and Slow.
Recently I read a fascinating book by Daniel Kahneman Thinking, Fast and Slow. It has tons of insights. Every chapter was a discovery. I learned so many new things.
I work in a software development company. It’s quite natural to apply new learned things to your domain. That’s what I’m doing in this post.
System 1 and System 2
The book is about two systems in the human brain. System 1 is fast, intuitive, alert and cheap. System 2 is lazy, analytical and expensive. Most daily activities and tasks are solved by System 1, while the most complex tasks are redirected to System 2.
When you use System 1 you feel nothing. It’s natural and effortless. When you use System 2, you feel strain and pressure. You notice that you are really thinking hard. System 2 is effortful.
There are many experiments that proves existence of these two systems. Make no mistake, there is no clear separation of these systems on a physical brain structure, but they describe how people think and make decisions.
If you are a software engineer, you’ll immediately grasp the nature of System 1 and System 2. System 1 is quite similar to cache, while System 2 is a business layer. Cache is cheap and fast, business opearation is slow and expensive.
Now I’ll take first concepts from the book and find examples in software development.
What You See Is All There Is (WYSIATI)
People usually don’t think about things they don’t see (in a broad sense). If you have some information, it’s unlikely that you’ll immediately question it and go look for some alternatives. Quite often people make decisions based on existing facts and limited evidence. This leads to several biases:
Estimation is not easy in software development. We make huge mistakes all the time. Estimation is a distribution.
Software estimation is the area where overconfidence flourishes.
“We need twitter integration — you ask — We want to accumulate all tweets with specific tags. When you can complete this?” Developers often don’t ask additional questions and just provide quick estimate like, well, 2 days. This is System 1 in action. It’s extremely unlikely the answer is correct. Lack of information somehow doesn’t impede immediate estimate. Developers should be smart and confident, oh yes! But also they should escape WYSIATI and think carefully about unknowns.
Problem wording does matter. If you describe the same problem using different words you can have different solutions or preferences. Consider two questions:
- “Do you think we need a very experienced developer to solve this simple problem?”
- “Do you think we need a very experienced developer to solve this complex problem?”
The question set the frame. It’s hard to answer “Yes” to the first question, it’s easy to answer “Yes” to the second one.
Some events are more likely than others. However, people have no good intuition about statistics.
Let’s say, you create a product that used by 1000 companies. You receive an email from a customer who think that his idea is really “must have” and should be implemented ASAP. The argumentation is perfect and does make real sense. You feel his pain and push this new feature to production with all your force. Well, you missed one thing: it may very well happen that 999 other customers have no need in this solution. You forget that customers base is large and a single request should be reviewed from that perspective.
It’s a very common mistake and I repeated it again and again myself. Now I know better and think very carefully about every request. No rush. In rush you delegate the decision to System 1, and this system do you no good in such a complex domain.
Answering an easier question
When we encounter a hard question, we tend to replace it in our mind with an easy one, that System 1 can handle. Here is an example.
You hear the question “How good will be progress on this project 3 months from now?” You replace this tricky question with “How good is progress right now?”
By doing this substitution, you don’t awake your lazy System 2 and allow System 1 to answer the easier question. We follow the path of least effort. The funny thing is that we don’t realize the substitution, we really think we answered the hard question!
If we like something or dislike something it affects our decisions greatly. For example, if you like node.js, you will be optimistic about choosing node.js for the new web application. If you hate node.js, you will provide so many arguments against it and will fight to death to apply another technology.
The affect heuristic is hard to overcome. People are not so rational, but clearly some reasonable comparison of technologies will be much better than a quick choice based on intuitive preferences.
I just scratches the surface, but it’s already obvious that our decisions in software development based on many biases and bad heuristics. What can we do about it?
Intuition vs. Models
Kahneman was very skeptical about our abilities to wake up System 2 when it’s really needed. Indeed, how can we change a human nature?
However, it seems we can do something. We can apply models.
Kano Model is a good example. It helps to make more intelligent decisions about new features.
Model is almost always better than intuition. Even a very simple linear model is better than most (all?) experts in a domain. Model forces you to think about the domain, about various facets of the problem, look at it from different angles — use System 2.
Here is the simple question: What feature should we start next? You can rely on your intuition and say “Advanced Search”. It may happen that this feature is not so important and there are dozen of more important features. You can build a simple model and evaluate your backlog. For example, the model can have parameters like:
- Kano model (basic, delighter, performance)
- Votes: How many customers and leads requested it
- Frequency: how often people will use this feature.
- Spread: how many people will use the feature.
- Effort (S, M, L, XL, …)
- Complexity. Will it increase usage complexity
With this simple model you’ll make much better decision for sure.
I believe models can be applied to many areas in software development. Decision making and problem solving should rarely rely on intuition, unless you are Steve Jobs.
There’s more and more buzz around estimates and #noestimates in software development. People like to write bold statements and go extreme about things in blogs. Usually, personal dialogues are much more balanced. Some hate estimates and believe it’s a useless activity. Some defend it with arguments of arguable truth.
I want to dig into intrinsic estimates complications, what people mean by “estimate” and what future directions we may attack → Read the article.