Complexity science is very young. Cybernetics, ecology, sociology, meteorology all study complexity. In general, a complex system consists of interacting components. The result of this interaction can't be predicted by observing an individual component. For example, human brain consists of neurons. The brain has 'memory', while each neuron doesn't. Ants’ colony behavior can’t be predicted from the behavior of an individual ant. And it is impossible to understand software development process looking at how just one developer works.
We are especially interested in a particular case of complex systems called Complex Adaptive Systems (CAS). The main difference is that CAS may learn and change (adapt) over time based on previous experience. CAS remember the history, and that is the main difference from chaotic systems.
There is no common definition of Complex Adaptive System (CAS). One of the most popular definition was offered by John H. Holland
CAS has quite many common properties. However, hardly a system should have all of them to be CAS. Moreover, researches define different sets of CAS properties. Let's try to review the most common properties and see how software development process can be described in terms of those properties.
|CAS Property||Software Development interpretation|
|People in development team and others involved (product owners, stakeholders).|
Each agent in CAS reacts to information flows. In return, the information that this agent will receive in the future is influenced by his actions. This feedback may strengthen or weaken agent actions.
|Development team relies on previous experience (previous project, previous iteration) for decision making. Decisions influence future actions.|
Emergence is the way complex systems and patterns arise out of a multiplicity of relatively simple interactions. Small actions of agents lead to unexpected emergent system behavior and it is impossible to predict system behavior based on the behavior of an individual agent.
|Small errors pile up and may cause huge problems, even stop project at all. |
Each person in a development team does not have all info about the project.
People communication and collaboration leads to unexpected results.
Feedback and emergence cause non-linear system behavior.
|Will a project success or fail? You never know from the start. Predictability in software development is hard (almost impossible in many cases). So, you can't create a big plan upfront, you can't create a big design upfront. Non linearity is a natural property of CAS, so all that BDUF, BPUF are just plain wrong attempts.|
|Sensitivity to initial conditions|
Also known as Butterfly Effect. . Even one small change may lead to significant effect. Again it plays against predictability, you can't predict weather for more than 2 weeks. Butterfly effect is a property of chaotic systems, so CAS is a chaotic system as well.
|A small bug in the code may cause huge problems to the end user. Unavailability of a key project person on some particular day may cause major slippage because a wrong decision was made while this person was away. I bet you have such examples from real life.|
CAS is very flexible and adaptable. It can keep its basic structures and behavior, reacting to changes in external environment and taking advantages of these changes.
|Development team should react to environment changes (new requirement, technology change, lead developer loss, etc). It should adapt to the new conditions and try to take advantage even from the worst events.|
Agents exchange information.
|People communicate in development team and outside development team (stakeholders, product owners, etc).|
Agents in CAS cooperate to reach the goal.
|People in development team cooperate to produce the software. BTW, game theory studies cooperation and may be applied to software development. That was already done by Alistair Cockburn.|
Agents may have specialties. Strong CASs have a number of various agent types. This provides more opportunities to use environment features and to adapt to them.
|Obviously, we have Developers, Testers, Scrum Masters, Product owners, System Administrators, etc. If a team consists of developers only, it will be less efficient than cross-functional team.|
There's no hierarchy of command and control in a complex adaptive system. There's no planning or managing, but there's a constant re-organizing to find the best fit to the environment. The system is continually self-organizing through the process of emergence and feedback.
|Traditionally, there is hierarchy in software development. There is a Project Manager who rules the project, there are Functional Managers, there is VP of Development, etc. In agile development there is no hierarchy. The goal is to have self-organizing team.|
|Edge of Chaos|
A system in equilibrium does not have the internal dynamics that enables it to respond to the environment and will slowly (or quickly) die. A system in chaos stops functioning as a system. The most productive state to be in is at the edge of chaos where there is maximum variety and creativity, leading to new possibilities.
|It is hard to create software in Order. Development team will not have enough creativity to resolve unexpected and complex problems. That's why heavy methodologies do not fit software development. It is impossible to create software in Chaos. We all know which results hack & fix brings. Ideally we should produce software using a relatively light methodology (agile?). A set of simple rules allows edge of chaos and powers creativity, flexibility and success.|
This is just a quick overview of common CAS properties. Later we will go into details and draw more parallels between CAS and software development. But even in this short list we've got some nice phrases like self-organization, feedback, information exchange, cooperation, adaptation - does it remind you something?
- Part 1. Agile Software Development and Complex Adaptive Systems: Intro
- Part 2. Software Development is Complex Adaptive System. No Doubt
- Part 3. Edge of Chaos and Hyper Productive Software Development Teams
- Part 4. Simple Rules, Complex Systems and Software Development