Many people claim that the software they are writing is “Complex”. For over 20 years, Ralph Stacy has been writing on the subject of complexity; a summary look at his work should be very enlightening. At the root level, a quadrant matrix is produced which yields the following four categorizations:
These are highly generalized, but will suffice for the purposes of this post. Readers are encouraged to look more deeply into the work of Ralph Stacy to gain a deeper understanding.
Software projects in the Simple quadrant are the easiest to address. We have a definitive goal, and know how to achieve it. To steal a quote, “Just To It!” is the general solution.
As the project moves into Complicated, there are two aspects which are common. There may be a need for multiple Simple actions and/or there may be some items which require additional expertise/knowledge. An example of a highly complicated project would be to create an exact functional clone of an existing program (such as Microsoft Word). It would be a huge effort (many Simple actions involved) and probable require significant research into some of the arcane capabilities of the existing product.
Skipping ahead to Chaos, we have entered the Wild West. If a project is in this quadrant there is high risk across multiple fronts. Bringing a measure of order to a chaotic situation is typically required before predictable progress can be made. Future posts will address various aspects in this area.
This leaves us with the remaining quadrant of Complex software. It is likely that a percentage of readers will have realized that their project(s) fall into one of the other three quadrants, but many will find themselves here. The move from heavy sequential processes [often referred to as Waterfall] to highly iterative approaches [Agile, Scrum, …] was driven by projects which fall into this quadrant.
Despite the preponderance of projects in this category, decades of experience has shown that the majority of them are compositions of elements from each of the four quadrants. Another way to say this is “While the project as a whole is Complex, many of the elements are merely Complicated or even Simple, while some of them may be Chaotic“. Decomposing the project along these lines will, in many cases, increase the manageability and improve the predictability for much of the effort, and help isolate the elements which are the least known.
One way to examine this conceptual decomposition is from a retrospective viewpoint. With the power of hindsight, a backwards path from the various artifacts (which are almost invariably simple, when viewed in small enough scope) can be quite revealing as to the evolution through this matrix. This type of retrospective has many other benefits and is the foundation of the “Build, Measure, Learn” paradigm.
Many organizations have come to the realization that the majority of their work is not actually Complex. Instead it is the initial perception of the project as a monolith that created the belief that the work was complex.
Like many (most?) means of handling a situation, trends tend to swing like a pendulum. The historical “Big Requirements Up Front” can be viewed as one end of the arc, which clearly introduced many problems as computers and software evolved. The Agile Manifesto  codified “We Welcome Changing Requirements” [and based on works dating back to 1948 at Lockheed] as a way to address the situation.
But a pendulum will swing past the desired point, and perhaps this has occurred with many software projects and processes. Missing and/or Incorrect information about requirements is distinct from actual changes to requirements. For the truly Complex and Chaotic elements [those which can not be decomposed] missing as well as changing information is unavoidable; but, for the Simple and even Complicated components the missing and incorrect types can and should be largely eliminated.