Wandering off into over-engineered results in a later effort to re-engineer or remove that initial implementation; this activity detracts from the liner iterative progression i.e. the desired progress.
In the same context, any poorly developed feature also pushes into re-engineering, again, pulling away from the linear path.
This is not to say the journey is set, it’s not! The important thing is that there is as little waste, in making progress, as possible; allowing for a shift in direction without having to do too much backtracking to get moving again.
Achieve this by making work items, and in turn iterations, complete vertical sections. Each vertical section and work item confirming to INVEST, making it possible to deliver and test very frequently. Gaining the maximum feedback for the minimum implementation effort.
It’s all too common in the early adoption of agile to focus on the iteration activity; without changing the holistic approach. Specifically, the identification, division and execution of work. The aim of agile is to do as little as possible and get maximum feedback as soon as possible. Output may not be immediately viable, as a product; but will provide enough feedback to confirm or refute the direction taken. When doing this it’s not possible to write all the back end first then do UI. Nor can you start with just data structures. It is this very approach that highlights one of the reasons why waterfall doesn’t work; making all the key decisions about structure and architecture when you know least about the problems you will face.
You have to think of an agile process as evolution through feedback and innovation; rather than a means to a pre-determined goal.
The agile journey for a car would possibly first produce a uni-cycle then a bicycle, a motorcycle, then a quadricycle; before anything close to the car we know, today takes shape.
Focus On The Cheap
In physical materials terms, the evolutionary approach is the more expensive journey, this is why the different approaches exist. Usually, in this context, these are mass production or repeated production tasks. Whereas, In software, because it is “soft”, evolution makes more sense; with much less risk of terminal failure, and the option to fail early or “just get out” if needed.
This is also the point at which the age-old comparisons to buildings and architecture start to fall down; as an agile approach to buildings would be incredibly wasteful and extremely slow going.
Where the cost of a rebuild is high the planning becomes the focus; this is where a waterfall type approach to managing a project works better. However, this does not stand up when working in software, where the cost of change is low. This then makes the feedback and change loop the cheaper way to progress.
Project vs Product
The hard shift required to properly take on agile development, of any technology-based solution, is moving the mindset from “Project” to “Product”. This shift applies regardless of the business focus, be that pure technology, with software as the end product, or something else supported by the software or technology. Projects do still exist in this context, but those technologies that underpin, support or are, the business, are “Products”.
A project by definition has an end, a final goal; this is only ever true with software, or technology when the goal is to remove it. While a business uses a product, and it adds value, it requires maintenance. The team that does that maintenance should persist and evolve with the product. Teams should not get stood down or diminished at some arbitrary delivery point denoted by a project.
Projects will come along that cut across several products, which requires collaboration across the teams owning those products. These projects should never stand up or tare down teams or external resources; instead, they should provide the platform to support the collaboration and feedback for the cross product, and wider project, goals.
The disturbing fact is much of this insight and perspective is not new. Much of the above is based on acknowledgements from over 4 decades ago. The poor adoption of agile is therefore not through ignorance or lack of credentials; simply a bias towards the times when success came through doing it wrong.