In the corners of every software development and technology conference there are small groups of experienced software practitioners muttering under their breath in frustration as yet another group of inexperienced, charismatic Agile zealots excitedly propound the latest fad technique.
The frustration of the more experienced practitioners is not that what the excitable noobs are proposing is necessarily wrong - Agile has a place for a wide variety of tools and techniques - but that quite often what is presented avoids the parts of Agile which are hard, but necessary.
In Agile, failing to prepare really is preparing to fail.
Agility takes both experience and ability to do well. There’s no easy button to designing and building good software, and anyone suggesting they have one should be treated with suspicion. Agile done poorly can readily produce low-quality, unworkable software, which can be described as like coral: beautiful to look at, but fragile, jagged around the edges, and full of holes.
If doing Agile poorly can easily lead to delivering a low-quality outcome, why is it so common for Agile to be badly executed? The reason is for many inexperienced developers (and customers) it’s fun to jump into “Agile” - whiteboarding the customer experience, running scrums, cranking out concepts - as it feels productive. Words like minimum viable product, customer experience focus, and fail fast get thrown around, often along with start-up mindset and entrepreneurial thinking.
Don’t believe the mythology of back-of-a-napkin designs, hackathons, and caffeine-fueled code binges. 90% of all start-ups fail not because of lack of effort, but because of lack of planning and preparation. Those that survive do so through designing to succeed.
Agile is a design-centric methodology which focuses defining the outcomes and then creating the minimal amount of useful documentation and performing the minimal amount of coding necessary to deliver it. This requires doing the preparation and planning to understand the business, development, and operating environment in which the software is going to be used. An essential part of this is to work out if Agile is in fact a viable approach. Once that’s been determined, there are some proven approaches to establishing a solid foundation for your Agile project.
Define and Design in Sprints.
In this context, the term “sprint” is being used as shorthand for any defined iteration of activity to produce an outcome. The initial solution definition – the scoping, design, and architecture – are well suited to this kind of approach. One of the common objections to planning and definition is that “design and architecture takes too long” and that “Agile eschews documentation” (it doesn’t). This rush to code often means shoddy - or non-existent - software and system architecture, and also misses a key opportunity to really understand the business value and end user.
A typical excuse given by software teams is “we needed to get the MVP out and didn’t have time to do the architecture”. The obvious flaw in this thinking is to be viable, software needs to be functional, stable, and performant. Skimping on architecture makes that difficult to achieve, and re-factoring spaghetti code that’s been hacked together is expensive and time-consuming, especially if the application is unstable and performing poorly, resulting in having to manage unhappy users and owners.
Taking the time to use the first sets of sprints to develop the epics and user stories, define the software and data architecture and integration points, and set the tool sets and coding standards to be followed will set the project team up for success. Involving the business owner, product owner, and end user representatives in this process also helps entrench Agile discipline, set the team culture, and – vitally – build alignment between customer and project team about what success is.
When defining success and building agreement, the four things to do before the first line of code is written are to know your purpose, product, personas, and process.
Know your Purpose.
Understanding the what the software is to achieve from a business perspective shapes the entire project. This includes identifying how success will be measured, and in the case of business software the specific objectives and metrics associated with adoption. The purpose is, essentially, a business case agreed to by the project team.
Not capturing and validating a clear set of objectives means proceeding rapidly with no real sense of direction and makes it harder to deliver high-quality software which achieves business goals.
Agreeing the business goals and capturing the top-level workflows and metrics builds a common understanding between the customer and the development team and is the starting point for prioritization and the definition of what a viable software product is.
Know your Product.
Sketching things on a whiteboard during a “customer experience” workshop is not design. Sometimes it’s not even helpful, especially if what’s being captured is unvalidated opinion.
Opting for customer experience over architecture is like choosing the color to paint a house without ensuring a solid foundation to build it on.
Without a well-considered solution architecture, the team will just be building a mess. The technology stack, data models, integration and interface definitions all need to be thought out in advance for the software to be stable and performant. This shouldn’t necessarily take a long time (and it’s where expertise is highly valuable) but it does need to be done thoroughly.
The “Agile doesn’t need documentation” mindset shows a lack of understanding of the core principle of having the right level of documentation at the right time. In parallel with the architecture, developing the initial Epics and Stories (and ideally first sketches and wireframes) which describe the software can be used to validate the initial priorities with representatives from the business and end-user communities. This starts to shape the product roadmap and establishes the relationship between product team and end user communities right from the start.
Know your Personas.
Developing an understanding the purpose and product also often helps build a more detailed understanding of the customer and user personas. Discussions refining the purpose and product help elicit important factors about the personas which will impact adoption. Do embrace or resist change? Are their objectives consistent or conflicting? Are the end user representatives truly reflecting the full community, or their own agenda and opinions? Do they understand what they want, or do they need educating in the options?
These questions are important as many end users may not really know what they want or have firmly held views which may not be aligned with the product direction and technical realities. Without a clear vision and an understanding of the people and culture of the user community, it becomes harder to deal with pushback against good product ideas which are different, rather than wrong, and maintain a coherent roadmap.
Another risk of poorly defined personas is design churn. If the users aren’t sure of what they want, design and validation sessions blur into a mess of constantly changing and conflicting inputs and requests. If the project team is trusting “the customer knows best”, rather than establishing a rigorous definition and design approach to resolve ambiguities early and define the “definition of done”, the team can spend many development cycles iterating around a poorly defined feature. This kills velocity and can sow frustration within both customer the product team.
Know your Process.
A project can do all the preparation to solidly define its purpose, product, and personas and still fail if an Agile discipline is not established and followed. Agility is a mindset, and Agile is a collection of techniques which can be adapted to fit each project and team. The specific approach, such as Extreme Programming (XP) or a combination or subset of Agile techniques need to be selected. The skills and experience of the team need to be assessed, roles assigned, and coding and documentation standards established. Management tools for the Agile process and artifacts need to be configured. The cadence and controls need to be defined and practiced. Then the discipline needs to be adhered to.
Often, the first thing out the window in a crisis is development discipline and Agile descends into “code like hell”. Rather than stop, assess, and re-baseline – or, as discussed above, move an ambiguous or poorly defined feature back into design – there is often an attempt to iterate out of a problem. “If we just write more code, we can fix it” is the death-knell of quality programming. Frustratingly, this is where quite a lot of Agile projects start, as well as where they end up. “Go slow to go faster” is the maxim of the experienced engineer.
If Agile were easy, everybody would do it and all projects would be successful. The reality is that doing Agile well is hard and takes practice to consistently do well. Failing to spend sufficient time on preparation and planning can make successful Agile development and delivery much harder.
Liam Speden is RG’s Chief Technology Officer, previously a start-up CEO and formerly a product line manager for Autodesk in San Francisco. He has been practicing rapid development software engineering for over 30 years, including applying Agile and its precursors, on projects ranging from 3 to 300+ people.