One of the base recommendations of the Defense Science Board’s paper on Design and Acquisition of Software for Defense Systems from February 2018 is the formation of Agile Software Factories which, as the name implies, are intended to consistently and repeatedly produce reliable, high-quality software applications.
Aside from the questionable assertion of holding Facebook or Google up as being exemplars of software production (both have historically had significant problems in building trusted systems and successfully bringing new products to market, and no demonstrable expertise in enterprise applications), the paper contains a lot of excellent explanations, analyses, and examples of Agile software development.
Kessel Run in particular has been seen as so successful that it was called out as a model to be extended across the Air Force in the 2019 Defense Bill. This has led to a sustained push for expanding Kessel Run and creating program offices to further promote the creation of more software factories.
Architecture is essential.
One of the points highlighted in the DSB report is the need for rigorous architecture and design to ensure that the integrity of the underlying technology framework and information architecture are robust.
This is an essential part of delivering a consistent, stable, and secure set of enterprise applications. (Facebook’s famous informal motto of “move fast and break things” is hopefully not regarded as a good motto for many defense applications).
In practice, defining a comprehensive enterprise architecture within an organization as complex as the Department of Defense is hard to do well and requires a high degree of expertise. The scale of the organization combined with the number of legacy systems is an additional complication. Even Amazon, a highly sophisticated business adept at solving the most complicated logistical problems, had the advantage of starting from a clean slate.
One challenge is that the architecture needs to be defined at the enterprise level, and is therefore part of the infrastructure of product development, rather than part of any individual software development project.
This often makes enterprise architecture harder to consistently fund and maintain, as the cost-benefit analysis for each project tends to ignore the costs to maintain the supporting framework.
Factories produce products.
There is a limitation in the DSB software factory approach. Factories, almost by definition, are very good at producing large numbers of the same - or at least the same type - of products. This is an approach that feasibly works well for discrete apps with limited function sets for a well-defined user persona. However, as a model, it becomes difficult to scale to enterprise systems.
Solutions with diverse, multi-disciplinary user communities that execute complex and interconnected workflows which are underpinned by sophisticated business rules and expansive regulations need a tailored approach.
If you have a production line manufacturing a car, then a washing machine, and then a rocket, you’re going to have a lot of challenges in maintaining quality and consistency. Each product requires a different set of skills, knowledge, and technologies in order to perform as required.
Producing software, while perhaps superficially similar in terms of being composed of code, is also subject to significant differences depending on the scale and complexity of the solution being delivered.
One of the surprising omissions in the DSB paper is the lack of any real emphasis on product management. Unless you’re familiar with the software or manufacturing industries, product management is likely the most important job you’ve never heard of, and likely also the most misunderstood.
The DSB paper describes the product manager as the “administrator of product responsible for the strategy, roadmap, and feature definition for said product or product line.” But this statement expands little on how important that role is in achieving the goals set out in the rest of the paper.
While product managers are mentioned in passing within the context of a specific software project, there is no elaboration on responsibilities besides creating an initial product - such as delivering sustained value to the enterprise.
A product manager needs to understand and coalesce industry trends, technological direction, user needs, and the enterprise mission into a coherent vision for the product (or set of products) under their responsibility. This must include how products will interact and integrate with other products and technologies.
Next, this vision is formulated into a strategy and business case which sets out why, how, and when a product capability is delivered and how success is measured, which then needs to be communicated to stakeholders for validation and support.
This requires a breadth of experience and the type of personality to be successful, which contrasts strongly with the common perception that a product manager simply gathers high level user requirements and puts them on a release roadmap.
Products need a market.
It’s a truism that there’s an enormous gap between writing good software and writing good software that people want to use. In an enterprise environment where system use can be mandated, the usability of a system often gets lost in the drive for conformance to operational policies and procedures. This often undermines both the potential productivity gains as well as the opportunity for innovation.
Enterprise software – even apps – require an understanding of the internal market products will serve. Understanding can come from answering a variety of questions such as:
- Beyond the day-to-day end users, who’s tracking use of the system?
- How is the work done being managed and reported?
- How is success measured?
- How will the application be sustained, and with who’s budget?
- What other systems use or need access to the application data?
- Who “owns” the data, and where does it go?
- What other systems are impacted?
- Are legacy systems being decommissioned, and if so, when?
- Will the application expand to serve multiple user communities?
- Who’s responsible for maintaining the roadmap and adapting it as the operating environment changes?
These are the kind of broader questions and business needs which may be missed when simply focusing on end-user requirements.
For software factories to produce quality products which both meet the needs of the user communities and continue to deliver sustained value to the enterprise, the DoD will need to develop a strong competence in genuine product management to supplement Agile acquisition, software development, and project management.
Liam Speden is RG’s Chief Technology Officer, previously a start-up CEO and formerly a product line manager for Autodesk in San Francisco. He has a background in enterprise architecture and management consulting for technology-based transformation and modernization for commercial and government organizations.