Assume variability; preserve options.

—SAFe Lean-Agile Principle #3

 

Set-Based Design

Set-Based Design (SBD) is a practice that keeps requirements and design options flexible for as long as possible during the development process. Instead of choosing a single point solution upfront, SBD identifies and simultaneously explores multiple options, eliminating poorer choices over time. It enhances flexibility in the design process by committing to technical solutions only after validating assumptions, which produces better economic results.

System development can be described as a process of continuously converting uncertainty to knowledge. No matter how well a system is initially defined and designed, real customer needs and technological choices are both uncertain and evolving. Therefore, understanding how a system needs to be implemented must adapt over time.

Details

SBD maintains multiple requirements and design options for a longer period in the development cycle. As the timeline advances, SBD uses empirical data to narrow the focus on the final design option. Through this approach, one can embrace Principle #3, Assume variability; preserve options for as long as possible, providing maximum flexibility.

Conversely, point-based design commits to a set of requirements and a single design strategy too early in the process. It often leads to late discoveries that require substantial rework as the deadline approaches. This may require shortcuts, quality compromises, and—worse—missed program commitments and deadlines. Using the Continuous Delivery Pipeline to provide feedback and learning along with SBD is often the optimal approach.

SBD is an important practice for economic efficiency in Lean product development, described further in references [1] and [2]. Figure 1 shows the conceptual difference between set- and point-based design approaches.

Figure 1. Comparing set-based and point-based design approaches

Increase Economic Efficiency with Set-Based Design

Employing SBD along with the teams, the System Architect/Engineering defines the subsystems and components for the solution and identifies architecture options for each. The teams then explore multiple alternative concepts for each subsystem, filtering out options that provide less economic value, are flawed in some way that cannot meet the targets, or violate laws of physics, etc. [1].

In SAFe, teams explore set-based design alternatives, applying a hypothesis-driven Minimum Viable Product (MVP) approach with a Lean Startup mindset (see Continuous Delivery Pipeline). Design alternatives include hypotheses and assumptions. MVPs may also define experiments to gain the knowledge that allows teams to validate or invalidate those hypotheses, filter alternatives, and arrive at the optimal economic decision.

Figure 2 provides an example of a significant learning Milestone (a deadline for the decision) in which designers of a future autonomous vehicle have to select the technology to support a new initiative that prevents forward collisions.

Figure 2. Example where a future autonomous vehicle has a significant learning Milestone

Figure 2 shows teams exploring alternatives for a new ‘Obstacle Detection System’ (ODS) vehicle subsystem using lidar, radar, and camera technologies. With their corresponding hypothesis statement, they create Enablers that explore cost trade-offs, support for environmental and weather conditions, impact on vehicle design and manufacturing, quality of detection, etc. Teams record the results in the Solution Intent and filter the design alternatives based on validated learning.

Of course, exploring multiple design options comes at a cost to develop and maintain those options, even if they are mostly model or paper-based. (Note: Reinertsen points out that maintaining multiple design options is one form of u-curve optimization, and sometimes the optimum number on the curve is one. [3])

However, if there’s a high degree of innovation, variability, or immovable deadlines (e.g., the crop combine must ship in January), a set-based design may be the best choice. In this case, design efficiency depends on several factors:

  • Flexibility – Preserving a broad set of design options for as long as possible
  • Cost – Minimizing the cost of multiple options through modeling, simulation, and prototyping
  • Speed – Facilitating learning through early and frequent validation of design alternatives

Recommended practices to achieve design efficiency are described next.

Increase Flexibility in Interfaces and Design

Complex systems are built out of subsystems and component elements that collaborate to produce system behavior. Traditionally, specifications are defined early with minimal understanding of what is possible and less concern for design trade-offs. A set-based approach makes trade-off decisions based on validated learning. Final designs and specifications emerge from teams exploring alternatives and understanding the trade-offs.

The System Architect/Engineering function typically defines the connections between subsystems and provides the context for teams to negotiate their own interfaces. In Figure 2, the system has a new ODS component responsible for preventing forward collisions. But teams must determine their own interfaces between the ODS and other subsystems—such as chassis (sensor mounting), powertrain (deceleration), braking, and lighting (brake lights).

However, interfaces are not rigid specifications. Lean allows interfaces to vary at the points of subsystem intersection. At these intersection points, system engineers may specify ranges for negotiation (e.g., what’s possible with an additional allocation of space, weight, or power?) This approach allows system engineers to manage the system-level allocations while teams make their own detailed decisions, creating a collaborative environment for system-level learning, negotiation, and economic decisions.

Modeling, Simulation, and Prototyping

The process of modeling, simulating, and prototyping allows early empirical system validation, and it provides the initial learning points that will help eliminate some design alternatives and confirm others. Model-Based Systems Engineering (MBSE) supports set-based design through a disciplined, comprehensive, and rigorous approach. It incorporates a broad spectrum of modeling techniques specific to the industry and product type, including design thinking and user-centered design. These techniques should be applied to the parts of the system where the risk is highest, significantly reducing the cost of maintaining design alternatives for a longer period of time.

Frequent Integration Points

During development periods when new designs are being explored, uncertainty abounds. Actual knowledge is scarce. The only way to resolve the uncertainty is to test the design through the early and frequent integration of the system components. Integration points are driven in part by System Demos, which occur on a fixed two-week cadence, and by Solution Demos, which typically occur on the longer Program Increment (PI) cadence.

In fact, without this frequent integration, SBD practices may create a false sense of security. They may even increase risk, as it’s possible none of the design alternatives will really meet the identified targets. Frequent integration supports empirical learning with new insights used to reduce options as the system evolves, as Figure 3 illustrates.

Figure 3. Frequent integration provides critical learning points that narrow design alternatives

Take a Systems View

On large systems, decisions may span many initiatives. For example, the ODS technology decision for the autonomous vehicle should consider more than the current initiative to avoid front collisions. System Architect/Engineering owns the technical vision and helps ensure that the system meets future goals. As Figure 4 illustrates, they must guide the teams’ understanding of the larger solution when making significant technology decisions.

Figure 4. Technology decisions must look beyond the current initiative

As mentioned above, a set-based design has a cost. Exploring options can even increase the overall Cost of Delay (CoD). System Architect/Engineering must balance the possibility of over-engineering the solution (e.g. wasteful future-proofing) with being unprepared for near-term capabilities that may incur larger costs and rework ahead. And like every decision in SAFe, how much exploration should be an economic decision.

Use Set-Based Design in Fixed-Schedule Programs

SBD is particularly effective for programs that require a high degree of fixed schedule commitments. Even if some of the more reliable design options don’t provide the degree of innovation or enhanced performance that the system developers would prefer, it makes sense to keep multiple design options present, since the schedule is unmovable. And when the deadline is unyielding, teams must do what they can within the schedule.

Adapting Planning

Explicit and regular planning provides the opportunity for evaluating different design alternatives and directly supports set-based thinking. PI Planning defines the overall intent for the PI, fostering alignment on the constraints and requirements that will govern the design alternatives under consideration. Iteration Planning plays a more tactical role. It allows teams to adjust during PI execution as they learn more from frequently integrating and reviewing increments of value.

Economic Trade-Offs in Set-Based Design

Different design options have different economic implications. So understanding SBD requires knowledge of the macroeconomic goals and benefits of the system. One way to look at this is to place the alternatives on a spectrum, where a certain weight can be associated with each option.

Some of the significant economic indicators may include:

  • Cost of development
  • Cost of manufacturing
  • Performance and reliability
  • Cost of support
  • Development time
  • Technical risks

These indicators help illustrate which design options provide the greatest benefits. For instance, in the earlier collision prevention example, the trade-off between the accuracy of various detection technologies and cost of manufacture can make a big difference, as Figure 5 demonstrates.

Figure 5. A trade-off curve between cost and a performance requirement (error margin) provides guidance for choosing among set-based designs

In summary, up-front commitment to a specific, detailed design can rarely survive contact with empirical evidence. A proper understanding of the economic trade-offs and SBD provides an adaptive approach with a wider systems perspective, better economic choices, and more adaptability to existing constraints.


Learn More

[1] Ward, Allen and Durward Sobek. Lean Process and Product Development. Second edition. Lean Enterprise Institute, Inc., 2014.

[2] Oosterwal, Dantar P. The Lean Machine: How Harley-Davidson Drove Top-Line Growth and Profitability with Revolutionary Lean Product Development. Amacom, 2010.

[3] Reinertsen, Don. Principles of Product Development Flow: Second Generation Lean Product Development. Celeritas Publishing, 2009.

 

Last update: 14 November 2017