Step 5: Evaluate and Iterate

Evaluation is where honesty meets the process. You hold your solutions up against the goals you set in Step 1 and ask: did we get there? If the answer is yes, you move forward. If not, you cycle again. The evaluation step is what makes SiD a learning process rather than a linear plan.

The evaluation step reflects on the activities of each cycle, compares solutions to goals, and determines whether they succeed and whether another cycle is necessary. This step has a different character at each stage of the process. Early on, evaluation may only estimate whether the process is headed in the right direction. In later stages, it becomes an extensive check of details against indicators, edge conditions, and systemic goals.

The Core Questions

Every evaluation asks two questions:

1. Did we achieve our goals with the chosen solutions, within the chosen boundaries? Score the solutions against the goals. Check each indicator. Identify strengths and weaknesses.

1. What are the potential unintended consequences of the solutions on the system and all stakeholders? This second question is equally important. A solution that achieves its stated goal while externalizing harm to another part of the system is not a solution. It is a displacement.

Methods of Evaluation

Evaluation matrix. List all goals and edge conditions. Score each solution from low to high for each criterion. This is especially useful for comparing competing solutions and identifying the strengths and weaknesses of each. It often leads to combining parts of different solutions into something better. Making the matrix for the first time may also surface entirely new criteria that should become part of the goals in a future cycle.

System impact check. With the entire team, formulate pitfalls, possible externalizations, and system dynamics that may influence the solutions positively or negatively. Draw additional system maps that trace the impact of each solution through the system. This is where you catch unintended consequences before they become real.

External evaluation (the Jury). Invite a panel of experts who are not part of the co-creation process but have been kept informed. Present them with a compressed SiD cycle: the goal, the analysis, the understanding, the solutions, and the roadmap in 15 to 20 minutes. Then ask the jury to reflect and criticize. Ask them to be constructive: what concerns do they have, and how would they address them? Record everything. It is usually invaluable information for the next day's work.

During a five-day co-creation session, juries are typically used twice. Once at the end of day one, after two quick method cycles (morning and afternoon), to validate whether the goal and preliminary mapping direction are sound. And once on the final day, when the team walks through the entire project and gets feedback to carry into implementation.

Mid-Session Harvests

Evaluation is not only a formal end-of-cycle activity. In the middle of sessions, regular harvests help check direction, cull excess ideas, and refocus. Gather the group every half day. Recap activities. Re-list goals and the best ideas so far. Write these on a sheet as a collection of "best ideas at the time." This tracks the shifting perspective of the group as the session progresses.

Also evaluate the process itself. Is the mapping sufficient? Is the understanding deep enough? Are the solutions addressing the right level? A rule of thumb: perform at least three full method cycles during any session, whether it lasts one day or five. This means evaluation also happens at least three times.

Iterate or Move On?

The evaluation step determines whether you need another cycle or can proceed to implementation. In early cycles, you often realize that goals should be redefined, additional maps are needed, or you still feel lost in the complexity. If so, continue cycling. If you are confident and enthusiastic about the solutions, if they score well against the goals and the team sees no critical gaps, it is time to move on.

Each cycle goes deeper. The first cycle is broad and rough. The second refines the analysis and begins to converge on solutions. The third tests those solutions against a more detailed understanding and produces a credible roadmap. Later cycles may engage additional stakeholders, incorporate new data, or address aspects that were intentionally deferred in earlier rounds.

The Fibonacci-inspired proportioning from theory applies here as well. Early cycles are short and cover all steps quickly. Later cycles are longer and more thorough. The evaluation step is what calibrates this rhythm. It tells you when to speed up and when to slow down.

Lessons from Practice

One of SiD's founding case studies illustrates why evaluation and stakeholder involvement are inseparable. In a project for a declining desert town in Western Australia, a brilliant systemic solution was designed: using abundant sunlight, cheap land, and saline groundwater to cultivate Spirulina algae, creating a multi-million dollar enterprise. The solution scored perfectly against all goals. But it was developed without involving the community. When presented, the town council dismissed it as a crackpot idea. The technical evaluation was flawless. The process evaluation was a failure.

The lesson: evaluation must assess not only whether the solutions work, but whether the process built the trust, understanding, and shared ownership needed for implementation. A perfect solution that nobody supports is not a solution. It is a fantasy.

A decade later, the same town revisited the feasibility study. The project was still profitable and feasible. The difference was that the community was ready. Timing, trust, and process matter as much as the quality of the analysis.

Setting Up an Evaluation Framework

For larger projects, build an evaluation framework in advance. This might include:

• A scoring matrix linked to the goals and indicators from Step 1
• Scheduled evaluation points tied to project milestones
• Designated external evaluators or jury members
• A personal evaluation component where each participant reflects on the process
• Documentation standards for capturing evaluation outcomes

At the end of each session day, and at the end of each session, ask every participant to personally evaluate the process. This strengthens mutual understanding and provides space for focused dialogue.