9.2 Model Evaluation

Although the concepts of ability, workability, viability and believability are useful for model analysis, often, especially in complicated models, it is difficult to assess these concepts in an ad-hoc manner. In order to help answer analysis questions using i* models in a systematic and consistent way, a qualitative evaluation procedure has been introduced. This procedure uses a set of qualitative evaluation labels that represent the satisfaction or denial level of each element in the model. Initial evaluation labels reflecting an analysis question are placed in the model. These labels are then propagated throughout the model using a combination of set propagation rules and human judgment. The results of this propagation are interpreted in order to answer the analysis question. This process is repeated for additional analysis questions. Often, analysis leads to a modification of the model, when areas of the model which do not convey the intended meaning are discovered and modified. The entire process is intended to be iterative, with cycles of model evaluation and modification until a viable design alternative, or routines for all goals have been found and the model is believable.

This section discusses the main concepts, notation, propagation rules and steps of the i* evaluation procedure. The provided rules are provided only as guidelines that can be adapted by intermediate and advanced users as appropriate in more advanced situations. A running example is used to make the explanation more concrete. The procedure is presented step-by-step:

• Step 1 Decide on an analysis question
• Step 2 Give initial Labels to elements based on the question that you have decided to apply in Step 1
• Step 3 Propagate Label values
• Step 4 Interpret the results
• Step 5 Iteration: Repeat steps 1 to 4 for each analysis question, modify the model if needed.

For each step, general recommendations and guidelines, associated with discussions and illustrations, are provided.
This section is intended only as a summary of the i* evaluation procedure. For additional discussions on using i* models for evaluation, including the general algorithm, convergence, termination, and human judgment see Horkoff (2006), Horkoff (2004), Horkoff and Yu (2005), Horkoff, Yu, and Liu (2004), Horkoff, Yu, and Liu (200), and Horkoff and Yu (2005).



Open Version



For questions regarding i* syntax please click on the "discuss" tab. This feature is available for registered users only. For registration information please see the Registration and User Rights Policy for the i* Wiki.