Examination Answers
Forecasting Using Structured Analogies
1. What is an analogy for the purpose of forecasting with structured analogies?
Forecasting with structured analogies requires the identification of real situations from the past that resemble the situation about which forecasts are required—the target situation.
2. Why does the structured analogies method tend to provide forecasts that are more accurate than forecasts obtained from experts who are using their unaided judgment?
Firstly, analogies provide information about real human behavior in situations similar to the target situation. Such information can often provide the most realistic representation of the target situation. This is particularly likely to be the case when the target situation is complex and causal relationships are not well understood. Secondly, structured analogies is a method for structuring expert judgment and, as research on forecasting has shown, structured approaches tend to provide forecasts that are more accurate than those that are provided by unstructured ones. See also the answer to question 10 about the effect on accuracy of combining the forecasts of independent experts.
3. What is the role of the administrator in structured analogies?
The administrator is responsible for describing the forecasting problem, and for selecting, recruiting, and briefing people who are likely to know about situations that are good analogies to the target situation (experts). When the experts have identified and analyzed their analogies, the administrator selects the outcome implied by each expert’s top-rated analogy as his or her forecast. The administrator combines the individual experts’ forecasts by averaging them following a predetermined scheme. This typically involves an unweighted average unless there are strong a priori reasons for using different weights.
4. How expert do the experts need to be when using structured analogies? (Discuss)
Evidence to date suggests that for the structured analogies method, an expert is someone who is able to identify at least two good analogies with which he or she is familiar. Ideally the familiarity should be from personal experience. Senior people are most likely to be familiar with relevant analogies, so the administrator should recruit senior people.
5. “For the purpose of structured analogies, an analogy can be a metaphor or a fable that has similarities to the target situation.”
True False
While fictional stories may seem realistic, there is no reason to believe that the characters in them make the same decisions as real people or that the outcomes of real situations will be like those of apparently similar fictional situations.
6. Describe how to use structured analogies to forecast decisions in conflict situations.
As was stated in the Method Guide, forecasting using the structured analogies method involves five main steps: An administrator (1) describes the target situation, and (2) selects experts; the experts independently (3) identify analogous situations and describe them, and (4) rate the similarity of the analogies to the target; the administrator (5) derives forecasts.
Descriptions of the forecasting problem should include all information that is relevant to the forecasting task, but no more. The descriptions should be short, factual, and unbiased. If possible define a set of up to four possible outcomes that are relevant to the decision maker. The possible outcomes may be ranges. Design a questionnaire to elicit the necessary information from the experts in a structured way. The questionnaire shown in Figure 1 in the Method Guide can be used as a template. Review and test the materials, where possible, before giving it to the experts.
Select experts who are likely to be personally familiar with situations similar to the target. Ask the experts to describe analogies and their similarities and differences to the target, to rate provide an overall rating of similarity for each analogy, and to identify the target outcome implied by the outcome of the analogy. Ask experts about the source of their knowledge for each analogy to help assess confidence in the analogies.
Derive forecasts by selecting the outcome implied by each expert’s top-rated analogy. Combine forecasts by averaging. Use weights only when there is good evidence that some experts have better knowledge than others.
7. What are the important characteristics of the description of a target situation?
The description should include the information needed to derive forecasts that will help the client to make the best decision for the situation. It should be objective and contain all historical and current information that is likely to be important for forecasting. Nevertheless, even for complex situations such as conflicts, descriptions need be no longer than a page or so and shorter descriptions are likely to be better for simpler situations.
When forecasting for conflicts, the description should stick to the facts and avoid speculation, for example, about the motives of protagonists.
8. “The primary reason structured analogies works is that analogies provide useful data when there is little relevant data about the target situation itself.”
True False
Analogies provide relevant data about real human behavior.
9. Describe real problems where structured analogies could help to improve forecast accuracy.
Examples might come from the following arenas:
a. union-management
b. buyer-seller negotiations
c. regulator-regulated interactions
d. military strategy development
e. counter-terrorism strategy,
f. project planning and costing, . . . etc
10. Is it a good idea to get more than one forecast for the one target situation? (Discuss)
There is much empirical evidence that combining between 5 and 20 independent forecasts helps to improve accuracy. Forecasts from diverse forecasters and from different methods are based on different information and different processing of the information. Forecasts from one forecaster may be biased but combining forecasts from diverse forecasters will tend to cancel biases.
Certificate of Completion
Forecasting Using Structured Analogies
I hereby certify that I have completed the prescribed self-certification course.
1. / I completed all tasks prescribed in the Course.2. / I kept a diary of my learning in which I described what I did and what I learned from each task.
3. / I completed the Pre-Course Test and the Certification Examination before looking at the Model Examination Answers.
4. / I fully understand the answers that I gave and I could explain them to another person.
5. / The time I spent on the tasks described in the Course Guide was… / minutes
6. / The number of times I applied the principles covered in this Course to a real problem was…
One of my applications was… / times
Completed on:
By: (Please print)
Address:
Date:
Please submit a copy of this certification and you will be listed as a special interest group member on conflictforecasting.com. If you are a member of the IIF, you can also be listed on forecastingprinciples.com as a consultant on Structured Analogies.