PDET Unit 7 Exercises
Assignments Unit 3:
Activity / Goal## Programming Notes:
Correct answers are revealed on last try (turn color of correct selection word red)
Completed correctly answered items have a blue check mark in leftmost column beside item number
Completed incorrectly answered items have a blue x mark in the leftmost column beside the item number
Each exercise page has a reset button so that users can clear the exercise and try again.
Last attempt is saved to the database when the users leaves the “site” so that when the users sign back on at a later date, they can see their answers (but they can always erase and try again with the reset button.
Exercise 7-1: Definition of a Successful Training Program
Lee – I changed the feedback for 5 a tad so that learners will have a clear correlation between what they checked off and what the feedback says – since we are using a radio button format – we are accepting one answer at a time – so feedback that tells them two answers are correct is frustrating for them.
If you prefer, we can change this to a checkbox (multiple select) type of coding. We can also make it two try if you want them to really zero in on definition 6.
DirectionsGiven the following list of choices, check the 1 choice that most closely fits your definition of a successful training program.
I define a successful training program as one that:
O1. Has lots of enrollees and no no-shows.
O2. Goes like clockwork, every topic getting its allotted time, and ends
on time.
O3. Brings high praise from participants on end-of-course evaluation.
O4. Meets predetermined learning objectives.
O5.Teaches skills, knowledge, or attitudes that are then used by
trainees when back at the job.
O6.Contributes to a positive end result for the organization.
O7.Causes management to ask for more and to support training with
higher budgets, more personnel, and modern facilities.
O8.Results in happier employees and higher morale.
O9.Other (please describe in the space below)
##CA == 4
Very good. Meeting predetermined objectives, based on a sound task analysis, is the first step in creating training that will have a positive impact on the organization (definition #6), which is what we want to ultimately achieve.
You have now completed Exercise 7-1. Continue working with your mentor and syllabus until you are ready for the next exercise.
##CA == 5
GreatYou are on the right track! Any time the skills and knowledge learned during the training are used by the learners back on the job, you are well on your way to contributing to a positive end result for the organization(definition #6) which is what we ultimately want to acheive.achieve.
You have now completed Exercise 7-1. Continue working with your mentor and syllabus until you are ready for the next exercise.
##CA == 6
Of course! Although Tthere are various ways to evaluate training success, but ultimately we, as instructional designers,we ultimately want to see our training designs meet the criteria expressed in definition #6.contribute to the positive end result of the organization (definition #6).
You have now completed Exercise 7-1. Continue working with your mentor and syllabus until you are ready for the next exercise.
##Exit
Okay. Although eachll of these definitions indicate various aspects and stages of training success, but ultimately we need to reach the criteria expressed in definition #6. we should ultimately try and reach a level of transferring new skills to the job (definition #5) that leads to contributing to the positive end result for the organization (definition #6).
You have now completed Exercise 7-1. Continue working with your mentor and syllabus until you are ready for the next exercise.
Ruth and Lee: We could provide them with feedback specific to each of these definitions, but that seemed like overkill. Also, please beef up the CA and Exit feedback to include more of the good stuff you tell them in the class.
Exercise 7-2: Four Levels of Evaluation
DirectionsBefore you complete this exercise, be sure to have read sections 7-1 through 7-2 as well as completed the previous reading assignments and exercises.
Select which of the four levels each evaluation scenario describes.
Scenario 1
After training was provided on accurate reporting of inventory charge-backs, the number of errors on charge-back forms declined by 35%.
O Level 1: Trainee-reaction / O Level 2: Achievement-of-objectivesO Level 3: Job-transfer / O Level 4: Bottom-line
##CA 3
##CA feedback
Great. This information reports on the transfer of the training to the job – where errors decreased by 35%.
##WA -- Anything else
Not exactly. As a hint, consider when the data is collected. It helps to remember that Level 1 Evaluation usually happens within a short time of the training event, as does evaluation of the trainee’s achievement of the objectives. If trainees are using their new skills in their jobs, it will show up in their performance measurements. Bottom-line evaluation occurs much later, when costs and profits associated with the training content are factored into the bigger corporate picture. Please try again.
##EXIT
Incorrect. This is a level 3 evaluation is a level 3, focusinges on how the trainees are using their new skills on the job – thus seeing an improvement in performance. In this case, the improvement is measured at 35% fewer errors.
(broad referent – This – this what? --needs a noun so that learners know specifically what’s being referred to – in a classroom one is pointing to something – in a book on an overhead etc.) We could say “This scenario is a level 3-evaluation” if you want.
Scenario 2
With fewer errors on inventory charge-back forms, corporate loss declines by $500,000 annually.
O Level 1: Trainee-reaction / O Level 2: Achievement-of-objectivesO Level 3: Job-transfer / O Level 4: Bottom-line
##CA -- 4
Perfect. That kind of monetary savings is a very good example of a bottom-line measurement.
##WA
Not exactly. Reconsider the type of data used for the evaluation. It helps to remember that Level 1 Evaluation focuses on the trainee’s reaction. Level 2 looks at test scores to see if the trainee can perform the task associated with the objective, usually in a controlled environment. Level 3 takes a look at performance measurements on the job in order to determine if the new skills have changed production numbers. Level 4 looks at the dollar and cents of the training impact on the organization. Please try again.
##EXIT
Incorrect. This is a level 4 evaluation is a level 4, focusinges on the dollar and cents value of the training and its successful implementation. Thus, it is a level 4 evaluation.
Scenario 3
At the end of the first day of the inventory charge-back class, the instructor asks learners to write her a note describing how they felt the class was going.
O Level 1: Trainee-reaction / O Level 2: Achievement-of-objectivesO Level 3: Job-transfer / O Level 4: Bottom-line
##CA -- 1
Of course. In this case, the instructor is simply looking for trainee-reaction at the end of the first day.
##WA
Not exactly. Reconsider when the data is collected. It helps to remember that Level 1 Evaluation usually happens within a short time of the training event or even during the training itself. Level 3 occurs sometime after the trainees are back on the job. Level 4 occurs much later, after data on the larger corporate financial picture is available. Please try again.
##EXIT
Incorrect. This is a level 1 evaluation is a level 1, focusinges the trainee’s reaction to the training – collected during the event itself. It is Level 1 evaluation.
Scenario 4
At the end of Unit 2, “Calculating Monthly Inventory Charge-back,” the instructor gives each learner a stack of loss forms to tally and calculate without any help.
O Level 1: Trainee-reaction / O Level 2: Achievement-of-objectivesO Level 3: Job-transfer / O Level 4: Bottom-line
##CA -- 2
Of courseVery good. By requiring learners to complete the task during training, and without help, you are determining if they can achieve the objectives.
##WA
Not exactly. As a quick reminder remember that Level 1 collects subjective data. Level 2 looks at test scores to see if the trainee can perform the task associated with the objective. Level 3 takes a look at performance measurements on the job. Level 4 looks at the financial impact on the organization. Please try again.
##EXIT
Incorrect. This is a level 2 evaluation is a level 2 focusingeson the trainee’s performance on tests in the controlled environment of the classroom. – Level 2 evaluation.
Scenario 5
After training the production department on a new quality-assurance program, the number of defective product returns to the department declines from 30% to 12% over the next six months.
O Level 1: Trainee-reaction / O Level 2: Achievement-of-objectivesO Level 3: Job-transfer / O Level 4: Bottom-line
##CA --3
Outstanding. The performance improvement indicates job transfer. And although in addition to athis is a good level 3 indicator, this evaluation itscenario is also very close to evaluating the bottom-line -- allowing for the some to argue it could also be level 4.
##CA – 4
Good. Your answer indicates that this scenario comes very close to evaluating the training’s impact on the organization’s bottom line. However, note that the evaluation focuses on job transfer as demonstrated in the decline in defective product returns (not on dollar amount impact on the company’s bottom line).
##WA
Not exactly. Note that this data is collected on the job, six months after the conclusion of the training and does not include any financial data. Please try again.
##EXIT
Incorrect. This is a level 3 evaluation is a level 3, focusinges on the trainees’ abilities to transfer to their jobs the skills acquired during training. Thus it is level 3 – Job Transfer evaluation.
Scenario 6
With the decline in product returns, the company sees an 8% increase in return customer sales along with an 18% decrease in sales refunds during the same six months.
O Level 1: Trainee-reaction / O Level 2: Achievement-of-objectivesO Level 3: Job-transfer / O Level 4: Bottom-line
##CA --4
Superlative. Increased sales and decreased refunds are solid bottom-line measurements.
##WA
Not exactly. Note that this evaluation requires the bigger view of the financial picture, based on more than just the individual decrease in errors. Please try again.
##EXIT
Incorrect. This is a level 4 evaluation is a level 4, focusingeson the bigger financial impact that successful training has on the company as a whole – Level 4 – Bottom Line evaluation..
Scenario 7
At the completion of the course, “Processing a Claim,” the trainees process ten simulated claims without assistance from the instructor.
O Level 1: Trainee-reaction / O Level 2: Achievement-of-objectivesO Level 3: Job-transfer / O Level 4: Bottom-line
##CA -- 2
Perfect. Completing the claims without help is a level 2 activity.
You have now completed Exercise 7-2. Continue working with your mentor and syllabus until you are ready for the next exercise.
##WA
Not exactly. Which level looks at how well trainees can perform tasks in a controlled environment? Please try again.
##EXIT
Incorrect. This is a level 2 evaluation is a level 2, focusingeson the trainee’s mastery of the training objectives through controlled tests. – Level 2 evaluation.
You have now completed Exercise 7-2. Continue working with your mentor and syllabus until you are ready for the next exercise.
Exercise 7-3: Establish Test Validity with the Content Performance Matrix
##MEF – This exercise is going to be particularly nasty in layout. Each scenario will be ##a page – but the page will be long – definitely have to scroll. So we will need next ##page, etc. as well as a top of page. Also, we will want to concatenate the learners ##input. Part A is two radio interactions (1 correct answer each) Part be is three radio ##interactions (1 correct answer each). Probably similar pattern as lesson 1 Exercise 1.
##PS. MEF – Don’t get confused by the Test Items for each of these four questions. ##Students don’t answer those test items -- they are examples.
Ruth and Lee – How do you want to handle the items where there is no revision necessary to the test item? Do you want to offer them the opportunity to revise it anyway – Provide additional enhancements, such as making the test more “job-centered”? I left these blank for now.
DirectionsBefore you complete this exercise, be sure to have read 5-20 through 5-26 as well as completed previous reading and exercise assignments. You may want to refer to page 5-25 as a reference when completing this exercise.
For each learning objective below:
a)First classify the objective itself – content type and performance level
b)Then classify the content type and performance level of the test item and determine if there is a match between objective and test item
c)Third, if there isn’t a match between the two, revise test item.
Hint: During this exercise there is a tendency to classify all remember-level items as facts. However, please remember that all 5 content types have a remember-level. The table on page 5-25 will be very helpful in completing this exercise.
Scenario 1
- Classify the Content Type and Performance Level of the scenario 1 learning objective below, then click OK.
Learning Objective 1: The student will complete a customer request for service by correctly entering the information on the TUON format on the computer.
O Concept O Fact O Procedure O Process O PrincipleO Use O Remember
##OK button
##CA Procedure + Use
## CA Feedback
Great. The learners will use their learning to follow a series of steps to actually complete a procedure. Now classify the test item.
##WA – Procedure + Remember
##WA1 Feedback
Not exactly. You are correct that this objective is a procedure – but does this objective ask the learner to apply or use the learning? Try again.
##WA2 – Anything else + Use
You are correct that this objective asks the learner to use the knowledge, but you haven’t correctly identified the content type. Try again.
##WA3 Anything else
No. You need to rethink both the content type and the performance level. Here’s a hint: The task asks the learner to use the knowledge in activity that is repeatableto follow a series of steps to complete an activity. Try again.
##Exit.
No. This objective contains content that is procedural and requires the learner to use the knowledge in a job-centered application. Now classify the test item.
- Classify the Content Type and Performance Level of the scenario 1 test item, then click OK.
Test Item 1: List the major steps necessary to complete a service order with 90% activity.
O Concept O Fact O Procedure O Process O PrincipleO Use O Remember
Test Item Matches Objective? O Yes O No
##OK button
##CA –Procedure + Remember + No
Great job. You have correctly identified that there is no match between the learning objective and the test item. The since the objective asks the learner to complete the procedure and the test only ask the learner to remember it.
Now, try your hand at revising the test item to better fit the learning objective.
##WA1 Procedure + Remember + Yes
##WA1 Feedback
Almost. As you correctly identified, the test item is a procedure and requires the learner to remember knowledgeat the remember-level. However, the learning objective for this scenario requires the learner to use knowledge. So -- do they match? Try again.
##WA2 Procedure +Use + No
Close. You are correct that the test item is a procedure and does not match the learning objective – consider the performance level and try again.
##WA# 3-- Anything Else
Not really. Here’s a hint: If the learner must follow steps to perform a repeatable task – it’s probably a procedure – tests that require the learner recall items by route are often Remember remember level of performance. Finally, if the objective is at a Uuse level of performance and the test is not – there is no match. Try again.
##Exit
No. Although both the test item and the objective are procedures – the objective asks for Use level performance and the test item asks for Remember level, so there is no match.
See if you can remedy this mismatch by revising the test item to better fit the learning objective.
##OK button
##Feedback
##Feedback – appears under boxes (just as in Unit 1 – Exercise )
Okay. Compare your revision to the one shown. Note that our sample revision requires either role-play or media so that the learner can responses, as realistically as if he or she were actually on the job, to a human voice’s request.
When you are ready, click Next Page to try another scenario.
Scenario 2
- Classify the Content Type and Performance Level of the scenario 2 learning objective below, then click OK.
Learning Objective 2: Given a picture of the rank insignia for some U.S. Army personnel, the student will state from memory the rank associated with the insignia.
O Concept O Fact O Procedure O Process O PrincipleO Use O Remember
##OK button
##CA Fact+ Remember
## CA Feedback
Great. You recognized the learner was being asked to identify a unique and specific piece of information.Now classify the test item.
##WA – Concept + Remember
##WA1 Feedback
Not exactly. Although you correctly identified the performance level as Remember, you need to rethink the concept content type. Concepts require that the learner identify characteristics – Facts are specific and unique items. Try again.
##WA2 – Fact + Use
You correctly identified the content type as fact, but reconsider the performance level. The distinction between remembering and using facts is trickyRemember that facts only have 1 performance level. Try again.
##WA3 Anything else
No. You need to rethink both the content type and the performance level. Here’s a hint: The task asks the learners to memorize recognize unique and specific details so that they known them when they see them. Try again.