This Script Assists Instructors to Have Students Peer Review Writing Assignments

PeerGradeDashboardV2-4.au3 Instructions

By D. Braun

Introduction

This script assists instructors to have students peer review writing assignments.

It assumes the instructor uses the following software:

1. Thunderbird Email Software

A) Using the AttachmentExtractor Add-on for Thunderbird makes it easier to grab the Excel files emailed from reviewers.

B) It helps to have the ImportExportTools (MboxImport enhanced) installed to enable saving multiple email messages as text files and allow AutoIt to control Thunderbird via scripting.

2. Microsoft Excel™ software

Before running PeerGradeDashboardV2.au3, have Thunderbird running and logged in to your email account. PeerGradeDashboardV2.au3 uses the default email account. See Appendix I for instructions how to adjust the source code to use other than the default email account.

Instructors who wish to tailor the software would also need to install AutoIt3 available from The SciTE4AutoIt3 editor has many helpful tools. See

Table of Contents

Introduction

Table of Contents

Table of Figures

Software Initialization

Setup Tab

Pick Rubric Template

Customizing Rubric File

Pick Scores Template

Set Emails Directory

Set Eval Forms Directory

Set Score Directory

Pick Classlist

Pick Number Reviewers

Email Info Tab

Step 1: Enter assignment information required for email communications

Assign Reviewers Tab

Step 2: Assign Reviewers

Compose Reviewer Emails Tab

Step 3: Compose Reviewer Emails

Step 4: Remind Reviewers

Collate Reviewer Scores Tab

Step 5: Receive Reviews

Step 6: Collate Reviewer Scores

Email Scores Tab

Step 7: Compose Emails With Scores

Send Open Emails

Close Open Emails

Appendix I – Source code adjustments for the brave

Using other than Thunderbird’s default email account

Extracting Rubric Score Data from non-default Excel Cells

Appendix II – Customizing email texts

Customizing Reviewer Emails

Default Reviewer Email Text

Customizing Scores Emails

Default Scores Email Text

To get something to appear in the Table of Contents requires Viewing the Outlining toolbar and promoting Table-of-Contents entries to Levels 1, 2, or 3 (since I pick a TOC to show these levels). The Table-of-Contents doesn’t automatically grab body text, even if you insert an appropriate field code!!

Table of Figures

Figure 1 Screenshot showing how to use ImportExportTools from Thunderbird to save emails as text files.

Figure 2 Screen shot shows Setup menu options to locate required files and directories.

Figure 3 Screen shot shows the Email Info menu options to assign reviewers, more reviewers, compose emails to reviewers, email scores to authors, and record scores (not implemented yet).

Figure 4 Screen shot shows the Assign Reviewers menu options to assign reviewers and assign more reviewers.

Figure 5 Screen shot shows the Compose Reviewer Emails menu options to compose emails to reviewers, compose more emails to reviewers, and compose reminder emails to reviewers.

Figure 6 Screen shot shows the Collate Reviewer Scores menu options to assign reviewers and assign more reviewers, compose emails to reviewers, and email scores to authors.

Figure 7 Screen shot shows the Thunderbird menu options to extract all review Excel files from reviewer email messages and store the files in a single folder. Must have the AttachmentExtractor Thunderbird Add-on installed.

Figure 8 Screen shot shows the Collate Reviewer Scores menu options to assign reviewers and assign more reviewers, compose emails to reviewers, and email scores to authors.

Figure 9 Screen shot of error message when the template scores spreadsheet doesn’t appear where the program seeks it.

Figure 10 Screen shot of error message when the reviewer’s evaluation spreadsheet doesn’t contain Author+Reviewer codes in either cell.

Figure 11 Screen shot shows the Email Scores menu options to email scores to authors.

Software Initialization

After assignment emails start arriving, place them all in one email subfolder. Use ImportExportTools from Thunderbird to save all email messages in the folder as text files in one folder.

Figure 1 Screenshot showing how to use ImportExportTools from Thunderbird to save emails as text files.

Setup Tab

After running PeerGradeDashboardV2.exe, the opening screen appears. The first time the script runs, it prepares an initialization (.ini) file. Press the OK button to allow. Select the appropriate directories and files using the various options under the Setup menu as shown in Figure 2.

Figure 2 Screen shot shows Setup menu options to locate required files and directories.

Pick Rubric Template

The Rubric Template contains the template Excel file rubric. Reviewers receive and use a personalized version of the Rubric stored in the \RubricsOUT sub-directory to score the assignments they read. To customize your own template rather than using the one provided, please read the following comments carefully:

Customizing Rubric File

This is an IMPORTANT paragraph: As built, PeerGradeDashboard pulls evaluation score data from rubric Excel file cells F9:F23, F27, A35 a.k.a. .Cells(9, 6) to .Cells(23,6), .Cells(27, 6), .Cells(35, 1). It identifies authors and reviewers in rubric cells B2:B6 a.k.a. .Cells(2, 2) to .Cells(6, 2). It doesn’t identify reviewers in the Scores Excel file. It reserves cells A37:F42, a.k.a. .Cells(37,1) to .Cells(42,6) for its own purposes. It leaves the option of using cells F24, F25, F31:F33 a.k.a. .Cells(24, 6), .Cells(25,6), .Cells(31, 6) to .Cells(33, 6).

You may change the contents of any RUBRIC Excel file template cells other than those listed in the previous paragraph.

Similar considerations apply to the SCORES Excel file template.

Also, please note that theRUBRIC Excel file template and the SCORES Excel file template must sit in the SAME folder.

Pick Scores Template

The Scores Template contains the template Excel file used to report reviewer scores and comments to the authors. This Scores Template should sit in the same folder as the Rubric Template. Just as PeerGradeDashboard makes a personalized Excel Evaluation File from the Rubric Template for each reviewer, it also makes a personalized Excel Score file for each author. Unlike the rubrics sent to each reviewer, which contain author and reviewer identities, the Scores files do NOT identify reviewers.

Set Emails Directory

The Emails Directory contains text versions of email submissions received from students when they turn in their assignments.

Set Eval Forms Directory

The Eval Forms Directory contains the rubrics received from each reviewer with their reviews and comments.

Set Score Directory

The Score Directory contains the Excel files each student author receives after all the scoring ends. These files contain scores for each assignment from the reviewers who read and commented on the assignment.

Pick Classlist

The Classlist is an Excel file listing each student’s Name in column A and email address in column C, starting in row 2, since row 1 contains the column headers.

Pick Number Reviewers

Not implemented yet. When implemented, it will allow the instructor to select how many reviewers should review each paper submitted.

Email Info Tab

Figure 3 Screen shot shows the Email Info menu options to assign reviewers, more reviewers, compose emails to reviewers, email scores to authors, and record scores (not implemented yet).

Step 1: Enter assignment information required for email communications

The Instructor Name, Instructor Email address, and Eval Due Date are self-explanatory. The Set Email Subject Reviewer routine prompts the user to enter a string at least 12 characters long to identify the project. PeerGradeDashboard uses this string in email message subject headers and to identify the Scores files produced to report to authors.

Assign Reviewers Tab

Figure 4 Screen shot shows the Assign Reviewers menu options to assign reviewersand assign more reviewers.

Step 2: Assign Reviewers

Bug report 2.3 Blank lines in the txt version of the email lead to blanks in the log files. Perhaps, the parser should delete blank lines, as observed

User needs to check the log file after running the assign reviewers command to make sure each user has an entry for first name, last name, email, and assignment url.

In the log file, the program highlights in Bold font with a yellow background any emails received whose calpoly email addresses don’t agree with the “from” address where the email originated.

Running the Assign Reviewers or Assign More Reviewers routines creates rubrics customized for each reviewer and containing information about the piece to review.The program places the derived rubrics in the RubricsOUT sub-directory of the directory containing the rubric template.

Compose Reviewer Emails Tab

Figure 5 Screen shot shows the Compose Reviewer Emails menu options to compose emails to reviewers, compose more emails to reviewers, and compose reminder emails to reviewers.

Step 3: Compose Reviewer Emails

This needs Thunderbird open and running with an email already sent by the user to ensure the outgoing email server already has the desired passwords.

Before running the Compose Reviewer Emails, check the language that will go into the emails in _ComposeReviewerEmails(),_ComposeMoreReviewerEmails(), and _ComposeReviewerReminderEmails()

Figure 6Screen shot shows the Collate Reviewer Scores menu options to assign reviewers and assign more reviewers, compose emails to reviewers, and email scores to authors.

Step 4: Remind Reviewers

Some won’t return evaluation forms by the assigned Due Date.

Collate Reviewer Scores Tab

Step 5: Receive Reviews

Store review Excel files in one folder ($sEvalFormDir). Either save files individually, message by message, or use ImportExportTools to save attachments from multiple selected email messages to the desired folder (“Message”: “Extract from Selected Messages to . . .” : “Browse . . .” [alt-m-t-0] ).

The program scans through the folder for Excel files, assuming each Excel file in the folder contains a review with scores.

Figure 8Screen shot shows the Collate Reviewer Scores menu options to assign reviewers and assign more reviewers, compose emails to reviewers, and email scores to authors.

Step 6: Collate Reviewer Scores

Collate Reviewer Scores collects the scores information from each article’s evaluations files and inserts it into the scores files for the article. The routine requires a template scores files located in the Rubric Directory. If it cannot find this file, the error message shown in Figure 6 appears:

Figure 9Screen shot of error message when the template scores spreadsheet doesn’t appear where the program seeks it.

The routine also announces the error shown in Figure 7, if it can’t find author+reviewer codes in the evaluation spreadsheet.

Figure 10Screen shot of error message when the reviewer’s evaluation spreadsheet doesn’t contain Author+Reviewer codes in either cell.

An error also appears, if more than one “+” appears in the Author+Reviewer spreadsheet cell.

After collecting scores information from the returned evaluation forms, move them into another directory, so the program doesn’t have to look through them again.

The main problems arising from returned evaluation forms:

1)The reviewers don’t use the forms the instructor emailed them, so the forms miss:

Author info, wiki url, or other labels
Author+Reviewer codes
Use .xlsx file format instead of .xls

2)Reviewers leave one or more cells blank, specifically the comment sections.

3)Reviewers assign full credit for items clearly missing from the author’s work.

Email Scores Tab

Figure 11Screen shot shows the Email Scores menu options to email scores to authors.

Step 7: Compose Emails With Scores

After emailing the scores to the authors, move the scores spreadsheets to another directory. The Compose Emails with Scores routine looks for all scores files in the scores directory.

Send Open Emails

The Compose Emails With Scores routine pauses every 15 email messages to prevent overloading the screen and taskbar with open email messages. The pause permits a quick check to determine, if the email messages make sense. If desired to send the messages, select the Send Open Emails routine.It sends all messages in open Thunderbird Email Compose windows whose Subject matches the Set Email Subject Reviewer string selected under Step 1.

Close Open Emails

The Compose Emails With Scores routine pauses every 15 email messages to prevent overloading the screen and taskbar with open email messages. The pause permits a quick check to determine, if the email messages make sense. If desired to close and delete the messages without sending, select the Close Open Emails routine. It closes all open Thunderbird Email Compose windows whose Subject matches the Set Email Subject Reviewer string selected under Step 1.

Appendix I – Source code adjustments for the brave

Using other than Thunderbird’s default email account

The software assumes you wish to use Thunderbird’s default email account to compose emailsautomatically. To adjust in the source code, if you want to use an email account other than the default account to send emails, requires changes to two functions:

_PutInEmailComposeWindow and

_PutInEmailComposeWindowAttach

In each function, replace the one occurrence of Send("^v") with Send("^v!r{DOWN}") containing enough {DOWN} commands to reach the desired email account, i.e. Send("^v!r{DOWN}") to reach the second account in the “From” pull-down menu, Send("^v!r{DOWN}{DOWN}") to reach the third account in the “From” pull-down menu.

Extracting Rubric Score Data from non-default Excel Cells

Of course, such changes would require corresponding edits to the rubric template Excel file.

PeerGradeDashboard assumes you wish to pull evaluation score data from rubric cells F9:F23, F27, A35. It also reserves cells A37:F42 a.k.a. .Cells(37,1) to .Cells(42,6) for its own purposes. To adjust in the source code requires manychanges to the function:

_CollateReviewerScores

1. Account for changes to the Rubric template cells used in the function, adjust by adjusting or adding to the many occurrences of expressions such as

$oExcelRubricReceived.Activesheet.Cells(9, 6).Value,

where $oExcelRubricReceived.Activesheet.Cells refers to the rubric received from the reviewer and providing score values, and

where (9, 6) refers to cell in Row 9, Column 6 (F9).

2. Account for changes to the Scores template cells used in the function, adjust by adjusting or adding to the many occurrences of expressions such as

With $oExcelScore.Activesheet, . . .

.Cells(27, 6).Value =

where $oExcelScore.Activesheet refers to the Scores Excel file to return to the author containing completed scores, and

where (27, 6) refers to cell in Row 27, Column 6 (F9).

3. Make appropriate changes to the Log file organization and contents.

This would require more changes than it makes sense to list here.

Appendix II – Customizing email texts

Customizing Reviewer Emails

To customize the email text sent to reviewers, change code in the following functions:

_ComposeReviewerEmails and

_ComposeMoreReviewerEmails

_ComposeReviewerReminderEmails

The email text code assigns the text to a string variable $sEmailMessage, with text in double quotes concatenated with variables and carriage return linefeed characters (@CRLF). The underscore character at a line’s end extends the instruction to the next line.

Default Reviewer Email Text

This section indicates the default email message sent to reviewers informing them which reports to review. PGD replaces the bold entries below with information provided by the instructor, the student, or based on its reviewer assignment algorithm.

Subject: 4E Sustainability Analysis Reviews for First_name5

Email Text:

“First_name,

This part of the sustainability analysis assignment involves evaluating reports

from two of your classmates. Please read carefully the following two

Sustainability Analyses selected by your instructor:

Please use the attached evaluation forms to enter your scores and comments

about the reports. In the attached Evaluation Forms:

Make entries in all blue shaded cells.

Do not change any cells that arrive with contents.

Keep the same filenames.

Save the file as an Excel .xls file, not .xlsx or any other format.

The evaluation form filenames are:

4E_Sust_Analysis_Eval_Form8+5.xls

4E_Sust_Analysis_Eval_Form3+5.xls

Submit your evaluation forms before class on Monday, November 14, 2011,

using the following email format:

1. To:

2. Subject: 4E Sustainability Analysis Reviews from

3. Message first line: Your last name

4. Message second line: Your first name

5. Message third line:

6. Message additional lines: optional text, but no images

7. Attachments: Attach the two evaluation form files to the email message.

Please don't change the filenames or suffix.

Thanks,

Instructor Name”

Attachments:

4E_Sust_Analysis_Eval_Form8+5.xls

4E_Sust_Analysis_Eval_Form3+5.xls

Customizing Scores Emails

To customize the email text sent to authors, change code in the following function:

_ComposeScoresEmails.

Default Scores Email Text