Accuracy of Collecting Statistical Data Using

Accuracy of Collecting Statistical Data using

Voice Recognition Software

An Experimental Design Project by:

Jason Lehmiller and Richard Lee

4/22/2004

I. Statement of Goals:

Using the dictation mode of the speech recognition feature of Microsoft Word 2003, we were looking to model the accuracy of collecting different types of data that could be utilized for statistical analysis purposes, under varying levels of computerized speech training. Three individuals participated in the experiment. Each utilized the same computer and microphone, but created and used their own unique speech profile.

We would like to see if any distinct type of data can be collected more accurately than others by using speech recognition software. We are also interested to see if an increase of training the computer to recognize your voice also increases the accuracy of data collected to a significant degree.

II. Description/Design/Execution of this Experiment:

Response Variable:

In this experiment, the response variable is a number from 0 to 25, which indicates the number of correct voice-to-computer recognition successes for a particular type of data dictated by an individual. Each of the types of data had a pre-compiled script which could be divided into 25 distinct units (of 10 characters each) in terms of measuring accuracy. After an individual dictated a script, their results were compared and graded based upon the initial script. After dictation of a unit, if all 10 characters are correct, the unit is counted as a success (+1). If at least one character is incorrect, the unit is counted as a failure (+0).

Description of Each of the Factors used in the Study:

One treatment factor is the type of data being collected. We’ve separated this factor into 4 levels. Each “unit” of data is comprised of 10 total characters (including commas, dollar signs and spaces). After each unit of the data is read, the participant will dictated the phrase “new line” to move the cursor to the next line in the Microsoft Word document.

o  LEVEL 1: (A) Whole Numbers

§  25 random numbers – generated by the Microsoft Excel RANDBETWEEN function. Examples are:

·  72,200,489

·  98,340,244

o  LEVEL 2: (B) Prices

§  25 random prices - generated by the Microsoft Excel RANDBETWEEN function. Examples are:

·  $2,300,988

·  $8,992,330

o  LEVEL 3: (C) College Course Subject Names (1 Word)

§  The 16 college course subject names listed below were taken from a university’s website. A random sample of 25 these college course subject names was taken, and made up the data script for this level. By coding each of the college course subject names with a unique number between 1 and 16, the Microsoft Excel RANDBETWEEN function was used to create this script. The random sample was taken from the population of 16 with replacement, so the same college course subject name can appear more than once in the data script. This also means that some of the college course subject names listed below did not appear at all within the data script.

·  Statistics

·  Psychology

·  Philosophy

·  Accounting

·  Technology

·  Proctology

·  Entomology

·  Humanities

·  Management

·  Literature

·  Portuguese

·  Journalism

·  Biometrics

·  Pediatrics

·  Psychiatry

·  Musicology

o  LEVEL 4: (D) Color Names (2 Words)

§  The 21 color names listed below were taken from a list of valid HTML color names. A random sample of 25 of these color names was taken, and made up the data script for this level. By coding the color names with a unique number between 1 and 25, the Microsoft Excel RANDBETWEEN function was used to create this script. The random sample was taken from this population of 21 with replacement, so the same color name can appear multiple times in the data script. This also means that some of the color names below did appear at all within the data script.

·  Cadet Blue

·  Indian Red

·  Lawn Green

·  Slate Blue

·  Alice Blue

·  Burly Wood

·  Dark Khaki

·  Light Blue

·  Light Cyan

·  Light Gray

·  Light Pink

·  Lime Green

·  Mint Cream

·  Misty Rose

·  Olive Drab

·  Pale Green

·  Peach Puff

·  Rosy Brown

·  Royal Blue

·  Slate Gray

·  Steel Blue

The replicate treatment factor is the level of voice recognition training provided to the computer by the participant. This factor is divided into 2 levels.

·  Under Training Part 1: (T1) Participant initially provided Microsoft Office 2003 with the default level of speech training by dictating the “Introduction to Microsoft Speech Recognition”. After completing the initial training, the participant read the 4 data scripts.

·  Under Training Part 2: (T2) In addition to the default level of training, the participant also completed supplementary training by dictating excerpts from “The Wonderful Wizard of Oz” by L. Frank Baum. After completing the training, the participant then read the same 4 data scripts again.

The blocking factor is the individual who dictated the data script. In this case there were 3 subjects:

·  Jason (JL)

·  Richard (RL)

·  Josephine (JD)

Please note the following excerpt from Microsoft Office Speech Recognition Help:

“After speech recognition is installed, you can increase speech recognition accuracy by taking a few minutes to train the computer to recognize how you speak.

When you read aloud the prepared training text, the training wizard can look for patterns in the way you speak, and gather voice data that helps interpret the words that you'll dictate into Office programs. The training session includes help with adjusting your microphone, and it should take less than 15 minutes to complete.

Additional training may increase speech recognition accuracy. At any time, you can go back to the training wizard to read additional training text.”

Potential Factors that were Controlled or Ignored:

To minimize any obviously potential flaws with using voice recognition technology, we did the following:

·  Each individual used the same computer and same microphone.

·  Each participant was instructed to hold the microphone the same distance from their mouth for all dictations.

·  Each participant was instructed to use a relatively constant tone of voice throughout the process (for example, no yelling or whispering).

·  Each participant developed and used his/her own unique speech profile within Microsoft Office 2003.

·  The dictations were done in a quiet environment where there is minimal background noise. The Statistics Department conference room was used for this purpose.

·  To discourage the participant from being affected by seeing the results of their voice recognition, while speaking, they were not permitted to view the computer screen that outputs their results.

·  One potential factor that we attempted to control was fatigue. A bottle of water was provided for the participant to drink between reading sessions. The participant was also given the option to take a five minute break in between sessions if they wished.

Experimental Units:

The experimental units are the individuals doing the dictation for each of the data scripts under the specified level of speech training. In this case there were 3 subjects:

·  Jason (JL)

·  Richard (RL)

·  Josephine (JD)

Design/Execution of the Experiment, and the Randomization Process:

We used a Replicated Randomized Complete Block Design. Please refer to the diagram below to view an example of the structure of our design:

Training Level 1 / Training Level 2
Data Type / Data Type
A / B / C / D / A / B / C / D
JL / JL
Participant / RL / RL
JD / JD

Below are the data scripts that were compiled for this experiment:

Numbers / Currency / Course Names / Colors
72,201,700 / $6,165,384 / Psychology / Slate Gray
63,229,761 / $4,154,801 / Humanities / Misty Rose
79,190,767 / $7,912,100 / Literature / Lime Green
71,264,126 / $3,219,341 / Portuguese / Indian Red
45,096,043 / $2,324,497 / Biometrics / Pale Green
60,216,331 / $7,007,470 / Portuguese / Steel Blue
47,550,319 / $1,458,164 / Management / Cadet Blue
39,289,708 / $3,400,919 / Psychiatry / Olive Drab
90,525,218 / $3,552,658 / Portuguese / Light Gray
63,886,189 / $8,645,039 / Humanities / Dark Khaki
55,298,701 / $3,084,958 / Philosophy / Lawn Green
63,688,628 / $3,654,456 / Psychiatry / Peach Puff
91,593,913 / $2,470,348 / Statistics / Indian Red
77,858,216 / $3,185,841 / Philosophy / Cadet Blue
97,745,705 / $1,699,053 / Technology / Lawn Green
94,861,256 / $4,623,913 / Psychiatry / Lime Green
70,447,435 / $5,441,836 / Psychology / Lawn Green
94,117,912 / $7,333,081 / Technology / Pale Green
36,599,368 / $6,848,778 / Proctology / Mint Cream
41,490,729 / $5,867,070 / Biometrics / Light Pink
44,008,456 / $2,157,440 / Portuguese / Cadet Blue
44,707,892 / $7,219,047 / Technology / Rosy Brown
43,006,239 / $8,395,704 / Portuguese / Olive Drab
51,384,294 / $7,427,647 / Philosophy / Light Blue
77,403,029 / $4,673,943 / Proctology / Alice Blue

The model for how the experiment was executed is as follows. The entire process (training/dictation) elapsed between 45 and 60 minutes per participant:

·  Participant RL:

o  Completed voice recognition training part 1.

o  Read the 4 data scripts in random order. The order was:

§  Currency

§  Colors

§  College Course Subject Names

§  Numbers

o  Completed voice recognition training part 2.

o  Read the 4 data scripts in a new random order. The order was:

§  Colors

§  College Course Subject Names

§  Numbers

§  Currency

·  Participant JL:

o  Completed voice recognition training part 1.

o  Read the 4 data scripts in random order. The order was:

§  College Course Subject Names

§  Colors

§  Numbers

§  Currency

o  Completed voice recognition training part 2.

o  Read the 4 data scripts in a new random order. The order was:

§  College Course Subject Names

§  Currency

§  Numbers

§  Colors

·  Participant JD:

o  Completed voice recognition training part 1.

o  Read the 4 data scripts in random order. The order was:

§  Currency

§  Numbers

§  Colors

§  College Course Subject Names

o  Completed voice recognition training part 2.

o  Read the 4 data scripts in a new random order. The order was:

§  Currency

§  Numbers

§  Colors

§  College Course Subject Names

Note: Randomization for the order in which the data scripts were read was done using a single, fair, die. Each of the 4 scripts were stapled together, 1 page per script. Before dictating a script, the participant would roll the die. It a 1, 2, 3 or 4 was rolled, the participant would read the script on that page number, if that script had not already been dictated within that segment of the session. If a 5, 6 or a number that was already used was rolled, then the participant re-rolled the die as long as necessary.

III. Raw Data and Analysis of the Data:

Raw Data

The raw data collected is presented below, and is color coded. A unit in the color green was counted as a success, and a unit in the color red was counted as a failure.

Participant RL – Training Level 1:

T1 - Numbers / T1 - Currency / T1 - Course Names / T1 - Colors
72,201,700 / $6,165,384 / Psychology / Slate gray
63,229,761 / $4,154,801 / Humanities / Mr. Rose
79,190,767 / $7,912,100 / Literature / Lime green
71,264 thousand 126 / $3,219,341 / For gypsies / Indian read
45,096,043 / $2,324,497 / Biometrics / The Oprean
60,216,331 / $7,007,470 / Portuguese / Seal blue
47,550,319 / $1,450,164 / Management / It that you
39,289,708 / $3,400,919 / Psychiatry / I looked around
90,525,218 / $3,552,688 / For duty / Light gray
63,886,189 / $8,645,039 / Anant is / There can be
85,298,701 / $3,084,950 / Full of city / Laundry
63,688,628 / $3,654,456 / Psychiatry / Each of the
91,593,913 / $2,470,348 / Statistics / Indian read
77,858,216 / $3,185,841 / Philosophy / Today:
97,745,705 / $1,699,083 / Technology / Long green
94,861,256 / $4,623,913 / Psychiatry / Fine grain
70,000,044 447,435 / $5,441,836 / Psychology / On Brini
94,117,912 / $7,333,081 / Technology / The Oprean
36 million 599,368 / $6,840,778 / Product elegy / The green
41,490,729 / $5,867,070 / Biometrics / White paint
44,008,456 / $2,157,440 / Portuguese / Kidd and blue
44,707 thousand 892 / $7,219,047 / Technology / Rosie brown
43,006,239 / $8,395,704 / Portuguese / I love (
81,384,294 / $7,427,647 / The loss of three / White and blue
77,403,029 / $4,673,943 / Product out / Alice Ballou
Total: 19 / Total: 19 / Total: 18 / Total: 3

Participant RL – Training Level 2:

T2 - Numbers / T2 - Currency / T2 - Course Names / T2 - Colors
72,201,700 / $6,165,384 / Psychology / Slate gray
63,229,761 / $4,154,801 / Humanities / Mr. Rose
79,190,767 / $7,912,100 / Literature / I’m drained
71,000 264,126 / $3,219,341 / Portuguese / Indian read
45,096,043 / $2,324,497 / Biometrics / The Oprean
16,216,331 / $7,007,470 / Portuguese / Seal blue
47,550,319 / $1,458,164 / Management / Today:
A 9,289,780 / $3,400,919 / Psychiatry / I love travel
90,525,218 / 3552 thousand $658 / Portuguese / Light gray
63,886,189 / $8,645,039 / Humanities / Dark khaki
55,298,701 / $3,084,958 / Philosophy / Long green
63,688,628 / $3,654,456 / Psychiatry / Each half
91,593,913 / $2,470,348 / Statistics / Indian read
77,858,216 / $3,185,841 / Philosophy / Today blue
To 97,745,705 / $1,699,053 / Technology / Long green
94,861,256 / $4,623,913 / Psychiatry / Lime green
70,447,435 / $5,441,836 / Psychology / Lorne Greene
94,117,912 / $7,333,081 / Technology / Pale green
36,599,368 / 6,848,778dollars / Product allergy / Mint green
41,490,729 / $5,867,070 / Biometrics / Light tanks
44,008,456 / $2,157,440 / Portuguese / Did that blew
44,707,892 / $7,219,047 / Technology / Rosie brown
43,006,239 / 8,395,000 / Portuguese / I love travel
51,384,294 / Philosophy / Light blue
77,403,029 / Product elegy / Alice blue
Total: 21 / Total: 20 / Total: 23 / Total: 7

Participant JL – Training Level 1:

T1 - Numbers / T1 - Currency / T1 - Course Names / T1 - Colors
72,201,700 / $6,165,384 / Psychology / Slate gray
63,229,761 / $4,154,801 / Humanities / Misty heroes
79,190,767 / $7,912,100 / Literature / Lime green
71,264,126 / $3,219,341 / Portuguese / Indian read
45,096,043 / $2,324,497 / Biometrics / Daniel green
60,216,331 / $7,007,470 / Portuguese / Steel plate
47,550,319 / $1,458,164 / Management / Did that play a
39,289,708 / $3,400,919 / Psychiatry / Olive drab
90,525,218 / $3,552,658 / Portuguese / Light gray
63,886,189 / $8,645,039 / Humanities / Dark khaki
55,298,701 / $3,084,958 / Philosophy / One green
63,688,628 / $3,654,456 / Psychiatry / Each half
91,593,913 / $2,470,348 / Statistics / Indian read
77,858,216 / $3,185,841 / Philosophy / Did that play
97,745,705 / $1,699,053 / Technology / One green
94,861,256 / $4,623,913 / Psychiatry / Lime green
70,447,435 / $5,441,836 / Psychology / Long grain
94,117,912 / $7,333,081 / Technology / Pale green
36,599,368 / $6,848,778 / From college the / Meant three
41,490,729 / $5,867,070 / Biometrics / White paint
44,008,456 / $2,157,440 / Portuguese / To get the lieu
44,707,892 / $7,219,047 / Technology / Rosie brown
43,006,239 / $8,395,704 / Portuguese / All of the trap
51,384,294 / $7,427,647 / Philosophy / Light blue
77,403,029 / $4,673,943 / From college a / Dallas blue
Total: 25 / Total: 25 / Total: 23 / Total: 8

Participant JL – Training Level 2: