CenterforProgramEvaluation

2014EvaluationReport

Contents

Overview...... 4

EvaluationMethods...... 6

DailyFormativeSurveys...... 8

Concerns,Confidence,CommitmentSurvey...... 10

FocusGroup...... 15

StudentPresentations...... 18

SummaryFindingsandRecommendations...... 20

Appendices(A-B)...... 22

CenterforProgramEvaluationandDevelopment

GeorgiaCollege CampusBox116

Milledgeville,GA31061

478-607-2730

AbouttheCenterforProgramEvaluationandDevelopment

TheCenterforProgramEvaluationandDevelopmentatGeorgiaCollegeprovidesevaluationsolutionstoorganizationsaddressing significantlocal,state,nationalandinternationalneeds.TheCenteroffersafullrangeofservicesforclientstohelpthemidentifyfunding sources,preparegrantproposals,andevaluatetheeffectivenessof theirefforts.

CenterforProgramEvaluationandDevelopmentTerrellHall

Overview

TheTeacherQualitygrantsprogramisdesignedto“strengthenanddeepen teachers’contentknowledgeintheir academicsubjectswithemphasison howdeepenedcontentknowledgeimpactsteachingpracticesand studentlearning.”Specificcontentareastargetedbytheprogramincludelanguagearts,mathematics,reading, scienceand/orsocialstudies.

Projectsmightincludeuseoftechnology,innovativeassessment,orteachingstrategiestohelpachievetheoverallgoalofimprovingteaching andlearning.

TheNanoscaleScienceand Engineering(NES)projectwasapartnershipbetweentheGeorgia

InstituteofTechnology’sInstituteofElectronicsandNanotechnology;areasschooldistricts(Cobb, DeKalb,andMontgomerycounties);andtheNationalNanotechnologyInfrastructureNetwork. TheNESprojecthadthefollowinggoals:

1.Provideteamsofteacherswithan understandingofnanoscalescienceand engineering;

2.Developteachers’understandingoftheconnectionsbetweennanoscalescienceand traditionallytaughtscience,math,and languagearts;

3.Providenanoscaleinstructionalmaterials linkedtoGeorgiaStandards(science,literacy,mathandEnglishlanguagearts);

4.Provideopportunitiestoexperiencedesign challengesthatwillsupportstudentlearning ofNESandincreasestudentinterestin STEM.

Toachievethesegoals,theNESprojectpursuedafour-stepapproach:1)backgroundreadingpriortosummerworkshop(twobooks);2)four-daysummerworkshopatGeorgia TechfocusingonNESandengineeringdesign;3)teamsimplementingNES designchallengeswiththeirstudentsduringfall

semester;and4)sharingofimplementation experiencesandstudentpresentationsatfollow-up sessioninDecember.

Participants

Asplanned,theNESprojectrecruitedteamsof elementaryschoolandmiddlegradesteachersfromCobb,DeKalb,andMontgomerycounties.Atotalof 20teachersparticipatedinthesummerworkshop, including9K-5thgradeteachersand116-8thgradeteachers.Theseincludedwhotaughtavarietyofthetargetedsubjectareas:EnglishLanguageArts5,Math5,andScience5.Oneteacherwasanmiddlegradesinstructionalcoordinatorand4elementary schoolparticipantstaughtallsubjects.Oneteacher droppedoutafterthesummerworkshopwhenhewasreassignedbyhisdistricttoahighschool.

SummerWorkshop

Thefour-daysummerworkshopwasfromJune10toJune13atGeorgiaTech.Theteamsparticipatedinhands-onactivitiestolearnaboutNSEandits curriculumconnectionstoGeorgiaPerformanceStandards,NextGenerationCommonCore,and8thgradeEngineeringDesignStandards.ThefirstthreeworkshopdaysincludedcombinationsofNES contentpresentationanddiscussions,includingafieldtriptoseecleanrooms;hands-onactivities

Overview

demonstratingthatcontent;and modelingactivitiestousein classrooms.Attheendofeach daytherewasa“MakingSense”sessioninwhichparticipants discussedhowcontentand activitiesfromthatdaymightbeincorporatedintotheircurriculumormodifiedtomeettheir students’needs.Teamswerealsogiventimeeachdaytoworkon theirdesignchallenges.Onthefinaldayoftheworkshop, participantsexploredthesocietalimplicationsofnanoscalescienceandengineering,discussedthedesignchallengetheywould conductwiththeirstudents,and presentedtheresultsoftheirteamdesignchallengework.A

detailedsummerworkshopscheduleispresentedin AppendixA.

Attendanceatthesummerworkshopwasexcellent. Onlyoneteachermissedoneday,andthatwasduetomedicalreasons.

Follow-UpSession

InDecember,participantsandtheirstudentsreturnedtoGeorgia Techtopresentworktheydidfortheirdesignchallenges.All19teachers,approximately 50students,andmanyoftheirparents/family membersattendedthepresentationsthattookplaceintheMarcusNanotechnologyBuilding.IntheafternoonteachersparticipatedinNEScontentsessionsandtheirstudentsworkedwithGeorgiaTechstudentsondifferentnanoscalescienceandengineeringdemonstrationsandcleanroomtours.

Evaluatorsobservedmorningstudentpresentations andconductedafocusgroupwithteacherstolearn moreabouthowthedesignchallengewas implementedfallsemester.ThecompleteagendamaybefoundinAppendixB.

TQEvaluationMethods

Theevaluationtargetedbothprocessevaluation,focusingonprojectimplementation,andoutcomesevaluation,examiningtheimpactoftheprojectonparticipants.Processevaluationactivitiessoughttoanswerthefollowingquestions:

P1:Weretheprojectactivitiesimplementedasplanned?P2:Whatwasthequalityoftheprojectactivities?

P3:Whoweretheprogramparticipantsandhowweretheyrecruited?P4:Whatfactorsaffectedprogramimplementation?

Avarietyofdatasourceswereusedtoanswertheseprocessevaluationquestions:

ProcessEvaluationQuestion / Datasource
P1:Implementation / WorkshopDocumentAnalysis/SummerWorkshopandGeorgiaTechFollowupSessionObservations/ParticipantAttendanceRecords
P2:QualityofProjectActivities / DailyFormativeSurveys;SummerWorkshopObservation;FollowupSessionObservation;Concerns,Confidence,andCommitmentSurvey
P3:Participants/Recruitingand Retention / ParticipantAttendanceRecords;Discussionswith ProjectLeadership
P4:FactorsAffectingImplementation / DiscussionsandCorrespondencewithProjectLeadership/DiscussionswithProjectParticipants duringSummerWorkshop andFollowupSession.

TheDailyFormativeSurveyssupplementedtherequiredTQSurveysbytargetingspecificprojectcomponentsonadailybasisandthusprovidingtimelyformativedatafortheprojectleadership. Participants’dailyreflectionswereanalyzedbyprojectstaffforplanningpurposes.TheWorkshopdocumentanalysisexaminedthecontentandthequalityofworkshopplansandmaterials.

Observingworkshopandfollow-upsessionsprovideddataforassessingthefidelityofprojectimplementation,thequalityoftraining,andprovidedcontextforunderstandingsurveyandfocus groupdata.Attendancerecordsweremonitoredthroughouttheproject.Evaluatorsalso correspondedwithprojectleadershipaboutaspectsoftheproject.Informationfromthesediscussions,alongwithdatafromtheothersources,helpedtheevaluatorsbetterunderstandtheproject,documentunexpectedoutcomes,andidentifyfactors/barriersthataffectedtheproject.

Thefiveoutcomeevaluationquestionsaredirectlycorrelatedwithprojectobjectives.Outcomeevaluationdatacollectionactivitiesweredesignedtogatherinformationaboutindicatorsassociated witheachoftheprogramobjectives.Evaluationanddatasourcesarelistedinthetable below.

TQEvaluationMethods

OutcomeEvaluationQuestion / DataSource
O1:Towhatextentdidparticipantsincreasetheircontentknowledgeofnanotechnologyand relatedcontentstandards? / Concerns,Confidence, CommitmentInstrument(Pre/Post)
O2:Towhatextentwereparticipantsabletoapplydeveloptheirunderstandingoftheconnections betweenNSEandtraditionallytaughtscience, math,andEnglishlanguagearts? / Concerns,Confidence, CommitmentInstrument(Pre/Post) ParticipantFocusGroup Observationofstudentpresentations
O3:Towhatextentwereparticipantsabletoapplytheirunderstandingofnanotechnologycontentandmaterialsandtheirunderstandingof contentstandardstoaddressdesign challenges? / Concerns,Confidence, CommitmentInstrument(Pre/Post) Observationofparticipantpresentations
Observationofstudentpresentations
O4:Towhatextentwereparticipantsabletoimplementdesignchallengesrequiringuseof nanoscalematerialswiththeirstudents? / Observationofparticipantpresentations
Observationofstudentpresentations ParticipantFocusGroup
O5:Towhatextentdidparticipationindesignchallengesanduseofnanoscalematerials increasestudentinterestinSTEM? / Observationofstudentpresentations ParticipantFocusGroup

Theprimarytoolsforanalyzingoutcomesweretheself-reportsurveys,observationofstudentpresentations,analysisparticipantdailyfeedback,andafocusgroup.Analysisofdataprimarily usedqualitativestrategiestoidentifypatternsofresponsesandsimpledescriptiveanalysesand graphicpresentationoffindings.Concerns,Confidence,Commitmentsurveyswereanalyzedusing simplet-teststodetermineifpre/postdifferencesindispositionsrelatedtoNEStopicsandactivitieswerestatisticallysignificant.

DailyFormativeSurveys

Attheendofeachdayoftheworkshop(June10-13),participantscompletedonlineformativesurveysabouttheday’sactivities.Specifically,theywereaskedtoratehoweffectivetheday’s activitieswereinequippingthemtoincorporatenanotechnology-relatedcontentorinquiry-basedlearningintotheirteaching.Theywerealsoaskedto describewhattheylikedmostandleastabouttheactivities.Theprojectusedthisformativefeedbacktomakeongoingjudgmentsregardingitseffectivenessandtoadjustworkshopplans.Belowisasummaryofthefeedbackfromparticipantsforeachday.

Tuesday,June10(n=19)

Notatall / Slightly / Somewhat / Quite / Very
TeamBuilding-­‐BannerMaking / 0.00% / 0.00% / 10.53% / 52.63% / 36.84%
IntroductiontoNSE,BigIdeas,EngineeringDesign,andConnectionstoSTEMandLearningGoalsforSize andScale / 0.00% / 0.00% / 10.53% / 21.05% / 68.42%
NoodlingAroundwithSI / 0.00% / 0.00% / 0.00% / 31.58% / 68.42%
Height-­‐Deci,Centi,Milli / 0.00% / 0.00% / 0.00% / 31.58% / 68.42%
SharingofIdeasfromreadings / 0.00% / 5.26% / 31.58% / 26.32% / 36.84%
IntroductiontoEngineeringDesign / 0.00% / 0.00% / 5.26% / 47.37% / 47.37%
GoogleSketch-­‐upTraining / 0.00% / 5.26% / 5.26% / 31.58% / 57.89%
BiomimicryDesignEngineeringfromNature / 0.00% / 0.00% / 10.53% / 47.37% / 42.11%
SenseMaking-­‐Ideasforincorporatingengineering intoyourcurriculum / 0.00% / 0.00% / 5.26% / 52.63% / 42.11%

Wednesday,June11(n=20)

Notatall / Slightly / Somewhat / Quite / Very
IntroductiontoUniqueProperties,SizeDependentProperties,andLearningGoals / 0.00% / 0.00% / 5.00% / 50.00% / 45.00%
OdorsAloft / 0.00% / 0.00% / 5.00% / 20.00% / 75.00%
SpaghettiandMeatballsforAll / 0.00% / 0.00% / 10.00% / 35.00% / 55.00%
AlkaSeltzer-­‐Whatmakesfasterreaction / 0.00% / 0.00% / 5.00% / 15.00% / 80.00%
DietCokeandMentos-­‐ / 0.00% / 0.00% / 5.00% / 20.00% / 75.00%
ClassificationofMixtures / 0.00% / 0.00% / 10.00% / 40.00% / 50.00%
TestingofSunscreens / 0.00% / 0.00% / 0.00% / 30.00% / 70.00%
CleanroomTour / 0.00% / 0.00% / 15.00% / 30.00% / 55.00%
Magnets-­‐Soap Bubbles / 0.00% / 0.00% / 0.00% / 31.58% / 68.42%
CarbonDioxideBubbles / 0.00% / 0.00% / 0.00% / 30.00% / 70.00%
Encapsulation / 0.00% / 5.00% / 5.00% / 45.00% / 45.00%
DesignChallengeTeamWorkTime / 0.00% / 0.00% / 10.00% / 50.00% / 40.00%
SenseMaking-­‐WheredoesNSEfitintoyourcurriculum / 0.00% / 0.00% / 15.79% / 36.84% / 47.37%

DailyFormativeSurveys

Thursday, June12(n=19) / Notatall / Slightly / Somewhat / Quite / Very
IntroductiontoStructureofMatterandToolsandInstrumentation / 0.00% / 0.00% / 5.26% / 57.89% / 36.84%
Whatdoesasolid,liquid,gaslooklikeattheatomic level / 0.00% / 0.00% / 5.26% / 31.58% / 63.16%
StateofMatterGame / 0.00% / 0.00% / 5.26% / 21.05% / 73.68%
BigIdeasataverysmallscale / 0.00% / 0.00% / 0.00% / 42.11% / 57.89%
SmartMetal / 0.00% / 0.00% / 0.00% / 36.84% / 63.16%
IntroductiontoSEM / 0.00% / 0.00% / 15.79% / 36.84% / 47.37%
MittenChallenge-­‐RefrigeratorMagnet / 0.00% / 0.00% / 5.26% / 42.11% / 52.63%
BlackBox / 0.00% / 0.00% / 5.26% / 31.58% / 63.16%
SEMMatchingGame / 0.00% / 0.00% / 5.26% / 26.32% / 68.42%
DesignChallengeTeam / 0.00% / 5.26% / 15.79% / 36.84% / 42.11%
WorkTime / 0.00% / 5.26% / 10.53% / 47.37% / 36.84%
SenseMaking-­‐WhatmakesNSESTEM? / 0.00% / 0.00% / 21.05% / 36.84% / 42.11%
Friday, June13(n=18) / Notatall / Slightly / Somewhat / Quite / Very
Video:InvisibilityCloak(Whodecideshow technologywillbeused?) / 0.00% / 0.00% / 5.56% / 44.44% / 50.00%
WouldYouBuyThat? / 0.00% / 0.00% / 0.00% / 44.44% / 55.56%
FromSmallScienceComesBigDecisions / 0.00% / 0.00% / 0.00% / 27.78% / 72.22%
SchoolDesignChallenge / 0.00% / 0.00% / 11.11% / 33.33% / 55.56%
TeamChallengePlanningtime / 0.00% / 0.00% / 16.67% / 38.89% / 44.44%
TeamPresentations / 0.00% / 0.00% / 0.00% / 55.56% / 44.44%
FinalSurveyandgivingoutofmaterials / 0.00% / 0.00% / 0.00% / 50.00% / 50.00%

Analysisofdailyresponsesshowsthatparticipantswereverypositiveabouttheeffectivenessofworkshopactivities.Therewereonlytwoactivities(SharingofideasfromreadingsandSensemaking-whatmakesNSESTEM)forwhichfewerthan80%ofrespondentsindicatedtheactivitywaseither“quite”or“very”effectiveinpreparingthemtoimplementnanotechnology-relatedcontentorinquiryintheirclassrooms.

Participantdescriptionsofwhattheylikedbestaboutactivitiesvariedwidely.Manycitedspecificactivitiesfromthatdayornotedthattheactivitieswerehandson,engagingorthoughtprovoking(“Allthenanosciencethatis involvedinstuffyouwouldn'tthink.”).Othersmentionedapplicabilityoftheactivitiestotheirclassrooms(“Ilovehowengagingtheywere.Icanreallyseemyselfdoingtheminmyclassroom.”).Severalalsomentioned theylikedtheopportunitytocollaborateandlearnfromothers(“Ilikedbeingabletoworkwithmyteamand cooperativelydesignourprojects.”).Askedwhattheydidnotlike,participantsmostfrequentlysaidtheyliked everythingthatday.Somewrotetheywouldliketohavehadmoretimeforactivities(“...moretimeduringtheweektoworkondesignprojects”)andafewindicatedthatthelevelofthematerialwaschallenginggiventheir lackofbackground.

Concerns,Confidence,CommitmentSurvey

ParticipantscompletedtheConcerns,Confidence,andCommitmentSurveyatthebeginningofthesummer workshopinJune(pre)andattheTQFollow-upWorkshop inDecember(post).TheCCCsurveyexaminedparticipants’self-reportedlevelsofconcern,confidence,andcommitmentregardingtwoareas:1)Nanoscalesciencetopicsand2)implementationofnanoscalescienceactivitieswiththeirstudents.Participantswereaskedto ratetheirfeelingsonitemsusingafivepointLikertScale(1=LowLevelto5=HighLevel).Likertscalesweretailoredforeacharea(e.g.,commitmentscalefrom1=“I’mreluctanttocommittimeandresources”to5=I’mfullypreparedtocommittimeandresources”)Surveyswerecollectedforall20participantsinJuneand19 participantsinDecember.Scalescoreswerecalculatedforconcerns,confidence,andcommitmentforeachofthetwoareasbyaddingrelateditemscores.Pre/postdifferenceswereanalyzedusingt-tests.

NanoscaleScienceEngineeringTopics

Analysisofscoresonthelevelofconcernscaleindicatedtherewasastatisticallysignificantreductionin participants’concernsfornanoscaletopics(t=8.22,p.01).Therewasalsoastatisticallysignificantincreasein theirconfidenceworkingwithnanoscaletopics(t=9.39,p.01).Thedifferencebetweenpre/postscoresonthecommitmentscalewerenotsignificantlydifferent.Notethatparticipants’levelsofcommitmenttoworkingwithnanoscalesciencetopicswerealreadyhighatthebeginningoftheproject.

Thefollowingtablesshowthemeanresponseforeachitemonthetopicsscale.Levelsofconcernwerereduced acrossalleighttopics,thelargestreductionbeingforsocietalimplicationofnanoscalescienceengineering.Foralltopics,levelofconfidenceincreasedfromthebeginningtotheendoftheproject.Thelargestincreasesinconfidencewereforsizeandscaleofnano;nanotechnologytoolsandinstrumentation;andsocietalimplicationof nanoscalescienceengineering.Levelsofcommitmentalsoincreasedacrossalltopicsalthoughtheseincreases weresmall,andasindicatedabove,therewasnosignificancebetweenpreandpostforthecombineditemsonthis scale.

CCCSurvey

CCCSurvey

NanoscaleScienceActivities

Analysisofscoresonthelevelofconcernscaleindicatedtherewasastatisticallysignificantreductionin participants’concernsforimplementingnanoscalescienceengineeringactivitieswiththeirstudents(t=6.73, p.01).Therewasalsoastatisticallysignificantincreaseintheirconfidenceusingtheseactivities(t=7.41, p.01).Thedifferencebetweenpre/postscoresonthecommitmentscalewerenotsignificantlydifferent.

Again,participants’levelsofcommitmenttousingnanoscalescienceactivitieswerealreadyhighatthebeginningoftheproject,almost4onthe5pointscale.

Concern / Confidence / Commitment
Pre / 3.85 / 1.96 / 3.97
Post / 2.11 / 3.98 / 4.22

CCCSurvey

AswasthecasewithNEStopics,levelsofconcernforimplementingnanoscaleengineeringactivitieswerereduced acrossallninetypesofactivities.Thelargestreductioninconcernswasforusingnanoscaleinstructionalmaterialsandforusingtheengineeringdesignprocess.Thelatterisveryimportantsincethiswasthefocusofstudent projectsduringthefallsemester.Foralltypesofactivities,levelsofconfidenceincreasedfromthebeginning totheendoftheproject.Thelargestincreasesinconfidencewereusingnanoscaleinstructionalmaterials andusingnanoscaleengineeringmodelsandsimulations.Levelsofcommitmentalsoincreasedacross alltypesofactivities.Again,theseincreasesweresmallbuttherewasahighlevelofcommitmentatthebeginning oftheproject.Teacher ratingswere3.9orhigher(ona5pointscale)foralltypesofactivities.

CCCSurvey

Afocusgroupwasconductedwith19 participantsaspartofthefollowupsession atGeorgiaTechUniversityintheMarcusNanotechnologyBuilding.Thepurposeof thefocusgroupwastogatherinformation aboutparticipantexperiencesimplementing thedesignchallengewiththeirstudents.

Thequestionswere:

•Howdidyouworkedasateamto takeyourstudentsthroughthedesignchallenge?

•Towhatextentdidthesummertrainingprepareyoutodothe

designchallengewithyour students?Whatelsewouldhavehelped?

•Whatwerethebiggestchallenges

yourstudentsfacedinthedesign challengeandhowdidyouaddress them?

•Whatdidstudentslearnfromdoingthedesignchallengethatwasdifferentfromwhattheynormallywouldlearninyour classes?

•Doyouplantodothisagainwithyour

students?Doyouhavesuggestionsfor modifications?Werethereanylessonslearned?

Theresultsfromthefocusgrouparesummarized belowandorganizedbyquestion.

Working asateam.Itwasclearatallfivesitesthatteachersworkedtogetheronthedesignchallenge, althoughthewaystheycollaboratedand implementedtheirdesignchallengevaried.Thiswas dueinlargeparttohowtheywereabletoscheduletimefortheactivity.Inoneschoolthechallenge wasdoneasanafterschoolclubactivitywith studentsingrades3-5.Theteachers(whoactually taughtgradesK,3,and4)workedtogetherintheafterschoolchallengeactivityandthenconductedthedesignchallengewiththeirownstudentsduringtheschoolday.Inanotherschoolwith1st,4thand5thgradeparticipants,theteacherplannedthedesign challengetogether,workedonalignmentofthe

challengewithgradelevelstandards,butimplementeditindependentlyintheirclassrooms.

Atonesiteteachersworkedonthedesignchallengeinatwo-hourmath/scienceblock.Thelanguagearts teacherservedmoreinaresourceroleby coordinatingwiththeotherteachersandconnecting contentinherlanguageartsclasstotopicsofthedesignchallenge.Forexample,studentswatchedYouTubevideosaboutamanbornwithouthands,lookedat3-Dprinters,andreadanarticlein Smithsonianabouthanddesign.Inanotherschool, thelanguageartsteacherworkeddirectlywith studentsontheirchallengeproject,comingintothesciencelabandworkingwithstudentsastheytested theirmodels.

Anothersiteimplementedthedesignchallengeinall4thand5thgradeclassesintheirschool.Oneteacher identifiedadesignchallengethatalignedtothestandardsandwasrelevanttostudents.Inthisway theyfelttheteamwasabletoengageallstudentsand thatmostofthestudentswereabletoexcelinthehandsonactivities.

Thelastsitehadatwo-hourSTEMblockthat

FocusGroup

coincidedwiththeplanningtimeofthelanguageartsteacher,sothatallthreeteacherswereabletoworkwithstudentsonthechallenge.Theteachers identifiedatopicthatalignedwith7thand8thgrademathandsciencestandards.Thelanguagearts teacherworkedwithstudentsonhowtoconductresearch.

SummerPreparation

All19participantsindicatedthat,asaresultofthesummerworkshop,theyfeltverypreparedto implementthedesignchallengewiththeirstudents. Oneteachersuggestedthathavinglessonplansfor thehandsonscienceactivitieswouldbehelpful,butshewasremindedbytheotherteachersthatthesewereincludedonaCDtheyreceived.Another teachersuggesteditwouldbehelpfultohavecharts withamoredetailedalignmentbetweenNEStopics andgradelevelstandards.Finally,ateacherrecommendedhavingalistofbusinessorindustry contactsthatcouldbeusedtogetaccesstoup-to- dateresourcestousefortheNESchallengeprojects. Theteachersaidthatitwasnotuntilafter completingthedesignchallengethatsherealized howusefulthislistmightbe.

BiggestChallenges

Overall,thebiggestchallengeforimplementingthedesignchallengeprojectwastime.Issuesrelatedto

timewere:

  • Workingtheprojectintotheiralreadybusydays;and
  • Fittingthedesignchallengeprojectintothecurriculumandstandardstheyalreadyhave.

Severalteachersmentionthatthedesignchallengewassomethingthatwasnewfortheir students,differentfromthenormalscienceexperimentsthathavearightanswers.Oneteachersaid, “It’sanewconceptforourkidsforthemtothinkforthemselves.”However,teacherscommentedthatstudentsresponded tothechallengeofhavingtofigureoutthings ontheirown.

Learningthatwasdifferent

Theparticipantswereaskedwhattheir studentslearnedfromdoingthedesign challengethatwasdifferentfromwhattheywouldnormallylearnintheirclasses.Mostoftheresponsescenteredaroundfourideas:

1)thevalueofproblem-solvingand engineeringdesignprocessversusfollowinga

recipeforanexperiment;2)havinganopportunityto think“outsidethebox”;3)beingabletowork

collaborativelyandlearnteamworkskills;and4) learningaboutnanoscaletopics.Seeexamplesofthecommentsbelow.

  • “[Students]workingtogetherandproblem

solving;knowingthattheyareactually

FocusGroup

capableofsolvingproblemsthatthey canapplytotherealworld.”

  • ‘Everythingisnothandedto

you..sometimesyoudon’tknowwhattheproblemisandyouhavetoaddress thattoo...Theywerenotusedtothataspectoflearning.”

  • “Tohearthemdiscussandsaywhatif

wediditthiswayinsteadofthatway,orhowwoulditbeifwetriedthis.”

  • “Onethingthattheycommentedonover andoveragainwasthefactthattheygottotestandretest.It wasn’tjust‘Oh.Ifailed.That’sabummer.IhateIwastedallthattimeonit.Itwaslike,waita

second,howdidwefail,whatdoweneedtodotochangeittomakeitbetter.”

  • “Andweweren’tconstantlytellingthem

howtodoit…weweren’tconstantlytellingthemtheyneededtobeontask.”

  • “Ithinkoverthesummertheyshowedusa

videoofnanojobsandwhatkindofjobs havetodowithnano-technologyandI showedmykidsthatasanintrotoitand theywereallfascinatedbythat.”

FutureDesignChallengeImplementationandModifications

Allparticipantsindicatedtheyplanonimplementing thedesignchallengewiththeirstudentsinthefuture. Havingmoretimeforthedesignchallengewas mentionmostfrequently.Twoparticipantssaidtheywouldaddmoretimeforformalresearchontheir topics.Anothercommentedthatthemoretimeteachersandstudentsareabletoputintothechallengethebettertheoutcomes.Whenasked aboutthegeneralamountoftimeittooktocompletetheirdesignchallenges,answersvariedwidely:

  • aboutamonth
  • twoweeks
  • fivedays
  • aboutthreeweeks
  • aboutfivedaysforactualconstruction

Oneteacherdescribedhowtheirdesignchallengeprojectdidn’talignwithwhattheywerestudyingaboutatthetimeandthat,inhindsight,sheprobably wouldhaveaddedmorelanguagearts.Anothersaid thattheirtopicwouldprobablybebettertopursuein thespring.Oneelementaryteachersuggestedthattheybeprovidedwithlessonplansofthemiddleschoolpresentationsfromearlierintheafternoonso theycouldbemodifiedfortheyoungergrades.

Inpreparationforthestudentpresentations, participantsweresentalistofquestionsforthestudentstoaddress.Thesequestionswere:

  • Tellmehowyouusedideasandskillsfor

differentsubjectareas(math,science,languagearts,socialstudies…)tohelpyou withyourdesignchallenge?

  • Whatwasdifferentaboutdoingthedesign

challengefromwhatyounormallydoin school?

  • Whatdidyoulearnfromdoingtheproject?

Mostofthegroupsdirectlyaddressedthesequestionsandotherssprinkledanswersthroughouttheirpresentations.Students’answersaresummarizedbelow.

IdeasandSkillsfromDifferentSubjectAreasStudentshadnodifficultydiscussingwaystheydrewonvarioussubjectareaswhenworkingontheir designchallenges.Mathapplicationsfocused primarilyonmeasurementandconversion(e.g. millimeterstonanometers).Englishlanguagearts applicationswerewritingofresearchprojects,

readingforbackgroundinformation,andhowto conductresearchontopics.Applicationsforsocialstudiesweresimilarwithstudentscitingreadingto learnmoreabouttopicsandculture.Scienceideas includednanoscalescience,microbiology,humananatomy(hand),factsaboutowls,andlearningaboutmicroscopesandhowtheyareengineered.

DifferentfromWhatyouNormallydo

Fewerteamsaddressedthisquestion,butresponses wereconsistent.Mostfrequentlystudentsdiscussed howdoingthedesignchallengewasdifferentinthatitwasopen-endedandtheirteachersdidnotgivethemasetofdirectionstofollowtocomeupwith theiranswers.Studentsmentioned:

  • Usuallytheyhavedirectionsandlimits
  • Theyhadtostartfromscratch
  • Itwasharderbecauseinschoolwearegiven directions.
  • Hadtodoitbythemselves

But,studentsalsosawthisasabenefit:

  • Morefunthanwhatisin school.
  • Theygottoexploredifferentideasanduseengineering

process.

  • Withthedesignchallengethestudentssaidtheycoulddo whatevertheywant.Inregular schooltheyhavetodowhatthe

teacherhasplanned.

Mostgroupsalsosaidthattheygotto workwithotherstudentstosolvethedesignchallenge.

WhatTheyLearned

Althoughstudentpresentationswererichwithsubjectmatter,whenitcametimetodiscusswhattheylearneddoing thedesignchallenge,mostmention thingstheylearnedthroughtheprocess.

Theseincluded:

  • Timemanagement(don’twastetime)
  • Teamwork
  • Teamworkskills(Don’targue,learntocompromise)
  • Benefitsofworkingasateam(easiertoworkinteamsthanalone)
  • Engineeringprocess
  • Persevere
  • Learndifferentthingsthroughtrialanderror
  • Letyourcreativityrunwild

Contentmentionedbystudentsincludedthesizeof nanometers,conversionofCelsiustoFahrenheit, solarenergy,and“nomatterwhattimeperiodyou’rein,nanosciencedominates.”

SummaryFindingsandRecommendations

Thisfinalsectionofthereportconsistsoftwoparts:1)Evaluationfindingsorganizedaroundtheevaluationquestionsand2)Recommendations.

PROCESS EVALUATION QUESTIONS

P1:Weretheprojectactivitiesimplementedasplanned?

AllmajorNESprojectactivitieswereimplementedasplanned.

P2:Whatwasthequalityoftheprojectactivities?

Accordingtoobservationdata,DailyFormativeSurveys,andfeedbackfromparticipants,thequalityoftheprojectactivitieswasexcellent.Workshopactivitieswerehighlyengagingandaddressedtheneedsofawiderangeofgradelevels.Participantsallindicatedtheywerepreparedtoimplementdesignchallengeactivitieswiththeirstudents.Participantsuggestions forimprovementsweremoreforenhancementsintheprogram(e.g.,providingcopiesof participants’lessonplans)thancorrectionsofanyshortcomings.

P3:Whoweretheprogramparticipantsandhowweretheyrecruited?

Projectparticipantswerefromthedistrictstargetedintheproposal.Recruitmentofinterdisciplinaryteamsproceededasplanned.Onlyoneparticipantdroppedoutoftheprogramandthiswasduetohimbeingreassignedtoahighschoolposition.

P4:Whatfactorsaffectedprogramimplementation?

Nofactorswereidentifiedthatsignificantlyaffectedprogramimplementation.

OUTCOME EVALUATION QUESTIONS

O1:Towhatextentdidparticipantsincreasetheircontentknowledgeofnanotechnologyandrelatedcontentstandards?

Participantpre/postCCCsurveyresultsindicatedastatisticallysignificantincreaseintheir confidenceandreductionsintheirconcernsaboutusingnanoscalescienceandengineering contentwiththeirstudents.Therewerepre/postgainsforalltopicareasaddressedintheworkshop.

O2:TowhatextentwereparticipantsabletoapplydeveloptheirunderstandingoftheconnectionsbetweenNSEandtraditionallytaughtscience,math,andEnglishlanguagearts?

Allwereabletointegratescience,math,andEnglishlanguage-artsintotheirprojectsand studentswereabletodescribehowthesedifferentsubjectareaideasandskillscontributedtotheirdesignchallengework.Designchallengeactivitieswere,forthemostpart,alignedwith subjectareastandards.Inthoseinstanceswheretheywerenot,itseemedtobemoreanissueof

timing-whenthedesignchallengehadtobecompletednotcoincidingwithwhattheyweretobestudyingatthattimeoftheyear.

O3:Towhatextentwereparticipantsabletoapplytheirunderstandingofnanotechnologycontentandmaterialsandtheirunderstandingofcontentstandardstoaddressdesignchallenges?

Allparticipantswereabletoimplementnanoscalescienceandengineeringdesignchallenges withtheirstudents.Studentpresentationsatthefollowupsessiondemonstratedtheir understandingofthedesignprocess.Asnotedabove,participantswereabletoaligndesign challengecontentandprocesseswithGeorgiacontentstandards.

O4:Towhatextentwereparticipantsabletoimplementdesignchallengesrequiringuseofnanoscalematerialswiththeirstudents?

Asindicatedbystudentpresentationsandteacherresponsestofocusgroupandsurveyquestions, allwereabletoincludeaspectsofnanoscalescienceandmaterialsinthedesignchallengeswith theirstudents.

O5:TowhatextentdidparticipationindesignchallengesanduseofnanoscalematerialsincreasestudentinterestinSTEM?

Thisquestionwasnotdirectlyaddressedbytheevaluation.Insteadoftheplannedstudent survey,weincludedquestionsforthestudentstoansweraspartoftheirpresentations.AlthoughwecannotmakeanyconclusionabouthowthedesignchallengeaffectedstudentinterestinSTEM,wecanstatethatstudentinterestedinnanoscalescienceandengineeringwaspositivelyaffectedasdemonstratedinstudentpresentationsatthefollowupsessionandbyobservationsoftheir enthusiasticinteractionswithGeorgiaTech studentsintheafternoonactivities.

RECOMMENDATIONS

Byallmeasures,theprojectwasverysuccessful.ParticipantslearnedaboutnanoscaletopicsandtheengineeringprocessandwereabletoincorporatetheseintheirNESdesignchallenges.Resultsofsurveys andaparticipantfocusgroup;observationofthesummerworkshop;andobservationofthefollow-upsession studentpresentationsindicatedthattheprogramwasimplementedasplannedandwasofveryhighquality.

Basedonoverallevaluationfindings,wemakethefollowingrecommendations.

1.First,keepupthegoodwork.Thisisoneofthebestprofessionaldevelopmentworkshopswehaveobserved.Thelevelofprofessionalism,thoughtfulnessofpreparation,attentionto participantneeds(e.g.,theirlackofbackgroundintheareaofnanoscalescience)was exceptional.Themanyhands-onactivitiesareengagingandnotonlyintroducenewconceptsto theteachers,butalsomodelhowtheseactivitiesmightbeusedwiththeirstudents.

2.TeachersrequestedamoredetailedcrosswalkbetweennanoscaletopicsandGeorgiaperformancestandards.Thismightbesomethingthattheteacherscontributetoastheyareintroducedtotopicsduringthesummerandworkontheirdesignchallengesbackintheir schools.IfdoneonaWikisiteitcouldbeanorganicdocumentthatteacherscontributetoandusethroughoutandbeyondtheirtimeintheproject.

3.Considerparticipants’requesttohaveaccesstodesignchallengelessonplansfromother teachersandtolistsofpossiblebusiness/industryconnectionsfornanoscalematerials.

Appendices

AppendixA:NESSummerWorkshopSchedule

Time / Tuesday / Wednesday / Thursday / Friday
8:30-­‐9:00 / WelcomePre-­‐survey / Introductionto UniqueProperties,SizeDependentProperties,and LearningGoals / Introductionto StructureofMatterand Tools and Instrumentation / Video:InvisibilityCloak(Whodecideshowtechnologywillbeused?)
9:00-­‐9:30 / TeamBuildingBannerMaking / OdorsAloftSpaghettiand MeatballsforAll / Whatdoesasolid,liquid,gaslooklikeattheatomiclevel / WouldYouBuyThat?
9:30-­‐10:00 / IntroductiontoNSE,BigIdeas,EngineeringDesign,andConnections toSTEMandLearningGoalsforSizeandScale / AlkaSeltzer-­‐Whatmakesfasterreaction / StateofMatterGame / FromSmallScienceComesBigDecisions
10:00-­‐10:30 / NoodlingAroundwithSI / DietCokeand Mentos-­‐ / BigIdeasataverysmallscale
10:30-­‐11:00 / Height-­‐Deci,Centi,Milli / Classificationof Mixtures / SmartMetal / SchoolDesignChallenge
11:00-­‐11:30 / SizeSorting-­‐What’s SmallerThanaPygmyShrew / TestingofSunscreens
11:30-­‐12:00 / SometimesWeNeed LargeNumbers
12:00-­‐12:30 / Lunch / Lunch / Lunch / Lunch-­‐PlanningforyourTeamChallenge
12:30-­‐1:00 / SharingofIdeasfromreadings / CleanroomTour / IntroductiontoSEM / TeamChallengePlanningtime
1:00-­‐1:30 / IntroductiontoEngineeringDesign / Magnets-­‐Soap Bubbles / MittenChallenge-­‐RefrigeratorMagnet / Team1presentation
1:30-­‐2:00 / CarbonDioxideBubbles / BlackBox / Team2presentation
2:00-­‐2:30 / GoogleSketch-­‐up Training / TastyEncapsulation / Team3presentation
2:30-­‐3:00 / SEMMatchingGame / Team4presentation
3:00-­‐3:30 / BiomimicryDesignEngineeringfromNature / DesignChallengeTeamWorkTime / DesignChallengeTeam
WorkTime / Team5presentation
3:30-­‐4:00 / Team6presentation
4:00-­‐4:30 / SenseMaking-­‐
Ideasforincorporatingengineeringintoyourcurriculum / Sense Making-­‐WheredoesNSEfitinto yourcurriculum / Sense Making-­‐Ideas forDesignChallengesforyourschool / FinalSurveyandgivingoutofmaterials

AppendixB:AgendaforITQFollow-­‐upWorkshop

Time / Teachers / Students
10:00-­‐10:15 / Registration / Registration
10:15-­‐10:30 / WelcomeandInstructionsfortheDay / WelcomeandInstructionsfortheDay
10:30-­‐10:45 / PresentationbyMidvaleElementary / PresentationbyMidvaleElementary
10:45-­‐11:00 / PresentationbyMontgomeryCountyElementary / PresentationbyMontgomeryCountyElementary
11:00-­‐11:15 / PresentationbyShallowfordFalls Elementary / PresentationbyShallowfordFallsElementary
11:15-­‐11:30 / StretchBreak / StretchBreak
11:30-­‐11:45 / PresentationbyMontgomeryCountyMiddle / PresentationbyMontgomeryCountyMiddle
11:45-­‐12:00 / PresentationbySimpsonMiddle / PresentationbySimpsonMiddle
12:00-­‐12:15 / PresentationbyTuckerMiddle / PresentationbyTuckerMiddle
12:15-­‐1:00 / Lunchwithteachers / LunchwithGradStudents
1:00-­‐1:30 / Ms.BonnieHarris / MidvaleElemandMontgomeryElemTabletopDemos
ShallowfordFallsandMontgomeryMiddleCleanroomTour
SimpsonMiddleandTuckerMiddleCharacterizationDemo
1:30-­‐2:00 / Dr.MindyDiSilvao / MidvaleElemandMontgomeryElemCleanroomTour
ShallowfordFallsandMontgomeryMiddleCharacterizationDemo
SimpsonMiddleandTuckerMiddleTabletopDemos
2:00-­‐2:30 / EvaluationFocusGroups / MidvaleElemandMontgomeryElemCharacterizationDemo
ShallowfordFallsandMontgomeryMiddleTabletopDemos
SimpsonMiddleandTuckerMiddleCleanroomtour
2:30-­‐2:45 / FinalSurvey / Break-­‐Snack
2:45-­‐3:00 / Wrap-­‐up / Wrap-­‐up