学生减负外文文献翻译.docx

学生减负外文文献翻译.docx

《学生减负外文文献翻译.docx》由会员分享，可在线阅读，更多相关《学生减负外文文献翻译.docx（9页珍藏版）》请在冰豆网上搜索。

学生减负外文文献翻译

学生减负外文文献翻译2019

英文

Apeerassessmentmethodtoprovidefeedback,consistentgradingandreducestudents'burdeninmassiveteachingsettings

OscarLuaces，JorgeDíez，AntonioBahamonde

Abstract

Togradeopen-responseanswersinamassivecourseisanimportanttaskthatcannotbehandledwithouttheassistanceofanintelligentsystemabletoextendtheabilitiesofexperts.Apeerassessmentmethodmaybeusedforthis.Thestudentswhowrotetheanswersalsoplaytheroleofgradersforareducedsetofanswersprovidedbyotherstudents.Thegradesthusobtainedshouldbeaggregatedtoprovideareasonableoverallgradeforeachanswer.However,thesesystemspresenttwocleardisadvantagesforstudents:

theyincreasetheiralreadyheavyworkload,andthegradesthatstudentsfinallyreceivelackfeedbackexplainingthereasonsfortheirscores.Thecontributionofthispapercomprisesaproposaltoovercometheseshortcomings.Thestudentsactingasgradersareaskedtoevaluateanumberofdifferentaspects.Oneofthemistheoverallgrade,butthereareotherannotationsthatcanbeincludedtoexplaintheoverallgrade.Moreover,werepresenttheresponsesgivenbythestudents（textdocuments）astheinputsinalearningtask,inwhichtheoutputsaretheaspectstobeassessed（labelswithanordinallevel）.Ourproposalistolearnalltheselabelsatonceemployingamultitaskapproachthatusesmatrixfactorization.Themethodpresentedinthispapershowsthatpeerassessmentcanprovidefeedbackandcanadditionallybeextendedtogradetheresponsesofstudentsnotinvolvedinthepeerassessmentloop,thussignificantlyreducingtheburdenonstudents.Wepresentthedetailsofthemethod,aswellasanumberofexperimentscarriedoutusingthreedatasetsobtainedfromcoursesbelongingtodifferentfieldsatouruniversity.

Keywords：

Peerassessment，Students'burdenreduction，Feedback，Factorization，Preferencelearning

Theassessmentofopen-responseassignmentsisfrequentlyaproblem.ThisisthecaseinmassivecourseslikeMOOCsorevenwhentherearealotofassignmentsduringacourse.Oneoftheoptionstoovercomethisproblemistoavoidopen-responseinfavorofmultiple-choicequestions.However,thissignificantlyreducesthecommunicationbetweenstudentsandinstructorsthatmayinvolvehandlingdifferentformsofdata,includingcomputerprograms,video,audio,andwrittentexts.Thealternativeisforthestudentsthatwrotetheanswerstoalsoplayaroleintheassessment.Peerassessmenthasbeenexploredasanefficientproceduretodealwiththisproblem;seeforinstance（Díez,Luaces,Alonso-Betanzos,Troncoso,&Bahamonde,2013; Formanek,Wenger,Buxner,Impey,&Sonam,2017; Kulkarnietal.,2015; Labutov&Studer,2016; Luaces,Díez,Alonso,Troncoso,&Bahamonde,2015a; Luaces,Díez,Alonso-Betanzos,Troncoso,&Bahamonde,2017, 2015b; Piechetal.,2013; Raman&Joachims,2014, 2015; Sadler&Good,2006; Shah,Bradley,Parekh,Wainwright,&Ramchandran,2013）.Ithasbeenacknowledgedasanactivitythatenhancesstudentlearningin Sun,Harris,Walther,andBaiocchi（2015）.

However,peerassessmenthasanumberofflawsthatshouldbeaddressedinordertobedeployedmoreextensively.Firstly,peerassessmentmayconsiderablyincreasetheburdenonstudents.Second,thequalityofthefeedbackreceivedbystudentsshouldbeimproved（Gielen,Peeters,Dochy,Onghena,&Struyven,2010; Hovardas,Tsivitanidou,&Zacharia,2014; Liu&Carless,2006; Tseng&Tsai,2007）.Inadditiontoagrade,studentsshouldobtainsomeannotationspointingtotheweakandstrongaspectsoftheiranswers. LuandLaw（2012） presentaninterestinganalysisontheeffectsofprovidingfeedbackbothforthestudentsbeingassessed,andforthestudentsactingasgraders.Theauthorsconcludethatfeedbackisclearlybeneficialandtheyalsopointoutsomeaspectsthatmustbeconsideredinordertodesigngoodpeerassessmentprocesses.

Finally,thefactthateachassignmentisevaluatedbydifferentpeerassessorsgivesrisetotheproblemofreachingaconsensustosummarizethosedifferentopinionsintoasinglegradeandafeedback.Thisisnottrivial,aswewillexplaininSection 2.

Inthispaperweexploreamethodtoaddressthementionedissuesinpeerassessmentwhenopen-responsesarewrittendocuments.Toimprovethefeedbackfromanautomatedperspective,weproposetouseasetoflabelsorannotationsthatmaybeattachedtoanswerswithalevel.Theselabelsshouldcovertheexplanationsthatastudentcouldobtainfromapersonalizedassessmentgivenbyaprofessionalinstructor.Wetestedthisproposalinthreecoursesatouruniversitybelongingtodifferentfields:

LawandEconomy.Instructorscouldeasilyexpressthepossibilitiesofannotationsintermsoflabelswithlevels.Ontheotherhand,thestudentswereabletounderstandtheassessmenttaskwithannotationseffortlessly.

Theoutputofpeerassessmentsisadatasetthatmustsomehowbefilteredtoaggregateorreconcilethegradesreceivedbyoneanswerfromseveralstudentsactingasgraderswithoutexperienceinthistask.ThisisusuallytackledusingMachineLearningmethods.Intheexperimentsreportedattheendofthepaper,weprovethatmodelslearnedtoaggregategradescanbeusedtoeasetheacademicworkloadofstudents.

Theideaistoextendtheassessmentmodeltoanswersnotinvolvedinpeerassessmentinanyway.Forthispurpose,weuseacontent-basedapproachsimilartothoseemployedinRecommenderSystems.Inthiscontext,contentsaredocuments（thestudents'answers）thatcanberepresentedusinga bagofwordsrepresentation.Severalapproacheshavebeenproposedtoovercomethelimitationsofthisrepresentationparadigm;seeforinstance（Deerwesteretal.,1990）.Inthispaper,weproposeamatrixfactorizationmethodtolearnhowtogradethatincludesamethodtoarrangetheanswersofstudentsinametricspaceaccordingtotheirgrades.

Noticethatwehavetolearntogradeeachoftheaspectsoftheanswerthatneedtobeconsidered:

theoverallgrade,andthelevelofeachofthelabelsorannotationsforfeedback.Wepresentamultitask（Caruana,1997）methodtosimultaneouslylearnalltheaspectstobeassessed,andweshowintheexperimentsthat,infact,thereisaninductivetransferthatimprovesthewholeMachineLearningprocess.

Thecontributionsofthisarticlecanbesummarizedasfollows.Theevaluationofopenresponseassignmentsisataskthatmustbecarriedoutbyanexpert.However,whenthereexistsalargenumberofassignments,whichwilltaketheinstructor（s）aprohibitivetimetoassess,peerassessmentiscommonlyused.Theinherentsubjectivityintheassessment,aswellasthefactthateachassignmentwillreceivedifferentmarksfromdifferentpeerassessors,posestheneedtoaggregatethosescores.Computingtheaverageisrisky,sincewehaveonlyafewassessmentsperassignment,soweneedtouseansmarterapproach.Thus,thehelpofanintelligentsystemcapableofperformingthistaskisneeded.

Generallyspeaking,anintelligentsystemisapieceofsoftwareabletoperformataskwhichrequiressomekindofintelligentbehaviour.Inthiscontext,ourproposedmethodisabletogeneralizethecriteriaofthepeerassessors（graders）,goingbeyondaveragingtheirscores,andgettingridoftheirsubjectivityintheassessment.

However,peerassessmententailsanaddedburdentothealreadylargevolumeofworkrequiredtothestudents.Thefirstcontributionofthispaperisamethodthatdoesnotrequireallstudentstoparticipateinthepeerassessment.Thesecondisthatourmethodprovidesfeedbacktostudentsintheformofasummarizedexplanationofthefinalgrade,somethingwhichisnotdoneautomaticallybyotherpeerassessmentmethods.

Thepaperisorganizedasfollows.Firstweexplainthewholeprocessasitisseenbystudentsandinstructorsandthenintroducetheinsightbehindtheapproachpresentedhere.Thefollowingsectionisdevotedtopresentingtheformalsetting.Wethenreporttheexperimentsconductedtoevaluatetheapproachpresentedinthepaper.Weendwiththeconclusionsofthisresearch.

Inthispaper,weaddresstwoimportantissuesinordertoincreasethequalityofthepeerassessmentofwrittenopen-responses:

theneedtoprovideusefulfeedbacktostudents,andtorelievetheirworkload.Theproposalrequiresgraderstoassessanumberofannotationsorlabelsabouttheanswerthattheyareassessing.Theoverallgradeisanotherlabelinthiscontext.Wehavepresentedamethodthatusesamultitaskapproachtosearchforgradingpatternsinalllabelsatthesametime.

Multitaskleveragestheaccuracyofabaselinethatsuccessivelyfocusesoneachlabelseparately.Thus,theassessmentsprovidedbystudentscanbeaggregatedinalistofgradedlabelsthatinformstheirpeersoftheiroverallgrade,aswellasprovidinganumberofreasonsexplainingweakandstrongpointsintheiranswers.

Ontheotherhand,modelslearnedusingthemultitaskapproachcanbeextendedtoanswersnotatallinvolvedinpeerassessment.Theconsequenceisthatapartofthestudentscanberelievedoftheassessmenttask,therebyreducingtheburdenonstudentsinprocessesofthiskind.

Thepaperthuspresentsanintelligentsystemthatallowsimplementingtheassessmentofopen-responseassignmentsinmassivecourses.Ourproposalpresentstwofundamentalcontributionswithrespecttotheexistingliterature.Ontheonehand,itconsiderablyreducesthestudentworkload.Infact,whenstudentsarerequiredtoassesstheworkofothers,theymustspendalotoftimeonthistask.Inourproposal,byexplicitlyusingthetextsoftheanswers,theintelligentsystemisableto