《影视制作》考试系统英文翻译文档.docx

《影视制作》考试系统英文翻译文档.docx

《《影视制作》考试系统英文翻译文档.docx》由会员分享，可在线阅读，更多相关《《影视制作》考试系统英文翻译文档.docx（24页珍藏版）》请在冰豆网上搜索。

《影视制作》考试系统英文翻译文档

毕业设计（论文）

英文翻译

学院名称：

计算机科学与技术学院

班级：

051班

学号：

20054440102

学生姓名：

周怡

2009年05月12日

AnOnlineTestingandAnalysisSystem

forCreativeProblem-SolvingAbilityinSciences

aChun-ChiehHuang,bHao-ChuanWang,aTsai-YenLiandcChun-YenChang

aNationalChengchiUniversity,Taipei,Taiwan

bCarnegieMellonUniversity,Pittsburgh,PA,USA

cNationalTaiwanNormalUniversity,Taipei,Taiwan

Email:

g9415@cs.nccu.edu.tw,haochuan@cs.cmu.edu,

li@nccu.edu.tw,changcy@ntnu.edu.tw

Abstract:

Thispaperproposesanonlinetestingandanalysissystemforstudyingstudents’CreativeProblem-Solving（CPS）abilityinsciences.Usinganopen-endedessay-question-typetest,studentsareaskedtoexpresstheirideaandimaginehowtosolveproblemsbetter.Basedonpreviousworks,weutilizeanautomatedscorerforevaluatingstudents’CPSability.Thissystemservesasareal-time（self-）assessmentforonlinelearnersandausefulresearchtooltogatherdataforCPSstudies.Wehavere-examinedtheinter-raterreliabilityofourautomatedscorerwithnewsamples,andconductedaquestionnairesurveyforusabilityinspection.Thepreliminaryresultsshowthesystemisreliableforautomatedscoringandsatisfactoryforsystemusability.Aroadmapforfuturedevelopmentisalsoproposed.

Keywords:

OnlineTesting,CreativeProblemSolving,AutomatedScoring,Web-basedLearning

1Introduction

TheassessmentofCreativeProblem-Solving（CPS）abilityistakenasausefulindextoinvestigatestudents’creativityandproblemsolvingabilitiesinpsychologicalandeducationalresearch（Chang&Weng,2002）.Wegenerallybelievethatusingassessmentofanopen-endedessay-question-typeformatisausefulapproachtoevaluatestudents’CPSability.Studentsarerequiredtoexpresstheirself-generatedthoughtsinnaturallanguages（e.g.,Chinese）insuchatest,andteachersorresearchersmayanalyzestudents’essays,oneofthemostcomprehensiveandauthenticrepresentationsofstudents’thinkingandreasoning,tostudytheircomprehension,competenceorcreativityregardingthedomainsubjects.Inthecontextofweb-basededucation（WBE）,itisconsideredessentialandbeneficialtoincorporatethistypeofassessmentsasavaluablecomponentofonlinelearningenvironmentsforbetterinstructionaldecision-makingandmoreadvancededucationalstudies.

Nevertheless,althoughonlinecourseshavebeendevelopedanddeployedextensivelymorethanever,onlinetesting,especiallyopen-endedessay-question-typetests,isstillunderemphasized.OnepossiblefactoristhattechnologiesrequiredforonlinetestingwerenotsystematicallydevelopedandincorporatedbypractitionersofWBE.Thoughthedevelopmentofcertaintypesoftesting,suchasdichotomous-,single-andmultiple-choicesquestions,maybeintuitiveandmature,thesituationisverydifferentfromthecaseofopen-endedessay-question-typetesting,inwhichtechnologiesforlanguageprocessingandlearnermodelingcanbemorecomplicatedandwerelessfamiliartotheAppearinProceedingsofthe10thGlobalChineseConferenceofComputersinEducation,June2006.WBEcommunity.Insum,webelievethatitiscrucialtoconductsystematicinvestigationonthetopicofonlinetestingforopen-endedproblemsolvingtasksinordertofurthertheeffectofonlinelearning.

Gradingisagreatburdentoteachers,especiallyforopen-endedtesting.Withlimitededucationalresourcesandgivenalargenumberofstudents,apromptandvalidwaytoevaluatestudentswouldbeveryusefultoteachers.Anapproachtosolvethisproblemisbyadoptinganautomatedscorer,madepossiblebyArtificialIntelligencetechnologies,togradealargevolumeofopen-endedtesting.

Inthisresearch,weaimtodevelopanonlinetestingsystemsuitableforscienceeducationtoassessstudents’CPSabilitiesthroughopen-endedessay-question-typetesting.Thesystemisdesignedtosupportthreemajorfunctions:

（1）students’onlineansweringtothetestquestions,

（2）reliablyautomatedprocessingandgradinguponstudents’responses,and（3）on-lineinspectionofstatisticalresultswithgraphicalvisualizationtoolsforteachersandresearchers.Morespecifically,thisworkconsistsofthreeparts:

（1）developingafullyintegratedsystemthatsupportstheaforementionedfunctionalities,

（2）conductingausabilitystudyoftheuserinterfaceandinteractiondesignofthissystem,and（3）re-testingthereliabilityofourautomatedscoringmodule,whichwaspreliminarilyshowntohaveasatisfactoryperformanceforasmallsampleofstudents（Wangetal.,2005a）.Inthiswork,wehaveconductedanewempiricalexplorationwith70studentsinvolvedinthestudiesofsystemusabilityandgradingreliability.Inaddition,wewillalsointroducesometechniquestoimplementsuchasystem,suchascontrollingstudents’onlineresourceaccessandcollectingstatisticsaboutstudents’scores.

2PreviousWorks

Thesystemdescribedinthispaperisbasedonourpreviousworksinthisseries,includingtheoreticalandempiricalinvestigationofstudents’CPSinsciences（Chang&Weng,2002）,usermodelingforCPS（Wangetal.,2005a）,andautomatedgradingtechnologiesdesignedforCPSstudies（Wangetal.,2005b）.Theworkpertainingtotheresearchinthisdirectionisdescribedasfollows.

2.1CPS:

CreativeProblem-Solving

Problem-solvingabilityisviewedasanabilitytothinkcritically,reasonanalytically,andcreateproductively,whichallinvolvesquantitative,communicationandcritical-responseskills（AmericanAssociationfortheAdvancementofScience,1993）.Thesystemwehavebuiltisbasedonthetheoriesofseveralmeasurementsofcreativityproposedintheliterature（Hocevar,1981;Yuetal.,2005）.

Inourpreviousworkondesigningthepaper-basedassessmentforCPS（Wu&Chang,2002）,wehavetakenProblem-FindingandProblem-Solvingasdifferenttestingphases,andeachphaseisfurtherseparatedastwoparts:

ideation（i.e.,divergentthinking）andexplanation（i.e.,convergentthinking）.TheseparationofsuchreasoningphaseswasoriginatedfromsometheoriesofCPS（Basadur,1995;Osborn,1963）.Thepaper-basedanswersheetwasdesignedwithatwo-columntableforstudentstoexpresstheirideasandexplanations,asshowedinFigure1.

Figure1.Asnapshotoftheanswersheetshowingthepair-wiserelationbetweenideasandreasons.

Ineachrowofthetable,studentsmayfillintwoconsecutivetextfieldswithanideaandthereason（s）explainingthatideaforaspecificproblem-solvingscenariodescribedinthetestingquestion.MoredetailsanddiscussionsofthedesignforthisassessmentcanbefoundinWu&Chang（2002）andWangetal.（2005a,2005b）.

2.2UPSAM:

UserProblem-SolvingAbilityModeler

Ourpriorworkhasalsoproposedausermodelingsystem,UserProblem-SolvingAbilityModeler（UPSAM）,whichaimsatabstractingandmodelingstudents'responsesinscientificproblem-solvingtasks（Wang,2005a）.Byrepresentingexperts’andstudents’thoughtsofproblem-solving,includingbothideasandreasons,intheformalismofbipartitegraph,ideationandexplanationaremodeledastwodisjointsetsconnectedwithundirectededges.UPSAMenablesustocompareanytwobipartite-graph-basedusermodelstoaddressresearchquestionssuchashowwellarethisstudent’sanswersbyusingtheexpert’sanswersasthecriteria（i.e.,essaygrading）andwhatconceptsareknown/unknownbythestudent（i.e.,learningdiagnosis）.

AnimportantfeatureoftheUPSAMisthatitmaintainsanexpertmodelrepresentingthedomainexperts’knowledgeandvocabularyagainsttheproblem-solvingtasks,anditservesasausefulmoduletoprocessstudents’open-endedanswersandperformautomatedgrading.

2.3AutomatedScorer

BasedonUPSAM,wehavedevelopedanautomatedscorerforgradingstudents’responsestowardtheCPStest.ThecoreconceptistouseInformationRetrieval（IR）technologiestomatchstudents’responsesagainstexperts’standardanswersandthenrepresenteachstudentasasubsetofthebipartite-graph-basedexpertmodel（Wangetal.,2005b）.Wehavecollectedmanyexperts’ideationsandexplanationsfromdifferentaspects,andorganizethemintoanexpertmodel,whichisaimedtobeobjectiveandequitable.BecauseofthestructuredanswersformatfromUPSAM,theautomatedscorercangraspstudents’keyconceptsinordertoevaluatetheirCPSabilitybasedoneachtextentry.Inaddition,wealsoinvitehumangraderstogrademanuallyforcomparisons.

Inthispriorwork,wehaveconductedanevaluationtocorroboratetheviabilityoftheautomatedscoringschemeempirically.WefoundthePearson’scorrelationstatisticsr,whichservesasanindexofinter-raterreliabilitybetweenhumangradersandourautomatedscorer,rangefrom.67to.82,whicharepositive,high,andstatisticallysignificant.Thepreliminaryexperimentsshowedthatautomatedscoringisreliabletobeusedhere,anditisalsosuggestedthatanintegratedonlinesystemshouldbedevelopedfortesting,gradingandfurtheranalyses（Wang,2005a）.

3OnlineTestingSystem:

WOTAS

Themainobjectiveofthispaperinthislineofresearchistobuildanonlinesystemintegratingourpreviouslydevelopedcomponentswiththenewlydesignedmodules（e.g.,UIfortestingandstatisticsreporting.）WehavedesignedandimplementedtheWeb-basedOnlineTestingandAnalysisSystem（WOTAS）suchthatagreatnumberofstudentscanaccessthesystemonlineconcurrentlywithgreatstabilityandusabilityandteachersorresearcherscaneasilydiagnosethelearningconditionfromthoselearners.ThearchitectureofWOTASisdescribedbelow.

3.1DesignConsiderations

Figure2.SystemarchitectureforWOTAS.

Thesystemthatwehavedesign,asdepictedinFigu