Stages and Activitiesin Language Test Development12页wordWord文件下载.docx
《Stages and Activitiesin Language Test Development12页wordWord文件下载.docx》由会员分享,可在线阅读,更多相关《Stages and Activitiesin Language Test Development12页wordWord文件下载.docx(5页珍藏版)》请在冰豆网上搜索。
Keywords:
stages,activities,language,test,development
1.Introduction
Testdevelopmentistheentireprocessofcreatingandusingatest,beginningwithitsinitialconceptualizationanddesign,andculminatinginoneormoreachievedtestsandtheresultsoftheiruse.Theamountoftimeandeffortweputintodevelopinglanguagetestswill,ofcourse,varydependinguponthesituation.Atoneextreme,withlow-stakestests,theprocessesmightbequiteinformal,asmightbethecaseifoneteacherwerepreparingashorttesttobeusedasoneofaseriesofweeklyquizzestoassigngrades.Attheotherextreme,withhigh-stakestests,theprocessesmightbehighlycomplex,perhapsinvolvingextensivetrailingandrevision,aswellascoordinatingtheeffortsofalargetestdevelopmentteam.Thismightbenecessaryifatestweretobeusedtomakeimportantdecisionsaffecting,alargenumberofpeople.Wewouldagainpointoutthatalthoughtheamountoftimeandeffortthatgoesintotestdevelopmentmayvary,dependingontheuseforwhichthetestisintended,thequalitiesofusefulnessneedtobecarefullyconsideredandthisconsiderationshouldnotbesacrificedineitherlow-stakesorhigh-stakessituations.
Weorganizetestdevelopmentconceptuallyintothreestages:
design,operationalization,andadministration.Wesay“conceptually”becausethetestdevelopmentprocessisnotstrictlysequentialinitsimplementation.Inpractice,althoughtestdevelopmentisgenerallylinear,withdevelopmentprogressingfromonestagetothenext,theprocessisalsoaniterativeone,inwhichthedecisionsthataremadeandtheactivitiescompletedatonestagemayleadustoreconsiderandrevisedecisions,andrepeatactivities,thathavebeendoneatanotherstage.Whiletherearemanywaystoorganizethetestdevelopmentprocess,wehavediscoveredovertheyearsthatthistypeoforganizationgivesabetterchanceofmonitoringtheusefulnessofthetestthroughoutthedevelopmentprocessandhenceproducingausefultest.
2.Stagesandactivitiesinlanguagetestdevelopment
Stage1:
Design
Inthedesignstagewedescribeindetailthecomponentsofthetestdesignthatwillenableustoinsurethatperformanceonthetesttaskswillcorrespondascloselyaspossibletolanguageuse,andthatthetestscoreswillbemaximallyusefulfortheirintendedpurposes.Designisingeneralalinearprocess,butinsomecasessomeactivitiesareiterative,thatis,willneedtoberepeatedanumberoftimes.Forexample,therearecertainpartsoftheprocess,suchasconsideringqualitiesofusefulnessandresourceallocationandmanagement,thatarerecurrentandwillneedtobeconsideredandthoughtaboutthroughouttheprocess.
Theproductofthedesignstageisadesignstatement,whichisadocumentthatincludesthefollowingcomponents:
1.adescriptionofthepurpose(s)ofthetest,
2.adescriptionoftheTLU(targetlanguageuse)domainandtasktypes,
3.adescriptionofthetesttakersforwhomthetestisintended,
4.adefinitionoftheconstruct(s)tobemeasured,
5.aplanforevaluatingthequalitiesofusefulness,and
6.aninventoryofrequiredandavailableresourcesandaplanfortheirallocationandmanagement.
Thepurposeofthisdocumentistoprovideuswithaprincipledbasisfordevelopingtesttasks,ablueprint,andtests.Itisimportanttopreparethisdocumentcarefully,forthisenablesustomonitorthesubsequentstagesofdevelopment.
Therearesixactivitiesinvolvedinthedesignstage,correspondingtothesixcomponentsofthedesignstatement,asindicatedabove.Thesearedescribedbrieflybelow.
Describingthepurpose(s)ofthetest
Thisactivitymakesexplicitthespecificusesforwhichthetestisintended.Itinvolvesclearlystatingthespecificinferencesaboutlanguageabilityorcapacityforlanguageuseweintendtomakeonthebasisoftestresults,andanyspecificdecisionswhichwillbebasedupontheseinferences.Theresultingstatementofpurposeprovidesabasisforconsideringthepotentialimpactoftestuse.
IdentifyinganddescribingtasksintheTLUdomain
ThisactivitymakesexplicitthetasksintheTLUdomaintowhichwewantourinferencesaboutlanguageabilitytogeneralize,anddescribesTLUtasktypesintermsofdistinctivecharacteristics.ItprovidesasetofdetaileddescriptionsoftheTLUtasktypesthatwillbethebasisfordevelopingactualtesttasks.Thesedescriptionsalsoprovideameansforconsideringthepotentialauthenticityandinteractivenessoftesttasks.
Describingthecharacteristicsofthelanguageusers/testtakers
Thisactivitymakesexplicitthenatureofthepopulationofpotentialtesttakersforwhomthetestisbeingdesigned.Theresultingdescriptionprovidesanotherbasisforconsideringthepotentialimpactoftestuse.
Definingtheconstructtobemeasured
Developingaplanforevaluatingthequalitiesofusefulness
Theplanforevaluatingusefulnessincludesactivitiesthatarepartofeverystageofthetestdevelopmentprocess.Aplanforassessingthequalitiesofusefulnesswillincludeaninitialconsiderationoftheappropriatebalanceamongthesixqualitiesofusefulnessandsettingminimumacceptablelevelsforeach,andachecklistofquestionsthatwewillaskabouteachtesttaskwedevelop.Assessingusefulnessinpretestingandadministeringwillincludecollectingfeedback.Thiswilldealwitharangeofinformation,bothquantitative,suchastestscoresandscoresonindividualtesttasks,andqualitative,suchasobservers’descriptionsandverbalself-reportsfromstudentsonthetesttakingprocess.Finally,theplanwillincludeproceduresforanalyzingtheinformationwehavecollected.Thiswillincludeproceduressuchasthedescriptiveanalysisoftestscores,estimatesofreliability,andappropriateanalysesofthequalitativedata.
Identifyingresourcesanddevelopingaplanfortheirallocationandmanagement
Thisactivitymakesexplicittheresources(human,material,time)thatwillberequiredandthatwillbeavailableforvariousactivitiesduringtestdevelopment,andprovidesaplanforhowtoallocateandmanagethemthroughoutthedevelopmentprocess.Thisactivityfurtherprovidesabasisforconsideringthepotentialpracticallyofthetest,andformonitoringthisthroughoutthetestdevelopmentprocess.
Stage2:
Operationalization
Operationalizationinvolvesdevelopingtesttaskspecificationsforthetypesoftesttaskstobeincludedinthetest,andablueprintthatdescribeshowtesttaskswillbeorganizedtoformactualtests.Operationalizationalsoinvolvesdevelopingandwritingtheactualtesttasks,writinginstructions,andspecifyingtheproceduresforscoringthetest.Byspecifyingtheconditionsunderwhichlanguageusewillbeelicitedandthemethodforscoringresponsestothesetasks,weareprovidingtheoperationaldefinitionoftheconstruct.
Developingtesttasksandablueprint
Indevelopingtesttasks,webeginwiththedescriptionsoftheTLUtasktypesprovidedinthedesignstatement,andmodifythese,againtakingintoconsiderationthequalitiesofusefulness,toproducetesttaskspecifications.Thesecompriseadetaileddescriptionoftherelevanttaskcharacteristics,andprovidethebasisforwritingactualtesttasks.Wewouldnotethattheparticulartaskcharacteristicsthatareincludedandtheorderinwhichtheyarearrangedinthetesttaskspecificationsarelikelytovarysomewhatfromonetestingsituationtoanother.
Ablueprintconsistsofcharacteristicspertainingtothestructure,oroverallorganization,ofthetest,alongwithtesttaskspecificationsforeachtasktypetobeincludedinthetest.Theblueprintdiffersfromthedesignstatementprimarilyintermsofthenarrownessofthefocusandtheamountofdetailincluded.Adesignstatementdescribesthegeneralparametersforthedesignofatest,includingitspurpose,theTLUdomainforwhichitisdesigned,theindividualswhowillbetakingthetest,whatthetestisintendedtomeasure,andsoforth.Ablueprint,ontheotherhand,describeshowactualtesttasksaretobeconstructed,andhowthesetasksaretobearrangedtoformthetest.
Writinginstructions
Writinginstructionsinvolvesdescribingfullyandexplicitlythestructureofthetest,thenatureofthetasksthetesttakerswillbepresented,andhowtheyareexpectedtorespond.Someinstructionsareverygeneralandapplytothetestasawhole.Otherinstructionsarecloselylinkedwithspecifictesttasks.
Specifyingthescoringmethod
Specifyingthescoringmethodinvolvestwosteps:
1.definingthecriteriabywhichthequalityofthetesttakers’responseswillbeevaluatedand
2.determiningtheproceduresthatwillbefollowedtoarriveatascore.
Stage3:
Testadministration
Thetestadministrationstageoftestdevelopmentinvolvesgivingthetestto