Stages and Activitiesin Language Test Development12页wordWord文件下载.docx

Stages and Activitiesin Language Test Development12页wordWord文件下载.docx

《Stages and Activitiesin Language Test Development12页wordWord文件下载.docx》由会员分享，可在线阅读，更多相关《Stages and Activitiesin Language Test Development12页wordWord文件下载.docx（5页珍藏版）》请在冰豆网上搜索。

　　Keywords:

stages,activities,language,test,development

　　1.Introduction

　　Testdevelopmentistheentireprocessofcreatingandusingatest,beginningwithitsinitialconceptualizationanddesign,andculminatinginoneormoreachievedtestsandtheresultsoftheiruse.Theamountoftimeandeffortweputintodevelopinglanguagetestswill,ofcourse,varydependinguponthesituation.Atoneextreme,withlow-stakestests,theprocessesmightbequiteinformal,asmightbethecaseifoneteacherwerepreparingashorttesttobeusedasoneofaseriesofweeklyquizzestoassigngrades.Attheotherextreme,withhigh-stakestests,theprocessesmightbehighlycomplex,perhapsinvolvingextensivetrailingandrevision,aswellascoordinatingtheeffortsofalargetestdevelopmentteam.Thismightbenecessaryifatestweretobeusedtomakeimportantdecisionsaffecting,alargenumberofpeople.Wewouldagainpointoutthatalthoughtheamountoftimeandeffortthatgoesintotestdevelopmentmayvary,dependingontheuseforwhichthetestisintended,thequalitiesofusefulnessneedtobecarefullyconsideredandthisconsiderationshouldnotbesacrificedineitherlow-stakesorhigh-stakessituations.

　　Weorganizetestdevelopmentconceptuallyintothreestages:

design,operationalization,andadministration.Wesay“conceptually”becausethetestdevelopmentprocessisnotstrictlysequentialinitsimplementation.Inpractice,althoughtestdevelopmentisgenerallylinear,withdevelopmentprogressingfromonestagetothenext,theprocessisalsoaniterativeone,inwhichthedecisionsthataremadeandtheactivitiescompletedatonestagemayleadustoreconsiderandrevisedecisions,andrepeatactivities,thathavebeendoneatanotherstage.Whiletherearemanywaystoorganizethetestdevelopmentprocess,wehavediscoveredovertheyearsthatthistypeoforganizationgivesabetterchanceofmonitoringtheusefulnessofthetestthroughoutthedevelopmentprocessandhenceproducingausefultest.

　　2.Stagesandactivitiesinlanguagetestdevelopment

　　Stage1:

Design

　　Inthedesignstagewedescribeindetailthecomponentsofthetestdesignthatwillenableustoinsurethatperformanceonthetesttaskswillcorrespondascloselyaspossibletolanguageuse,andthatthetestscoreswillbemaximallyusefulfortheirintendedpurposes.Designisingeneralalinearprocess,butinsomecasessomeactivitiesareiterative,thatis,willneedtoberepeatedanumberoftimes.Forexample,therearecertainpartsoftheprocess,suchasconsideringqualitiesofusefulnessandresourceallocationandmanagement,thatarerecurrentandwillneedtobeconsideredandthoughtaboutthroughouttheprocess.

　　Theproductofthedesignstageisadesignstatement,whichisadocumentthatincludesthefollowingcomponents:

　　1.adescriptionofthepurpose（s）ofthetest,

　　2.adescriptionoftheTLU（targetlanguageuse）domainandtasktypes,

　　3.adescriptionofthetesttakersforwhomthetestisintended,

　　4.adefinitionoftheconstruct（s）tobemeasured,

　　5.aplanforevaluatingthequalitiesofusefulness,and

　　6.aninventoryofrequiredandavailableresourcesandaplanfortheirallocationandmanagement.

　　Thepurposeofthisdocumentistoprovideuswithaprincipledbasisfordevelopingtesttasks,ablueprint,andtests.Itisimportanttopreparethisdocumentcarefully,forthisenablesustomonitorthesubsequentstagesofdevelopment.

　　Therearesixactivitiesinvolvedinthedesignstage,correspondingtothesixcomponentsofthedesignstatement,asindicatedabove.Thesearedescribedbrieflybelow.

　　Describingthepurpose（s）ofthetest

　　Thisactivitymakesexplicitthespecificusesforwhichthetestisintended.Itinvolvesclearlystatingthespecificinferencesaboutlanguageabilityorcapacityforlanguageuseweintendtomakeonthebasisoftestresults,andanyspecificdecisionswhichwillbebasedupontheseinferences.Theresultingstatementofpurposeprovidesabasisforconsideringthepotentialimpactoftestuse.

　　IdentifyinganddescribingtasksintheTLUdomain

　　ThisactivitymakesexplicitthetasksintheTLUdomaintowhichwewantourinferencesaboutlanguageabilitytogeneralize,anddescribesTLUtasktypesintermsofdistinctivecharacteristics.ItprovidesasetofdetaileddescriptionsoftheTLUtasktypesthatwillbethebasisfordevelopingactualtesttasks.Thesedescriptionsalsoprovideameansforconsideringthepotentialauthenticityandinteractivenessoftesttasks.

　　Describingthecharacteristicsofthelanguageusers/testtakers

　　Thisactivitymakesexplicitthenatureofthepopulationofpotentialtesttakersforwhomthetestisbeingdesigned.Theresultingdescriptionprovidesanotherbasisforconsideringthepotentialimpactoftestuse.

　　Definingtheconstructtobemeasured

　　Developingaplanforevaluatingthequalitiesofusefulness

　　Theplanforevaluatingusefulnessincludesactivitiesthatarepartofeverystageofthetestdevelopmentprocess.Aplanforassessingthequalitiesofusefulnesswillincludeaninitialconsiderationoftheappropriatebalanceamongthesixqualitiesofusefulnessandsettingminimumacceptablelevelsforeach,andachecklistofquestionsthatwewillaskabouteachtesttaskwedevelop.Assessingusefulnessinpretestingandadministeringwillincludecollectingfeedback.Thiswilldealwitharangeofinformation,bothquantitative,suchastestscoresandscoresonindividualtesttasks,andqualitative,suchasobservers’descriptionsandverbalself-reportsfromstudentsonthetesttakingprocess.Finally,theplanwillincludeproceduresforanalyzingtheinformationwehavecollected.Thiswillincludeproceduressuchasthedescriptiveanalysisoftestscores,estimatesofreliability,andappropriateanalysesofthequalitativedata.

　　Identifyingresourcesanddevelopingaplanfortheirallocationandmanagement

　　Thisactivitymakesexplicittheresources（human,material,time）thatwillberequiredandthatwillbeavailableforvariousactivitiesduringtestdevelopment,andprovidesaplanforhowtoallocateandmanagethemthroughoutthedevelopmentprocess.Thisactivityfurtherprovidesabasisforconsideringthepotentialpracticallyofthetest,andformonitoringthisthroughoutthetestdevelopmentprocess.

　　Stage2:

Operationalization

　　Operationalizationinvolvesdevelopingtesttaskspecificationsforthetypesoftesttaskstobeincludedinthetest,andablueprintthatdescribeshowtesttaskswillbeorganizedtoformactualtests.Operationalizationalsoinvolvesdevelopingandwritingtheactualtesttasks,writinginstructions,andspecifyingtheproceduresforscoringthetest.Byspecifyingtheconditionsunderwhichlanguageusewillbeelicitedandthemethodforscoringresponsestothesetasks,weareprovidingtheoperationaldefinitionoftheconstruct.

　　Developingtesttasksandablueprint

　　Indevelopingtesttasks,webeginwiththedescriptionsoftheTLUtasktypesprovidedinthedesignstatement,andmodifythese,againtakingintoconsiderationthequalitiesofusefulness,toproducetesttaskspecifications.Thesecompriseadetaileddescriptionoftherelevanttaskcharacteristics,andprovidethebasisforwritingactualtesttasks.Wewouldnotethattheparticulartaskcharacteristicsthatareincludedandtheorderinwhichtheyarearrangedinthetesttaskspecificationsarelikelytovarysomewhatfromonetestingsituationtoanother.

　　Ablueprintconsistsofcharacteristicspertainingtothestructure,oroverallorganization,ofthetest,alongwithtesttaskspecificationsforeachtasktypetobeincludedinthetest.Theblueprintdiffersfromthedesignstatementprimarilyintermsofthenarrownessofthefocusandtheamountofdetailincluded.Adesignstatementdescribesthegeneralparametersforthedesignofatest,includingitspurpose,theTLUdomainforwhichitisdesigned,theindividualswhowillbetakingthetest,whatthetestisintendedtomeasure,andsoforth.Ablueprint,ontheotherhand,describeshowactualtesttasksaretobeconstructed,andhowthesetasksaretobearrangedtoformthetest.

　　Writinginstructions

　　Writinginstructionsinvolvesdescribingfullyandexplicitlythestructureofthetest,thenatureofthetasksthetesttakerswillbepresented,andhowtheyareexpectedtorespond.Someinstructionsareverygeneralandapplytothetestasawhole.Otherinstructionsarecloselylinkedwithspecifictesttasks.

　　Specifyingthescoringmethod

　　Specifyingthescoringmethodinvolvestwosteps:

　　1.definingthecriteriabywhichthequalityofthetesttakers’responseswillbeevaluatedand

　　2.determiningtheproceduresthatwillbefollowedtoarriveatascore.

　　Stage3:

Testadministration

　　Thetestadministrationstageoftestdevelopmentinvolvesgivingthetestto