英语测试专四口语考试.docx

英语测试专四口语考试.docx

《英语测试专四口语考试.docx》由会员分享，可在线阅读，更多相关《英语测试专四口语考试.docx（8页珍藏版）》请在冰豆网上搜索。

英语测试专四口语考试

ACritiquetotheValidityandReliabilityofOralTestinTEM4in2012

June2014

CourseName:

LanguageTesting

Lecturer:

Dr.LiuMin

StudentID&Name:

ACritiquetotheValidityandReliabilityofOralTestinTEM4in2012

1.Introduction

Listening,speaking,reading,andwritingarethe4basicskillsoflearningaforeignlanguage.Listening,reading,andwriting,these3languageskillshavealreadygothighattentioninTEM4andTEM8.However,theoraltest,representingthespeakingskill,ishardtoworkoneffect.Asasubjectiveexam,reliabilityassessalwaysrestrictsthedevelopmentoforaltests.Reliabilityassessandvalidityassessareofequalimportanceinexaminationtheory.

TEM4OralEnglishTestconsistsofthreeparts:

retellingastory（listentothestorytwiceandretellfor3minutes）,talkingonagiventopicfor3minutesandrole-playingfor4minutes.ThescoringmethodsynthesizestheadvantageofspeakingtestofTOEFLandoraltestinHongkong.Itgivesspecificexplanatorynotesforeverypartsandmacrocontrolofunfairscoringproblemsofthejudgment.Theaudiotapesofthecandidatesarerandomlyassignedintogroups.Eachaudiotapeisscoredbytwoteachersandthefinalscoreistheaveragegradeofthetwoteachers.

Throughtest-theorystudyandstatisticanalysis,thispaperprobeintothevalidityandreliabilityoforaltestinTEM4in2012.Reliabilityparticularlyemphasizesonthetestcontent,testingandgrading.Validitywillfocusonthecontentvalidity,facevalidity,standardvalidityandtheoryvalidity.Inaddition,byusingreliabilitycoefficienttotestreliability,thispaperwillcometoaconclusionofreliabilityandvalidityofthecontentandtestinginOralLanguageTestthroughdataanalysis.

OrallanguagetestisavitalpartinLanguageTesting.Also,itisapartoflinguisticstudy,thatis,applicationofforeignlanguageoraltheoryinforeignlanguageoralteachingtesting.OralLanguageTestofTEM4hasalreadyspreadmorethan10years.Butwhetheritefficientlyreflectsorallevelsofexaminees?

Guidedbythelanguagetestingtheoryandorallinguisticstheory,thispaperwillanalyzequalityofthisorallanguagetestin2012fromreliabilityandvalidity.

2.TheReliability

Henning（2001）definesreliabilityas“ameasureofaccuracy,consistency,dependabilityorfairnessscoresresultingfromadministrationofaparticularexamination”.Thetendencytowardconsistencyfoundinrepeatedmeasurementsisreferredtoasreliability（Carmines&Zeller,1979）.Atestisreliableifitisconsistentacrossdifferentcharacteristicsofthetestingsituation.

2.1ContentReliability

Factorsthataffectreliabilityarethelength,difficultyanddiscriminationofthecontent.（Bachman,1999）.Basicallyspeaking,themorequestionsthetestcontains,thelargeritcoversandthethelongerofitslength,thehigherofthetestwillbe.Theoraltestwithafixedlengthnotonlyoffersabundantlanguageusingexamples,butalsolimitstheinfluencefromtheprejudiceofthejudgment（HuangYonghong,2006）.Fromthisperspective,thereliabilityofTEM4oralEnglishtestiswellguaranteed.Thetotaltimeoftheoraltestisabout19minutesandreachesthelengthrequirement.

Ifthetestistooeasyortoodifficult,thediscriminationwillbedeclined.TEM4oralEnglishtesthasagoodcommandofdifficultyanddiscrimination（LiZhaoqing,2005）.Theformertwopartsarerathereasyandthelastpartisratherdifficult,whichmakessureofthediscriminationofthescoresonthewhole.

2.2AdministrativeReliability

Administrativereliabilityisdefinedasthereliabilityofthepreparationformandtestprocedureinthetest.Inthisaspect,TEM4oralEnglishtesthasachievedhighreliability.Firstly,theoraltestistakenatthesametime.Second,thecandidatestakethetestinlanguagelaboratoryandstartrecordingatthesametimetoensurefairandconfidentiality（LiuRunqing,1991）.

2.3ScorerReliability

Inthefirstplace,scorerreliabilitydependsonwhetherthescoringcriteriaissimpletooperate,concreteandaccurate.TEM4oraltesthasveryconcretescoringstandards.Thetestpaperin2012forexampleoffers25pointsinthefirstpartofretelling,onepointforoneright.Theothertwopartsalsohavespecificscoringcriteria.

Moreover,scorerreliabilitydependsonthescoringfoundation.JinYan（2002）inherresearchofreliabilityandvalidityoftape-assistedoralEnglishtestpointsoutthatoralEnglishtestcarriedoutbyrecordingachievesadmirableconsistency.Thistestmethodnotonlysavestimeandhumanresources,butalsoprovidesaobjectivescoringfoundation.Thejudgmentcanlistentotherecordingoverandoveragainandmakecarefulcomparisontochoosetheexcellences（LiuRunqing,1991）.Asaresult,someadverseimpactscanbeavoidedsuchasthepre-conceivedimageofcandidatesandignoranceofsomecontentsoutoffatigue.

2．4TheresearchonthereliabilityofSpeakingTestinTEM4

Reliabilityistheoverallconsistencyofameasure.Ameasureissaidtohaveahighreliabilityifitproducessimilarresultsunderconsistentconditions.Inordertotestthereliabilityforresearchpurpose,thepreciousscholarsusuallyemployedTheRetestingMethodorTheReevaluatingmethod.ButthefactthatstudentjusttaketheSpeakingTestinTEM4oncemakesTheRetestingMethodseemnotsopractical.Asforthemethodofreevaluation,onlytheExaminationCenterhastheauthorityofusingit.Therefore,howtotestthereliabilityofSpeakingTestinTEM4?

WenQiufang,aprofessorinNanJingUniversity，basedonreliabilitycoefficient（ifReliablecoefficientislowerthan0.4,itmeansthereliabilityofthetestislowtosomeextent.）,usesthemethodofformat:

ReliableCoefficient=

Note:

Nreferstothetotalnumberoftestingsections;mreferstotheaveragescoreoftheexaminees;xreferstostandarddeviation（S．d．）

S.d.=

Note:

dreferstothedeviationtotheaveragescoreofeveryexaminee.（CaiZhengying1999:

167）

totestthereliabilityofSpeakingTestinTEM4.Andshegetsthereliabilitycoefficientoftwoclassesin1999grade,whichare2.26and1.68respectivelyandbotharegreatlyhigherthan0.4.So,shedrawtoconclusionthatSpeakingTestinTEM4ishighlyreliable.However,fromtheperspectiveofourgroup,itisstilldoubtfultosaythatSpeakingTestinTEM4ishighlyreliablesincethatthefigureswhichbaseononlytwoclassesdonhavetherepresentation.

3．ThevalidityofSpeakingTestinTEM4

Testvalidityistheextenttowhichatestaccuratelymeasureswhatitpurportstomeasure.AndaccordingtoBachmanandLiuRunqing,thevaliditycanbeclarifiedintofourparts:

（a）contentvalidity,（b）facevalidity,（c）criterion-relatedvalidity,and（d）constructvalidity.（Bachman1999：

243—255；LiuRunqing1991：

16—18）。

3．1contentvalidity

Contentvalidityincludestwoaspects:

therelevanceofthecontentandthecourageofthecontent.

Asfortherelevanceofcontent,Pophampointsoutitshouldincludethreeelements:

（a）thepurposeofthetest（b）thetraitsofgivingexamineesinspirations（c）thetraitsofknowingpossiblequestionsraisedbyexaminees.AccordingtotheCollegeTeachingOutlineforEnglishMajor,theexamineesmustmeetsomerequirementsinTEM4Speakingtestsuchastheyoughttohavetheabilityofcommunicatewiththenativeenglishspeakersingeneralsocialoccasions.Andtheyshouldbeabletoexpresstheirideasaccuratelywitharightpronunciationandanaturaltone.What’smore,theyshouldmakesentencewithoutgravegrammaticerrors.BasedonsuchanOutline,theSpeakingTestinTEM4isdesignedtoincludethreeparts,whicheffectivelytesttheexaminees’abilityofconveyingtheirthoughtstoothers,speakingwitharightpronunciationandnaturaltoneandmakingnoseriousgrammatic-errorsentences.HuangYonghong,ayoungscholarcomingfromHeiLongjiangUniversity,howeverclaimsthattheSpeakingTesteffectivelytesttheexaminees’abilityexceptforcommunicatingwithnativeEnglishspeakers,whichislimitedbysomeobjectiveconditions.

Intermsofthethecourageofcontent,Wenqiufang,WuCaixiawhostudyinNanJingUniversityandLydiaSowhocomesfromtheUniversityofHongKong,maintainthatSpeakingTestinTEM4usuallyemploysseveralbutnotjustonequestionsinordertotestthetrueoralEnglishlevelofexaminees.But,itscontentignoresculturalelementtosomeextent,whichhasbeenimprovedinTheNewTeachingOutlinethatitrequirestheexamineestohavebetteracquaintancewiththeGeography,History,LiteratureandCultureoftheEnglishspeakingcountries.Andasforthedecencyoflanguage,SpeakingTestinTEM4ismorereasonablesincethedecencyoflanguageiscloselyassociatedwiththedifferentcontextsoftheconversations,whichSpeakingTestinTEM4coversalot.

3．2FaceValidity

Facevalidityreferstoatestappearstobeappropriateatleastonthesurface.ThespeakingtestinTEM-4isasemidirectoraltest,foritguaranteesitsfairnessthroughtaperecording.AccordingtoPangJixianandChenChan（2005）,theexamcontentinasemidirectoraltestisunifiedandtheprocessoftestandscoringisseperatefromeachother,thereforeitisnoteasilyaffectedandhasahighlevelofvalidity.Theyclaimthatbecauseofthelackofinteraction,thefacevalidityislow.Huangyonghong（2006）arguesthequalityoftherecordedtapeshouldbeimprovedbecausecandidatescanhearsomewhisperduringthebreak,whichmayhavesomeeffectontheperformanceofcandidates.Besides,thetest-makermakesitclearinthetestspecificationthatthecandidatesmustusetheirownwordswhenretellandrecitationw