A RiskBased Test Strategy.docx

A RiskBased Test Strategy.docx

《A RiskBased Test Strategy.docx》由会员分享，可在线阅读，更多相关《A RiskBased Test Strategy.docx（18页珍藏版）》请在冰豆网上搜索。

ARiskBasedTestStrategy

ARisk-BasedTestStrategy

Dr.IngridB.Ottevanger

IQUIPInformaticaB.V.

POBox263,1110AGDiemen,TheNetherlands

Tel:

+31206606600

Fax:

+31206953298

E-mail:

i.b.ottevanger@iquip.nl

1.Introduction

Thedevelopmentofateststrategyisameansofcommunicationwiththecustomercommissioningthetestonsuchmattersastheorganizationoftestingandthestrategicchoicesthatgowithit.Theteststrategyindicateshowtestingistobecarriedout.Inordertomakethebestpossibleuseofresourcesandtime,itisdecidedonwhichpartsandaspectsofthesystemtheemphasisshouldfall.Theteststrategyformsanimportantbasisforastructuredapproachtotestingandmakesamajorcontributiontoamanageabletestprocess.

Thecustomerwhocommissionsthetestwillexpectspecificqualitiesofthesystemwheninproduction,andwantstoknowwhetherthereleasedsystemwillmeettheserequirements.Ifthesystemqualitativelydoesnotmeettherequirementsoronlytoalimitedextent,thisimplieshighdamagefortheorganization,forinstancesincehighreworkcostswillbeneededorclients/userswillbeunsatisfied.Therefore,thissituationformsariskfortheorganization.'Risk'inthispaperisdefinedas:

Ariskisthechanceofanerroroccurring（chanceoffailure）relatedtothedamageexpectedwhenthiserrordoesoccur

Testingcoverssuchrisksbygivinginsightintotheextenttowhichthesystemmeetsthequalitydemands.Whenqualityturnsouttobeinsufficienttimelymeasurescanbetaken,e.g.reworkbydevelopers.Iftheshippingofthesystemimpliesmanyrisksfortheorganization,bettertestingisobviousasasolution.Andthereversealsoholds:

Norisk,notest

Althoughintheabovewerefertoqualityandrisksinageneralsense,theremaybelargedifferencesdependingonthesituation.Itisofgreatimportancetodiscussthiswiththecustomer,andtotranslatethecustomer'swishesinthisrespectintothewaytestingwillbeperformed.Thus,theteststrategyisdirectedtowardsfindingtheoptimalbalancebetweenthetestefforttobeexertedandthecoveragerequiredfortherisks.Tothispurposetherisksarespecifieduptothelevelofqualitycharacteristicsandseparatesubsystem.Indoingsoitbecomespossibletofindasuitabletestcoveragefortheassessedrisks.Hereahighertestcoverageusuallyresultsinmoretesteffort.Inordertoreachatthevariationintestcoverageneeded,theuseofmorethanonetestspecificationtechnique（testdesigntechnique）,eachofferingaspecifiedtestcoverage,iscrucial.

Ananalogywithinsurancesmayclarifythismatterabitmore.Apersonwantstocoverarelevantriskandtakesaninsurancewithacoveragefittingthisriskasbestaspossible.Thisinsurancetakesacertainpremium.Ifthepersonwantstopayless,aninsurancewithalowercoverageisbought.Theconsequenceisthattherewillbenopaymentiftheuncoveredriskoccurs.Ontheotherhand,ifcoverageweretolarge,thentoomuchpremiumispaid,sinceasituationhasbeeninsuredwhichisunlikelytooccurforthisperson.

Thebalancebetweenbudgetandriskcoverage

2.RiskAssessment

Teststrategyisbasedonriskassessment.Thismeansassessingthedamageoftheconsequencesofdefects,bothundetectedpriortooperationandoccurringduringoperation.

Riskassessmenttakesplaceonthebasisofqualitycharacteristicsandsubsystems.Forinstance,ifthesystemisinsufficientlyuser-friendly,whatwillbethenegativeconsequences.Andwhatwillbethedamagewhenthesalarycalculationmoduleinapayrollsystemdoesnotworkcorrectly.

Inordertobeabletoperformthisassessmentwell,theseparateaspectsofariskareconsidered:

Risk=chanceoffailurexdamage,

wherechanceoffailureisrelatedtoaspectsincludingfrequencyofuseandthechanceofanerroroccurring.

Theseaspectsarelistedbelow:

∙Frequencyofuse

Inafunctionwhichisuseddozensoftimeseachdaythechanceofanerrordemonstratingitselfismuchbiggerthanwithafunctionusedonceayear.

∙Chanceoferror

Fortheassessmentofthechanceoferrorsthefollowinglistcanbehelpful.Itpresentsthelocationswhereerrorstendtocluster.ItispartlybasedonH.Schaefer,1996（Survivingundertimeandbudgetpressure,in:

ConferenceProceedingEuroSTAR1996,Amsterdam,theNetherlands）:

◊Complexfunctions;

◊Completelynewfunctions;

◊（Especiallyfrequently）adjustedfunctions;

◊Functionsforwhichcertaintoolsortechniqueswereemployedforthefirsttime;

◊Functionswhichweretransferredfromonedevelopertoanotherduringdevelopment;

◊Functionsthatwererealizedunderextremetimepressure;

◊Functionswhichhadtobeoptimizedmorefrequentlythanonaverage;

◊Functionsinwhichmanydefectswerefoundearlier（e.g.inpreviousreleasesorduringearlierreviews）;

◊Functionswithmanyinterfaces;

◊Inexperienceddevelopers;

◊Insufficientinvolvementofusers;

◊Insufficientqualityassuranceduringdevelopment;

◊Insufficientqualityoflow-leveltests;

◊Newdevelopmenttoolsanddevelopmentenvironment;

◊Largedevelopmentteams;

◊Developmentteamswithsub-optimalcommunication（e.g.owingtogeographicalspreadorpersonalcauses）;

∙Damage

Ifandwhentheerrormanifestsitself,whatwillbethedamagefortheorganization.Aspectsarecostsofrepair（bothofthesystemandoftheconsequences）,forgoneincomeandlossofclientsorofconfidence.Usuallythedamageincreasesiftheerrorhasitsimpactonotherfunctionsorsystems.Inthecaseoferrorsoccurringinbatchprocessestheremaybeapossibilitytopreventthemfromhamperingusers,sothattheeventualdamagewillbesmallerthanwithsimilaron-lineprocesses.Ofcourse,thisonlyholdsiferrorsaredetectedontime.

Becauseofthecomplexityofthematter,itisimpossibletoassessriskswithcompleteobjectivityandindetail:

itisaglobalassessment.Itisthereforeimportantfortheriskassessmentnottobecarriedoutbythetestmanageralone.Alargenumberofpeopleinvolvedintheschemeshouldcontribute:

customer,users,developmentteam,accountants,ITauditorsandsoon.Thisnotonlyincreasesthequalityofthestrategy,butitalsohastheadvantagethatthedifferentpartiesaremoreawareoftherisksandtheextenttowhichtestingcontributestomakingtheserisksmanageableinabetterway.

Thedeveloperoftheteststrategyshouldrealizethat'users'arethebestpeopletoassessthedamageandthefrequencyofusewhenvaluingtherisks（end-users,systemmanagersandapplicationmanagers,linemanagement）,whereasprojectteammembersarebesttoassessthechanceoferror（projectmanagers,designers,programmers,projectqualitystaff,testmanager）.

Thefocusinriskassessmentisonproductrisks,or,inotherwords,whatistheriskfortheorganizationiftheproductdoesnotdemonstratetheexpectedquality.Inadditiontothis,therearealso（test）projectrisks.IfthesystemmustbeinproductiononJanuary1st,iffunctionalspecificationsareproducedtoolate,ifnoexperiencedtestersareavailable,orifthetestinfrastructureisnotreadyontime,thenwespeakof（test）projectrisks.Thesearenottakenintoaccountindeterminingtheteststrategy;theydoplayaroleinthetestplan.

Indevelopingateststrategytheaimistoseetoitthatthetestwillbeorganizedinsuchawaythatwithacertainextentofreliability

∙themostimportantproblemswillbefound;

∙theproblemswillbefoundinanearlystage;

∙theproblemsthatrequirethemostreworktimewillbefoundfirst:

∙efficientuseismadeofresources;

∙andeventuallyanaccuratequalityadvicecanbegiven.

Thiscanbesummarizedas:

Teststrategyaimsatfindingthemostimportanterrorsasearlyaspossibleagainstthelowestcosts

Inpractice,thedevelopmentofateststrategyisoftenplannedtocoincidewithpreparingthebudget,forexamplewiththehelpoftestpointanalysis.Theadvantageisthattheconsequencesoftheadoptedstrategyareimmediatelytranslatedintotimerequiredfortesting,andconsequentlythecostoftesting,whichmakesthestrategicchoicesmanageable.Ifthetimeavailablefortestingismoreorlessfixed,itisalsopossibletouseteststrategycombinedwithtestpointanalysistodeterminewhatisachievablewithinthetimelimits.Itisprobablyevenmoreimportanttomakeitclearatthistimewhichpartscannotbetested,orcannotbefullytested,andwhatriskswillthereforebeincurred.

3.QualityCharacteristics

Thequalitycharacteristicswedistinguishcanbedividedintodynamicandstaticqualitycharacteristics.Thedynamicqualitycharacteristicsdealwithfeaturesoftheinformationsysteminuse;examplesaresecurity,usability,continuity,traceability,functionality,userfriendliness,suitability,efficiency,performance.Thestaticareconcernedwithintrinsiccharacteristicsoftheinformationsystemandthedocumentation,asconsideredfromthestandpointofdevelopersandfuturesystemmanagers.Examplesaremanageability,maintainability,connectivity,reusability,portability,testability.

4.Procedure

Indevelopingateststrategywedistinguishbetweenmastertestplanningandatestplanforaspecifictestlevel,e.g.acceptancetestorsystemtest.

Theprocedurecanbefollowedbothfordevelopmentofnewsystemsandformaintenancesituations.Forthelatter,however,itisbesttomakeafewadjustmentsinthebasicprocedure（cf4.4）.

Thedevelopmentofateststrategyisnotsomethingthatcanbedonepurelymethodicallyorformally.Thebelowstepsareaidsandindicators.Experienceandskillsoftheperformerofthisactivityintheareaoftesti