Chapter 3 The Realities of Software Testing.docx

Chapter 3 The Realities of Software Testing.docx

《Chapter 3 The Realities of Software Testing.docx》由会员分享，可在线阅读，更多相关《Chapter 3 The Realities of Software Testing.docx（17页珍藏版）》请在冰豆网上搜索。

Chapter3TheRealitiesofSoftwareTesting

Chapter3.TheRealitiesofSoftwareTesting

INTHISCHAPTER

∙TestingAxioms

∙SoftwareTestingTermsandDefinitions

InChapter1,"SoftwareTestingBackground,"andChapter2,"TheSoftwareDevelopmentProcess,"youlearnedaboutthebasicsofsoftwaretestingandthesoftwaredevelopmentprocess.Theinformationpresentedinthesechaptersofferedaveryhigh-levelandarguablyidealisticviewofhowsoftwareprojectsmightberun.Unfortunately,intherealworldyouwillneverseeaprojectperfectlyfollowanyofthedevelopmentmodels.Youwillneverbegivenathoroughlydetailedspecificationthatperfectlymeetsthecustomer'sneedsandyouwillneverhaveenoughtimetodoallthetestingyouneedtodo.Itjustdoesn'thappen.But,tobeaneffectivesoftwaretester,youneedtounderstandwhattheidealprocessissothatyouhavesomethingtoaimfor.

Thegoalofthischapteristotemperthatidealismwitharealitycheckfromasoftwaretester'sperspective.Itwillhelpyouseethat,inpractice,trade-offsandconcessionsmustbemadethroughoutthedevelopmentcycle.Manyofthosetrade-offsaredirectlyrelatedtothesoftwaretesteffort.Thebugsyoufindandtheproblemsyoupreventallsignificantlyaffecttheproject.Afterreadingthischapter,you'llhaveamuchclearerpictureoftheroles,theimpact,andtheresponsibilitiesthatsoftwaretestinghasandyou'llhopefullyappreciatethebehind-the-scenesdecisionsthatmustbemadetocreateasoftwareproduct.

Thehighlightsofthischapterinclude

∙Whysoftwarecanneverbeperfect

∙Whysoftwaretestingisn'tjustatechnicalproblem

∙Thetermscommonlyusedbysoftwaretesters

TestingAxioms

Thisfirstsectionofthischapterisalistofaxioms,ortruisms.Thinkofthemasthe"rulesoftheroad"orthe"factsoflife"forsoftwaretestingandsoftwaredevelopment.Eachofthemisalittletidbitofknowledgethathelpsputsomeaspectoftheoverallprocessintoperspective.

It'sImpossibletoTestaProgramCompletely

Asanewtester,youmightbelievethatyoucanapproachapieceofsoftware,fullytestit,findallthebugs,andassurethatthesoftwareisperfect.Unfortunately,thisisn'tpossible,evenwiththesimplestprograms,duetofourkeyreasons:

∙Thenumberofpossibleinputsisverylarge.

∙Thenumberofpossibleoutputsisverylarge.

∙Thenumberofpathsthroughthesoftwareisverylarge.

∙Thesoftwarespecificationissubjective.Youmightsaythatabugisintheeyeofthebeholder.

Multiplyallthese"verylarge"possibilitiestogetherandyougetasetoftestconditionsthat'stoolargetoattempt.Ifyoudon'tbelieveit,considertheexampleshowninFigure3.1,theMicrosoftWindowsCalculator.

Figure3.1.EvenasimpleprogramsuchastheWindowsCalculatoristoocomplextocompletelytest.

AssumethatyouareassignedtotesttheWindowsCalculator.Youdecidetostartwithaddition.Youtry1+0=.Yougetananswerof1.That'scorrect.Thenyoutry1+1=.Youget2.Howfardoyougo?

Thecalculatoracceptsa32-digitnumber,soyoumusttryallthepossibilitiesupto

1+99999999999999999999999999999999=

Onceyoucompletethatseries,youcanmoveonto2+0=,2+1=,2+2=,andsoon.Eventuallyyou'llgetto

99999999999999999999999999999999+99999999999999999999999999999999=

Nextyoushouldtryallthedecimalvalues:

1.0+0.1,1.0+0.2,andsoon.

Onceyouverifythatregularnumberssumproperly,youneedtoattemptillegalinputstoassurethatthey'reproperlyhandled.Remember,you'renotlimitedtoclickingthenumbersonscreenyoucanpresskeysonyourcomputerkeyboard,too.Goodvaluestotrymightbe1+a,z+1,1a1+2b2,….Thereareliterallybillionsuponbillionsofthese.

Editedinputsmustalsobetested.TheWindowsCalculatorallowstheBackspaceandDeletekeys,soyoushouldtrythem.12+2shouldequal4.Everythingyou'vetestedsofarmustberetestedbypressingtheBackspacekeyforeachentry,foreachtwoentries,andsoon.

Ifyouoryourheirsmanagetocompleteallthesecases,youcanthenmoveontoaddingthreenumbers,thenfournumbers,….

Therearesomanypossibleentriesthatyoucouldnevercompletethem,evenifyouusedasupercomputertofeedinthenumbers.Andthat'sonlyforaddition.Youstillhavesubtraction,multiplication,division,squareroot,percentage,andinversetocover.

Thepointofthisexampleistodemonstratethatit'simpossibletocompletelytestaprogram,evensoftwareassimpleasacalculator.Ifyoudecidetoeliminateanyofthetestconditionsbecauseyoufeelthey'reredundantorunnecessary,orjusttosavetime,you'vedecidednottotesttheprogramcompletely.

SoftwareTestingIsaRisk-BasedExercise

Ifyoudecidenottotesteverypossibletestscenario,you'vechosentotakeonrisk.Inthecalculatorexample,whatifyouchoosenottotestthat1024+1024=2048?

It'spossibletheprogrammeraccidentallyleftinabugforthatsituation.Ifyoudon'ttestit,acustomerwilleventuallyenterit,andheorshewilldiscoverthebug.It'llbeacostlybug,too,sinceitwasn'tfounduntilthesoftwarewasinthecustomer'shands.

Thismayallsoundprettyscary.Youcan'ttesteverything,andifyoudon't,youwilllikelymissbugs.Theproducthastobereleased,soyouwillneedtostoptesting,butifyoustoptoosoon,therewillstillbeareasuntested.Whatdoyoudo?

Onekeyconceptthatsoftwaretestersneedtolearnishowtoreducethehugedomainofpossibletestsintoamanageableset,andhowtomakewiserisk-baseddecisionsonwhat'simportanttotestandwhat'snot.

Figure3.2showstherelationshipbetweentheamountoftestingperformedandthenumberofbugsfound.Ifyouattempttotesteverything,thecostsgoupdramaticallyandthenumberofmissedbugsdeclinestothepointthatit'snolongercosteffectivetocontinue.Ifyoucutthetestingshortormakepoordecisionsofwhattotest,thecostsarelowbutyou'llmissalotofbugs.Thegoalistohitthatoptimalamountoftestingsothatyoudon'ttesttoomuchortoolittle.

Figure3.2.Everysoftwareprojecthasanoptimaltesteffort.

YouwilllearnhowtodesignandselecttestscenariosthatminimizeriskandoptimizeyourtestinginChapters4through7.

TestingCan'tShowThatBugsDon'tExist

Thinkaboutthisforamoment.You'reanexterminatorchargedwithexaminingahouseforbugs.Youinspectthehouseandfindevidenceofbugsmaybelivebugs,deadbugs,ornests.Youcansafelysaythatthehousehasbugs.

Youvisitanotherhouse.Thistimeyoufindnoevidenceofbugs.Youlookinalltheobviousplacesandseenosignsofaninfestation.Maybeyoufindafewdeadbugsoroldnestsbutyouseenothingthattellsyouthatlivebugsexist.Canyouabsolutely,positivelystatethatthehouseisbugfree?

Nope.Allyoucanconcludeisthatinyoursearchyoudidn'tfindanylivebugs.Unlessyoucompletelydismantledthehousedowntothefoundation,youcan'tbesurethatyoudidn'tsimplyjustmissthem.

Softwaretestingworksexactlyastheexterminatordoes.Itcanshowthatbugsexist,butitcan'tshowthatbugsdon'texist.Youcanperformyourtests,findandreportbugs,butatnopointcanyouguaranteethattherearenolongeranybugstofind.Youcanonlycontinueyourtestingandpossiblyfindmore.

TheMoreBugsYouFind,theMoreBugsThereAre

Thereareevenmoresimilaritiesbetweenrealbugsandsoftwarebugs.Bothtypestendtocomeingroups.Ifyouseeone,oddsaretherewillbemorenearby.

Frequently,atesterwillgoforlongspellswithoutfindingabug.He'llthenfindonebug,thenquicklyanotherandanother.Thereareseveralreasonsforthis:

∙Programmershavebaddays.Likeallofus,programmerscanhaveoffdays.Codewrittenonedaymaybeperfect;codewrittenanothermaybesloppy.Onebugcanbeatell-talesignthattherearemorenearby.

∙Programmersoftenmakethesamemistake.Everyonehashabits.Aprogrammerwhoispronetoacertainerrorwilloftenrepeatit.

∙Somebugsarereallyjustthetipoftheiceberg.Veryoftenthesoftware'sdesignorarchitecturehasafundamentalproblem.Atesterwillfindseveralbugsthatatfirstmayseemunrelatedbuteventuallyarediscoveredtohaveoneprimaryseriouscause.

It'simportanttonotethattheinverseofthis"bugsfollowbugs"ideaistrue,aswell.Ifyoufailtofindbugsnomatterhowhardyoutry,itmayverywellbethatthefeatureyou'retestingwascleanlywrittenandthatthereareindeedfewifanybugstobefound.

ThePesticideParadox

In1990,BorisBeizer,inhisbookSoftwareTestingTechniques,SecondEdition,coinedthetermpesticideparadoxtodescribethephenomenonthatthemoreyoutestsoftware,themoreimmuneitbecomestoyourtests.Thesamethinghappenstoinsectswithpesticides（seeFigure3.3）.Ifyoukeepapplyingthesamepesticide,theinsectseventuallybuildupresistanceandthepesticidenolongerworks.

Figure3.3.Softwareundergoingthesamerepetitivetestseventuallybuildsupresistancetothem.

RememberthespiralmodelofsoftwaredevelopmentdescribedinChapter2?

Thetestprocessrepeatseachtimearoundtheloop.Witheachiteration,thesoftwaretestersreceivethesoftwarefortestingandruntheirtests.Eventually,afterseveralpasses,allthebugsthatthosetestswouldfindareexposed.Continuingtorunthemwon'trevealanythingnew.

Toovercomethepesticideparadox,softwaretestersmustcontinuallywritenewanddifferentteststoexercisedifferentpartsoftheprogramandfindmorebugs.

NotAlltheBugsYouFindWillBeFixed

Oneofthesadrealitiesofsoftwaretestingisthatevenafterallyourhardwork,noteverybugyou