A Model and Sensitivity Analysis of the Quality Economics.docx

A Model and Sensitivity Analysis of the Quality Economics.docx

《A Model and Sensitivity Analysis of the Quality Economics.docx》由会员分享，可在线阅读，更多相关《A Model and Sensitivity Analysis of the Quality Economics.docx（47页珍藏版）》请在冰豆网上搜索。

AModelandSensitivityAnalysisoftheQualityEconomics

AModelandSensitivityAnalysisoftheQualityEconomics

ofDefectDetection

Techniques

StefanWagner

Institutf╱rInformatik

TechnischeUniversit╝tM╱nchen

Boltzmannstr.3,D85748

Garchingb.M╱nchen,Germany

wagnerst@in.tum.de

ABSTRACT

Oneofthemaincostfactorsinsoftwaredevelopmentis

thedetectionandremovalofdefects.However,therelationships

andinfluencingfactorsofthecostsandrevenues

ofdefect-detectiontechniquesarestillnotwellunderstood.

Thispaperproposesananalytical,stochasticmodelofthe

economicsofdefectdetectionandremovaltoimprovethis

understanding.Themodelisabletoincorporatedynamic

aswellasstatictechniquesincontrasttomostothermodels

ofthatkind.Weespeciallyanalysethemodelwithstate-ofthe-

artsensitivityanalysismethodsto

（1）identifythemost

relevantfactorsformodelsimplificationand

（2）prioritise

thefactorstoguidefurtherresearchandmeasurements.

CategoriesandSubjectDescriptors

D.2.9[SoftwareEngineering]:

Management;D.2.8[Soft-

wareEngineering]:

Metrics;D.2.5[SoftwareEngineer-

ing]:

TestingandDebugging

GeneralTerms

Economics,Verification,Reliability

Keywords

Softwarequalityeconomics,qualitycosts,cost/benefit,defectdetection

techniques,sensitivityanalysis

1.INTRODUCTION

Thequalityofasoftwaresystemcanbedescribedusing

differentattributessuchasreliability,maintainabilityetc.

Therearealsovariousapproachestoimprovethequalityof

softwarewithdifferingemphasisontheseattributes.Constructive

methodscompriseonegroupthattriestoimprove

theoveralldevelopmentprocessinordertopreventtheintroduction

offaults.However,theprevalentapproachisstill

touseanalyticalmethods,alsocalleddefect-detectiontech-

Permissiontomakedigitalorhardcopiesofallorpartofthisworkfor

personalorclassroomuseisgrantedwithoutfeeprovidedthatcopiesare

notmadeordistributedforprofitorcommercialadvantageandthatcopies

bearthisnoticeandthefullcitationonthefirstpage.Tocopyotherwise,to

republish,topostonserversortoredistributetolists,requirespriorspecific

permissionand/orafee.

ISSTA?

6,July17?

0,2006,Portland,Maine,USA.

Copyright2006ACM1595932631/

06/0007...$5.00.

niques,tofindandremovefaults.Themainrepresentatives

ofthisapproacharetestsandreviews.

Anoftencitedestimate[26]relates50%oftheoveralldevelopment

coststotesting.Jones[14]stillassigns30?

0%to

qualityassuranceanddefectremoval.Hence,defect-detection

techniquesareapromisingfieldforcostoptimisations.

However,tobeabletooptimisetheusageofdefect-detection

techniques,weneedasuitableeconomicalmodelfirst.There

aresomeapproachesthatmodelsoftwarequalitycostsbut

mostlyonahighlevelofabstraction.Theeffectsofindividual

faultsandtheeffectivenessofdifferentdefect-detection

techniquesregardingthesefaultsarenottakenintoaccount.

Alsoin[28]itisdiscussedthat揷ostisclearlyacentralfactor

inanyrealisticcomparisonbutitishardtomeasure,

dataarenoteasytoobtain,andlittlehasbeendonetodeal

withit.?

Raietal.identifyin[31]mathematicalmodels

oftheeconomicsofsoftwarequalityassuranceasanimportant

researcharea.揂betterunderstandingofthecostsand

benefitsofSQAandimprovementstoexistingquantitative

modelsshouldbeusefultodecision-makers.?

1.1Problem

Theunderlyingquestionishowwecanoptimallyusedefect-

detectiontechniquestoimprovethequalityofsoftware.

Inparticular,weinvestigateinthispaperhowtheeconomical

relationshipsofdefect-detectiontechniquesandquality

canbemodelledandtheimportanceofthefactorsinterms

oftheinfluenceontheoutputandespeciallyitsvariance.

1.2Contribution

Weproposeananalyticalmodeloftheeconomicsofdefect-

detectiontechniquesincorporatingdifferenttypesofdefect

costs,thedifficultyoffindingafaultofdifferenttechniques

andtheprobabilityoffailureforafault.Thisallows

anevaluationofdifferenttechniquesandgivesabetterunderstanding

oftherelationships.Theusedinputfactorsare

prioritisedtosimplifythemodelandtoidentifythefactors

thataremostbeneficialtobefurtherinvestigated.Furthermore,

amodelbasedondefecttypesisderivedtoallow

asimplerapplicationonrealworldprojects.Thismodel

couldbeusedtopredictoptimalusageofdefect-detection

techniquesinthefuturebasedonoldprojectdata.

1.3Outline

Westartbydescribingsoftwarequalitycostsingeneral

andourunderstandingofthevariouscostfactorsinSec.2.

Sec.3proposesananalyticalmodeloftheeconomicsofdefect-

detectiontechniquesthatcontainsthecostsassociated

73

witheachfault.Thismodelissubjecttoasensitivityanalysis

basedonthedatafromanolderstudyinSec.4.Forthe

practicalapplicationofthemodelasimplifiedversionbased

ondefectclassesisderivedinSec.5.Sec.6givesrelated

workandfinalconclusionscanbefoundinSec.7.

2.SOFTWAREQUALITYCOSTS

Qualitycostsarethecostsassociatedwithpreventing,

finding,andcorrectingdefectivework.Basedonexperience

fromthemanufacturingarea[15,5]similarqualitycostmodels

havebeendevelopedexplicitlyforsoftware[18,35,19].

Thesecostsaredividedintoconformanceandnonconfor-

mancecosts,alsocalledcontrolcostsandfailureofcontrol

costs.Theformercomprisesallcoststhatneedtobespent

tobuildthesoftwareinawaythatitconformstoitsquality

requirements.Thiscanbefurtherbrokendowntopreven-

tionandappraisalcosts.Preventioncostsareforexample

developertraining,toolcosts,orqualityaudits,i.e.costs

formeanstopreventtheinjectionoffaults.Theappraisal

costsarecausedbytheusageofvarioustypesoftestsand

reviews.

Thenonconformancecostscomeintoplaywhenthesoftware

doesnotconformtothequalityrequirements.These

costsaredividedintointernalfailurecostsandexternalfail-

urecosts.Theformercontainscostscausedbyfailuresthat

occurduringdevelopment,thelatterdescribescoststhat

resultfromfailuresattheclient.Agraphicaloverviewis

giveninFig.1.Becauseofthedistinctionbetweenprevention,

appraisal,andfailurecoststhisisoftencalledPAF

model.

costofquality

preventioncostsappraisalcostsexternalfailure

conformancenonconformance

internalfailure

setupexecutionfaultremovaleffect

Figure1:

Overviewoverthecostsrelatedtoquality

WeaddfurtherdetailtothePAFmodelbyintroducing

themaintypesofconcretecoststhatareimportantfor

defect-detectiontechniques.Notethattherearemoretypes

thatcouldbeincluded,forexample,maintenancecosts.

However,weconcentrateonamorereliability-orientedview.

Theappraisalcostsaredetailedtothesetupandexecution

costs.Theformerconstitutingallinitialcostsforbuying

testtools,configuringthetestenvironment,andsoon.The

lattermeansallthecoststhatareconnectedtoactualtest

executionsorreviewmeetings,mainlypersonnelcosts.

Onthenonconformanceside,wehavefaultremovalcosts

thatcanbeattributedtotheinternalfailurecostsaswell

astheexternalfailurecosts.Thisisbecauseifwefounda

faultandwanttoremoveit,itwouldalwaysresultincosts

nomatterwhethercausedinaninternalorexternalfailure.

Actually,theredoesnothavetobeafailureatall.

Consideringcodeinspections,faultsarefoundandremoved

thathavenevercausedafailureduringtesting.Itisalso

agoodexamplethattheremovalcostscanbequitedifferent

regardingdifferenttechniques.Whenatestidentifiesa

failure,thereneedstobeconsiderableeffortspenttofind

thecorrespondingfault.Duringaninspection,faultsare

founddirectly.Faultremovalcostsalsocontainthecosts

fornecessaryre-testingandre-inspections.

Externalfailuresalsocauseeffectcosts.Theseareall

furthercostswiththefailureapartfromtheremovalcosts.

Forexample,compensationcostscouldbepartoftheeffect

costs,ifthefailurecausedsomekindofdamageatthe

customersite.Wemightalsoincludefurthercostssuchas

lossofsalesbecauseofbadreputationintheeffectcostsbut

donotconsideritexplicitlybecauseitsoutofscopeofthis

paper.

3.ANALYTICALMODEL

Wedescribeageneral,analyticalmodelofdefect-detection

techniquesinthefollowing.Itisgeneralwithrespect

tothevarioustypesoftechniquesitisabletoanalyse.We

mainlyanalysedifferenttypesoftestingwhichessentially

detectfailuresandstaticanalysistechniquesthatreveal

faultsinthecodeorotherdocuments.Amodelthatincorporates

allimportantfactorsforthesedifferingtechniques

needstousetheuniversalunitofmoney,i.e.,unitssuchas

euroordollar.Wefirstdescribethemodelanditsassumptions

ingeneral,andthengiveequationsforeachcomponent

ofthemodelforasingletechniqueandforthecombination

ofseveraltechniques.

3.1General

Inthissection,weconcentrateonanidealmodelofquality

economicsinthesensethatwedonotconsiderthepractical

useofthemodelbutwanttomirrortheactualrelationships

asfaithfullyaspossible.

3.1.1Components

Wedividethemodelinthreemaincomponents:

?

DirectcostsdA

?

FuturecoststA

?

Revenues/savedcostsrA

Thedirectcostsarecharacterisedbycontainingonlycosts

thatcanbedirectlymeasuredduringtheexecutionofthe

technique.Thefuturecostsandrevenuesarebothconcerned

withthe（potential）costsinthefieldbutcanbedistinguished

becausethefuturecostscontainthecoststhatare