yyu.docx

yyu.docx

《yyu.docx》由会员分享，可在线阅读，更多相关《yyu.docx（6页珍藏版）》请在冰豆网上搜索。

yyu

论文附件

Internetlinksoperateathighspeeds,andpasttrendspredictthatthesespeedswillcontinuetoincreaserapidly.RoutersandintrusiondetectiondevicesthatoperateatuptoOC-768speeds（40Gb/s）arecurrentlybeingdeveloped.Whilethemainbottlenecks（e.g.,lookups,classification,andqualityofservice）inatraditionalrouterarewellunderstood,whatarethecorrespondingfunctionsthatshouldbehardwiredinthebravenewworldofsecurityandmeasurement?

Ideally,wewishtoabstractoutfunctionsthatarecommontoseveralsecurityandmeasurementapplicationandfindefficientalgorithmsforthesefunctions,especiallyalgorithmsforthesefunctions,especiallyalgorithmswithacompacthardwareimplementation.

Towardthisgoal,thispaperisolatesandprovidessolutionsforanimportantproblemthatoccursinvariousnetworkingapplications；countingthenumberofactiveflowsamongpacketsreceivedonalinkduringaspecifiedperiodoftime.Aflowisdefinedbyasetofheaderfields;twopacketsbelongtodistinctflowiftheyhavedifferentvaluesforthespecifiedheaderfieldsthatdefinetheflow.Forexample,ifwedefineaflowbyasourceanddestinationIPaddress,wecancountthenumberofdistinctsource-destinationIPaddress,wecancountthenumberofdistinctsource-destinationIPaddresspairsseenonalinkoveragiventimeperiod.Ouralgorithmsmeasuresthenumberofactiveflowsusingaveryasmallamountofmemorythatcaneasilybestoredinon-chipSRAMorevenprocessorregisters.Bycontrast,nativealgorithmsdescribedbelowwouldrequiremassiveamountsofmemorynecessitatingtheuseofslowDRAM.

Forexample,anativemethodtocountsource-destinationpairswouldbetokeepacountertogetherwithahashtablethatstoresallthedistinct64-bitsourcedestinationaddresspairsseenthusfar.Whenapacketarriveswithsourceanddestinationaddresspairssay〈S,D〉,wesearchthehashtablefor〈S,D〉;ifthereisnohashmatch,thecounterisincrementedand〈S,D〉isaddtothehashtable.Unfortunately,giventhatbackbonelinkscanhaveuptoamillionflowstoday,thisnativeschemewouldminimallyrequire64Mbofhigh-speedmemory.SuchlargeSRAMmemoryisexpensiveornotfeasibleforamodemrouter.

Therearemoreefficientgeneral-purposealgorithmsforcountingthenumberofdistinctvaluesinamultiset.Inthispaper,wenotonlypresentageneral-purposecountingalgorithms-multiresolutionbitmap-thathasbetteraccuracythanthebestknownprioralgorithm,probabilisticcountingalgorithms,probabilisticcountingalgorithmsthatfurtherimproveperformancebytakingadvantageofparticularitiesofthespecificcountingapplication.Ouradaptivebitmap,usingthefactthatnumberofthenumberofactiveflowsdoesnotchangeveryrapidly,cancountthenumberofactiveflowsdoesnotchangeveryrapidly,cancountthenumberofactiveflowsdoesnotchangeveryrapidly,cancountthenumberofdistinctflowsonalinkthatcontainsanywherefrom0to100millionflowswithanaverageerroroflessthan1%usingonly2KBofmemory.Ourtriggeredbitmap,whichisoptimizedforrunningmultipleconcurrentinstancesofthecountingproblem,manyofwhichhavesmallcounts,issuitablefordetectingportscansandusesevenlessmemorythanrunningadaptivebitmaponeachinstance.

Aflowisdefinedbyanidentifiergivenbythevaluesofcertainheaderfiled.Theproblemwewishtosolveiscountingthenumberofdistinctflowidentifiers（flowIDs）seeninaspecifiedmeasurementinterval.Forexample,anintrusiondetectionsystemlookingforportscanscouldforeachactivesourceaddresstheflowsdefinedbydestinationIPandsuspectanysourceIPthatopensmorethanthreeflowsin12sofscanning.

Also,whilemanyapplicationdefineflowsatthegranularityofTCPconnection,onemaywanttouseotherdefinition.Forexample,whendetectingDoSattackswemaywishtocountthenumberofdistinctsource,notthenumberofTCPconnections.Thus,inthispaper,weusethetermflowinthismoregenericway.

Aswehaveseen,anativesolutionusingahashtableofflowIDsisaccuratebuttakestoomuchmemory.Inhigh-speedrouters,itisnotonlythecostoflarge,fastmemoriesthatisaproblembutalsotheirpowerconsumptionandtheboardspacetheytakeuplinecards.Thus,weseeksolutionsthatuseasmallamountofmemoryandhavehighaccuracy.Wewanttofindalgorithmswherethesetradeoffsarefavorable.Also,sinceathighspeedstheper-packetprocessingtimeislimited,itisimportantthatthealgorithmsuseonlyoneortwomemoryaccessesandaresimpleenoughtobeimplementedinhardware.

Whyisinformationaboutthenumberofflowsuseful?

Wedescribefourpossiblecategoriesofuse.

Detectingportscans:

Intrusiondetectionsystemwarnofportscanswhenasourceopenstoomanyconnectionwithinagiventime.TheywidelydeployedSnortintrusiondetectionsystem（IDS）usesthenativeapproachofstoringarecordforeachactiveconnection.Thisisanobviouswastesincemostoftheconnectionsarenotapartscan.Evenforactualportscans,iftheIDSonlyreportsthenumberofconnections,wedonotneedtokeeparecordforeachconnection.Sincethenumberofsourcescanbeveryhigh,itisdescribetofindalgorithmsthatcountthenumberofconnectionsofeachsourceusinglittlememory.Further,ifanalgorithmscandistinguishquicklybetweensuspectedportscannersandnormaltraffic,theIDSneednotperformexpensiveoperations（e,g.logging）onmostofthetraffic,thusbecomingmorescalableintermsofmemoryusageandspeed.Thisisparticularlyimportantinthecontextoftherecentracetoprovidewire-speedintrusiondetection.

Detectingdenialofservice（DoS）attracks:

FlowScanbyPlonkainapopulartoolforvisualizingnetworktraffic.Itusesthenumberofactiveflows（seeFig.1）todetectongoingdenialofserviceattacks.Whilethisworkswellattheedgeofthenetwork（i.ethelinkbetweenalargeuniversitycampusandtherestoftheInternet）,itdoesnotscaletothecore.Also,itreliesonmassiveintermediatedata（NetFlow）tocomputecompactresult-couldweobtaintheusefulinformationmoredirectly?

Mahajanetal.proposeamechanismthatallowsbackbonerouterstolimittheeffectof（distributed）DoSattacks.Whilethemechanismassumethattheserouterscandetectanongoingattackitdoesnotgiveaconcretealgorithmsforit.EstanandVarghesepresentalgorithmsthatcandetectdestinationaddressorprefixesthatreceivelargeamountsoftraffic.Todifferentiatebetweenlegitimatetrafficandanattack,wecanusethefactthatDoStoolsusefakesourceaddresschosenatrandom.Ifforeachsuspectedvictimwecountthenumberofsourcesofpacketsthatcomefromsomenetworksknowntobesparelypopulated,alargecountisastrongindicationthataDoSattackisinprogress.

Generalmeasurement:

CountingthenumberofactiveconnectionsandthenumberofconnectionassociatedwitheachsourceanddestinationIPaddressisapartoftheCoralReeftrafficanalysissuite.Otherwaysofcountingdistinctvaluesingivenheaderfieldscanalsoprovideusefuldata.Onecouldmeasurethenumberofsourcesusingaprotocolversionorvarianttogetanaccurateimageofprotocoldeployment.Alternatively,bycountingthenumberofconnectionsassociatedwitheachoftheprotocolsgeneratingsignificanttraffic,wecancomputetheaverageconnectionlengthforeachprotocol,thusgettingabetterviewofitsbehavior.Dimensioningthevariouscachesinrouters（packetclassificationcaches,multicastroutecachesforsource-group（S-G）state,andARPcaches）alsobenefitsfrompriormeasurementsoftypicalworkload.

Estimatingthespreadingrateofaworm:

FromAugust1toAugust12,2001,whiletryingtotracktheCodeRedworm,collectingpacketheadersforCodeRedtrafficona/8networkproduced0.5GBperhourofcompressesdata.Todeterminetherateatwhichthewormwasspreading,itwasnecessarytocountthenumberofdistinctCodeRedsourcespassingthroughthelink.Thiswasactuallydoneusingalargelogandahashtablewhichwasexpensiveintimeandalsoinaccurate（becauseoflossesinthelog）.

Thus,whilecountingthenumberofflowsisusuallyinsufficientbyitself,itcanprovideausefulbuildingblockforcomplextasks.Thispaperextendsanearlierconferenceversion.Themostimportantadditionsareadiscussionofhardwareimplementationofthebitmapandprobabilisticcounting,andadiscussionofmorerecentrelatedwork.

Thenetworkingproblemofcountingthenumberofdistinctflowshasawell-studiedequivalentinthedatabasecommunity:

countingthenumberofdistinctbaserecords（ordistinctvaluesofanattribute）.Thus,themajorpieceofrelatedworkisaseminalalgorithm,probabilisticcounting,duetoFlajoletandMartin,introducedinthecontextofdatabase.Weuseprobabilisticcountingasabaseagainstwhichtocompareouralgorithms.Whangetal.addressthesameproblemandproposeanalgorithmequivalenttothesimplestalgorithmswedescribe（directbitmap）。

Theinsightbehindprobabilisticcountingistocomputeametricofhowuncommonacertainrecordisandkeeptrackofthemostuncommonrecordsseen.Ifthealgorithmsseesveryuncommonrecords,itconcludesthatthenumberofrecordsislarge.Moreprecisely,foreachrecord,thealgorithmcomputesahashfunctionthatmapsittoanLbitstring.ItthencountsthenumberofconsecutivezeroesstartingfromtheleastsignificantpositionofthehashresultandsetsthecorrespondingbitinabitmapofsizeL.Ifthealgorithmsseesrecordsthathashtovaluesendinginzero,one,andtwo0’s（thefirstthreebitsinthebitmapareset,andtherest