yyu.docx
《yyu.docx》由会员分享,可在线阅读,更多相关《yyu.docx(6页珍藏版)》请在冰豆网上搜索。
![yyu.docx](https://file1.bdocx.com/fileroot1/2023-1/22/deb80be8-8216-49db-bc70-ad96d85c542a/deb80be8-8216-49db-bc70-ad96d85c542a1.gif)
yyu
论文附件
Internetlinksoperateathighspeeds,andpasttrendspredictthatthesespeedswillcontinuetoincreaserapidly.RoutersandintrusiondetectiondevicesthatoperateatuptoOC-768speeds(40Gb/s)arecurrentlybeingdeveloped.Whilethemainbottlenecks(e.g.,lookups,classification,andqualityofservice)inatraditionalrouterarewellunderstood,whatarethecorrespondingfunctionsthatshouldbehardwiredinthebravenewworldofsecurityandmeasurement?
Ideally,wewishtoabstractoutfunctionsthatarecommontoseveralsecurityandmeasurementapplicationandfindefficientalgorithmsforthesefunctions,especiallyalgorithmsforthesefunctions,especiallyalgorithmswithacompacthardwareimplementation.
Towardthisgoal,thispaperisolatesandprovidessolutionsforanimportantproblemthatoccursinvariousnetworkingapplications;countingthenumberofactiveflowsamongpacketsreceivedonalinkduringaspecifiedperiodoftime.Aflowisdefinedbyasetofheaderfields;twopacketsbelongtodistinctflowiftheyhavedifferentvaluesforthespecifiedheaderfieldsthatdefinetheflow.Forexample,ifwedefineaflowbyasourceanddestinationIPaddress,wecancountthenumberofdistinctsource-destinationIPaddress,wecancountthenumberofdistinctsource-destinationIPaddresspairsseenonalinkoveragiventimeperiod.Ouralgorithmsmeasuresthenumberofactiveflowsusingaveryasmallamountofmemorythatcaneasilybestoredinon-chipSRAMorevenprocessorregisters.Bycontrast,nativealgorithmsdescribedbelowwouldrequiremassiveamountsofmemorynecessitatingtheuseofslowDRAM.
Forexample,anativemethodtocountsource-destinationpairswouldbetokeepacountertogetherwithahashtablethatstoresallthedistinct64-bitsourcedestinationaddresspairsseenthusfar.Whenapacketarriveswithsourceanddestinationaddresspairssay〈S,D〉,wesearchthehashtablefor〈S,D〉;ifthereisnohashmatch,thecounterisincrementedand〈S,D〉isaddtothehashtable.Unfortunately,giventhatbackbonelinkscanhaveuptoamillionflowstoday,thisnativeschemewouldminimallyrequire64Mbofhigh-speedmemory.SuchlargeSRAMmemoryisexpensiveornotfeasibleforamodemrouter.
Therearemoreefficientgeneral-purposealgorithmsforcountingthenumberofdistinctvaluesinamultiset.Inthispaper,wenotonlypresentageneral-purposecountingalgorithms-multiresolutionbitmap-thathasbetteraccuracythanthebestknownprioralgorithm,probabilisticcountingalgorithms,probabilisticcountingalgorithmsthatfurtherimproveperformancebytakingadvantageofparticularitiesofthespecificcountingapplication.Ouradaptivebitmap,usingthefactthatnumberofthenumberofactiveflowsdoesnotchangeveryrapidly,cancountthenumberofactiveflowsdoesnotchangeveryrapidly,cancountthenumberofactiveflowsdoesnotchangeveryrapidly,cancountthenumberofdistinctflowsonalinkthatcontainsanywherefrom0to100millionflowswithanaverageerroroflessthan1%usingonly2KBofmemory.Ourtriggeredbitmap,whichisoptimizedforrunningmultipleconcurrentinstancesofthecountingproblem,manyofwhichhavesmallcounts,issuitablefordetectingportscansandusesevenlessmemorythanrunningadaptivebitmaponeachinstance.
Aflowisdefinedbyanidentifiergivenbythevaluesofcertainheaderfiled.Theproblemwewishtosolveiscountingthenumberofdistinctflowidentifiers(flowIDs)seeninaspecifiedmeasurementinterval.Forexample,anintrusiondetectionsystemlookingforportscanscouldforeachactivesourceaddresstheflowsdefinedbydestinationIPandsuspectanysourceIPthatopensmorethanthreeflowsin12sofscanning.
Also,whilemanyapplicationdefineflowsatthegranularityofTCPconnection,onemaywanttouseotherdefinition.Forexample,whendetectingDoSattackswemaywishtocountthenumberofdistinctsource,notthenumberofTCPconnections.Thus,inthispaper,weusethetermflowinthismoregenericway.
Aswehaveseen,anativesolutionusingahashtableofflowIDsisaccuratebuttakestoomuchmemory.Inhigh-speedrouters,itisnotonlythecostoflarge,fastmemoriesthatisaproblembutalsotheirpowerconsumptionandtheboardspacetheytakeuplinecards.Thus,weseeksolutionsthatuseasmallamountofmemoryandhavehighaccuracy.Wewanttofindalgorithmswherethesetradeoffsarefavorable.Also,sinceathighspeedstheper-packetprocessingtimeislimited,itisimportantthatthealgorithmsuseonlyoneortwomemoryaccessesandaresimpleenoughtobeimplementedinhardware.
Whyisinformationaboutthenumberofflowsuseful?
Wedescribefourpossiblecategoriesofuse.
Detectingportscans:
Intrusiondetectionsystemwarnofportscanswhenasourceopenstoomanyconnectionwithinagiventime.TheywidelydeployedSnortintrusiondetectionsystem(IDS)usesthenativeapproachofstoringarecordforeachactiveconnection.Thisisanobviouswastesincemostoftheconnectionsarenotapartscan.Evenforactualportscans,iftheIDSonlyreportsthenumberofconnections,wedonotneedtokeeparecordforeachconnection.Sincethenumberofsourcescanbeveryhigh,itisdescribetofindalgorithmsthatcountthenumberofconnectionsofeachsourceusinglittlememory.Further,ifanalgorithmscandistinguishquicklybetweensuspectedportscannersandnormaltraffic,theIDSneednotperformexpensiveoperations(e,g.logging)onmostofthetraffic,thusbecomingmorescalableintermsofmemoryusageandspeed.Thisisparticularlyimportantinthecontextoftherecentracetoprovidewire-speedintrusiondetection.
Detectingdenialofservice(DoS)attracks:
FlowScanbyPlonkainapopulartoolforvisualizingnetworktraffic.Itusesthenumberofactiveflows(seeFig.1)todetectongoingdenialofserviceattacks.Whilethisworkswellattheedgeofthenetwork(i.ethelinkbetweenalargeuniversitycampusandtherestoftheInternet),itdoesnotscaletothecore.Also,itreliesonmassiveintermediatedata(NetFlow)tocomputecompactresult-couldweobtaintheusefulinformationmoredirectly?
Mahajanetal.proposeamechanismthatallowsbackbonerouterstolimittheeffectof(distributed)DoSattacks.Whilethemechanismassumethattheserouterscandetectanongoingattackitdoesnotgiveaconcretealgorithmsforit.EstanandVarghesepresentalgorithmsthatcandetectdestinationaddressorprefixesthatreceivelargeamountsoftraffic.Todifferentiatebetweenlegitimatetrafficandanattack,wecanusethefactthatDoStoolsusefakesourceaddresschosenatrandom.Ifforeachsuspectedvictimwecountthenumberofsourcesofpacketsthatcomefromsomenetworksknowntobesparelypopulated,alargecountisastrongindicationthataDoSattackisinprogress.
Generalmeasurement:
CountingthenumberofactiveconnectionsandthenumberofconnectionassociatedwitheachsourceanddestinationIPaddressisapartoftheCoralReeftrafficanalysissuite.Otherwaysofcountingdistinctvaluesingivenheaderfieldscanalsoprovideusefuldata.Onecouldmeasurethenumberofsourcesusingaprotocolversionorvarianttogetanaccurateimageofprotocoldeployment.Alternatively,bycountingthenumberofconnectionsassociatedwitheachoftheprotocolsgeneratingsignificanttraffic,wecancomputetheaverageconnectionlengthforeachprotocol,thusgettingabetterviewofitsbehavior.Dimensioningthevariouscachesinrouters(packetclassificationcaches,multicastroutecachesforsource-group(S-G)state,andARPcaches)alsobenefitsfrompriormeasurementsoftypicalworkload.
Estimatingthespreadingrateofaworm:
FromAugust1toAugust12,2001,whiletryingtotracktheCodeRedworm,collectingpacketheadersforCodeRedtrafficona/8networkproduced0.5GBperhourofcompressesdata.Todeterminetherateatwhichthewormwasspreading,itwasnecessarytocountthenumberofdistinctCodeRedsourcespassingthroughthelink.Thiswasactuallydoneusingalargelogandahashtablewhichwasexpensiveintimeandalsoinaccurate(becauseoflossesinthelog).
Thus,whilecountingthenumberofflowsisusuallyinsufficientbyitself,itcanprovideausefulbuildingblockforcomplextasks.Thispaperextendsanearlierconferenceversion.Themostimportantadditionsareadiscussionofhardwareimplementationofthebitmapandprobabilisticcounting,andadiscussionofmorerecentrelatedwork.
Thenetworkingproblemofcountingthenumberofdistinctflowshasawell-studiedequivalentinthedatabasecommunity:
countingthenumberofdistinctbaserecords(ordistinctvaluesofanattribute).Thus,themajorpieceofrelatedworkisaseminalalgorithm,probabilisticcounting,duetoFlajoletandMartin,introducedinthecontextofdatabase.Weuseprobabilisticcountingasabaseagainstwhichtocompareouralgorithms.Whangetal.addressthesameproblemandproposeanalgorithmequivalenttothesimplestalgorithmswedescribe(directbitmap)。
Theinsightbehindprobabilisticcountingistocomputeametricofhowuncommonacertainrecordisandkeeptrackofthemostuncommonrecordsseen.Ifthealgorithmsseesveryuncommonrecords,itconcludesthatthenumberofrecordsislarge.Moreprecisely,foreachrecord,thealgorithmcomputesahashfunctionthatmapsittoanLbitstring.ItthencountsthenumberofconsecutivezeroesstartingfromtheleastsignificantpositionofthehashresultandsetsthecorrespondingbitinabitmapofsizeL.Ifthealgorithmsseesrecordsthathashtovaluesendinginzero,one,andtwo0’s(thefirstthreebitsinthebitmapareset,andtherest