3A CongestionAware Network File System.docx

3A CongestionAware Network File System.docx

《3A CongestionAware Network File System.docx》由会员分享，可在线阅读，更多相关《3A CongestionAware Network File System.docx（25页珍藏版）》请在冰豆网上搜索。

3ACongestionAwareNetworkFileSystem

CA-NFS:

ACongestion-AwareNetworkFileSystem

AlexandrosBatsakis

NetApp

JohnsHopkinsUniversity

RandalBurns

JohnsHopkinsUniversity

ArkadyKanevsky

NetApp

JamesLentini

NetApp

ThomasTalpey

NetApp

Abstract

Wedevelopaholisticframeworkforadaptivelyschedulingasynchronousrequestsindistributedfilesystems.Thesystemisholisticinthatitmanagesallresources,includingnetworkbandwidth,serverI/O,serverCPU,andclientandservermemoryutilization.Itaccelerates,defers,orcancelsasynchronousrequestsinordertoimproveapplication-perceivedperformancedirectly.Weemploycongestionpricingviaonlineauctionstocoordinatetheuseofsystemresourcesbythefilesystemclientssothattheycandetectshortagesandadapttheirresourceusage.WeimplementourmodificationsintheCongestion-AwareNetworkFileSystem（CA-NFS）,anextensiontotheubiquitousnetworkfilesystem（NFS）.OurexperimentalresultshowsthatCA-NFSresultsina20%improvementinexecutiontimeswhencomparedwithNFSforavarietyofworkloads.

1Introduction

Distributedfilesystemclientsconsumeserverandnetworkresourceswithoutconsiderationforhowtheiroperationsinterferewiththeirfuturerequestsandotherclients.Eachclientrequestincursacosttothesystem,expressedinincreasedloadtooneormoreofitsresources.Asmorecapacity,moreworkload,ormoreusersareaddedcongestionrises,andallclientoperationssharethecostindelayedexecution.However,clientsremainoblivioustothecongestionlevelofthesystemresources.

Whenthesystemisundercongestion,networkfileserverstrytomaximizethroughputacrossclients,assumingthattheirbenefitincreaseswiththeflowrate.Thispracticedoesnotcorrespondwellwithapplication-perceivedperformancebecauseitfailstodistinguishtheurgencyandrelativepriorityoffilesystemoperationsacrosstheclientpopulation.Fromtheserver’sperspective,allclientoperationsatanygiventimeareequallyimportant.Thisisafallacy.Filesystemoperationscomeatdifferentprioritiesimplicitly.Whilesomeneedtobeperformedondemand,manycanbedeferred.Synchronousclientoperations（metadata,reads）benefitmorefromtimelyexecutionthanasynchronousoperations（mostwrites,read-aheads）,becausetheformerblockthecallingapplicationuntilcompletion.Also,certainasynchronousoperationsaremoreurgentthanothersdependingontheclient’sstate.Forexample,whenaclient’smemoryconsumptionishigh,allofitswriteoperationsbecomesynchronous,leadingtoadegradationinsystemperformance.

Inthispaper,wedevelopaperformancemanagementframeworkfordistributedfilesystemsthatdynamicallyassessessystemload,managessystemresources,andschedulesasynchronousclientoperations.Whenthesystemresourcesapproachcriticalcapacity,weapplypriorityscheduling,preferringblockingtonon-blockingrequests,andpriorityinheritance,e.g.performingwritesthatblockreadsathighpriority,sothatnon-time-critical（asynchronous）I/Otrafficdoesnotinterferewithon-demand（synchronous）requests.Ontheotherhand,ifthesystemloadislow,weperformasynchronousoperationsmoreaggressivelyinordertoavoidthepossibilityofperformingthesameoperationsatalatertime,whentheserverresourceswillbecongested.

Theframeworkisbasedonaholisticcongestionpricingmechanismthatincorporatesallcriticalresourcesamongallclientsandservers,fromclientcachestoserverdisksubsystems.Holisticgoesbeyondend-to-endinthatitbalancesresourceusageacrossmultipleclientsandservers.（End-to-endalsoconnotesnetworkendpointsandholisticmanagementgoesfromclientapplicationstoserverdisksystems.）Theholisticapproachallowsthesystemtoaddressdifferentbottlenecksindifferentconfigurationsandrespondtochangingresourcelimitationsovertime.

Serversencodetheirresourceconstraintsbyincreasingordecreasingthepriceofasynchronousreadsandwritesinthesysteminorderto“pushback”atclients.Astheserverpricesincrease,theclientsthatarenotresourceconstrainedwilldeferasynchronousoperationsforalatertimeand,thus,reducetheirpresentedload.ThishelpstoavoidcongestioninthenetworkandserverI/Osystemcausedbynon-criticaloperations.

Theunderlyingpricingalgorithm,basedonresourceutilization,providesalog-kcompetitivesolutiontoresourcepricingwhencomparedwithanofflinealgorithmthat“knows”allfuturerequests.Incontrasttoheuristicmethodsformovingthresholds,thisapproachissystemandworkloadindependent.

WeevaluateourproposedchangesinCA-NFS（Congestion-AwareNetworkFileSystem）,anextensionoftheNFSprotocol,implementedasmodificationstotheLinuxNFSclient,server,andmemorymanager.ExperimentalresultsshowthatCA-NFSoutperformsNFSandimprovesapplication-perceivedperformancebymorethan20%inawidevarietyofworkloads.

2SystemOperation

Inthissection,wegivetheintuitionbehindschedulingasynchronousoperationsandtheeffectthesehaveonsystemresourceutilization.Wethendemonstratehowclientsadapttheirbehaviorusingpricingandauctions.

2.1AsynchronousWrites

Theeffectivenessofasynchronouswriteoperationsdependsontheclient’scurrentmemorystate.Writesareasynchronousonlyifthereisavailablememory;asystemthatcannotallocatememorytoawrite,blocksthatwriteuntilmemorycanbefreed.Thishampersperformanceseverelybecauseallsubsequentwritesbecomeeffectivelysynchronous.Italsohasanadverseeffectonreads.Allpendingwritesthatmustbewrittentostorageinterferewithconcurrentreads,whichresultsinqueuingdelaysatthenetworkanddisk.

CA-NFSchangesthewaythatasynchronouswritesareperformedcomparedtoregularNFS.NFSclientswritedatatotheserver’smemoryimmediatelyuponreceivingawrite（）systemcallandalsobufferthewritedatainlocalmemory.Thebufferedpagesaremarkedasdirtyatboththeclientandtheserver.Tohardenthesedatatodisk,theclientsendsacommitmessagetotheserver.Thedecisionofwhentocommitthedatatotheserverdependsonseveralfactors.Traditionally,systemsusedaperiodicupdatepolicyinwhichindividualdirtyblocksareflushedwhentheiragereachesapredefinedlimit[32].Modernsystemsdestagedirtypageswhenthenumberofdirtypagesinmemoryexceedsacertainpercentage（flushingpoint）,whichistypicallyasmallfractionoftheavailablememory（e.g10%）.Then,adaemonwakesupandstartsflushingdirtypagesuntilanadequatenumberofpageshavereachedstablestorage.

IncontrasttoregularNFS,CA-NFSclientsadapttheirasynchronouswritebehaviorbyeitherdeferringoracceleratingawrite.CA-NFSclientsacceleratewritesbyforcingtheCA-NFSservertosyncthedatatostablestoragesothattheclientdoesnotneedtobufferallofthecorrespondingdirtypages.Theideabehindwriteaccelerationisthatiftheserverresourceutilizationislow,thereisnoneedtodeferthecommittoalatertime.Also,clientsmayelecttoacceleratewritesinordertopreservetheircachecontentsandmaintainahighcachehitrate.Notethatacceleratingawritedoesnotmakethewriteoperationsynchronous.Instead,itinvokesthewrite-backdaemonattheclientimmediately.

Writeaccelerationpossiblyincreasestheserverdiskutilizationandusesnetworkbandwidthimmediately.Inwrite-behindsystems,manywritesarecanceledbeforetheyreachtheserver[5, 34],e.g.writingthesamefilepagerepeatedly,orcreatinganddeletingatemporaryfile.Thus,theloadimposedtotheserverasaresultofwriteaccelerationcouldbeavoided.However,writeaccelerationhasalmostnonegativeeffectonsystemperformance,becauseCA-NFSaccelerateswritesonlywhentheserverloadislow.

Deferringawriteavoidscopyingdirtydatatoservermemoryuponreceivingawriterequest.Instead,clientskeepdatainlocalmemoryonly,untilthepriceofusingtheserverresourcesislow.Clientspriceasynchronouswritesbasedontheirabilitytocachewrites,i.e.availablememory.Aclientwithscarcememory,becauseofwritedeferral,willincreaseitslocalpriceforwritessothatitsbufferedpageswillbetransferredtotheserverassoonaspossible.Tomakewritedeferralpossible,wemodifytheoperationofthewrite-backdaemonontheclientsbydynamicallychangingtheflushingpointvaluebasedonthepricingmechanismtodictatewhenthewrite-backofdirtypagesshouldbegin.

Deferringawriteconsumesclientmemorywithdirtypages,savesservermemory,anddelaystheconsumptionofnetworkbandwidthandserverdiskI/O.However,itfacestheriskofimposinghigherlatencyforsubsequentsynchronouscommitoperations.Thisisbecauseafilesyncmayrequireanetworktransferofthedirtybuffersfromtheclienttoservermemory.Notethatdeferringawritedoesnotguaranteethattheserverpriceforthesameoperationwillbelowerinthefuture.Instead,thispolicygivesprioritytooperationsoriginatingfromresource-constrainedclients.

CA-NFSfollowsNFS’sclose-to-openconsistencymodel.DeferringoracceleratingwritesdoesnotviolatetheconsistencysemanticsofNFS,becauseCA-NFSdoesnotchangethesemanticsoftheCOMMIToperation.Asynchronouswrite-backinNFSincludesadeadlinethat,whenitelapses,escalatestheoperationtoasynchronouswrite.CA-NFSdoesthesame.

Theserverpricesasynchronouswritesbasedonitsmemory,diskandnetworkutilization.Iftheservermemorycontainsblocksthatarecurrentlyaccessedbyclients,settinghighpricesforcesclientstodeferwritesinordertopreservecachecontentsandma