Harrison Kinsley, Daniel Kukiea - Neural Networks from Scratch in Python (2020).pdf

Harrison Kinsley, Daniel Kukiea - Neural Networks from Scratch in Python (2020).pdf

《Harrison Kinsley, Daniel Kukiea - Neural Networks from Scratch in Python (2020).pdf》由会员分享，可在线阅读，更多相关《Harrison Kinsley, Daniel Kukiea - Neural Networks from Scratch in Python (2020).pdf（658页珍藏版）》请在冰豆网上搜索。

Preface-NeuralNetworksfromScratchinPython2NeuralNetworksfromScratchinPythonHarrisonKinsley&DanielKukieaPreface-NeuralNetworksfromScratchinPython3AcknowledgementsHarrisonKinsley:

Mywife,Stephanie,forherunfailingsupportandfaithinmethroughouttheyears.Youveneverdoubtedme.Eachandeveryviewerandpersonwhosupportedthisbookandproject.Withoutmyaudience,noneofthiswouldhavebeenpossible.ThePythonprogrammingcommunityingeneralforbeingawesome!

DanielKukieaforyourunwaveringeffortwiththismassiveprojectthatNeuralNetworksfromScratchbecame.FromlearningC+tomakemodsinGTAV,toPythonforvariousprojects,tothecalculusbehindneuralnetworks,theredoesntseemtobeanyproblemyoucannotsolveanditisapleasuretodothisforalivingwithyou.Ilookforwardtoseeingwhatsnext!

Preface-NeuralNetworksfromScratchinPython4DanielKukiea:

Myson,Oskar,forhispatienceandunderstandingduringthebusydays.Mywife,Katarzyna,fortheboundlesslove,faithandsupportinallthethingsIdo,haveeverdone,andplantodo,thesunlightduringmoststormydaysandthemorningcoffeeeverysingleday.HarrisonforchallengingmetolearnPythonthenpushingmetowardslearningneuralnetworks.Forshowingmethatthingsdonothavetobeperfectlydone,allthesupport,andmakingmeapartofsomanyinterestingprojectsincluding“letsmakeatutorialonneuralnetworksfromscratch,”whichturnedintoonethebiggestchallengesofmylifethisbook.IwouldntbeatwhereIamnowifallofthatdidnthappen.ThePythoncommunityformakingmeabetterprogrammerandforhelpingmetoimprovemylanguageskills.Preface-NeuralNetworksfromScratchinPython5CopyrightCopyright2020HarrisonKinsleyCoverDesigncopyright2020HarrisonKinsleyNopartofthisbookmaybereproducedinanyformorbyanyelectronicormechanicalmeans,withthefollowingexceptions:

1.Briefquotationsfromthebook.2.PythonCode/software（stringsinterpretedaslogicwithPython）,whichishousedundertheMITlicense,describedonthenextpage.Preface-NeuralNetworksfromScratchinPython6LicenseforCodeThePythoncode/softwareinthisbookiscontainedunderthefollowingMITLicense:

Copyright2020Sentdex,KinsleyEnterprisesInc.,https:

/nnfs.ioPermissionisherebygranted,freeofcharge,toanypersonobtainingacopyofthissoftwareandassociateddocumentationfiles（the“Software”）,todealintheSoftwarewithoutrestriction,includingwithoutlimitationtherightstouse,copy,modify,merge,publish,distribute,sublicense,and/orsellcopiesoftheSoftware,andtopermitpersonstowhomtheSoftwareisfurnishedtodoso,subjecttothefollowingconditions:

TheabovecopyrightnoticeandthispermissionnoticeshallbeincludedinallcopiesorsubstantialportionsoftheSoftware.THESOFTWAREISPROVIDED“ASIS”,WITHOUTWARRANTYOFANYKIND,EXPRESSORIMPLIED,INCLUDINGBUTNOTLIMITEDTOTHEWARRANTIESOFMERCHANTABILITY,FITNESSFORAPARTICULARPURPOSEANDNONINFRINGEMENT.INNOEVENTSHALLTHEAUTHORSORCOPYRIGHTHOLDERSBELIABLEFORANYCLAIM,DAMAGESOROTHERLIABILITY,WHETHERINANACTIONOFCONTRACT,TORTOROTHERWISE,ARISINGFROM,OUTOFORINCONNECTIONWITHTHESOFTWAREORTHEUSEOROTHERDEALINGSINTHESOFTWARE.Preface-NeuralNetworksfromScratchinPython7ReadmeTheobjectiveofthisbookistobreakdownanextremelycomplextopic,neuralnetworks,intosmallpieces,consumablebyanyonewishingtoembarkonthisjourney.Beyondbreakingdownthistopic,thehopeistodramaticallydemystifyneuralnetworks.Asyouwillsoonsee,thissubject,whenexploredfromscratch,canbeaneducationalandengagingexperience.Thisbookisforanyonewillingtoputinthetimetositdownandworkthroughit.Inreturn,youwillgainafardeeperunderstandingthanmostwhenitcomestoneuralnetworksanddeeplearning.ThisbookwillbeeasiertounderstandifyoualreadyhaveanunderstandingofPythonoranotherprogramminglanguage.Pythonisoneofthemostclearandunderstandableprogramminglanguages;wehavenorealinterestinpaddingpagecountsandexhaustinganentirefirstchapterwithabasicsofPythontutorial.Ifyouneedone,wesuggestyoustarthere:

https:

/citethismaterial:

HarrisonKinsley&DanielKukieaNeuralNetworksfromScratch（NNFS）https:

/nnfs.ioPreface-NeuralNetworksfromScratchinPython8Chapter1IntroducingNeuralNetworksWebeginwithageneralideaofwhatneuralnetworksareandwhyyoumightbeinterestedinthem.Neuralnetworks,alsocalledArtificialNeuralNetworks（thoughitseems,inrecentyears,wevedroppedthe“artificial”part）,areatypeofmachinelearningoftenconflatedwithdeeplearning.Thedefiningcharacteristicofadeepneuralnetworkishavingtwoormorehiddenlayersaconceptthatwillbeexplainedshortly,butthesehiddenlayersareonesthattheneuralnetworkcontrols.Itsreasonablysafetosaythatmostneuralnetworksinuseareaformofdeeplearning.Fig1.01:

Depictingthevariousfieldsofartificialintelligenceandwheretheyfitinoverall.Preface-NeuralNetworksfromScratchinPython9ABriefHistorySincetheadventofcomputers,scientistshavebeenformulatingwaystoenablemachinestotakeinputandproducedesiredoutputfortaskslikeclassificationandregression.Additionally,ingeneral,theressupervisedandunsupervisedmachinelearning.Supervisedmachinelearningisusedwhenyouhavepre-establishedandlabeleddatathatcanbeusedfortraining.Letssayyouhavesensordataforaserverwithmetricssuchasupload/downloadrates,temperature,andhumidity,allorganizedbytimeforevery10minutes.Normally,thisserveroperatesasintendedandhasnooutages,butsometimespartsfailandcauseanoutage.Wemightcollectdataandthendivideitintotwoclasses:

oneclassfortimes/observationswhentheserverisoperatingnormally,andanotherclassfortimes/observationswhentheserverisexperiencinganoutage.Whentheserverisfailing,wewanttolabelthatsensordataleadinguptofailureasdatathatprecededafailure.Whentheserverisoperatingnormally,wesimplylabelthatdataas“normal.”Whateachsensormeasuresinthisexampleiscalledafeature.Agroupoffeaturesmakesupafeatureset（representedasvectors/arrays）,andthevaluesofafeaturesetcanbereferredtoasasample.Samplesarefedintoneuralnetworkmodelstotrainthemtofitdesiredoutputsfromtheseinputsortopredictbasedonthemduringtheinferencephase.The“normal”and“failure”labelsareclassificationsorlabels.Youmayalsoseethesereferredtoastargetsorground-truthswhilewefitamachinelearningalgorithm.Thesetargetsaretheclassificationsthatarethegoalortarget,knowntobetrueandcorrect,forthealgorithmtolearn.Forthisexample,theaimistoeventuallytrainanalgorithmtoreadsensordataandaccuratelypredictwhenafailureisimminent.Thisisjustoneexampleofsupervisedlearningintheformofclassification.Inadditiontoclassification,theresalsoregression,whichisusedtopredictnumericalvalues,likestockprices.Theresalsounsupervisedmachinelearning,wherethemachinefindsstructureindatawithoutknowingthelabels/classesaheadoftime.Thereareadditionalconcepts（e.g.,reinforcementlearningandsemi-supervisedmachinelearning）thatfallundertheumbrellaofneuralnetworks.Forthisbook,wewillfocusonclassificationandregressionwithneuralnetworks,butwhatwecoverhereleadstootheruse-cases.Neuralnetworkswereconceivedinthe1940s,butfiguringouthowtotrainthemremainedamysteryfor20years.Theconceptofbackpropagation（explainedlater）cameinthe1960s,butneuralnetworksstilldidnotreceivemuchattentionuntiltheystartedwinningcompetitionsin2010.Sincethen,neuralnetworkshavebeenonameteoricriseduetotheirsometimesseeminglyPreface-NeuralNetworksfromScratchinPython10magicalabilitytosolveproblemspreviouslydeemedunsolvable,suchasimagecaptioning,languagetranslation,audioandvideosynthesis,andmore.Currently,neuralnetworksaretheprimarysolutiontomostcompetitionsandchallengingtechnologicalproblemslikeself-drivingcars,calculatingrisk,detectingfraud,andearlycancerdetection,tonameafew.WhatisaNeuralNetwork?

“Artificial”neuralnetworksareinspiredbytheorganicbrain,translatedtothecomputer.Itsnotaperfectcomparison,butthereareneurons,activations,andlotsofinterconnectivity,eveniftheunderlyingprocessesarequitedifferent.Fig1.02:

Comparingabiologicalneurontoanartificialneuron.Asingleneuronbyitselfisrelativelyuseless,but,whencombinedwithhundredsorthousands（ormanymore）ofotherneurons,theinterconnectivityproducesrelationshipsandresultsthatfrequentlyoutperformanyothermachinelearningmethods.Preface-NeuralNetworksfromScratchinPython11Fig1.03:

Exampleofaneuralnetworkwith3hiddenlayersof16neuronseach.Anim1.03:

https:

/nnfs.io/ntrTheaboveanimationshowstheexamplesofthemodelstructuresandthenumbersofparametersthemodelhastolearntoadjustinordertoproducethedesiredoutputs.Thedetailsofwhatisseenherearethesubjectsoffuturechapters.Itmightseemrathercomplicatedwhenyoulookatitthisway.Neuralnetworksareconsideredtobe“blackboxes”inthatweoftenhavenoideawhytheyreachtheconclusionstheydo.Wedounderstandhowtheydothis,though.Denselayers,themostcommonlayers,consistofinterconnectedneurons.Inadenselayer,eachneuronofagivenlayerisconnectedtoeveryneuronofthenextlayer,whichmeansthatitsoutputvaluebecomesaninputforthenextneurons.Eachconnectionbetweenneuronshasaweightassociatedwithit,whichisatrainablefactorofhowmuchofthisinputtouse,andthisweightgetsmultipliedbytheinputvalue.Oncealloftheinputsweightsflowintoourneuron,theya