神经网络概述.docx

神经网络概述.docx

《神经网络概述.docx》由会员分享，可在线阅读，更多相关《神经网络概述.docx（12页珍藏版）》请在冰豆网上搜索。

神经网络概述

NeuralNetworkIntroduction

1.Objectives

Asyoureadthesewordsyouareusingacomplexbiologicalneuralnetwork.Youhaveahighlyinterconnectedsetofsome1011neuronstofacilitateyourreading,breathing,motionandthinking.Eachofyourbiologicalneurons，arichassemblyoftissueandchemistry,hasthecomplexity,ifnotthespeed,ofamicroprocessor.Someofyourneuralstructurewaswithyouatbirth.Otherpartshavebeenestablishedbyexperience.

Scientistshaveonlyjustbeguntounderstandhowbiologicalneuralnetworksoperate.Itisgenerallyunderstoodthatallbiologicalneuralfunctions,includingmemory,arestoredintheneuronsandintheconnectionsbetweenthem.Learningisviewedastheestablishmentofnewconnectionsbetweenneuronsorthemodificationofexistingconnections.

Thisleadstothefollowingquestion:

Althoughwehaveonlyarudimentaryunderstandingofbiologicalneuralnetworks,isitpossibletoconstructasmallsetofsimpleartificial“neurons”andperhapstrainthemtoserveausefulfunction?

Theansweris“yes.”Thisbook,then,isaboutartificialneuralnetworks.

Theneuronsthatweconsiderherearenotbiological.Theyareextremelysimpleabstractionsofbiologicalneurons,realizedaselementsinaprogramorperhapsascircuitsmadeofsilicon.Networksoftheseartificialneuronsdonothaveafractionofthepowerofthehumanbrain,buttheycanbetrainedtoperformusefulfunctions.Thisbookisaboutsuchneurons,thenetworksthatcontainthemandtheirtraining.

2.History

Thehistoryofartificialneuralnetworksisfilledwithcolorful,creativeindividualsfrommanydifferentfields,manyofwhomstruggledfordecadestodevelopconceptsthatwenowtakeforgranted.Thishistoryhasbeendocumentedbyvariousauthors.OneparticularlyinterestingbookisNeurocomputing:

FoundationsofResearchbyJohnAndersonandEdwardRosenfeld.Theyhavecollectedandeditedasetofsome43papersofspecialhistoricalinterest.Eachpaperisprecededbyanintroductionthatputsthepaperinhistoricalperspective.

Historiesofsomeofthemainneuralnetworkcontributorsareincludedatthebeginningofvariouschaptersthroughoutthistextandwillnotberepeatedhere.However,itseemsappropriatetogiveabriefoverview,asampleofthemajordevelopments.

Atleasttwoingredientsarenecessaryfortheadvancementofatechnology:

conceptandimplementation.First,onemusthaveaconcept,awayofthinkingaboutatopic,someviewofitthatgivesclaritynottherebefore.Thismayinvolveasimpleidea,oritmaybemorespecificandincludeamathematicaldescription.Toillustratethispoint,considerthehistoryoftheheart.Itwasthoughttobe,atvarioustimes,thecenterofthesoulorasourceofheat.Inthe17thcenturymedicalpractitionersfinallybegantoviewtheheartasapump,andtheydesignedexperimentstostudyitspumpingaction.Theseexperimentsrevolutionizedourviewofthecirculatorysystem.Withoutthepumpconcept,anunderstandingoftheheartwasoutofgrasp.

Conceptsandtheiraccompanyingmathematicsarenotsufficientforatechnologytomatureunlessthereissomewaytoimplementthesystem.Forinstance,themathematicsnecessaryforthereconstructionofimagesfromcomputer-aidedtopography（CAT）scanswasknownmanyyearsbeforetheavailabilityofhigh-speedcomputersandefficientalgorithmsfinallymadeitpracticaltoimplementausefulCATsystem.

Thehistoryofneuralnetworkshasprogressedthroughbothconceptualinnovationsandimplementationdevelopments.Theseadvancements,however,seemtohaveoccurredinfitsandstartsratherthanbysteadyevolution.

Someofthebackgroundworkforthefieldofneuralnetworksoccurredinthelate19thandearly20thcenturies.Thisconsistedprimarilyofinterdisciplinaryworkinphysics,psychologyandneurophysiologybysuchscientistsasHermannvonHelmholtz,ErnstMuchandIvanPavlov.Thisearlyworkemphasizedgeneraltheoriesoflearning,vision,conditioning,etc.,anddidnotincludespecificmathematicalmodelsofneuronoperation.

Themodernviewofneuralnetworksbeganinthe1940swiththeworkofWarrenMcCullochandWalterPitts[McPi43],whoshowedthatnetworksofartificialneuronscould,inprinciple,computeanyarithmeticorlogicalfunction.Theirworkisoftenacknowledgedastheoriginofthe

neuralnetworkfield.

McCullochandPittswerefollowedbyDonaldHebb[Hebb49],whoproposedthatclassicalconditioning（asdiscoveredbyPavlov）ispresentbecauseofthepropertiesofindividualneurons.Heproposedamechanismforlearninginbiologicalneurons.

Thefirstpracticalapplicationofartificialneuralnetworkscameinthelate1950s,withtheinventionoftheperceptionnetworkandassociatedlearningrulebyFrankRosenblatt[Rose58].Rosenblattandhiscolleaguesbuiltaperceptionnetworkanddemonstrateditsabilitytoperformpatternrecognition.Thisearlysuccessgeneratedagreatdealofinterestinneuralnetworkresearch.Unfortunately,itwaslatershownthatthebasicperceptionnetworkcouldsolveonlyalimitedclassofproblems.（SeeChapter4formoreonRosenblattandtheperceptionlearningrule.）

Ataboutthesametime,BernardWidrowandTedHoff[WiHo60]introducedanewlearningalgorithmandusedittotrainadaptivelinearneuralnetworks,whichweresimilarinstructureandcapabilitytoRosenblatt’sperception.TheWidrowHofflearningruleisstillinusetoday.（SeeChapter10formoreonWidrow-Hofflearning.）

Unfortunately,bothRosenblatt'sandWidrow'snetworkssufferedfromthesameinherentlimitations,whichwerewidelypublicizedinabookbyMarvinMinskyandSeymourPapert[MiPa69].RosenblattandWidrowwere

awareoftheselimitationsandproposednewnetworksthatwouldovercomethem.However,theywerenotabletosuccessfullymodifytheirlearningalgorithmstotrainthemorecomplexnetworks.

Manypeople,influencedbyMinskyandPapert,believedthatfurtherresearchonneuralnetworkswasadeadend.This,combinedwiththefactthattherewerenopowerfuldigitalcomputersonwhichtoexperiment,

causedmanyresearcherstoleavethefield.Foradecadeneuralnetworkresearchwaslargelysuspended.Someimportantwork,however,didcontinueduringthe1970s.In1972TeuvoKohonen[Koho72]andJamesAnderson[Ande72]independentlyandseparatelydevelopednewneuralnetworksthatcouldactasmemories.StephenGrossberg[Gros76]wasalsoveryactiveduringthisperiodintheinvestigationofself-organizingnetworks.

Interestinneuralnetworkshadfalteredduringthelate1960sbecauseofthelackofnewideasandpowerfulcomputerswithwhichtoexperiment.Duringthe1980sbothoftheseimpedimentswereovercome,andresearch

inneuralnetworksincreaseddramatically.Newpersonalcomputersand

workstations,whichrapidlygrewincapability,becamewidelyavailable.Inaddition,importantnewconceptswereintroduced.

Twonewconceptsweremostresponsiblefortherebirthofneuralnetworks.Thefirstwastheuseofstatisticalmechanicstoexplaintheoperationofacertainclassofrecurrentnetwork,whichcouldbeusedasanassociativememory.ThiswasdescribedinaseminalpaperbyphysicistJohnHopfield[Hopf82].

Thesecondkeydevelopmentofthe1980swasthebackpropagationalgorithmfortrainingmultilayerperceptronnetworks,whichwasdiscoveredindependentlybyseveraldifferentresearchers.ThemostinfluentialpublicationofthebackpropagationalgorithmwasbyDavidRumelhartandJamesMcClelland[RuMc86].ThisalgorithmwastheanswertothecriticismsMinskyandPaperthadmadeinthe1960s.（SeeChapters11and12foradevelopmentofthebackpropagationalgorithm.）

Thesenewdevelopmentsreinvigoratedthefieldofneuralnetworks.Inthelasttenyears,thousandsofpapershavebeenwritten,andneuralnetworkshavefoundmanyapplications.Thefieldisbuzzingwithnewtheoreticalandpracticalwork.Asnotedbelow,itisnotclearwhereallofthiswillleadUS.

Thebriefhistoricalaccountgivenaboveisnotintendedtoidentifyallofthemajorcontributors,butissimplytogivethereadersomefeelforhowknowledgeintheneuralnetworkfieldhasprogressed.Asonemightnote,theprogresshasnotalwaysbeen"slowbutsure."Therehavebeenperiodsofdramaticprogressandperiodswhenrelativelylittlehasbeenaccomplished.

Manyoftheadvancesinneuralnetworkshavehadtodowithnewconcepts,suchasinnovativearchitecturesandtraining.Justasimportanthasbeentheavailabilityofpowerfulnewcomputersonwhichtotestthesenewconcepts.

Well,somuchforthehistoryofneuralnetworkstothisdate.Therealquestionis,"Whatwillhappeninthenexttentotwentyyears?

"Willneuralnetworkstakeapermanentplaceasamathematical/engineeringtool,orwilltheyfadeawayashavesomanypromisingtechnologies?

Atpresent,theanswerseemstobethatneuralnetworkswillnotonlyhavetheirdaybutwillhaveapermanentplace,notasasolutiontoeveryproblem,butasatooltobeusedinappropriatesituations.Inaddition,rememberthatwestillknowverylittleabouthowthebrainworks.Themostimportantadvancesinneuralnetworksalmostcertainlylieinthefuture.

Althoughitisdifficulttopredictthefuturesuccessofneuralnetworks,thelargenumberandwidevarietyofapplicationsofthisnewtechnologyareveryencouraging.Thenextsectiondescribessomeoftheseapplications.

3.Applications

ArecentnewspaperarticledescribedtheuseofneuralnetworksinliteratureresearchbyAstonUniversity.Itstatedthat"thenetworkcanbetaughttorecognizeindividualwritingstyles,andtheresearchersusedittocompareworksattributedtoShakespeareandhiscontemporari