信息理论和 知识集聚.docx

信息理论和 知识集聚.docx

《信息理论和 知识集聚.docx》由会员分享，可在线阅读，更多相关《信息理论和 知识集聚.docx（16页珍藏版）》请在冰豆网上搜索。

信息理论和知识集聚

InformationTheoryandKnowledge-Gathering

信息理论和知识集聚

ABSTRACT

Itisassumedthathumanknowledge-buildingdependsonadiscretesequentialdecision-makingprocesssubjectedtoastochasticinformationtransmittingenvironment.ThisenvironmentrandomlytransmitsShannontypeinformation-packetstothedecision-maker,whoexamineseachofthemforrelevancyandthendetermineshisoptimalchoices.Usingthissetofrelevantinformation-packets,thedecision-makeradapts,overtime,tothestochasticnatureofhisenvironment,andoptimizesthesubjectiveexpectedrateof-growthofknowledge.Thedecision-maker’soptimalactions,leadtoadecisionfunctionthatinvolveshisviewofthesubjectiveentropyoftheenvironmentalprocessandotherimportantparametersateachstageoftheprocess.Usingthismodelofhumanbehavior,onecouldcreatepsychometricexperimentsusingcomputersimulationandrealdecision-makers,toplayprogrammedgamestomeasuretheresultinghumanperformance.

KEYWORDS

decision-making,dynamicprogramming,entropy,epistemology,informationtheory,knowledge,sequentialprocesses,subjectiveprobability

摘要

人们一般认为人类知识构造是基于一些互相分离的不相关的决策程序，

ThinkingAboutKnowledge

Chebyshev,astranslatedbyKhinchin（1957）,expressedthemagicalrelationshipbetweenscientifictheoryandpractice:

“Thebringingtogetheroftheoryandpracticeleadstothemostfavorableresults;notonlydoespracticebenefit,butthesciencesthemselvesdevelopundertheinfluenceofpractice,whichrevealsnewsubjectsforinvestigationandnewaspectsoffamiliarsubjects.”Anobjectiveforascientististounderstandthephenomenaofhisorherenvironment.Oneofthemeansforunderstandingistobuildhypotheticalathematicalmodelsthatreflectourviewofrealityandhopefullyreachthelawsofnature.Withoutthisunderstanding,howarewetoconductexperimentstovalidateourhypothesisandthenreachoutforevenmoretruthsaboutnature?

Theriskforascientificmodelbuilderistobeabletosteeranarrowpathbetweenthenaivetyofoversimplificationandthemorassofovercomplication.Thesubjectofknowledge-gatheringinhumandecisionmakersisthebasisofthispaper.Recently,thestudyofknowledge-gatheringinmachineshasbeenamajorpartofthenewscienceofArtificialIntelligence.MostofthisAIresearchhasbeentothestudyknowledgeasexpressedalongsemanticorlogicnetworklinestoenableexpertsystems.Knowledgesystems,aslearningsystems,havealsohadalongandsuccessfulhistoryinthefieldofpsychometrics.Theinfluenceofinformationtheoryonknowledgegatheringsystemsthateffecthumandecision-makinghasresultedintheapplicationofseveralmathematicaldiscretesequentialsystemanalysistechniquesthatstemfromtheclassicalworksofH.Poincaré,G.D.BirkhoffandontotherecentworksofthelateRichardBellman（1960）oftheRANDCorporation.These“dynamicprogramming”techniqueshavealsobeensuccessfullyinappliedtomanyotheriversescientificendeavors.Applyingthesesametechniquestotheanalysisofknowledgegatheringinhumandecision-makersmayappeartobea“tailwaggingthedog”tosome,butsometimesnewideascomebystrangepaths.ThesubjectofwhathumanknowledgeishasalwaysoccupiedtheoldphilosophersfromPlatotoHusserl.Recently,ithasbecomepossibletoanalyzeknowledge-gatheringasadiscrete,sequentialstochastic,adaptivedecision-makingprocess,subjectedtothelawsofprobability,andthestep-bystepactionsofhumandecision-makers,whoinKennethArrow’swords,arelearningbydoing（Arrow（1961）.

Inthispaper,initssimplestform,Iwilldevelopthemathematicalstructureofahumanknowledge-gatheringmathematicalprocessofthistype.Despitethesimplicityofthismodel,themathematicaldetailsarestillextensiveand,forthesakeofthebalancebetweenthesimpleandthecomplex,IhavedelegatedthosedetailstotheAppendix,hereinafter.

Observationofahuman,decision-maker,behavinginaknowledge-gatheringprocess,indicatesthat,ontheaverage,hisintelligenceincreasesovertimeandtendstosimplifythedecision-maker’sgatheringprocessbydecreasinghisrelativeentropy.Thisprocesscanoccurinthefaceofastochasticenvironment,a“deusexmachina”thatfeedsthedecision-makeravarietyofrandominformation-packets.Someoftheseinformationpacketsarevaluabletothedecision-maker,butmostareirrelevant.Somehow,thedecision-makerisabletosort,theserandominformationpacketsintoalistofincreasingacceptancefromtheirrelevanttotherelevant,dependingonthedecision-maker’spreviousexperience,storedinhismemory.Inthisprocess,thedecision-makertendstodecrease,bothhisrelativeentropy（hisuncertaintyorchaos）,inthefaceofaworldwherethenetentropyisbelievedtobealwaysincreasing.Non-intelligentdynamicsystems,forexample,Brownianmotionofmicro-particlesinsuspension,donotseemtobehaveinthismanner.Itappearsthatthesenon-intelligentsystemslackameansoforganizingagainst,oradaptingtothehiddenmechanismsintheirenvironment.Forthemostpart,exceptforsomechemicalprocessesthattendoscillatebackandforthfromchaostoorder,withobviousconsequencestotheeffectontheirentropy,thesenon-intelligentprocesses,intheend,alwaysincreasetheirlocalentropy.IlyaPrigogine（1996）hasgivenmanyexamplesofthiskindofpseudolifelikebehavior.

Inthispaper,Ipostulatethatlifelikesystemshaveseveralnecessaryuniquefacilitiestoeffectrelativeentropydecreasesthroughtheirorganizedknowledgegatheringactivity.Thefirstoftheseisthebasicabilitytomakedecisions.Thisability,inthefaceofcomplexworld,enableslivingbeingstoadaptandcontroltheirecisionmakingthroughtheacceptanceorrejectionofrandominformation-packetstransmittedbytheirenvironment.Withinthesesystems,wefindthehumanknowledge-gatheringactivityenablestheoptimalacceptanceorrejectionofrandominformation-packets,receivedintotheirsensesfromtheirimmediatestochasticenvironmentandstoredintheirmind.AsShakespeareaptlyputitforpoets（andmathematicians）,

“Andasimaginationbodiesforth

Theformsofthingsunknown,thepoet’spen

Turnsthemtoshapes,andgivestoairynothing

Alocalhabitationandaname.”

“AMidsummerNight’sDream,”

ActV,SceneI.

Ingeneral,humanknowledge-gatheringsystemspossessauniqueandessentialmemorycapabilitythatenablesthemtooptimizetheirpersonalandsocialdecisionmakingbehavior.TheresearchofthelateHansReichenbach（1999）inhisbook“TheDirectionofTime”hasshownthatmemory（ofsomekind）isanessentialelement,forcingtimetoalwaysseemtoflowinanincreasingdirection.Thishumanmemoryisthesecondnecessaryelementthatenablesthehumandecision-maker,toreducehis

relativeentropyandoptimizehisdecisions.Themathematicalmodel,describedherein,isaembodimentofhowthesekeydependences,decision-makingandmemory,enablethehumanknowledge-gatheringprocessandproduceitsdecreasingeffectonhisrelativeentropy.

Ittakesnerveforaneconomistandcomputerscientisttoleapintoaworlddominatedbycognitivescientistsandphilosophers,andworseyet,proposeyetanothermathematicalmodelforman’scapabilitytoaccumulateknowledge.Mypreviousresearchinmathematicalstatistics,informationtheoryandeconomicdecisionprocessesforcedmepassdownthisdangerouspath.ItallstartedwhenClaudeShannon（1948）developedanoriginalidea.Inhispaper,heintroducedarationalmeasureforinformationbasedontheneedsofthecommunicationengineersatBellTelephoneLabsandthuslaunchedtheformalideaswenowfondlycall“InformationTheory.”Inhisoriginalconcept,Shannonconsideredallinformation-packetshadequalvalue.Sincehisprimaryfocuswasontheefficiencyofthetransmissionofelectronicinformationoverabandwidthlimitedcommunicationschannel,hewasnotdirectlyconcernedwiththevalueofthatinformation.InShannon’soriginalviewthevalueofatransmittedinformationpacketwasalwaysintheeyesofthesenderorreceiver-notthecommunicationsserviceprovider.LeonBrillouin（1963）pointedoutthatthevalueofShannon’smeasureofinformationisdependentonthepotentialuseofthatinformationbyadecision-maker.Shannonandhisassociatessoonrectifiedthatomissionintheirlaterpapers.J.Marshack（1960）advancedtheclassicaltheoryofeconomicsbyhisconsiderationtheroleofinformationasanewkindof“factor”intheproductionfunctionandwassoonfollowedbyseveralothersimilarpapers.Recently,KennethArrow（1996）extendedShannon’scostofinformationtheorytoaremarkablelevel.Arrowhasshownhowinformationtheory,appliedtomoderneconomictheory,causesthegenerationofstrongincreasingeconomicreturnstoscaleinproductionthatdominatestoday’sneweconomicgrowthinthis“information”age.VittorioSomenzi（1965）hasextendedtheaspectsofentropyfoundintheinformationtheorytothe“Mind-Body”philosophiccontroversy.Finally,speciallyforthenon-mathematicalphilosopher,FredDretske（1999）hasrecentlydrawntogether,ingreatdetail,alloftheseknowledgeandinformationfragments.Theseneweconomicandphilosophicapplicationsofinformationtheoryprovidethejumpingoffpointforthispaper.

Intheearly1900’sHenriPoincarégavealecturebeforetheSocietedePsychologieinParisonDiscoveryinMathematics.ThislectureformschapterIIIofhisbook,“ScienceandMethod.”Poincaré（1953）,agreeingwithHelmholtz,feltthatthephenomenonofknowledgegatheringwascomposedofthreedistinctphases.AfterattendingPoincaré’sfamouslecture,JacquesHadamand（1954）pulled