HowComputersKnowWhatWeWant BeforeWeDo.docx

HowComputersKnowWhatWeWant BeforeWeDo.docx

《HowComputersKnowWhatWeWant BeforeWeDo.docx》由会员分享，可在线阅读，更多相关《HowComputersKnowWhatWeWant BeforeWeDo.docx（4页珍藏版）》请在冰豆网上搜索。

HowComputersKnowWhatWeWantBeforeWeDo

HowComputersKnowWhatWeWant-BeforeWeDo

Here'sanexperiment:

trythinkingofasongnotasasongbutasacollectionofdistinctmusicalattributes.Maybethesonghaspoliticallyrics.Thatwouldbeanattribute.Maybeithasapolicesireninit,oraprominentbanjopart,orpairedvocalharmony,orpunkroots.Anyoneofthosewouldbeanattribute.Asongcanhaveasmanyas400attributes-thosearejustafewoftheonesfiledunderp.

ThiscuriousideaoriginatedwithTimWestergren,oneofthefoundersofanInternetradioservicebasedinOakland,Calif.,calledPandora.Everytimeanewsongcomesout,someoneonPandora'sstaff-aspeciallytrainedmusicianormusicologist-goesthroughalistofpossibleattributesandassignsthesonganumericalratingforeachone.Analyzingasongtakesabout20minutes.

ThepeopleatPandora-norelationtothealienplanet-analyze10,000songsamonth.They'vebeendoingitfor10yearsnow,andsofarthey'veamassedadatabasecontainingdetailedprofilesof740,000differentsongs.WestergrencallsthisdatabasetheMusicGenomeProject.

Thereisapointtoallthis,apartfromsettlingbarbetsaboutwhichsonghasthemostprominentbanjopartever.ThepurposeoftheMusicGenomeProjectistomakepredictionsaboutwhatkindofmusicyou'regoingtolikenext.PandorausestheMusicGenomeProjecttopowerwhat'sknowninthebusinessasarecommendationengine:

oneofthosepiecesofsoftwarethatgivesyouadviceaboutwhatyoumightenjoylisteningtoorwatchingorreadingnext,basedonwhatyoujustlistenedtoorwatchedorread.TellPandorayoulikeSpoonandi'tllplayyouModestMouse.TellityoulikeCajunaccordionvirtuosoAlphonse“BoisSec”Ardoinandit'lltryyououtonsomeIryLeJeune.EnoughpeopleliketellingPandorawhattheylikethattheserviceadds2.5millionnewusersamonth.

Overthepastdecade,recommendationengineshavebecomequietlyubiquitous.Attheappropriatemoment-generallywhenyou'reabouttoconsummatearetailpurchase-theyappearatyourshoulder,whisperingsuggestivelyinyourear.Amazonwasthepioneerofautomatedrecommendations,butNetflix,Apple,YouTubeandTiVohavethemtoo.Inthemusicspacealone,Pandorahasdozensofcompetitors.Agoodrecommendationengineisworthalotofmoney.AccordingtoareportbyindustryanalystForrester,one-thirdofcustomerswhonoticerecommendationsonane-commercesitewindupbuyingsomethingbasedonthem.

Thetroublewithrecommendationenginesisthatthey'rereallyhardtobuild.Theylooksimpleontheoutside-ifyoulikedX,you'llloveY!

-butthey'reactuallydoingsomethingfiendishlycomplex.They'reprocessingastoundingquantitiesofdataanddoingsowithseriouslyhigh-levelmath.That'sbecausethey'reattemptingtosecond-guessamysterious,perverseandprofoundlyhumanformofbehavior:

thepersonalresponsetoaworkofart.They'retryingtoreverse-engineerthesoul.They'realsochangingthewayourcultureworks.Weusedtolearnaboutnewworksofartfromfriendsandcriticsandvideo-storeclerks-frompeople,inotherwords.Nowwelearnaboutthemfromsoftware.There'sanewclassoftastemakers,andthe'yrenothuman.LearningtoLoveDolphLundgren

Pandoramakesrecommendationsthesamewaypeopledo,moreorless:

byknowingsomethingaboutthemusicit'srecommendingandsomethingaboutyourmusicaltaste.Butthat'sactuallyprettyunusual.It'saverylabor-intensiveapproach.Mostrecommendationenginesworkbackwardinstead,usinginformationthatcomesnotfromtheartbutfromitsaudience.

It'satechniquecalledcollaborativefiltering,anditworksontheprinciplethatthebehaviorofalotofpeoplecanbeusedtomakeeducatedguessesaboutthebehaviorofasingleindividual.Her'estheidea:

if,statisticallyspeaking,mostpeoplewholikedthefirstSexandtheCitymoviealsolikeMammaMia!

thenifweknowthataparticularindividuallikedSexandtheCity,wecanmakeaneducatedguessthatthatindividualwillalsolikeMammaMia!

Itsoundssimpleenough,butthecloseryoulook,theweirderandmorecomplicateditgets.TakeNetflix'srecommendationengine,whichithasdubbedCinematch.Thealgorithmicgutsofarecommendationengineareusuallyafiercelyguardedtradesecret,butin2006Netflixdecideditwasn'tcompletelyhappywithCinematch,andittookanunusualapproachtosolvingtheproblem.Thecompanymadepublicaportionofitsdatabaseofmovieratings-around100millionofthem-andofferedaprizeof$1milliontoanybodywhocouldimproveitsengineby10%.TheNetflixcompetitionopenedawindowontoaworldthat'susuallylockedawaydeepinthebowelsofcorporateR&Ddepartments.Theeventualwinner-whichclinchedtheprizelastfall-wasaseven-man,four-countryconsortiumcalledBellKor'sPragmaticChaos,whichincludedBobBellandChrisVolinsky,twomembersofAT&'Tsresearchdivision.Talkingtothem,youstarttoseehowdifficultitistomakeapieceofsoftwareunderstandthevagariesofhumantaste.Youalsoseehow,oddly,softwareunderstandsthingsaboutourtasteinmoviesthatahumanvideoclerknevercould.

Thekeypointtograspaboutcollaborative-filteringsoftwareisthatitknowsabsolutelynothingaboutmovies.Ithasnopreconceptions;itworksentirelyonthebasisoftheaudience'sreaction.Soifalargeenoughgroupofpeopleclaimtohaveenjoyed,say,bothSawVandOnGoldenPond,thesoftwarewouldbeforcedtoinferthatthosetwomoviessharesomecommonqualitythattheviewersenjoyed.Crazy?

Orcrazygenius?

Insuchacase,thesoftwarewouldhavediscoveredanaestheticpropertythatwemightnotevenbeawareoforhaveanameforbutwhichinamathematicalsensemustbesaidtoexist.EvenBellandVolinskydon'talwaysknowwhatthepropertiesare.“Wemightbeabletodescribethem,orwemightnotbeableto,”Bellsays.“Theymightbesubtletieslike‘actionmoviesthatdon'thavealotofblood,don'thavealotofprofanitybuthaveastrongfemalelead.'Thingslikethat,whichyouwouldneverthinktocategorizeonyourown.”AsVolinskyputsit,“Alotoftimes,wedon'tcomeupwithexplanationsthatareexplainable.”

Thatmakesrecommendationenginessoundpracticallypsychic,buteverydayexperiencetellsusthatthey'reactuallyprettyfallible.EverybodyhasfelttheoutragethatcomeswhenarecommendationengineaccusesoneofasecretdesiretowatchRockyIV,theonewithDolphLundgreninit.In2006,WalmartwaschargedwithracismwhenitsrecommendationenginepairedPlanetoftheApeswithadocumentaryaboutMartinLutherKing.Butgenerallyspeaking,theweaklinkinarecommendationengineisn'tthesoftware;it'sus.Collaborativefilteringworksonlyaswellasthedataithasavailable,andhumansproducenoisy,low-qualitydata.

Theproblemisconsistency:

we'rejustnotgoodatexpressingourdesiresinratingform.Weratethingsdifferentlyafterabaddayatworkthanwewouldifwewereonvacation.Somepeoplearenaturallystingywiththeirstars;othersaregenerous.Weratemoviesdifferentlydependingonwhetherweratethemrightafterwatchingthemorifwewaitaweek,anddifferentlyagaindependingonwhetherwesawalousymovieoragoodmovieinthatinterveningweek.Weevenratedifferentlydependingonwhetherwerateawholebatchofmoviestogetheroroneatatime.

Allthismeansthatthere'saceilingtohowaccuratecollaborativefilteringcanget.“There'salotofrandomnessinvolved,”Volinskyadmits.“There'ssomeintrinsicleveloferrorassociatedwithtryingtopredicthumanbehavior.”

TheGreatChoiceEpidemic

Recommendationenginesarearesponsetothestrangenewworldofonlineretail.It'saworldcharacterizedbyasurplusofsomethingweusuallycan'tgetenoughof:

choice.

We'redrowninginit.AsSheenaIyengarpointsoutinherbookTheArtofChoosing,in1994therewere500,000differentconsumergoodsforsaleintheU.S.NowAmazonaloneoffers24million.Whenfacedwithsuchanoversupplyofchoice,ourlittlelizardbrainsgostraighttovaporlock.“Wethinktheprofusionofpossibilitiesmustmakeitthatmucheasiertofindthatperfectgiftforafriend'sbirthday,”Iyengarwrites,“onlytofindourselvesparalyzedinthefaceofrowuponrowofpotentialpresents.”We'relivingthroughanepidemicofchoice.Werequireaninformationalprosthesistonavigateit.Therecommendationengineisthatprosthesis:

itwinnowsthemillionsofoptionsdowntoamanageablehandful.

Butthere'satrade-offinvolved.Recommendationenginesintroduceanewvoiceintotheculturalconversation,onethatspeakstouswhenwe'reatourmostvulnerable,whichistosayatthepointofpurchase.Whatisthatvoicesaying?

Recommendationenginesare'ntdesignedtogiveuswhatwewant.They'redesignedtogiveuswhattheythinkwewant,basedonwhatweandotherpeoplelikeushavewantedinthepast.

Whichmeanstheydon'tsurpriseus.Theydon'ttakeusoutofourcomfortzone.Arecommendationengineisn'tthespousewhodragsyoutoanartfilmyouwouldn'thavebeencaughtdeadatbutthenunexpectedlylove.Itwon'tforceyoutoreadthe18thcenturycanon.It'snosubstituteforstumblingontoagreatCDjustbecauseithascoolcoverart.RecommendationenginesaretheenemyofserendipityandGreatBooksandtheavant-garde.A19thcenturyrecommendationenginewouldneverhavesaid,IfyoulikedMonet,you'llloveVanGogh!

Impressionismwouldhavelastedforever.

Theriskyourunwithrecommendationenginesisthatthey'llkeepyouinarut.Theydothatbecauserutsarecomfyplaces-thoughoftenthey'redeeperthantheylook.“Bydefinition,wekeepyouinthesam