03Lecture noteLinear and Logistic Regression.docx

03Lecture noteLinear and Logistic Regression.docx

《03Lecture noteLinear and Logistic Regression.docx》由会员分享，可在线阅读，更多相关《03Lecture noteLinear and Logistic Regression.docx（18页珍藏版）》请在冰豆网上搜索。

03LecturenoteLinearandLogisticRegression

Lecturenote3:

LinearandLogisticRegressioninTensorFlow

CS20:

TensorFlowforDeepLearningResearch（cs20.stanford.edu）

PreparedbyChipHuyen（chiphuyen@cs.stanford.edu）

ThanksLaurenceMoroney,AkshayAgrawalforreviewingthenote

Phew,thelastlecturewaslong.Goodnewsisthatwe’redonewithallthealphabeticallearningandcannowbeofftobuildingthings.Beforestartingtheadventures,let’smakesurethatwe’refamiliarwithallthekeyconcepts.Ifyou’reunsureaboutanyofthese,youshouldgobacktotheprevioustwolectures.

Graphsandsessions

TFOps:

constants,variables,functions

TensorBoard

Lazyloading

LinearRegression:

Predictlifeexpectancyfrombirthrate

Let’sstartwithasimplelinearregressionexample.Ihopeyouallarealreadyfamiliarwithlinearregression.Ifnot,youcanreadaboutitonWikipedia.Basically,we’llbebuildingaverysimpleneuralnetworkconsistingofonelayertoinferthelinearrelationshipbetweenoneexplanatoryvariableXandonedependentvariableY.

Problem

Irecentlycameacrossthevisualizationoftherelationshipbetweenbirthratesandlifeexpectanciesofdifferentcountriesaroundtheworldandfoundthatfascinating.Basically,itlookslikethemorechildrenyouhave,theyoungeryouaregoingtodie!

YoucanplaythevisualizationcreatedbyGooglebasedonthedatacollectedbytheWorldBankhere.

Myquestionis,canwequantifythatrelationship?

Inotherwords,ifthebirthrateofacountryisanditslifeexpectancyis,canwefindalinearfunctionfsuchthat?

Ifweknowthatrelationship,giventhebirthrateofacountry,wecanpredictthelifeexpectancyofthatcountry.

Forthisproblem,wewillbeusingasubsetoftheWorldDevelopmentIndicatorsdatasetcollectedbytheWorldBank.Forsimplicity,wewillbeusingdatafromtheyear2010only.Youcandownloadthedatafromclass’sGitHubfolderhere.

DatasetDescription

Name:

Birthrate-lifeexpectancyin2010

X=birthrate.Type:

float.

Y=lifeexpectancy.Type:

foat.

Numberofdatapoints:

190

Approach

First,assumethattherelationshipbetweenthebirthrateandthelifeexpectancyislinear,whichmeansthatwecanfindwandbsuchthat.

Tofindwandb（inthiscase,theyarebothscalars）,wewillusebackpropagationthroughaonelayerneuralnetwork.Forthelossfunction,wewillbeusingmeansquarederror.Aftereachepoch,wemeasurethemeansquareddifferencebetweentheactualvalueYsandthepredictedvaluesofYs.

Youcandownloadthefileexamples/03_linreg_starter.pyfromtheclass’sGitHubrepotogiveitashotyourself.Afteryou’redone,youcancomparewiththesolutionbelow.Youcanalsovisitexamples/03_linreg_placeholder.pyonGitHubfortheexecutablescript.

importtensorflowastf

importutils

DATA_FILE="data/birth_life_2010.txt"

#Step1:

readindatafromthe.txtfile

#dataisanumpyarrayofshape（190,2）,eachrowisadatapoint

data,n_samples=utils.read_birth_life_data（DATA_FILE）

#Step2:

createplaceholdersforX（birthrate）andY（lifeexpectancy）

X=tf.placeholder（tf.float32,name='X'）

Y=tf.placeholder（tf.float32,name='Y'）

#Step3:

createweightandbias,initializedto0

w=tf.get_variable（'weights',initializer=tf.constant（0.0））

b=tf.get_variable（'bias',initializer=tf.constant（0.0））

#Step4:

constructmodeltopredictY（lifeexpectancyfrombirthrate）

Y_predicted=w*X+b

#Step5:

usethesquareerrorasthelossfunction

loss=tf.square（Y-Y_predicted,name='loss'）

#Step6:

usinggradientdescentwithlearningrateof0.01tominimizeloss

optimizer=tf.train.GradientDescentOptimizer（learning_rate=0.001）.minimize（loss）

withtf.Session（）assess:

#Step7:

initializethenecessaryvariables,inthiscase,wandb

sess.run（tf.global_variables_initializer（））

#Step8:

trainthemodel

foriinrange（100）:

#run100epochs

forx,yindata:

#Sessionrunstrain_optominimizeloss

sess.run（optimizer,feed_dict={X:

x,Y:

y}）

#Step9:

outputthevaluesofwandb

w_out,b_out=sess.run（[w,b]）

Aftertrainingfor100epochs,wegottheaveragesquarelosstobe30.04withw=-6.07,b=84.93.Itconfirmsourbeliefthatthere’sanegativecorrelationbetweenthebirthrateandthelifeexpectancyofacountry.Andno,itdoesn’tmeanthathavingachildtakesoff6yearsofyourlife.

YoucanmakeotherassumptionsabouttherelationshipbetweenXandY.Forexample,ifwehaveaquadraticfunction:

Tofindw,u,andbforthismodel,weonlyhavetoaddanothervariableuandchangetheformulaforY_predicted.

#Step3:

createvariables:

weights_1,weights_2,bias.Allareinitializedto0

w=tf.get_variable（'weights_1',initializer=tf.constant（0.0））

u=tf.get_variable（'weights_2',initializer=tf.constant（0.0））

b=tf.get_variable（'bias',