斯坦福大学机器学习第七讲Lecture7.docx
《斯坦福大学机器学习第七讲Lecture7.docx》由会员分享,可在线阅读,更多相关《斯坦福大学机器学习第七讲Lecture7.docx(11页珍藏版)》请在冰豆网上搜索。
斯坦福大学机器学习第七讲Lecture7
Theproblemof
overfitting
MachineLearning
Example:
Linearregression(housingprices
Overfitting:
Ifwehavetoomanyfeatures,thelearnedhypothesistogeneralizetonewexamples(predictpricesonnewexamples.
Price
SizePrice
SizePrice
Size
Example:
Logisticregression
(=sigmoidfunction
x
1
x
2
x
1
x
2
x
1
x
2
P
r
i
c
e
Sizesizeofhouse
no.ofbedrooms
no.offloors
ageofhouse
averageincomeinneighborhood
kitchensize
Options:
1.Reducenumberoffeatures.
―Manuallyselectwhichfeaturestokeep.
―Modelselectionalgorithm(laterincourse.
2.Regularization.
―Keepallthefeatures,butreducemagnitude/valuesofparameters.
―Workswellwhenwehavealotoffeatures,eachofwhichcontributesabittopredicting.
Costfunction
MachineLearning
Intuition
Supposewepenalizeandmake,reallysmall.
PriceSizeofhousePrice
Sizeofhouse
Smallvaluesforparameters
―“Simpler”hypothesis
―LesspronetooverfittingHousing:
―Features:
―Parameters:
P
r
ic
eSizeo
fhouse
Whatifissettoanextremelylargevalue(perhapsfortoolargeforourproblem,say?
-Algorithmworksfine;settingtobeverylargecan’thurtit
-Algortihmfailstoeliminateoverfitting.
-Algorithmresultsinunderfitting.(Failstofiteventrainingdatawell.
-Gradientdescentwillfailtoconverge.
Whatifissettoanextremelylargevalue(perhapsfortoolargeforourproblem,say?
Price
Sizeofhouse
Regularizedlinear
regression
MachineLearning
Regularizedlinearregression
Gradientdescent
Repeat
Normalequation
Suppose,
Non-invertibility
(optional/advanced.(#examples(#featuresIf,
Regularized
logisticregression
MachineLearning
Regularizedlogisticregression.x2x1Costfunction:
AndrewNg
GradientdescentRepeatAndrewNg
Advancedoptimizationfunction[jVal,gradient]=costFunction(thetajVal=[codetocompute];];gradient(1=[codetocomputegradient(2=[codetocompute];gradient(3=[codetocompute];gradient(n+1=[codetocompute];AndrewNg