中科院计算所Recent advances in deep learning.docx

中科院计算所Recent advances in deep learning.docx

《中科院计算所Recent advances in deep learning.docx》由会员分享，可在线阅读，更多相关《中科院计算所Recent advances in deep learning.docx（6页珍藏版）》请在冰豆网上搜索。

中科院计算所Recentadvancesindeeplearning

Recentadvancesindeeplearning

（biased&bynomeanscomplete）

Goingdeeper

imagefromK.He’sslides

Goingdeeper

“DeepResidualLearningforImageRecognition”,He,Zhang,Ren,Sun.CVPR2016.arXiv:

1512.03385.Dec.2015

“HighwayNetworks”,Srivastava,Greff,Schmidhuber,arXiv:

1505.00387.May2015“TrainingVeryDeepNetworks”,Srivastava,Greff,Schmidhuber.NIPS2015

imagefromK.He’sslides

Deepresiduallearning

ProbablythemostsignificantDeepLearningworkin2015.

“DeepResidualLearningforImageRecognition”.He,Zhang,Ren,Sun,CVPR2016.arXiv:

1512.03385.Dec.2015

imagefromK.He’sslides

Detection

~0.3sperimage

“FastR-CNN”,Girshick.ICCV2015

“FasterR-CNN:

TowardsReal-TimeObjectDetectionwithRegionProposalNetworks”,Ren,He,Girshick,Sun.NIPS2015

“YouOnlyLookOnce:

Unified,Real-TimeObjectDetection”,Redmon,

Divvala,Girshick,Farhadi.CVPR2016

imagefromGirshick2015.

Segmentation

“FullyConvolutionalNetworksforSemanticSegmentation”,Long,Shelhamer,Darrell.CVPR2015.（CVPRbestpaperhonorablemention）

“EfficientPiecewiseTrainingofDeepStructuredModelsforSemanticSegmentation”,Lin,Shen,vandanHengel,Reid.CVPR2016.

“ConditionalRandomFieldsasRecurrentNeuralNetworks”,Zhengetal.ICCV2015.

“High-performanceSemanticSegmentationUsingVeryDeepFullyConvolutionalNetworks”,Wu,Shen,vandenHengel.arXiv:

1604.04339.April2016.imagefromLongetal.2015

Vision&Language

（c）Baidu

Imagecaptioning

Imagecaptioning

Visualquestionanswering

“AskMeAnything:

Free-formVisualQuestionAnsweringBasedonKnowledgefromExternalSources”,Wuetal.arXiv:

1506.01144,CVPR2016

“WhatValueHighLevelConceptsinVisiontoLanguageProblems?

”,Wuetal.CVPR2016“VQA:

VisualQuestionAnswering”,Antoletal.ICCV2015.

“AskYourNeurons:

ANeural-BasedApproachtoAnsweringQuestionsAboutImages”,Malinowskietal.ICCV2015.

andmanyotherpapers…

Methodology

（WhatIbelieveisimportant）

RNN

http:

//karpathy.github.io/2015/05/21/rnn-effectiveness/

RNN

SceneLabelingWithLSTMRecurrentNeuralNetworks,Byeonetal.CVPR2015.

Deepstructuredlearning

LearningDeepStructuredModels,ICML2015

EfficientPiecewiseTrainingofDeepStructuredModelsforSemanticSegmentation,CVPR2016

ConditionalRandomFieldsasRecurrentNeuralNetworks,Zhengetal.ICCV2015

DeeplyLearningtheMessagesinMessagePassingInference,Linetal.NIPS2015

StructuredlearningasRNN

“CNN-RNN:

AUnifiedFrameworkforMulti-labelImageClassification”,Wangetal.CVPR2016.（Baidu）

“SemanticObjectParsingwithGraphLSTM”,Liang,Shen,Feng,Lin,Yan.arXiv:

1603.07063

Speedinguptesting/training

Low-rankApproximation:

“AcceleratingVeryDeepConvolutionalNetworksforClassificationandDetection”,Zhangetal.,TPAMI2015（MSRA）

Pruning:

“LearningbothWeightsandConnectionsforEfficientNeuralNetworks”,Hanetal.,NIPS2015（NVIDIA）

BinarizedNeuralNetworks:

“BinaryConnect:

TrainingDeepNeuralNetworkswithbinaryweightsduringpropagations”,Courbariauxetal.,NIPS2015

“BinarizedNeuralNetworks:

TrainingDeepNeuralNetworkswithWeightsand

ActivationsConstrainedto+1or-1”,Courbariauxetal.arXiv:

1602.02830

Thanks.Questions?