ðð»ââãã£ãŒãã©ãŒãã³ã°ã®åºæ¬çãªéªšæ Œã§ããMLPããCNNãRNNãŸã§ç°¡åãã€è¿
éã«åŠã³ãŸãã
Inflearnã§è¬çŸ©ããå
容ããŸãšãããPythonãã£ãŒãã©ãŒãã³ã°ãã€ããŒãããšããæ¬ã§åºçããŸããã
å€ãã®èå³ããé¡ãããŸã :)
(2020.10.06 åºæº Inflearn è¬çŸ©æŽæ°ãããŸãããç¶ç¶çãªè¬çŸ©æŽæ°ããããŠããã ããŸã)
http://m.yes24.com/Goods/Detail/93376077?ozsrank=10
http://mbook.interpark.com/shop/product/detail?prdNo=339742291&is1=book&is2=product
PyTorchã§ç°¡åãã€è¿
éã«åŠç¿ãããã£ãŒãã©ãŒãã³ã°
ðè¬çŸ©ã¢ããããŒã
- PyTorchã«æ
£ããŠããªã人ã®ããã«PyTorch backgroundè¬çŸ©ïŒ1æéã»ã©ïŒã远å ããŸããã
- è«æã¬ãã¥ãŒè¬çŸ©ïŒ1æéïŒã远å ããŸããã
ðè¬çŸ©ç޹ä»
æè¿è©±ããŠãã人工ç¥èœã¯ãã»ãšãã©ãã£ãŒãã©ãŒãã³ã°ã¢ãã«ã掻çšããŠããŸãããã£ãŒãã©ãŒãã³ã°ã®åºç€ãšãªããã¥ãŒã©ã«ãããã¯ãŒã¯ã¯æ°ããã¢ã«ãŽãªãºã ã§ã¯ãããŸããã以åããååšããŠããã¢ã«ãŽãªãºã ã§ãããåŠç¿ã®ç¹æ§äžããŸã䜿ãããŸããã§ããããã®åºæ¬ç㪠neural network ããå§ããŠããªããã£ãŒãã©ãŒãã³ã°ãæµ®ãã³å§ãããã£ãŒãã©ãŒãã³ã°ã®ç¹åŸŽãäœãªã®ããããã«åºæ¬çãªãã£ãŒãã©ãŒãã³ã°ã¢ãã«ãšåŒã°ãã convolution neural network, recurrent neural network ã«ã€ããŠåŠç¿ããŸãã
ð Multi Layer Perceptron(MLP)
æåã®äººå·¥ç¥èœãšåŒã°ããperceptronãšperceptronã®éçç¹ããããŠãããå
æããMLPã«ã€ããŠåŠã³ãŸãã
MLPã¯neural networkã®åºæ¬æ§é ãšèŠãªãããšãã§ããŸãã MLPã®åŠç¿ã¢ã«ãŽãªãºã ãæ¬¡ã
ãšãç¥ããããŸãã
Feed forwardãšback propagationã«ã€ããŠè©±ããé·æãšçæã«ã€ããŠè©±ããŸãã
ð Deep Learning ã®å®çŸ©
ãã£ãŒãã©ãŒãã³ã°ã®å®çŸ©ã¯äœã§ãããäžè¬çãªãã¥ãŒã©ã«ãããã¯ãŒã¯ãšã¯äœãéãã®ããéäžçã«åŠã³ãŸãã
NNã®æ¬ ç¹ã§ããgradient vanishing/overfittingåé¡ãç·©åã§ããactivation functionãdrop outãBatch normalizationã«ã€ããŠåãäžããŸãã
ããã«ãåã«åé¡ããããšã¯ã§ããŸããããæ°ããæ©èœã«ã€ããŠåŠã¶ããšãã§ããAuto-Encoderã«ã€ããŠã説æããŸãã
ð Convolutional Neural Network (CNN)
ãã£ãŒãã©ãŒãã³ã°ã®æŽå²ãèŠããšããæãçºå±ããã¢ãã«ããã®CNNã¢ãã«ã§ã¯ãªãããšæããŸããã€ã¡ãŒãžåé¡ããå§ãŸããèšå€§ãªçºå±ãéããŠããCNNã¢ãã«ã«ã€ããŠåãäžããŸããåŠç¿ã¢ã«ãŽãªãºã ã®ç¹æ§ã«ã€ããŠè©±ããäžè¬çãªNNãšã®éãã«ã€ããŠèª¬æããŸãã
ããã«ãCNNã®ããã©ãŒãã³ã¹ãåäžãããããã®ããŸããŸãªã¢ãŒããã¯ãã£ïŒResnetãDensenetïŒãåæåããªããã£ãã€ã¶æè¡ãããã³ transfer learning ã«ã€ããŠã話ããŸãã
ð Recurrent Neural Network (RNN)
ããã¹ãã¢ãã«ïŒLanguage modelïŒã®åºç€ã¢ãã«ã§ããRNNãšLSTMã«ã€ããŠèª¬æããŸãã
ðãã£ãŒãã©ãŒãã³ã°ã®ããŸããŸãªåé
ãã£ãŒãã©ãŒãã³ã°ã䜿ãããããŸããŸãªåéã«ã€ããŠè©±ããŸãã
ðãã£ãŒãã©ãŒãã³ã°ã®äžè¬å
ãã£ãŒãã©ãŒãã³ã°ãæ©æ¢°åŠç¿ãªã©ã®Generalizationã¯ãAIå
šäœçã«éåžžã«å€§ããªåé¡ã§ãããŸã 解決ãããŠããªãé£é¡ã§ããããã§ããã解決ããããã®æ§ã
ãªç ç©¶ãé²ãã§ããŠããŸãããã®ãžã§ããªãŒãŒã·ã§ã³ã«é¢ããããŸããŸãªç ç©¶ã«ã€ããŠã玹ä»ããŸãã
åŸæ¥ã® Generlization æ§èœã¯ Training error - Eest error ã§èšç®ããŠããŸãããåŠç¿ããŒã¿ã®æ§èœãšãã¹ãããŒã¿ã®æ§èœã䌌ãŠããã°äŒŒãŠããã»ã©è¯ããšèããããŠããããã§ãããšããããã®æ¢åã®é念ãç Žãç ç©¶çµæãåºãŠããŠãã£ãŒãã©ãŒãã³ã°ã¯åŠç¿ãããã®ã§ã¯ãªããèŠããããšã ããšè«æã®èè
ãã¡ã¯äž»åŒµããŸãã
- Understanding deep learning requires rethinking generalization
CNNã¯ç»ååé¡ã§éåžžã«å¹æçãªã¢ãã«ã§ããããšã¯èª°ã§ãç¥ã£ãŠãããšããäºå®ã§ãã Graphical Representation Learning ã§ã€ã¡ãŒãžã®ç¹åŸŽãããæããããã§ãããšããã§ããã®CNNãã€ã¡ãŒãžã®ShapeãåŠç¿ãããã®ã§ã¯ãªãããã¯ã¹ãã£ãåŠç¿ããããã®ã§ãããšããç ç©¶çµæããããŸãããã®è«æã¯åè¡ã®æ°åŒããªãããã å®éšçµæã ããæã£ãŠAI Top Conferenceã§ããICLRã«Oralã§ Acceptãããè«æã§ããéåžžã«éåžžã«åŠè¡ç䟡å€ãé«ããšãã話ã§ãã
- ImageNET-Trained CNNS Are Biased Towards Texture
人éãšãã£ãŒãã©ãŒãã³ã°ãGeneralizationã«ã¯äœããã®éãããããã©ã®ããã«ããã°ãã£ãŒãã©ãŒãã³ã°ã人éãšåæ§ã«Generalizationã§ããã®ãã«é¢ããè«æã玹ä»ããŸãã
- Generalisation in humans and deep neural networks
ç»ååé¡ã®ããã©ãŒãã³ã¹ãåäžãããäžè¬åã«å¯Ÿããããã©ãŒãã³ã¹ãåäžãããããã®æãåºæ¬çãªæ¹æ³ãData augmentationã§ããããã«ãå
çãåŠçã«ç¥èãäŒããæŠå¿µã§ããKnowledge Distillationã®æŠå¿µã«ã€ããŠã玹ä»ããŸãã
- Cutout/Cutmix, Knowledge Distillation
ãå°å¥³ãã®ã€ã¡ãŒãžãGirlã§åŠã¶ã®ã¯æ£ããã§ããããïŒãããšããWomanã§åŠç¿ããã®ãæ£ããã®ã§ããããïŒãã®ãããªæ¹æ³ã¯Label SofteningãšåŒã°ããKnowledge Distillationãå©çšããŠlabel softeningãé²è¡ããããšããããŸãããããã®æ¹æ³ãçšã㊠Regularization ããç ç©¶ã玹ä»ããŸãã
- Regularizing Class-wise Predictions via Self-knowledge Distillation
ðãã®ä» èªã¿ãããè«æã¬ãã¥ãŒ
èªã¿ãããç°¡åã§è¯ãè«æã®ããã€ããã¬ãã¥ãŒããŸãïŒ
è«æã¬ãã¥ãŒã¯åŒãç¶ãæŽæ°ãããäºå®ã§ãïŒ
象ã®åçã«å°ãã®ãã€ãºãå ãããšã人éã®ç®ã«ã¯è±¡ã®ããã«èŠããŸããããã£ãŒãã©ãŒãã³ã°ã¯KoalaãšããŠäºæž¬ãããŸãããã€ãºãéããŠãã£ãŒãã©ãŒãã³ã°ã欺ãåéãçãŸããé¢é£ç ç©¶ãå€ãç»å ŽããŸãããããã®ãã¡ã®1ã€ã®ãã€ãºã ãã远å ããã°ã»ãŒãã¹ãŠã®ã€ã¡ãŒãžããã£ãŒãã©ãŒãã³ã°ãããŸãåãããããšãã§ãããšããç ç©¶ãããã玹ä»ããŸãã
ãã£ãŒãã©ãŒãã³ã°ã®ã¢ãã«æ§é ã¯ãåçŽãªåé¡ã«ã¯åçŽãªã¢ãã«è€éãªåé¡ã«ã¯è€éãªã¢ãã«ãæããªããã°ãªããªãããšãç¥ãããŠããŸããããã«é¢ããŠããã£ãŒãã©ãŒãã³ã°ã®ã¢ãã«ãµã€ãºãšåŠç¿ãšããã¯æ°ãšãªãŒããŒãã£ããã£ã³ã°ã®é¢ä¿ã«ã€ããŠç ç©¶ããè«æããããŸãã
äžã®å³ã§ã¯ã4ã€ã®çµµã¯ãã¹ãŠãåç¬ãã§ããããšãããããŸãããããããã¹ãŠãç°ãªãã¹ã¿ã€ã«ãæã£ãŠããŸãã CNNã¯ããããã¡ã€ã³ã®åç¬ãåŠç¿ããŠããä»ã®ãã¡ã€ã³ã®åç¬ãåãããããšã¯ã§ããŸãããåŠç¿ããŒã¿ãšãã¹ãããŒã¿ãåãååžã«ãããšããä»®å®ãããããã§ããè€æ°ã®ãã¡ã€ã³ã«ã€ã㊠Generalization ã Domain Generalization ãšãããããã«é¢ããç ç©¶ã玹ä»ããŸãã
- Domain Generalization With MixStyle
ðð»ââïžè¬çŸ©ã«é¢ããäºæ³ããã質å
Q. æ°åŠçãªç¥èãããããå¿
èŠã§ããïŒ
A. MLPéšåã§å°ãå¿
èŠã§ããããªããŠãå
šäœçãªè¬çŸ©ãèãã«ã¯æ¯éããããŸããã
Q. pythonãæ±ãããšãç¥ã£ãŠããã¹ãã§ããïŒ
A. ã¯ããç§ã¯pythonãããçšåºŠç¥ã£ãŠãããšããåæã§ã¯ã©ã¹ãé²ããŸãããã ãããã£ãŒãã©ãŒãã³ã°ã®ããã®PyTorchãã¯ããã¯ã1æéã»ã©åãäžããŠããŸãã
Q. ã©ã®è«æãã¬ãã¥ãŒããŸããïŒ
A. ãã£ãŒãã©ãŒãã³ã°ã«ã€ããŠèªã¿ããããå¯äžåºŠãéåžžã«é«ãè«æãäž»ã«ã¬ãã¥ãŒããŸããæ°ããè¯ãè«æãçºèŠãããã³ã«ãè«æã¬ãã¥ãŒæ åã远å ããäºå®ã§ãã
å®ç¿ç¥èå
±æè
玹ä»
ðð»ââ Justin
- å»¶äžå€§ç£æ¥å·¥åŠç§ä¿®å£«èª²çš
- DataScience/ãã£ãŒãã©ãŒãã³ã°ç ç©¶
- https://github.com/Justin-A