{"id":515,"date":"2018-11-18T14:55:04","date_gmt":"2018-11-18T14:55:04","guid":{"rendered":"https:\/\/datagradient.com\/?p=515"},"modified":"2019-09-15T16:32:24","modified_gmt":"2019-09-15T16:32:24","slug":"deep-learning-introduction","status":"publish","type":"post","link":"https:\/\/datasciencediscovery.com\/index.php\/2018\/11\/18\/deep-learning-introduction\/","title":{"rendered":"Deep Learning &#8211; Introduction"},"content":{"rendered":"\n<p>What is this mythical beast I keep hearing about? Today, Deep Learning is a buzzword for a well deserved reason. Let&#8217;s do a deep dive into this subject and slay this beast.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"deep-learning\">Deep Learning<\/h2>\n\n\n\n<p>What is this mythical beast I keep hearing about? Today, Deep Learning is a buzzword for a well deserved reason. Some of it\u2019s applications include:<\/p>\n\n\n\n<ul><li>Automatic Machine Translation<\/li><li>Object Classification in Photographs<\/li><li>Adding Sounds to silent Movies<\/li><li>Image Caption Generation<\/li><li>Voice Search &amp; Voice-Activated Assistants<\/li><\/ul>\n\n\n\n<p>We all must have come across at least one application on our phone that uses deep learning. We also keep on hearing about Self-driving cars, reminds me of back to the future, I-robot and so on. There are so many movies where self driving cars where imagined. Now, we finally have them as a reality. To further understand what made all this possible and how it works, let\u2019s tackle this topic layer by layer:<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Series:<\/h2>\n\n\n\n<ul><li><strong>The Origins<\/strong>: Every superhero has an Origin story! We will take a look at the overall motive and the inspiration behind neural networks.\u00a0<a href=\"https:\/\/datasciencediscovery.com\/index.php\/2018\/11\/20\/deep-learning-origins\/\">Link<\/a><\/li><li><strong>Architecture<\/strong>: What are Neural Networks made of? Understanding the components of Neural Networks.\u00a0<a href=\"https:\/\/datasciencediscovery.com\/index.php\/2018\/12\/01\/deep-learning-architecture\/\">Link<\/a><\/li><li><strong>Inside the Black box<\/strong>: What is going on inside this Black box algorithm? Trying to build intuition and understanding of what is going on in the different layers of a neural network.\u00a0<a href=\"https:\/\/datasciencediscovery.com\/index.php\/2018\/11\/22\/deep-learning-black-box\/\">Link<\/a><\/li><li><strong>Mechanics of Deep Learning<\/strong>: Understand the concept of <strong>Gradient Descent and Back-propagation<\/strong>\u00a0to get some idea of how Neural Networks work. Warning some math involved! Don\u2019t worry, we will first try to explain it in an intuitive manner and then explore some math behind it.\u00a0<a href=\"https:\/\/datasciencediscovery.com\/index.php\/2018\/12\/04\/mechanics-of-deep-learning\/\">Link<\/a><\/li><li><strong>Activation<\/strong>: Understand the different types of activation functions and explore their characteristics.\u00a0<a href=\"https:\/\/datasciencediscovery.com\/index.php\/2018\/11\/21\/deep-learning-activation-function\/\">Link<\/a><\/li><li><strong>Learning Rates<\/strong>: An optimal learning rate is not only helpful in managing the computational load but also helps prevent over-fitting. It is one of the most important parameters and we will go over some techniques on how to choose the Learning Rates. I am working on this post and will update the link soon.<\/li><li><strong>Invasive Species<\/strong>: Don\u2019t get alarmed, we are going to put it into practice on a playground\u00a0<strong>kaggle<\/strong>\u00a0data set explaining the code along the way.\u00a0<a href=\"https:\/\/datasciencediscovery.com\/index.php\/2018\/11\/25\/deep-learning-invasive-species\/\">Link<\/a><\/li><li><strong>Convolution Neural Networks<\/strong>: CNNs are a bit different. In the post titled \u201cInvasive Species\u201d we use CNN to train on a data set of images and classify them as invasive\/harmless. Let\u2019s take a look at what was happening in more detail. However, I am still in the process of  putting together a post to explain this concept. <\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"references\">References<\/h2>\n\n\n\n<p>This blog series has been put together by using several references and it is necessary to point out some of them so that other readers can also take inspiration and understanding from these sources.<\/p>\n\n\n\n<ul><li><a href=\"http:\/\/www.cse.chalmers.se\/~coquand\/AUTOMATA\/mcp.pdf\" target=\"_blank\" rel=\"noopener\">Walter Pitts and Warren McCulloch<\/a><\/li><li><a href=\"https:\/\/www.google.co.in\/urlsa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=5&amp;cad=rja&amp;uact=8&amp;ved=2ahUKEwin6If6j_neAhWLxLwKHQ3MAGQQFjAEegQICRAC&amp;url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.335.3398%26rep%3Drep1%26type%3Dpdf&amp;usg=AOvVaw1xFgUDVyYrUDhnnH2LVQoD\" target=\"_blank\" rel=\"noopener\">Frank Rosenblatt\u2019s Perceptron<\/a><\/li><li><a href=\"http:\/\/www.deeplearningbook.org\/\" target=\"_blank\" rel=\"noopener\">Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning<\/a><\/li><li><a href=\"http:\/\/cs231n.github.io\/neural-networks-1\/\" target=\"_blank\" rel=\"noopener\">CS 231n<\/a><\/li><li><a href=\"https:\/\/arxiv.org\/pdf\/1511.06422.pdf\" target=\"_blank\" rel=\"noopener\">He-et-al<\/a><\/li><li><a href=\"http:\/\/www.asimovinstitute.org\/neural-network-zoo\/\" target=\"_blank\" rel=\"noopener\">Fjodor Van Veen<\/a><\/li><li><a href=\"https:\/\/arxiv.org\/abs\/1311.2901\" target=\"_blank\" rel=\"noopener\">Neural Networks Feature Visualization<\/a><\/li><li><a href=\"http:\/\/yosinski.com\/media\/papers\/Yosinski__2015__ICML_DL__Understanding_Neural_Networks_Through_Deep_Visualization__.pdf\" target=\"_blank\" rel=\"noopener\">Neural Networks Deep Visualization<\/a><\/li><li><a href=\"http:\/\/yann.lecun.com\/exdb\/publis\/pdf\/lecun-98b.pdf\" target=\"_blank\" rel=\"noopener\">Efficient Backprop<\/a><\/li><li><a href=\"https:\/\/course.fast.ai\/\" target=\"_blank\" rel=\"noopener\">fastai<\/a><\/li><li><a href=\"http:\/\/proceedings.mlr.press\/v15\/glorot11a\/glorot11a.pdf\" target=\"_blank\" rel=\"noopener\">Deep Sparse Rectifier<\/a><\/li><li><a href=\"http:\/\/neuralnetworksanddeeplearning.com\/\" target=\"_blank\" rel=\"noopener\">Nielson<\/a><\/li><li><a href=\"http:\/\/yann.lecun.com\/exdb\/publis\/pdf\/lecun-01a.pdf\" target=\"_blank\" rel=\"noopener\">Gradient Based Learning for Document Recognition<\/a><\/li><li><a href=\"https:\/\/arxiv.org\/pdf\/1609.04112.pdf\" target=\"_blank\" rel=\"noopener\">Understand CNN<\/a><\/li><li><a href=\"https:\/\/arxiv.org\/pdf\/1502.01852.pdf\" target=\"_blank\" rel=\"noopener\">Imagenet Classification<\/a><\/li><li><a href=\"https:\/\/arxiv.org\/abs\/1811.03804\" target=\"_blank\" rel=\"noopener\">Gradient Descent<\/a><\/li><li><a href=\"https:\/\/arxiv.org\/abs\/1506.01186\" target=\"_blank\" rel=\"noopener\">Cyclical Learning Rates for Training Neural Networks<\/a><\/li><\/ul>\n\n\n\n<p>I will continue to update this list of references and add more articles as we progress on our deep learning exploration.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"about-us\">About Us<\/h4>\n\n\n\n<p>Data science discovery is a step on the path of your data science journey. Please follow us on&nbsp;<a href=\"https:\/\/www.linkedin.com\/company\/data-science-discovery\/\" target=\"_blank\" rel=\"noopener\">LinkedIn<\/a>&nbsp;to stay updated.<\/p>\n\n\n\n<p>About the writers:<\/p>\n\n\n\n<ul><li><a href=\"http:\/\/linkedin.com\/in\/gadiankit\/\" target=\"_blank\" rel=\"noopener\">Ankit Gadi<\/a>: Driven by a knack and passion for data science coupled with a strong foundation in Operations Research and Statistics has helped me embark on my data science journey.<\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>What is this mythical beast I keep hearing about? Today, Deep Learning is a buzzword for a well deserved reason. Let&#8217;s do a deep dive into this subject and slay this beast. Deep Learning What is this mythical beast I keep hearing about? Today, Deep Learning is a buzzword for a well deserved reason. Some [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":516,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[72],"tags":[79,80,74,75,82,73,77,83,76,81,78],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/datasciencediscovery.com\/wp-content\/uploads\/2019\/09\/artificial-intelligence.jpg?fit=1280%2C853&ssl=1","jetpack_publicize_connections":[],"_links":{"self":[{"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/posts\/515"}],"collection":[{"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/comments?post=515"}],"version-history":[{"count":4,"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/posts\/515\/revisions"}],"predecessor-version":[{"id":561,"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/posts\/515\/revisions\/561"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/media\/516"}],"wp:attachment":[{"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/media?parent=515"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/categories?post=515"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datasciencediscovery.com\/index.php\/wp-json\/wp\/v2\/tags?post=515"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}