{"id":15185,"date":"2018-01-15T17:36:14","date_gmt":"2018-01-15T16:36:14","guid":{"rendered":"https:\/\/blog.trifork.com\/?p=15185"},"modified":"2018-01-15T17:36:14","modified_gmt":"2018-01-15T16:36:14","slug":"deep-learning-for-natural-language-processing-part-ii","status":"publish","type":"post","link":"https:\/\/trifork.nl\/blog\/deep-learning-for-natural-language-processing-part-ii\/","title":{"rendered":"Deep Learning for Natural Language Processing \u2013 Part II"},"content":{"rendered":"

Author \u2013 Wilder Rodrigues<\/em><\/p>\n

Wilder continues his series about NLP.  This time he would like to bring you to the Deep Learning realm, exploring Deep Neural Networks for sentiment analysis.<\/p>\n

If you are already familiar with those types of network and know why certain choices are made, you can skip the first section and go straight to the next one.<\/p>\n

I promise the decisions I made in terms of <\/em>train \/ validation \/ test split won\u2019t disappoint you. As a matter of fact, training the same models with different sets got me a better result than those achieved by Dr. Jon Krohn, from untapt, in his Live Lessons.<\/em><\/p>\n

From what I have seen in the last 2 years, I think we all have already been through a lot of explanations about shallow, intermediate and deep neural networks. So, to save us some time, I will avoid revisiting them here. We will dive straight into all the arsenal we will be using throughout this story. However, we won\u2019t just follow a list of things, but instead, we will understand why those things are being used.<\/em>
\n
\nWhat do we have at our disposal?<\/strong><\/p>\n

There is a famous image that does not appear only on several blog posts about Machine Learning, but also on Meetup<\/a> meetings, conferences and company-closed presentations. Although I wouldn\u2019t add it here just for the sake of it, I think it\u2019s interesting to get the audience to see some ofthe network architectures we currently have.<\/p>\n

The people behind the @asimovinstitute came up with a quite interesting compilation. See for yourself:<\/p>\n

\"\"<\/p>\n

The Neural Network Zoo, by the Asimov Institute \u2014 https:\/\/www.asimovinstitute.org\/<\/a><\/p>\n

 <\/p>\n

There are many more curiosities and things to learn about the Neural Network Zoo. If that\u2019s something that would make you more interested in Neural Networks and their intrinsic details, you can find it all in their own blog post here: https:\/\/www.asimovinstitute.org\/neural-network-zoo\/<\/a>.<\/p>\n

For this story, we are going to focus on a Feedforward Neural Network. If you paid attention to the image above, it won’t be difficult to spot it. Otherwise, please refer to the image below:<\/p>\n

\"\"<\/p>\n

That is a pretty simple network! But no worries, we will be stretching it a bit, making it deeper. However, before we do that, let\u2019s try to get a couple of things clear:<\/p>\n