A lot has changed since my February article on starting out in machine learning. The impacts of the coronavirus are being felt across the globe.

Artificial Intelligence and Machine Learning are helping efforts to detect, track and find a treatment for the virus. As the importance of this tool increases, a machine learning and deep learning education should be accessible to everyone, no matter what their background.

My February article covered the machine learning foundational work; learning Python, exploratory data analytics, machine learning practice and machine learning theory. It was time to move onto the next phase of the plan, having reached a good level of knowledge. Practice my machine learning skills and move into deep learning.

This article covers my motivations, explanations of deep learning, recommended courses for theory and practice, cloud platforms, AI governance and AI ethics.

Why Deep Learning?

· I have always had a strong desire for continuous learning, and I saw deep learning as a new challenge and opportunity for growth.

· I see deep learning as a tool that can augment human capabilities, translating theory into practice to solve real problems with human-centred values.

At the beginning of my deep learning education, it was quite daunting to know where to begin. It’s a vast topic, and it’s easy to get bogged down and frustrated with the many topics and mathematics. The key was perseverance.

What Is Deep Learning ?

The definition of deep learning has led to many debates. Still, for now, “deep learning is part of a broader family of machine learning methods based on artificial neural networks”. (Wikipedia)

The explosion in deep learning over recent years has been made possible by three things, the vast amount of data available, the increase in computational power and algorithm innovation.

Examples of deep learning:

Self Driving Cars, Object detection, decision making.

Object Classification, visual search, face recognition.

Natural language processing, spam filters, Siri, Alexa, or Google Assistant.

Health Care. MRI scans, CT scans, records analysis.

What Does A Neural Network Look Like ?

Google has developed this playground where you can “tinker” with neural network in your browser. Check out the playground TensorFlow site.

My Deep Learning Approach

My approach to deep learning was threefold, understand enough of the theory to interpret the algorithms, learn the coding and then practice them. This approach led me to the deep learning speciality (The theory), Fast.Ai (The coding) and Kaggle (The practice).

Deeplearning.ai Speciality on Coursera

Andrew Ng’s original machine learning course was clear, well-structured and made the mathematics understandable. It gave me the confidence to tackle deep learning with Andrew Ng’s latest speciality.

The deep learning speciality covers the building blocks of Neural networks, Convolutional neural networks (e.g. Image recognition, radiology imagery), Sequence models (e.g. speech recognition, chatbots, language processing), advice on how to improve your models (hyper-parameter tuning), structuring your deep learning projects and machine learning strategies.

The programming exercises are in Python and help clarify the content of the lectures. You will build a neural network, image classification, facial recognition, natural language processing and trigger word detection (e.g. hey Siri or Alexa).

Building a neural network with mathematics as opposed to programming libraries provides a level of understanding that some of the more practical courses don’t cover.

While this level of depth may not be required, and it is useful to learn what happens under the hood.

Over 16 weeks the five courses are layered out, but you can take them at your own pace. This specialisation teaches the most essential and foundational principles of deep learning. If you’re a strictly hands-on learner, then this course is not for you. The blend of theory in this course is complementary to the practical Fast.ai course.

Fast.ai Practical Deep Learning For Coders

Fast.ai Practical deep learning for Coders v3 is a free, hands-on course that aims to teach people to code in Python, state of the art, deep learning techniques.

Fast.ai aims to make deep learning simpler to use by getting more people from all backgrounds involved through free courses, research and a community. Quoting from the Fast.Ai website.

“What I hope is that lots of people will realise that state-of-the-art results of deep learning are something they can achieve even if they’re not a Stanford University deep learning PhD.”

Fast.ai covers data cleaning, image classification, multi-label classification; Segmentation NLP; Tabular data; Collaborative filtering; Embeddings, Regularisation, data ethics and Residual networks.

There are no formal programming exercises or quizzes. Still, you are encouraged to make your projects from what you have learned in the lectures. Using the latest state of the art technology and the inbuilt Fast.ai libraries, you’re taught to code your models.

The fast.ai libraries are built on top of open-source PyTorch library. After two lessons I had written my image classifier on AWS Sagemaker.

There is a strong community to help with your challenges and to critique your work. If you are a hands-on learner, this course is for you, even when techniques such as backpropagation are covered they are presented practically as opposed to the underlying mathematics.

Kaggle

Partway through the deep learning specialisation, I took a break and entered some machine learning competitions on Kaggle. Kaggle is a data science community with powerful tools and resources to help you achieve your data science goals.

Kaggle is a great way to test your skills. There is a leaderboard for each competition where individuals or teams compete against each other. Also, there are examples of winning code that helps develop your skills in this crucible of data science.

There are four beginners competitions. Data classification: predicting house prices. Image recognition using hand-written digits, and natural language processing analysing tweets for the sentiment. Jeremy Howard from Fast.ai, a Kaggle master himself, suggests that “if you’re any good you should get into the top 10% of a competition”.

My first score put me in the top 73%, a rude awakening after a long period of studying! After repeated attempts late into the night, I reached a level of scores that I was happy with, for now!

• Introduction to machine learning classifications. Top 5% 749/16523
• Advanced Regression Techniques predict sales prices Top 14% 605/4550
• Computer vision hand-written digit recogniser Top 16% 335/2230
• Natural Language processing predicting tweet outcomes. Top 44% 1472/3423

What can you learn from Kaggle ?

· The importance of data preparation cannot be understated. Learning how to clean the data, feature engineering, transformations, randomisation, train/test split have a massive impact on your outputs.

· How do you choose and train the right model? Different algorithms are for different tasks.

· The skills to evaluate and tune the model. Advice on selecting the right metrics and the art form of hyper-parameter tuning

· Don’t fall down the rabbit hole trying for the 10%

Summary

Deep learning.ai and Fast.ai explain everything so clearly that you feel confident enough to take up the code and algorithms and start using them. The best way to learn, practice and reinforce my knowledge came from alternating between the two courses and data science competitions in Kaggle.

Cloud Platforms

At some point, your home computer will not be fast enough to run deep learning models. Here are two options to mitigate that.

· First, you can reduce the complexity of your models, reduce the training data sets or fine-tune parameters and hyper-parameters.

· Second, get a Graphics processor. The good news is you don’t need to purchase a GPU, enterprise-grade GPUs are available to everyone in the cloud but be careful about the costs!

Google CoLab: This is a free platform that gives access to Google’s compute and GPU and Python libraries Keras, TensorFlow, PyTorch, and OpenCV.

Kaggle: Has a free coding environment, and you get 30hours a month GPU time. For beginners, this is enough, in my opinion.

Azure Studio: This eliminates the complexity of data engineering and Python coding, giving amateur data analysts and data scientists a more natural way to build models. Azure will charge based on your capacity so to keep costs down, make sure you shut down the instances.

AWS Sagemaker — Is an all in one AI platform. You can use AWS models, build your own or use AWS will host your models. Like Azure, you also get enterprise-grade graphics processing power, and there is a cost depending on the type of processing unit and memory. To keep costs down, make sure you shut down your instance when you finish. I created a simple AWS Lambda job to shutdown my notebook at specific intervals.

Kaggle and AWS Sagemaker were the best fit for my education workflows and experience. I developed the code on a laptop, ran with a small training/test set, fixed bugs, tuned and then uploaded to either platform for larger training/test sets and further tuning.

AI Governance and AI Ethics

The courses here will give you a deep technical understanding of how machine learning and deep learning work. As you start working on machine learning and deep learning in the real world, strong governance and ethical oversight cannot be added on later.

There are a burgeoning number of tools and information that help organisations understand and implement robust governance and ethical oversight. All of these sites have links to additional content.

World economic forum AI Toolkit

The toolkit has in-depth information on governance, ethics, risk, cybersecurity and more. The toolkit goes into detail on board-level responsibilities for the adoption of AI.

Berkman Klein Center at Harvard

The Principled Artificial Intelligence has eight key themes. Privacy, accountability, safety and security, transparency and explainability, fairness and non-discrimination, human control, professional responsibility and promotion of human values.

The Deep Mind Ethics and society

The team addresses the complex ethical and social exploring the real-world impacts of AI.

“With the right focus on ethical standards and safety, we have better chances of finding AI’s potential benefits. By researching the ethical and social questions involving AI, we ensure these topics remain at the heart of everything we do.” (deep mind).

Whats Next ?

In machine learning, there is a concept called overfitting, and this is when your model fits too well to the training data and cannot generalise on test data. In other words, it’s time for me to put the learning to one side for a while. Its time to focus on the exploration of real-world applications of machine learning and deep learning to continue to build my skills.

Deep Learning Videos.

Lex Fridman

A researcher in human-centred AI, autonomous vehicles, and deep learning from MIT. Excellent interviews with machine learning experts.

Deeplizard

This series explains concepts that are fundamental to deep learning and artificial neural networks for beginners. Useful as an alternative viewpoint if you get stuck.

Convolutional Neural Networks by Andrej Karpathy

Recorded lectures on Convolutional Neural Networks from Stanford, taught by Andrej Karpathy the director of artificial intelligence and Autopilot Vision at Tesla