Machine Learning is about creating algorithms that can learn from and make predictions or decisions based on data. The algorithm and data co-evolve with one another. The big data era (2000 to present) brought breakthroughs in deep learning (2010 to present), such as neural network architecture and training techniques. Enabled by advances in GPU technology and availability of massive datasets, deep learning dominates in areas like computer vision and natural language processing. This evolution has not made traditional ML obsolete. Traditional ML remains relevant and often preferable for structured data problems.  In this blog post, we’ll compare the characteristics, advantages, and use cases of both traditional machine learning and deep learning and discuss applications in drug development and healthcare.

Traditional Machine Learning:

Traditional machine learning algorithms learn patterns from data, making predictions or decisions based on statistical models. The key characteristic of traditional ML is its reliance on feature engineering, where domain experts identify and extract relevant features from raw data.

Common algorithms in traditional machine learning include Decision Trees, Random Forests, Support Vector Machines (SVM), and the popular XGBoost. The strengths of traditional machine learning lie in its interpretability, efficiency on smaller datasets, and lower computational requirements compared to more complex models. Common use cases include virtual screening and predicting drug toxicities in early-stage drug discovery, as well as risk stratification, prognosis prediction and diagnosis support in clinical settings.

However, traditional ML has limitations when dealing with complex, high-dimensional data such as images or natural language. This is where deep learning often steps in to fill the gap, offering powerful techniques for handling complex, unstructured data.

Deep Learning:

Deep Learning is a subset of machine learning inspired by the structure and function of the human brain. It uses artificial neural networks with multiple layers (hence “deep”) to progressively extract higher-level features from raw input. Unlike traditional ML, deep learning algorithms can automatically learn hierarchical representations of data, eliminating the need for manual feature engineering in many cases. This ability to learn complex patterns directly from raw data has led to breakthrough performances in various domains, particularly those involving unstructured data like images, audio, and text.

Popular deep learning architectures include Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformers. CNNs are particularly effective for image-related tasks, using convolutional layers to capture spatial hierarchies in data. Transformers, introduced more recently, have revolutionized NLP tasks by efficiently handling long-range dependencies in sequences and have now been applied in computer vision tasks as well. Common use cases include medical imaging analysis such as for X-rays, CT scans, MRIs and histopathology slides, de novo drug design such as protein structure-based drug design with Alpha-Fold, medical text analysis powered by natural language processing and smart wearable device for predictive healthcare.

Summary on Key Differences

1. Data Requirements
   • Traditional ML: Often works well with smaller datasets
   • Deep Learning: Typically requires large amounts of data
2. Feature Engineering
   • Traditional ML: Requires manual feature extraction and selection
   • Deep Learning: Automatic feature learning
3. Computational Resources
   • Traditional ML: Generally less computationally intensive
   • Deep Learning: Often requires significant computational power (GPUs)
4. Model Interpretability
   • Traditional ML: Many algorithms produce interpretable models
   • Deep Learning: Often considered a “black box”
5. Performance on Complex Tasks
   • Traditional ML: May struggle with very complex patterns
   • Deep Learning: Excels at learning complex patterns in data