> Markdown version of [/videos/14-the-pitfalls-of-deep-learning-when-neural-networks-are-not-the-solution?t=807](https://www.wearedevelopers.com/videos/14-the-pitfalls-of-deep-learning-when-neural-networks-are-not-the-solution?t=807). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# The pitfalls of Deep Learning - When Neural Networks are not the solution

Deep learning isn't a silver bullet for enterprise data. Discover why tree-based models often outperform neural networks and when to avoid their massive implementation costs.

- **Speakers:** Adrian Spataru, Bohdan Andrusyak
- **Event:** WeAreDevelopers LIVE
- **Published:** June 17, 2020
- **Duration:** 32:54
- **URL:** https://www.wearedevelopers.com/videos/14-the-pitfalls-of-deep-learning-when-neural-networks-are-not-the-solution

## Summary

The presentation explores the boundaries of artificial intelligence by contrasting classical machine learning with modern deep learning frameworks. While deep learning has driven monumental breakthroughs in self-driving cars, language translation, and generative media through powerful representation learning, it is rarely a silver bullet. High-profile failures—such as disastrous facial recognition mismatches and contextual translation errors—demonstrate that neural networks are fundamentally unsuited for certain problems without proper data constraints.

Several core pitfalls limit the practical application of deep learning in typical business environments. First, deep learning algorithms demand massive volumes of high-quality data and cannot fix "garbage in, garbage out" pipeline scenarios. Second, for standard tabular data found in most enterprise spreadsheets, tree-based models like XGBoost and LightGBM consistently outperform neural networks, especially when paired with manual feature engineering. The inherent "black box" nature of automatic feature extraction also sacrifices model explainability, making simple algorithms like decision trees preferable when transparency is required. Furthermore, deep learning introduces severe engineering complexity and prohibitive resource costs, with training massive natural language processing models running into millions of dollars.

To mitigate these limitations, the industry is advancing techniques like transfer learning, which reduces data dependency by adapting pre-trained models to new domains, alongside interpretability frameworks like SHAP, LIME, and DeepLIFT. Emerging research into tabular deep learning using attention mechanisms also shows long-term promise. Ultimately, organizations must align their AI investments with actual business value. Rather than adopting deep learning by default, teams dealing with small, tabular, or low-quality datasets should establish a baseline with classical machine learning to determine if the marginal accuracy gains of neural networks justify their immense implementation costs.

**Keywords:** deep learning limitations, classical machine learning, manual feature extraction, representation learning, tabular data modeling, tree-based algorithms, xgboost and lightgbm, model explainability, data quality constraints, neural network complexity, model training costs, transfer learning applications, tabular deep learning, shap and lime, business value of ai, natural language processing

## Chapters

1. **Contrasting classical machine learning with deep learning approaches** (00:17) — An overview of the differences between manual feature extraction in classical algorithms and automatic extraction in neural networks.
1. **Successful applications and representation learning in neural networks** (03:05) — How deep learning drives breakthroughs in autonomous driving, language translation, and music generation through powerful feature representations.
1. **Real-world failures and unrealistic expectations of deep learning** (06:52) — How out-of-context translations and flawed facial recognition demonstrate the severe consequences of algorithmic limitations.
1. **Evaluating data quantity constraints in complex neural networks** (09:38) — Why deep learning struggles to identify accurate patterns when provided with limited or scarce data sets.
1. **The critical requirement for high quality training data** (13:27) — How garbage or incomplete data negatively impacts model accuracy regardless of algorithmic sophistication.
1. **Overcoming deep learning challenges with tabular business data** (15:25) — Why tree-based boosting methods consistently outperform neural networks on limited historical business spreadsheets.
1. **Comparing model explainability between algorithms and neural networks** (18:54) — The importance of understanding algorithm decisions and discarding irrelevant features by leveraging simple decision trees.
1. **Production complexity and engineering costs of oversized models** (22:45) — How excessive model constraints and large file sizes prevent improved architectures from being deployed into production.
1. **Analyzing computational resources and cloud training costs** (24:19) — The astronomical financial requirements needed to train massive natural language processing models from scratch.
1. **Calculating the business value and return on investment** (27:17) — Determining if marginal accuracy improvements from deep learning justify the additional technical complexity and overhead.
1. **Leveraging transfer learning to reduce data dependency issues** (28:43) — Reusing pre-trained models on new datasets accelerates development while avoiding high data and processing requirements.
1. **Exploring interpretability and future advancements in tabular processing** (30:42) — Discovering recent research around transparent tabular transformers and assessing readiness before integrating neural architectures.

## Related Moments

- [Root causes of underlying AI initiative failures](https://www.wearedevelopers.com/videos/100328-the-limits-of-llms-in-real-world-applications) (from "The Limits of LLMs in Real-World Applications")
- [Understanding core ingredients for successful AI adoption](https://www.wearedevelopers.com/videos/1473-ai-or-ko-is-hr-ever-going-to-use-intelligent-technology) (from "AI or KO: Is HR ever going to use intelligent technology?")
- [Audience questions on practical machine learning operational strategies](https://www.wearedevelopers.com/videos/262-is-my-ai-alive-but-brain-dead-how-monitoring-can-tell-you-if-your-machine-learning-stack-is-still-performing) (from "Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing")
- [Business motivations and adoption challenges for generative AI](https://www.wearedevelopers.com/videos/1250-from-traction-to-production-maturing-your-llmops-step-by-step) (from "From Traction to Production: Maturing your LLMOps step by step")
- [Uncovering the hidden technical debt in machine learning](https://www.wearedevelopers.com/videos/392-mlops-what-s-the-deal-behind-it) (from "MLOps - What’s the deal behind it?")
- [The gap between data science maturity and business value](https://www.wearedevelopers.com/videos/185-effective-machine-learning-managing-complexity-with-mlops) (from "Effective Machine Learning - Managing Complexity with MLOps")

## Related Articles

- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)
- [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere)
- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)

## Related Jobs

- [Founding Data Scientist](https://www.wearedevelopers.com/jobs/ext/2731830-founding-data-scientist) at **Almedia**
- [Partner Sales Director - AI Alliances - Model Providers](https://www.wearedevelopers.com/jobs/48429-partner-sales-director-ai-alliances-model-providers) at **Dynatrace**
- [LLM Dataset Engineer](https://www.wearedevelopers.com/jobs/48419-llm-dataset-engineer) at **Sciforium**
- [AI Software Engineer (Germany)](https://www.wearedevelopers.com/jobs/48317-ai-software-engineer-germany) at **Sunhat**
- [TikTok LIVE - AI Data Operation Specialist](https://www.wearedevelopers.com/jobs/ext/3184453-tiktok-live-ai-data-operation-specialist) at **TikTok**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/2141919-machine-learning-engineer) at **Twilio**