techleft.blogg.se

1. #Revisiting deep learning models for tabular data update#
2. #Revisiting deep learning models for tabular data series#

#Revisiting deep learning models for tabular data series#

Methods developed for time series data.Classic deep learning methods (plain multilayer perceptrons, etc.).Please note that I am excluding the following topics from this list since they are slightly out of scope: Please let me know if you know of some additional papers I may have missed! I will try to keep this list up to date as new publications arrive. There is a reason why I cover conventional machine learning before deep learning in my book.ĭeep Learning For Tabular Data In Reverse Chronological Order #īelow is a (growing) list of relevant papers along with links and short summaries. I want to emphasize that no matter how interesting or promising deep tabular methods look, I still recommend using a conventional machine learning method as a baseline. However, to me, multilayer perceptrons are not really “deep learning,” so I am not listing those.)

(By the way, many earlier papers use multilayer perceptrons on tabular datasets and refer to it as “deep learning” – several computational biology papers that train multilayer perceptrons on molecular fingerprint data come to mind.

#Revisiting deep learning models for tabular data update#

I am happy to curate and update this list for future reference, so please let me know if there is something I missed. It is possible that I skipped or forgot a few. So, with this short post, I aim to briefly summarize the major papers on deep tabular learning I am currently aware of. Often, people ask for additional methods or counterexamples. Occasionally, I share research papers proposing new deep learning approaches for tabular data on social media, which is typically an excellent discussion starter. Nonetheless, many researchers recently tried developing special-purpose deep learning methods for tabular datasets. Most tabular datasets already represent (typically manually) extracted features, so there shouldn’t be a significant advantage using deep learning on these.

Deep learning is sometimes referred to as “representation learning” because its strength is the ability to learn the feature extraction pipeline. In my lectures, I emphasize that deep learning is really good for unstructured data (essentially, that’s the opposite of tabular data). We also compare the best DL models with Gradient Boosted Decision Trees and conclude that there is still no universally superior solution. Both models are compared to many existing architectures on a diverse set of tasks under the same training and tuning protocols. The second model is our simple adaptation of the Transformer architecture for tabular data, which outperforms other solutions on most tasks. The first one is a ResNet-like architecture which turns out to be a strong baseline that is often missing in prior works. In this work, we perform an overview of the main families of DL architectures for tabular data and raise the bar of baselines in tabular DL by identifying two simple and powerful deep architectures. Additionally, the field still lacks effective baselines, that is, the easy-to-use models that provide competitive performance across different problems. As a result, it is unclear for both researchers and practitioners what models perform best. However, the proposed models are usually not properly compared to each other and existing works often use different benchmarks and experiment protocols.

Abstract: The existing literature on deep learning for tabular data proposes a wide range of novel architectures and reports competitive results on various datasets.