Shivaji Dutta
Reputation: 41
Google Transformer Architecture (Attention is all you need), is it good for Time Series problems?
I have reading the paper on Transformer Architecture. GRU/LSTM have done well with Time Series problems. I was wondering if anyone has used Transformers for Time Series problems.
Upvotes: 1
Views: 237
Answers (1)
Allohvk
Reputation: 1374
Yes, In the past 2 years several papers have come out using Transformer architecture for time-series problems across business domains. These have been found to work particularly well compared to existing models. For e.g. see one such use case in medical domain - https://arxiv.org/pdf/2001.08317.pdf. This is a more generic writeup: https://towardsdatascience.com/attention-for-time-series-classification-and-forecasting-261723e0006d
Upvotes: 1
Related Questions
- BERT for time series classification
- Computational Complexity of Self-Attention in the Transformer Model
- Sinusoidal embedding - Attention is all you need
- Using EluetherAPI GPT models for NLP tasks
- Using the embedding layer as the input for an encoder
- Positional Encoding for time series based data for Transformer DNN models
- Verifying the implementation of Multihead Attention in Transformer
- Why use multi-headed attention in Transformers?
- Sequence to Sequence - for time series prediction
- How do attention network works?