Forecast Anything with Transformers

Chronos and PatchTST Models

A new direction from Trelis, covering the application of transformers to time series models, for forecasting.

Previously, transformers did not beat simpler models for forecasting things such as weather, power or prices. Models like PatchTST and Chronos from Amazon have changed this.

I explain how these models work, how to forecast, how to train from scratch, and how to fine-tune.

Lots more in the full video on the Trelis Research channel on YouTube.

And get access to the scripts here:

Cheers, Ronan

P.S. 🛠️ (NEW) Trelis Seminars - learn more here.

---

Trelis Links:

🤝 Are you a talented developer? Work for Trelis

💡 Need Technical or Market Assistance? Book a Consult Here

💸 Starting a New Project/Venture? Apply for a Trelis Grant

---

Using Transformer Models for Numerical Forecasting

Transformer models, known primarily for powering language models like ChatGPT, can now effectively forecast numerical sequences. While simple linear models were historically preferred for forecasting tasks, recent developments in transformer architectures have produced models that excel at predicting numerical patterns.

Key Models

Two notable transformer-based forecasting models have emerged:

1. PatchTST

2. Chronos

These models can process various types of numerical data, including:

1. Traffic patterns

2. Weather measurements

3. Financial time series

New Capabilities

Modern transformer models can now:

1. Process numerical sequences instead of text

2. Generate continuous number predictions

3. Handle multiple types of time series data

4. Adapt to custom applications through fine-tuning

Data Preparation

To use these models effectively, numerical data must be:

1. Formatted as sequential inputs

2. Preprocessed for transformer compatibility

3. Structured for time series prediction

Applications

These forecasting models can be applied to:

1. Traffic flow prediction

2. Weather forecasting

3. Financial market analysis

4. Any sequential numerical data

Model Selection

Choosing between PatchTST and Chronos depends on:

• Inference speed required: PatchTST is faster

• One-short performance (no tuning): Chronos is stronger