Summary

AWS has introduced Chronos, an innovative time series forecasting framework that leverages tokenized time series data and pretrained language models to simplify forecasting tasks and provide probabilistic forecasts with minimal model modifications.

Abstract

Chronos represents a significant advancement in time series forecasting by employing tokenization and transformer-based language models, such as T5, to treat time series data as sequential data akin to natural language. This approach allows for the application of large language models (LLMs) to the domain of time series, enabling the generation of probabilistic forecasts that are beneficial for estimating prediction intervals and quantifying uncertainty. The framework requires minimal architectural changes to existing language models and demonstrates strong performance in both traditional forecasting and zero-shot scenarios across diverse datasets. Chronos's ability to generalize without dataset-specific fine-tuning and its provision of competitive performance make it a promising tool for various industries, including finance, energy, healthcare, and climate science.

Opinions

The authors of Chronos believe that time series forecasting can significantly benefit from the application of language models, which traditionally have been used for natural language processing tasks.
The effectiveness of Chronos is highlighted through benchmarks showing its superiority over classical local models and deep learning approaches in in-domain forecasting tasks.
The authors are optimistic about the zero-shot forecasting capabilities of Chronos, suggesting that it can generalize well to new, unseen datasets without additional fine-tuning.
The provision of a minimal example and the availability of the Chronos package indicate the authors' intention to make Chronos accessible and user-friendly for practitioners who wish to implement it for their forecasting needs.
The mention of Generative AI's role in writing the article suggests that the authors acknowledge and embrace the integration of AI in the research and publication process.
The inclusion of a promotional link for SoFi Bank and a discount code for the author's digital products implies an endorsement of these services and products by the author.

The latest advancement in Time Series Forecasting from AWS: Chronos(Python Code included)

Citation: @article{ansari2024chronos,author = {Ansari, Abdul Fatir and Stella, Lorenzo and Turkmen, Caner and Zhang, Xiyuan, and Mercado, Pedro and Shen, Huibin and Shchur, Oleksandr and Rangapuram, Syama Syndar and Pineda Arango, Sebastian and Kapoor, Shubham and Zschiegner, Jasper and Maddix, Danielle C. and Mahoney, Michael W. and Torkkola, Kari and Gordon Wilson, Andrew and Bohlke-Schneider, Michael and Wang, Yuyang},title = {Chronos: Learning the Language of Time Series},journal = {arXiv preprint arXiv:2403.07815},year = {2024}}

Full Article: 2403.07815.pdf (arxiv.org)

Time series forecasting is a critical task in various domains, ranging from finance and energy to healthcare and climate science. Traditionally, statistical models like ARIMA and ETS have been the go-to methods for forecasting. However, with the advent of deep learning, there has been a paradigm shift towards leveraging neural network architectures for improved forecasting accuracy. In particular, the rise of large language models (LLMs) has sparked interest in developing foundation models for time series forecasting.

Chronos, a novel framework for pretrained probabilistic time series models. Developed by researchers, Chronos aims to simplify time series forecasting tasks by leveraging existing transformer-based language models, such as those in the T5 family, with minimal modifications. The core idea behind Chronos is to tokenize time series data and train language models on these tokenized sequences using cross-entropy loss.

Chronos Framework Overview

Chronos operates on the premise that despite the differences between natural language and time series data, both are fundamentally sequential in nature. By tokenizing time series values through scaling and quantization, Chronos transforms continuous-valued time series into discrete tokens, enabling the application of language models without significant architectural changes.

The tokenization process involves two main steps:

Scaling: Time series values are normalized to a suitable range to facilitate quantization. Chronos employs mean scaling, a technique commonly used in deep learning models, to normalize individual time series entries.
Quantization: Scaled time series values are mapped to a fixed number of bins through quantization. This step involves selecting bin centers and edges to discretize the continuous domain into a finite set of tokens.

Training and Inference

Once time series data is tokenized, Chronos employs pretrained language models, such as those in the T5 family, for training and inference. During training, the model learns to predict the next token in the sequence using the categorical distribution over the token vocabulary. The objective function is the cross-entropy loss between the predicted distribution and the ground truth tokens.

For forecasting, Chronos generates probabilistic forecasts by sampling from the predicted distribution autoregressively. Multiple sample paths are obtained, which are then mapped back to real values and unscaled to obtain the final forecasts.

Key Advantages of Chronos

Minimal Modifications: Chronos requires minimal modifications to existing language model architectures, making it easy to integrate with popular transformer-based models.
Probabilistic Forecasts: By design, Chronos generates probabilistic forecasts, allowing for the estimation of prediction intervals and uncertainty quantification.
Zero-shot Forecasting: Chronos demonstrates impressive zero-shot forecasting performance, even on unseen datasets, without the need for dataset-specific fine-tuning.

Evaluation and Benchmarking

The effectiveness of Chronos is demonstrated through a comprehensive benchmark comprising 42 datasets from diverse domains. Results show that Chronos outperforms both classical local models and task-specific deep learning approaches on in-domain forecasting tasks. Furthermore, Chronos achieves competitive zero-shot performance, highlighting its ability to generalize to new forecasting tasks without additional fine-tuning.

Chronos presents a simple yet effective framework for pretrained probabilistic time series models. By leveraging existing language model architectures and tokenizing time series data, Chronos streamlines the forecasting pipeline and offers competitive performance across various forecasting tasks. With its minimal modifications and robust performance, Chronos is poised to become a valuable tool for simplifying time series forecasting in diverse domains.

Start harnessing the power of Chronos for your time series forecasting needs today!

Code

To utilize Chronos models for time series forecasting, users can install the Chronos package and easily perform inference using the provided APIs. Below is a minimal example demonstrating how to forecast time series data using Chronos:

# Import necessary libraries
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import torch
from chronos import ChronosPipeline

# Load a pretrained Chronos model
pipeline = ChronosPipeline.from_pretrained(
  "amazon/chronos-t5-small",
  device_map="cuda",
  torch_dtype=torch.bfloat16,
)
# Load time series data
df = pd.read_csv("https://raw.githubusercontent.com/AileenNielsen/TimeSeriesAnalysisWithPython/master/data/AirPassengers.csv")
# Prepare input context for forecasting
context = torch.tensor(df["#Passengers"])
prediction_length = 12
# Perform forecasting
forecast = pipeline.predict(
  context,
  prediction_length,
  num_samples=20,
  temperature=1.0,
  top_k=50,
  top_p=1.0,
)
# Visualize the forecast
forecast_index = range(len(df), len(df) + prediction_length)
low, median, high = np.quantile(forecast[0].numpy(), [0.1, 0.5, 0.9], axis=0)
plt.figure(figsize=(8, 4))
plt.plot(df["#Passengers"], color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()

This example demonstrates how to load a pretrained Chronos model, prepare input data, perform forecasting, and visualize the forecasted results. Users can easily customize the input parameters to suit their specific forecasting tasks.

Full Code Example: https://colab.research.google.com/drive/1m2KucF7Kx0Pom0azBtt9hR1R2LqK2p_S?usp=sharing

Parts of this article were written using Generative AI
Subscribe/leave a comment if you want to stay up-to-date with the latest AI trends.
Earn $25 and 4.60% APY for FREE through my referral at SoFi Bank Here

Plug: Checkout all my digital products on Gumroad here. Please purchase ONLY if you have the means to do so. Use code: MEDSUB to get a 10% discount!