avatarAbish Pius

Summarize

The latest advancement in Time Series Forecasting from AWS: Chronos(Python Code included)

Chronos

Citation: @article{ansari2024chronos,author = {Ansari, Abdul Fatir and Stella, Lorenzo and Turkmen, Caner and Zhang, Xiyuan, and Mercado, Pedro and Shen, Huibin and Shchur, Oleksandr and Rangapuram, Syama Syndar and Pineda Arango, Sebastian and Kapoor, Shubham and Zschiegner, Jasper and Maddix, Danielle C. and Mahoney, Michael W. and Torkkola, Kari and Gordon Wilson, Andrew and Bohlke-Schneider, Michael and Wang, Yuyang},title = {Chronos: Learning the Language of Time Series},journal = {arXiv preprint arXiv:2403.07815},year = {2024}}

Full Article: 2403.07815.pdf (arxiv.org)

Time series forecasting is a critical task in various domains, ranging from finance and energy to healthcare and climate science. Traditionally, statistical models like ARIMA and ETS have been the go-to methods for forecasting. However, with the advent of deep learning, there has been a paradigm shift towards leveraging neural network architectures for improved forecasting accuracy. In particular, the rise of large language models (LLMs) has sparked interest in developing foundation models for time series forecasting.

Chronos, a novel framework for pretrained probabilistic time series models. Developed by researchers, Chronos aims to simplify time series forecasting tasks by leveraging existing transformer-based language models, such as those in the T5 family, with minimal modifications. The core idea behind Chronos is to tokenize time series data and train language models on these tokenized sequences using cross-entropy loss.

Chronos Framework Overview

Chronos operates on the premise that despite the differences between natural language and time series data, both are fundamentally sequential in nature. By tokenizing time series values through scaling and quantization, Chronos transforms continuous-valued time series into discrete tokens, enabling the application of language models without significant architectural changes.

The tokenization process involves two main steps:

  1. Scaling: Time series values are normalized to a suitable range to facilitate quantization. Chronos employs mean scaling, a technique commonly used in deep learning models, to normalize individual time series entries.
  2. Quantization: Scaled time series values are mapped to a fixed number of bins through quantization. This step involves selecting bin centers and edges to discretize the continuous domain into a finite set of tokens.

Training and Inference

Once time series data is tokenized, Chronos employs pretrained language models, such as those in the T5 family, for training and inference. During training, the model learns to predict the next token in the sequence using the categorical distribution over the token vocabulary. The objective function is the cross-entropy loss between the predicted distribution and the ground truth tokens.

For forecasting, Chronos generates probabilistic forecasts by sampling from the predicted distribution autoregressively. Multiple sample paths are obtained, which are then mapped back to real values and unscaled to obtain the final forecasts.

Key Advantages of Chronos

  1. Minimal Modifications: Chronos requires minimal modifications to existing language model architectures, making it easy to integrate with popular transformer-based models.
  2. Probabilistic Forecasts: By design, Chronos generates probabilistic forecasts, allowing for the estimation of prediction intervals and uncertainty quantification.
  3. Zero-shot Forecasting: Chronos demonstrates impressive zero-shot forecasting performance, even on unseen datasets, without the need for dataset-specific fine-tuning.

Evaluation and Benchmarking

The effectiveness of Chronos is demonstrated through a comprehensive benchmark comprising 42 datasets from diverse domains. Results show that Chronos outperforms both classical local models and task-specific deep learning approaches on in-domain forecasting tasks. Furthermore, Chronos achieves competitive zero-shot performance, highlighting its ability to generalize to new forecasting tasks without additional fine-tuning.

Chronos presents a simple yet effective framework for pretrained probabilistic time series models. By leveraging existing language model architectures and tokenizing time series data, Chronos streamlines the forecasting pipeline and offers competitive performance across various forecasting tasks. With its minimal modifications and robust performance, Chronos is poised to become a valuable tool for simplifying time series forecasting in diverse domains.

Start harnessing the power of Chronos for your time series forecasting needs today!

Code

To utilize Chronos models for time series forecasting, users can install the Chronos package and easily perform inference using the provided APIs. Below is a minimal example demonstrating how to forecast time series data using Chronos:

# Import necessary libraries
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import torch
from chronos import ChronosPipeline

# Load a pretrained Chronos model
pipeline = ChronosPipeline.from_pretrained(
  "amazon/chronos-t5-small",
  device_map="cuda",
  torch_dtype=torch.bfloat16,
)
# Load time series data
df = pd.read_csv("https://raw.githubusercontent.com/AileenNielsen/TimeSeriesAnalysisWithPython/master/data/AirPassengers.csv")
# Prepare input context for forecasting
context = torch.tensor(df["#Passengers"])
prediction_length = 12
# Perform forecasting
forecast = pipeline.predict(
  context,
  prediction_length,
  num_samples=20,
  temperature=1.0,
  top_k=50,
  top_p=1.0,
)
# Visualize the forecast
forecast_index = range(len(df), len(df) + prediction_length)
low, median, high = np.quantile(forecast[0].numpy(), [0.1, 0.5, 0.9], axis=0)
plt.figure(figsize=(8, 4))
plt.plot(df["#Passengers"], color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()

This example demonstrates how to load a pretrained Chronos model, prepare input data, perform forecasting, and visualize the forecasted results. Users can easily customize the input parameters to suit their specific forecasting tasks.

Full Code Example: https://colab.research.google.com/drive/1m2KucF7Kx0Pom0azBtt9hR1R2LqK2p_S?usp=sharing

  • Parts of this article were written using Generative AI
  • Subscribe/leave a comment if you want to stay up-to-date with the latest AI trends.
  • Earn $25 and 4.60% APY for FREE through my referral at SoFi Bank Here

Plug: Checkout all my digital products on Gumroad here. Please purchase ONLY if you have the means to do so. Use code: MEDSUB to get a 10% discount!

If you enjoyed this article, consider trying out the AI service I recommend. It provides the same performance and functions to ChatGPT Plus(GPT-4) but more cost-effective, at just $6/month (Special offer for $1/month). Click here to try ZAI.chat.

Time Series Analysis
Python
AWS
Time Series Forecasting
Recommended from ReadMedium