Why Discord is Moving an Many Services from Go to Rust

Discord has always been serious about performance. With millions of people chatting, calling, streaming, and gaming together at any given moment, even tiny slowdowns can lead to a frustrating user experience. Although Go (Golang) served us well for a long time, we found ourselves running into certain performance and latency issues that were proving stubborn to solve. After extensive experimentation and success with Rust elsewhere in our stack, we decided to try something bold: reimplementing a critical backend service entirely in Rust.

Before we dive into the tips, here’s a special link for non-paid members to read my article for free: Read it here for free!

This post will walk you through the reasoning behind this decision, the technical hurdles we faced, and how Rust’s unique memory management and async ecosystem helped us achieve smoother, more consistent performance. We’ll also dive into some sample code and discuss lessons learned along the way. This blog from Discord motivated me to write this article.

The Service: Tracking “Read States”

Before we jump into the “why” and “how,” let’s set the stage. The service in question handles “Read States.” Essentially, whenever you view a channel or read a message, this service records that state so Discord knows which messages you’ve read and which notifications you’ve cleared. It might not sound glamorous, but it’s a lynchpin of the user experience. Without this service, you’d never know at a glance what’s new since your last visit.

Key points:

• Volume: Billions of Read States entries, each corresponding to a (User, Channel) pair.
• Access Frequency: Every time you open Discord, send a message, or mark something as read, this service does work. It’s constantly in the hot path.
• Performance Goals: Near-instantaneous updates. Think in terms of microseconds or a few milliseconds at worst — especially under load.

Our previous Go-based implementation was fast most of the time, but we kept seeing periodic latency spikes every couple of minutes. These weren’t small blips, either — some spikes were large enough to cause noticeable slowdowns for our users. After digging deep, we identified the Go garbage collector (GC) and memory management model as key factors in these transient slowdowns.

Understanding the Go Performance Bottleneck

In Go, memory is managed by a garbage collector. This means unused memory isn’t immediately reclaimed once you stop using it; instead, the GC periodically scans the heap to find memory that can be freed. For systems dealing with a massive in-memory cache (like our Read States), these periodic scans can become costly. Even if you code efficiently and minimize allocations, the GC will still run at certain intervals — often resulting in noticeable, if temporary, latency hits.

For our Read States service, we implemented a huge LRU (Least Recently Used) cache to ensure that the most frequently accessed user-channel pairs would be quick to retrieve. While this cache improved average performance, it also created a large in-memory footprint for the GC to scan. The result? Every couple of minutes, when the forced GC cycle ran, we saw a CPU and latency spike.

We tried various GC tuning strategies — adjusting the GOGC environment variable and trying different cache sizes. Making the cache smaller reduced GC overhead (less memory to scan) but increased cache misses, leading to more database lookups and slower overall performance. It was a tough balancing act, and while we found a “good enough” setting, it never felt truly satisfying.

Enter Rust: A Different Memory Model

We’d already dipped our toes into Rust for other parts of the Discord ecosystem, including video encoding pipelines and certain Elixir NIFs. Rust had proven its mettle in performance-critical tasks, and its unique memory management model promised a solution to the GC-related issues we were facing.

How Rust Differs:

• No Garbage Collector: Rust uses a compile-time ownership model. When data goes out of scope, Rust frees it immediately — no runtime scanning required.
• Predictable Performance: Without periodic GC pauses, latency can remain more consistent. Memory is reclaimed as soon as it’s not needed, resulting in fewer unpredictable slowdowns.
• Strict Compile-Time Checks: Rust ensures memory safety and data race freedom at compile time, reducing the likelihood of tricky bugs or the need for certain runtime checks.

In other words, Rust allowed us to write a highly performant backend service with more predictable performance under heavy load.

Async Rust: Embracing the New Era

Rewriting a networked service in Rust doesn’t just mean switching languages. We also had to consider how to handle asynchronous operations — critical for a system that may wait on I/O (like database queries or RPC calls) millions of times a day.

When we started this rewrite, Rust’s async ecosystem was still maturing. Libraries like tokio were relatively new, and stable async/await syntax was just around the corner. We decided to be early adopters, running on Rust’s nightly releases to take advantage of async/await before it was fully stabilized. This approach came with challenges—occasional breaking changes, complex error messages, and the need for frequent updates—but it paid off. By the time we were done, the Rust ecosystem had solidified around a stable async model, and our service got to reap the benefits.

Rewriting the Service: A Step-by-Step Overview

The actual rewrite began with a straightforward translation of concepts from Go to Rust. We didn’t try to optimize prematurely. Instead, we focused on correctness first, and then iteratively refined our data structures and caching logic to leverage Rust’s strengths.

Data Structures and Caching

In Go, we used a large LRU cache keyed by (User, Channel) pairs. In Rust, we did something similar but were more careful with our choice of data structures. For instance, we experimented with HashMap and BTreeMap variants, ultimately finding that a BTreeMap sometimes offered more stable performance characteristics under our workload due to improved memory locality and predictable iteration patterns.

Example Rust snippet of a simplified LRU structure:

use std::collections::{BTreeMap, VecDeque};
use std::time::{Instant, Duration};

struct ReadState {
    last_read_message_id: u64,
    mention_count: u32,
    updated_at: Instant,
}
struct LruCache {
    store: BTreeMap<(u64, u64), ReadState>, // (UserID, ChannelID)
    recency_queue: VecDeque<(u64, u64)>,    // Tracks which keys were recently accessed
    capacity: usize,
}
impl LruCache {
    fn new(capacity: usize) -> Self {
        Self {
            store: BTreeMap::new(),
            recency_queue: VecDeque::with_capacity(capacity),
            capacity,
        }
    }
    fn get(&mut self, key: (u64, u64)) -> Option<&ReadState> {
        if let Some(value) = self.store.get(&key) {
            // Update recency
            self.recency_queue.retain(|&k| k != key);
            self.recency_queue.push_back(key);
            Some(value)
        } else {
            None
        }
    }
    fn insert(&mut self, key: (u64, u64), value: ReadState) {
        if self.store.len() == self.capacity {
            // Evict least recently used item
            if let Some(evicted_key) = self.recency_queue.pop_front() {
                self.store.remove(&evicted_key);
            }
        }
        self.store.insert(key, value);
        self.recency_queue.push_back(key);
    }
}

This snippet is a simplified illustration. In the actual implementation, we’d also handle concurrency (using something like tokio::sync::RwLock or parking_lot::RwLock) and possibly tune the data structures further. But it shows the kind of careful, memory-conscious design Rust encourages.

Async I/O and Database Access

Our service doesn’t just manage memory; it also interacts with a persistence layer (Cassandra or another distributed DB). Using async Rust allowed us to efficiently handle thousands of concurrent I/O operations without blocking threads unnecessarily.

Example async Rust snippet for fetching a read state from the DB if not cached:

use tokio::task;

async fn fetch_read_state_from_db(user_id: u64, channel_id: u64) -> Option<ReadState> {
    // Simulated DB call (in practice, use a real client)
    task::spawn_blocking(move || {
        // Imagine a synchronous DB client here
        // let row = db_client.query("SELECT ...");
        // Parse row into ReadState
        Some(ReadState {
            last_read_message_id: 42, // Fetched from DB
            mention_count: 0,
            updated_at: Instant::now(),
        })
    })
    .await
    .ok()?
}

This async pattern allowed our threads to handle other tasks while waiting on the database, improving resource usage and throughput.

Load Testing, Optimization, and Results

Once we had a working prototype, we subjected it to massive load tests simulating real-world conditions. Immediately, we noticed something exciting: even with minimal optimization, the Rust version matched Go’s performance and eliminated those periodic latency spikes. Our CPU usage also became more stable over time.

We decided to push it further. With profiling tools (like perf or tokio-console), we identified hotspots and made small tweaks:

• Switching Data Structures: For certain internal caches, switching from HashMap to BTreeMap improved memory locality and reduced cache miss penalties.
• Concurrency Primitives: Using more efficient synchronization primitives cut down on thread contention.
• Fewer Memory Copies: Carefully examining and reducing unnecessary data cloning gave us tighter control over memory usage.

Each of these changes shaved off a few percentage points of CPU usage or a handful of microseconds in latency. Combined, they resulted in a significantly more efficient system.

Final Launch and Lessons Learned

After validating performance in a canary environment (a small subset of real traffic), we gradually rolled out the Rust-based Read States service to the entire user base. The improvement was immediate and noticeable: smoother latency, no more periodic spikes, and better tail latencies (those 99th and 99.9th percentiles were looking much healthier).

With this success, we started raising our cache capacity again — something we couldn’t comfortably do with Go. Without GC overhead, a larger cache didn’t cause periodic slowdowns. Instead, we saw even fewer DB lookups and faster responses.

Key Takeaways:

1. Predictable Performance Matters: Eliminating GC-related pauses leveled out our performance profile.
2. Rust’s Memory Model Helps: By enforcing safety and ownership at compile time, Rust made it easier to write efficient, correct code.
3. Async Ecosystem is Maturing: Despite early challenges, Rust’s async story is now stable and powerful.
4. Refactoring is Straightforward: Rust’s strong type system makes it simpler to rewrite or refactor code with confidence.

Looking Ahead

Today, Rust powers a growing portion of Discord’s backend. From NIFs in Elixir to dedicated services like this one, Rust has earned its place as a first-class citizen in our tech stack. When we start new projects, Rust is no longer an “exotic” choice — we give it serious consideration right alongside more traditional options.

We’re excited about the direction the Rust ecosystem is heading. Libraries like tokio have matured, compile times have improved, and the community is continually refining best practices. For us, the investment in rewriting this service was well worth it.

P.S.: Fun fact: The Rust language development community itself actively uses Discord to coordinate. You can find helpful folks in the Rust community servers, always ready to guide newcomers and seasoned developers alike.

Thanks for sticking with me till the end! If you enjoyed this content and want to support my work, the best ways are:

• Follow me here on Medium.
• Connect with me on LinkedIn.