Reddit Data Tech Stack
Explore how Reddit's data infrastructure powers 120M daily users, 600M annual posts, and 40M daily searches across 138,000 communities, covering ingestion, processing, storage, and analytics.
Explore how Reddit handles one of the internet’s largest volumes of user-generated content, powering community discovery, real-time feeds, ads, and content safety for over a billion users. This article covers the core tools and architecture Reddit uses for data ingestion, processing, storage, and analytics.
Metrics
120+ million daily active users as of Q4 2025, source.
600+ million posts and 3+ billion comments published in 2025, source.
~40 million search queries per day on the platform as of December 2024, source.
~140k active subreddits and $2+ billion in revenue in 2025, source.
500+ Kafka brokers holding more than a petabyte of live data, source.
Tens of millions of Kafka messages processed every second across hundreds of business-critical services, source.
Content is based on multiple sources, including the Reddit Engineering Blog, third-party articles, and employee experiences. You will find references throughout.
Platform
AWS
Reddit decommissioned its last physical servers in 2009 and has run on AWS ever since. They were early adopters of S3 and EC2, and AWS continues to host all core online services; databases, Kafka, and Kubernetes.
GCP
GCP adoption was driven by a commercial deal, Google and Reddit expanded their partnership to include a Cloud agreement alongside access to Reddit's Data API. This brought BigQuery (data warehouse) and Vertex AI (powering Reddit Answers) into the stack.
📖 Recommended Reading: An expanded partnership with Reddit
Messaging Systems
Kafka
Kafka is the central nervous system of Reddit's data platform, receiving events from multiple sources including user actions, application logs, and database change events via CDC. It fans out data to both real-time and offline consumers:
Real-time path: Kafka → Flink (Snooron) for stream processing and content safety rules
Reporting path: Kafka → Spark → Druid for analytics
Offline path: Debezium (Kafka Connect) → Kafka for database replication to downstream stores
At scale, the Kafka fleet comprises 500+ brokers serving tens of millions of messages per second with over a petabyte of live topic data. Kafka runs on Kubernetes via Strimzi, migrated from EC2.
Processing
Flink (Snooron)
In 2021, Reddit's Safety Engineering team built Snooron on Flink Stateful Functions to replace their legacy rules engine with REV2, running on Kubernetes with GitHub-managed rules and real-time event processing. It was later extended with Signals-Joiner, which enriches live event streams with ML signals to detect more sophisticated policy violations.
Spark
Alongside Flink, multiple teams at Reddit rely on Spark for batch processing. The primary confirmed use case is the reporting pipeline, Spark validates and transforms events from Kafka, writes them to S3 as Parquet files, and prepares them for ingestion into Druid.
Data Replication
Debezium
Reddit uses Debezium with Kafka Connect to capture row-level changes from PostgreSQL via logical replication, streaming them to Kafka and into the data warehouse in real time. Schema changes are propagated automatically via a Schema Registry. Debezium runs as lightweight Kubernetes pods, and for large initial snapshots where sequential snapshotting is too slow, an EMR cluster with Spark is used to backfill instead.
📖 Read More: Change Data Capture with Debezium
Orchestrator
Airflow
Reddit uses Airflow as their pipeline orchestration tool, with custom operators built on top for internal workflows. Scale and hosting details (e.g. self-managed vs managed Airflow on GCP/AWS) have not been publicly disclosed.
Data Warehouse
BigQuery
Reddit migrated from Redshift to BigQuery as part of their GCP partnership. The migration was facilitated by BigQuery Data Transfer Service, which supports seamless migration from Amazon Redshift directly into BigQuery.
📖 Read More: BigQuery leads the way toward modern data analytics
Data Lake
S3
S3 serves as Reddit's data lake, storing raw events from Kafka, intermediate outputs, and Spark/Flink-processed data as Parquet files. There is no public confirmation of a modern open table format (Iceberg, Delta Lake, or Hudi) on top of S3 in Reddit's stack.
Data Store
Druid
Druid (via Imply) powers Reddit’s customer-facing real-time analytics platform. Data flows from Kafka → Spark → Druid, with Druid handling aggregations at ingestion time for fast query performance. As of 2021, Druid was updated hourly, though this is likely near real-time today given Druid’s native Kafka ingestion capability.
📖 Recommended Reading: Scaling Reporting at Reddit
Related Content:
💬 Reddit uses a mix of open-source and commercial solutions, and has increasingly shifted toward managed services, driven as much by strategic partnerships as by technical considerations.