Subscribe
Sign in
Home
š§āš» Become a DE
š Deep Dive
ā Tech Stack
š Growth
šļø Archive
ā About
Inside Data Engineering with HasanĀ Geren
Follow Hasan Geren as he explores the landscape of data engineering, offering insights, tackling challenges, and highlighting emerging industry trends.
Jan 10
Ā
ā¢
Ā
Junaid Effendi
Ā andĀ
Hasan Geren
12
2
2
Latest
Top
Discussions
Solving Sparkās Small File Problem for 100x Faster Reads
Understand the Spark common small file problem, learn how to solve in the modern open table formats through offline and online optimizations.
Dec 6, 2025
Ā
ā¢
Ā
Junaid Effendi
16
2
2
Shopify Data Tech Stack
Explore what tech stack is used at Shopify to process 284 million peak requests per minute generating $11+ billions in sales.
Nov 8, 2025
Ā
ā¢
Ā
Junaid Effendi
9
4
2
Inside Data Engineering with Erfan Hesami
Join Erfan Hesami as he shares his experience in the world of data engineering, offering insights, exploring challenges, and highlighting emergingā¦
Oct 11, 2025
Ā
ā¢
Ā
Junaid Effendi
Ā andĀ
Erfan Hesami
13
3
How Delta Lake Works
Understand how Delta Lake handles reads and writes using the transaction log, ensuring ACID guarantees through snapshot isolation and optimisticā¦
Sep 6, 2025
Ā
ā¢
Ā
Junaid Effendi
18
4
1
Spotify Data Tech Stack
Learn how Spotify ingests 1.4T+ events daily on GCP via 38K+ data pipelines, leveraging BigQuery, Dataflow, and Flyte to power ~5K dashboards and scaleā¦
Aug 16, 2025
Ā
ā¢
Ā
Junaid Effendi
22
2
Inside Data Engineering with Julien Hurault
Consultant Julien Hurault takes you inside the world of data engineering, sharing practical insights, real-world challenges, and his perspective onā¦
Jul 26, 2025
Ā
ā¢
Ā
Junaid Effendi
Ā andĀ
Julien Hurault
20
2
1
Benchmarking Spark - Open Source vs EMRs
Diving into four approaches from Spark Operator to EMR (EKS, EC2, and Serverless), sharing benchmarking results and key insights to help you choose theā¦
Jul 5, 2025
Ā
ā¢
Ā
Junaid Effendi
5
2
2
See all
Junaid Effendi | Sharing knowledge for Engineers
Covering tech, career, data, growth experiences from my journey.
Subscribe
Recommendations
View all 13
VuTrinh.
Vu Trinh
Data Tinkerer
Data Tinkerer
Data Gibberish
Yordan Ivanov
Leadership Letters
Akash Mukherjee
Pipeline To Insights
Pipeline to Insights
Junaid Effendi | Sharing knowledge for Engineers
Subscribe
About
Archive
Recommendations
Sitemap
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts
