Inside Data Engineering with Julien Hurault
Consultant Julien Hurault takes you inside the world of data engineering, sharing practical insights, real-world challenges, and his perspective on where the field is headed.
Today, we're joined by
from , who’s been working in Data Engineering for the last 10 years, mainly as a consultant, he will dive deep into his experience from the consultancy perspective.To recap: the series follows a Q&A format, featuring professionals who share their journeys, insights, and challenges.
What to Expect:
Inside the Role – Get a real-world look at what data engineers do day in and day out.
Getting Started – Discover the essential skills, tools, and career routes to break into the field.
Tech Trends – Stay in the loop with evolving technologies and shifts shaping data engineering.
Debunking Myths – Clear up common misconceptions about the data engineering profession.
Voices from the Field – Hear firsthand insights and experiences from seasoned data engineers.
⭐ If you're curious about data engineering or considering it as a career, this series is for you!
Let’s dive into Inside Data Engineering:
How would you describe Data Engineering?
Data engineering is the art of moving data from point A to point B as efficiently as possible. “Efficient” means choosing the tools, designs and processes that best fit the context, your company’s scale, the nature of the data, and the latency required.
How did you end up being a Data Engineer?
I started out in 2015 as a data scientist. Like many in that role, I spent about 90 % of my time cleaning, reshaping and piping data for my models. At some point I realised I was already doing data engineering work, just without the title. From there, I leaned fully into that specialty and transitioned step-by-step into a dedicated data engineering position, which is where I am today.
What's your day-to-day look like?
I work as a freelance data engineer. I spend most mornings on client projects, and in the afternoons, I focus on sales, marketing, or content creation.
What are some stakeholders that you work with?
When I engage with a client, my first contacts are usually the Head of Engineering, Head of Data, or Data Team Lead. After the initial discussions, I work mainly with the data engineering team.
What kind of projects do you work on?
I support startups by defining their data strategy, selecting the right tools, and designing data architecture. I also work with larger companies in a more hands-on role, helping them build data platforms.
What kind of data do you work with?
Most of my projects involve tabular data:
Time-series
Events
User data/profiles
What data size do you work with?
It largely depends on the company and its data volume, which can range anywhere from tens of GB to 100+ TB.
What tech stack do you use?
Tech Stack depends on client/projects, typically:
What is your favorite area of Data Engineering?
Data Platform Design
How can Data Engineering benefit from GenAI?
V0 of a data pipeline: With a solid foundation in place, GenAI can generate the v0 of a pipeline. However, GenAI struggles with anything beyond ~20 lines of code unless it’s tightly constrained.
That’s exactly why I launched boringdata.io. The templates provide 80% of the pipeline code out of the box; GenAI fine-tunes another 10%, and the engineer refines the final 10%.
What advice would you give your past self as a beginner Data Engineer?
If you hate AWS IAM, that's normal — you're not alone. 😄
What are some challenging aspects of Data Engineering?
Design your pipeline with maintenance in mind. Building a pipeline's v0 is easy; making it robust is not.
What is the next big thing according to you in Data Engineering?
Open table formats are transforming data platform design: We can keep all data in a single storage layer while choosing the best compute engine for each task.
What are some common misconceptions about data engineering?
Moving data from A to B may sound boring, but there are countless ways to design the same pipeline, which makes it fascinating. Technology evolves so quickly that you must keep learning new tools and approaches.
Reach out if you like:
To be the guest and share your experiences & journey.
To provide feedback and suggestions on how we can improve the quality of questions.
To suggest guests for the future articles.
 | A guest post by
|
Love that collaboration. Great work, guys!