Types of Data Engineers

Learn different types of focus areas for a Data Engineer in the modern industry with future articles covering the specific roadmaps.

Jul 27, 2024

Data Engineering is relatively still a new discipline with lot of demand as the data grows, its been called “data is the new oil”. Data Engineering has always been there in some form technically speaking, but the proper definition or title came into the picture around 2008 with the rise of Hadoop Ecosystem.

Today, I will share different types of core areas for Data Engineering which will help you decide which one suits best for you if you are looking to break into this. Whether you are a Software Engineer, Data Scientist or Data Analyst you are likely going to work in one or more of the following areas.

⭐This article will set the base for future articles where I will dive into details in the form of roadmap covering specifics.

Data Infrastructure Expert

An expert focusing on setting up Data Platform or Data Infrastructure, works closely with DevOps and Platforms Teams. The goal is to provide the infra/Platform As-a-Service.

Data Tooling Expert

An expert focusing on data related tooling and services covering data quality libraries, common data packages and APIs typically using the infra provided by Data Infra Experts. The goal is to provide easy to use tools for downstream users.

Data Pipeline Expert

An expert focusing on ETL or ELT Pipelines covering batch, streaming like full load and incremental pipelines typically using the infra and tooling provided by the above two experts. The goal is to provide quality data to end users.

Data Modeling Expert

An expert focusing on how to model data in the Warehouse or Lakehouse, e.g. Kimball Data Modeling on very huge datasets coming from variety of sources. The goal is to make data at rest easily accessible.

Data Visualization Expert

An expert focusing on building dashboards for variety of usecases from business metrics to data quality and monitoring. The goal is to provide easy to interpret charts for different stakeholders, from Finance to Senior Executives.

Data Domain Expert

An expert focusing on analyzing and experimenting data for different use cases e.g. Machine Learning, Data Quality etc. The goal is to have solid domain knowledge that helps other teams better understand and interpret data through great storytelling.

This may have been a bit boring article, but in the future, you will enjoy the visualization which will represent the technologies in the form of roadmap for the following transitions.

Software Engineer to Data Engineer
Data Scientist to Data Engineer
Data Analyst to Data Engineer

📖If interested in similar content checkout: My take on Data Engineering from the past.

💬 Let me know in the comment section which area(s) do you focus, you can be great at multiple areas. If you disagree, feel free to ping or comment to have healthy conversation to learn from each other.