Data Engineer Vs Analytics Engineer Vs Analyst - Which Position Is Right For You?
1 Video View·Nov 4, 2024
In this video we will focus on three roles and talk about how they all compare. Title - Data Engineer Vs Analytics Engineer Vs Analyst - Which Position Is Right For You? If you enjoyed this video, check out some of my other top videos.
Earlier in 2021, a sort of new company raised $150 million, all focused around kind of a new role
called analytics engineering. That company, which was originally a consulting company,
had developed a tool called DBT. Perhaps you've heard of it, and perhaps in the future I'll make
a video about what exactly it is, but today I wanted to focus on what exactly is the difference
between a data engineer, an analytics engineer, and an analyst. Because all of these roles have
a lot of similarities in terms of tools as well as function. And so there's a lot of blurry lines
between each of these jobs. Not to mention the fact that we have so many other similar titles
like BI engineer, data engineer, which if you've ever interviewed at Amazon, you'll know that they
actually give you the same study guide for both their BI as well as their DE position. So clearly,
it looks like they don't even know the difference. And truthfully, I could keep going on with this
kind of concept with roles like data architect, ETL developer, and all these other concepts along
the way that have kind of all morphed and traded jobs and traded different responsibilities over
the years. And of course, the fact that the word engineer is even in the title of the role,
analytics engineer, will probably make some people very upset because, you know, especially people
that are like mechanical engineers, chemical engineers, electrical engineers, who feel like
they do pure engineering, and are all upset because for some reason they make a lot less
than some of these roles. Don't ask me why people that make our planes and rocket ships get paid
less than people that, you know, make you click ads. But that's not the focus for this video. The
focus for this video will be talking about the differences between analytics engineers,
data engineers, and analysts. Because again, there's a lot of crossover. So let's kind of
talk about this forward assembly line of data. If you look at DBT's kind of what is analytics
engineering, I think they do a good job at least of summarizing the differences between these
positions, honestly, in one graphic. But in this video, we're gonna go a little more in depth,
as well as talk a little bit about salary. So hopefully we add a little more enrichment to
this one picture. But essentially, the role data engineers will play in a modern company with a
sizable data team is that they will make the main kind of data pipelines, you know, especially a lot
of the custom integrations, maybe create some more of the streaming components. So anything
that has a lot of technical overhead, that's where data engineers will play their main
responsibility, right, like getting that data into the data warehouse, kind of creating maybe that
first and second layer of data. For those of you unfamiliar, that first layer might be a little
more raw and staging. And that second layer will be that like core data set. So before doing any
form of fancy transformations, you're really just trying to show the main transaction data and
dimensional data that basically represents a company's day to day transactions and information.
This also fits kind of in that narrative I've talked about with the difference between software
engineers and data engineers, in the fact that data engineers kind of sometimes can either lean
more towards the software engineering side or the analytical side. So this I think puts a better
split in between the data engineer and analytics engineer, where the data engineer will start
leaning even more heavily towards being more software focused, whereas the analytics engineer
will be focused around creating data sets. And that kind of leads us into that responsibility
of the analytics engineer, which their focus is about creating those core data sets. So they
might be applying some of the more complex business logic, as well as applying like software
best practices, like version control to ensure that, you know, as you're creating these different
data sets, that if you're writing SQL, it's maintained over a long period of time, and
you're not just changing it quickly, but you're doing things like writing unit tests and maybe
doing some sort of CICD or something similar to make sure that whatever data comes out is accurate,
especially once you start applying more complex business logic, that's kind of their role.
They are creating these key data sets that analysts can then rely on without them then
having to apply their own logic on top. The benefit here with not having analysts apply
too much logic on top of the data, which is one of the major issues that can occur when you start
having lots and lots of data and lots of complex business logic, is you might have three or four
teams applying the same business logic, which can lead to three or four different outcomes.
So I think the goal here, at least envisioned by maybe DBT and other companies, is by having kind
of one team that can manage this one layer of data, especially the more complex business logic,
you're going to reduce the amount of issues in terms of syncing data, or maybe just having
slightly different data, even though you're trying to apply the same logic. So the analytics
engineer are going to apply a lot of, again, these best practices that we know from software
engineering to analytics. They will also include things like data governance, as well as maybe
hopefully tracking some things about KPIs and this complex business logic. So that way,
as things change over time, all of that can be tracked more efficiently and just in a cleaner
fashion. Looking at the analyst's responsibilities in this, again, forward assembly line of data,
an analyst will do a lot of the ad hoc work. Their focus is going to be drilling into the data and
really asking tons of questions and not having to be hindered by just creating clean data sets.
That takes a ton of time, and so you can't necessarily go as deep as you want to in all
your analysis. But by having someone prior in that analytics engineering role do a lot of this
heavy lifting, hopefully data analysts can then in turn focus a lot more on the business problems,
as well as doing things like creating dashboards, which also require a lot of engineering best
practices, which aren't always applied because there's not often a lot of time. So in turn,
hopefully this split of responsibilities can make everyone's life a little easier,
but that's just looking at responsibilities. Next, let's talk about tools. So when we look at the
tools between these three different roles, there's definitely some crossover. All three of these
roles probably use something like Python and SQL, but more than likely they use them at different
kind of weightings. A data engineer is going to be very focused on writing code and some SQL,
where more than likely a analytics engineer is going to be very focused on writing SQL,
and an analyst will probably use a combination of SQL, Tableau, Excel, and a couple other tools
that maybe aren't as code heavy to develop their analytics and analysis. In this new framework,
each of these different roles has the ability to better specialize in each of the tools that
they're really focused on. You can allow the data engineers that like programming to focus more on
that area, while you can allow the analytics engineers who maybe like SQL a little better
to focus on that whole section. They can focus a lot more of their time on things like SQL,
versus having to spend a lot of times on like DevOps and Python and writing code,
which gives them the ability to really focus in on, again, complex logic that can be very easy
to get wrong. And on the flip side, analysts can focus on answering questions with whatever tools
they like really in this case, because to me, an analyst's job is really to do some form of
analysis and generally answer a ton of questions that can then lead the business to better business
decisions. There may or may not be the need to write code in that role. They might be able to
answer a lot of their questions using things like Excel or maybe a quick Jupyter notebook,
but the point of the matter is each of these different roles can focus on their core tools.
On top of that, there's just so many solutions like Fivetran and Snowflake to focus on for both
the data engineer and the analytics engineer that they're going to be spending a ton of time just
either learning new tools, migrating their tools to better and more efficient tools,
and spending their time really making sure that those tools are utilized to the maximum capacity.
You know, you're not just utilizing dbt to use it, but you're actually using it in such a way
that makes your analytics more maintainable. So that's kind of the split of tools. Again,
more than likely data engineers will be coding, analytics engineers will be writing SQL,
and analysts will be answering questions using whatever tools that best suit them.
Next, let's talk about salaries. Now, if you've seen my previous video about the five most in
demand data jobs for 2022 and their salaries, you probably already know a little bit about
this answer. If you recall, data engineers make somewhere in the range of like 90 to 120k,
unless you're looking at a company like a Fang, where data engineers can easily make upwards of
a quarter of a million dollars, even at a pretty low level, which can be two or three times higher
than again, some of these other companies. So data engineers do tend to command a pretty high
salary. From what I've seen for analytics engineers, it's actually not that different
than compared to data engineers. On average, they're making somewhere in the range of 80 to 110k.
And there actually are, if you look at this chart, some that are making pretty near a quarter of a
million dollars. I think this makes sense because a lot of the analytics engineer positions that
I've seen are actually more of a combination of a data engineer, plus an analytics engineer.
We're often using a lot more tools to do the exact same work that a data engineer would do.
And so in that regard, you're providing the exact same value and thus should be paid the same. And
honestly, if you're also being expected to do analyst work, you might even think you should
get paid more, but we'll see where that goes in the future. Again, you can kind of see the salary
is pretty decent as well. And then finally for analysts, again, if you recall from my previous
video, it's somewhere in the range of like 67 to like 80, 88k, which is a, again, a great salary.
Again, if you go to a big tech company, this might be upwards of the range of a hundred K.
In fact, I'm pretty sure some things match the base salaries of most of their tech or analytics
employees. And the one differentiator then is stock. Of course, that's always going to be the
differentiator. I've talked about that before, even between like software and data engineers
at a lot of tech companies, it's all about those stonks. The truth of the matter is that not a lot
of companies can afford all three of these positions. Yes, it's nice in theory to have this
forward assembly line of data where everyone can specialize, but most companies are going to need
you to play a role of each. You're likely, especially as a data engineer, going to be
spending a lot of time, not just pulling in data and creating data pipelines with custom code,
but also using tools like Fivetran and other solutions in order to pull in data into your
warehouse. And then depending on your analyst skill level, we'll likely spend a lot of time
writing views as well as applying like business logic to create kind of core datasets that
analysts can then use and parse through for their analysis. And now you're probably wondering which
role is the one for you. I think the best way to answer this for yourself is to figure out what
you like doing. For example, when I first started in data, I thought I really liked the idea of
data science, but after spending about a year, just trying and pushing in that area, I kind of
found out I liked the idea of engineering better. I liked data, that was for sure, but I also enjoyed
kind of building infrastructure and data pipelines and things of that nature and not doing as much
analysis. It just wasn't as exciting for me. I really am not a huge fan of writing code in
Jupyter notebooks other than doing maybe some quick ad hoc stuff. And I really like writing
code that's more infrastructure based and more longstanding. I also really like dealing with
the things that are at that solution or system level where you're having to like coordinate
things among different servers and maybe having to deal with like on-prem servers and the cloud
and other components along that whole system. So I really like that picture and that side of data,
and I'm not as big of a fan of just doing analytics and doing some sort of math and statistics.
And that's something you need to be honest with yourself about. Do you prefer doing things like
programming in SQL or are you more SQL heavy? And where do you kind of fit on the data spectrum?
Are you more on the data engineering side or the analyst side? It's really a broad spectrum
and even amongst the spectrum, you can kind of fit as a mishmash in between.
So the question is, what problems do you like solving and what tools do you like using? Please
just take a moment to comment and answer that question below. And I just want to say thank you
so much for watching. I hope this helped answer the question. What's the difference between a
analytics engineer, a data engineer, and a analyst? With that guys, I will see you next time and
goodbye.

