How to Become an Analytics Engineer! Beginner's Guide to Starting a Career in Analytics Engineering
1 Video View·Nov 4, 2024
Considering a career in Analytics Engineering? Dive into this comprehensive beginner's guide on how to become an Analytics Engineer. We'll cover essential skills, tools, and steps to kick-start your journey in this booming field. From data manipulation to understanding business needs, discover what it takes to excel in this dynamic profession. Whether you're a fresh graduate or looking to pivot into a tech role, this video is your roadmap to success in Analytics Engineering. Don't forget to like, share, and subscribe for more career guidance videos!
Hey y'all, Data Guy here. So something that I've been hearing a lot about, you know,
from people out in the field, from just talking to everyone and just in general,
is the newer kind of concept of an analytics engineer. Maybe it's not as new as I'm thinking,
maybe I just didn't really hear about it before, but basically what an analytics engineer is kind
of a hybrid between a traditional, you know, data scientist and a data engineer. So they're really,
you know, the link between data engineering and data analysis. So they have skills that
comprise both of those. So they essentially are almost like a one-man army. You know,
they're able to set up pipelines to extract, transform, and get great data for them to use.
And then they also have the skills to visualize, interpret, and actually gain insights out of that
data. So if they're actually, you know, capable, it can be an incredibly powerful position,
and they can be a real, real up level for a business. And so their primary responsibility,
as you might imagine, is transforming all that raw data from whatever sources their business is
collecting it from into a format that can be used for analysis by data analysts, data scientists,
and other stakeholders. Maybe they're not even actually doing the analysis themselves. Maybe
they're just, because they're an analytics engineer, they know the format that other
analysts might need. And so instead of, you know, needing to have this constant push and pull
between a data engineer and a business analyst, you actually have someone that understands the
needs of the business analyst and can proactively serve them data that is ready for consumption by
them that can be used to make effective decisions without the business analyst needing to explicitly
ask for it. So they're a really great link to kind of bridging those two fields. And so, you know,
by ensuring that data is accurate, reliable, easily analyzable, they really provide that
strong foundation for data-driven decision-making in organizations. And some people see it as a
data engineer in isolation or just being a data scientist and not really having control over the
data you're getting. So it's a really interesting position. And so now we kind of have an overview
of, you know, what they are doing. I want to talk about some of the tools that they use and, you
know, what your day-to-day would look like. So here I have kind of an image of the typical data
engineering stack, or sorry, not data engineering, analytics engineering stack. So while a traditional
data engineer will probably be staying everything below the data science layer, you know, they'll be
using dbt for transformation. They'll be interacting with the database is that the company's using,
using tools like, you know, Airflow, Fivetran, Matillion, Airbyte to extract data from different
sources, manipulate it. They typically won't touch the higher level data science and analytics tools
that are building on top of this data. Now that's where an analytics engineer will come into play
and actually act across this entire stack. So they're going to be leveraging your databases
that are, you know, they're gonna be building databases to store all that structure data that
they're going to then feed to analytics users or use themselves. And they're also going to make use
of, you know, dbt for best practices on data transformations, modeling directly in the data
warehouse to use that more efficient compute, using tools like Airflow, like Fivetran, just as,
you know, a traditional data engineer would. And, but they're also pairing that with the skills to
then present that information in an actual understandable format for end users. And so
they really intimately know how to hook in, you know, their visualization tools or data science
tools into these databases in a way that can efficiently consume the data on a regular basis,
you know, provide things like live streaming, live updating dashboards, and really getting into the
minutia of optimizing those databases for really large scale analytics queries. And that's the
reason why I think why analytics engineers have become so popular is because as you're doing
things like AI and ML, those are almost just fancy words for really, really advanced data statistics,
data science. And so that's why analytics engineers are becoming so popular is because to be able to
make use of these massive scales of data that are now present in the ecosystem, you really need to
have a much more specialized skill set, because you have to wrangle all this data from many different
sources, bring it in at really large scale, and you need to do it constantly. So you can keep
updating your models, getting the latest predictions on the latest data. And then I think just
it's worth mentioning, you know, some skills that you'll need across all of these different tool
layers are just Python and SQL. Obviously, you're going to need a SQL for interacting databases,
and then Python is, in my opinion, or my favorite way to actually interact with data for, you know,
the end line manipulations, when I want to transform it into different visualizations,
you know, and just do things like that. So these are the so we've gone over the kind of tools,
the analytics engineer going to use, but what are their responsibilities and tasks? So here,
I have kind of a graphic actually from the great team over at dbt, which I thought was pretty
relevant, because they're going to be used across most data teams these days, describing what the
analytics engineers responsibilities are kind of in the context of what a data engineer does,
and what a data analyst does to really kind of highlight its, you know, link between the two
roles. And so at the core of their role is data modeling. And so that's why, you know, providing
clean transform data ready for analysis. This comes from analytics engineers meticulously designing,
maintaining data models that are robust and scalable for the business. So bringing in their
knowledge about, you know, what clean data looks like at the best, but also marrying that with the
best practices of data engineering, creating scalable pipelines that can manage these
transformations automatically at really large scale. And so these models will, you know,
be the ones that result in top level views, you know, providing visualizations to and business
users. And so ensuring data quality is incredibly paramount, they're actively monitoring create
checks to ensure data, you know, data accuracy and consistency. So that is where you have something,
you know, like applying software and hearing best practices to really rigidly, almost go through all
of your code, make sure that it is never going to go break anything, you know, break all your downstream
dashboard, just because you made a messy poll. So, you know, bringing in things like CI CD,
source control versioning, to, you know, make sure that, hey, even if something does go wrong,
we can easily revert it, you know, and kind of having that knowledge of what these different
models actually translate to on the data analyst side. So knowing, hey, you know, this is why we
need to have reversion, because this report is going to the CEO, and if it breaks, we're all
getting fired. Right. And so, as you might imagine, you know, with all these responsibilities,
tying two different aspects, you know, they got to get the data or relate the data is the raw data
from data engineers, you know, and also work with data analysts, and also train business users and
help them understand the reports that they're generating. They are collaborating a ton throughout
their daily tasks. So they're like liaising with I don't know, I'm not gonna use the fancy French
They are frequently meeting, you know, with data scientists, business analysts,
and business users, executives, as well as back end engineers to make sure, hey, what are the
needs that these end users have? What business insights are we trying to drive, and then taking
those requirements and translating them so the data engineers can go pull out whatever data they
need, whatever kind of data sources they need to start integrating into the stack. And, you know,
taking the business, you know, because you're not going to be talking to people that are
super well versed in like, you know, hey, I want all the information about our customers.
Well, that can involve, you know, 20 different platforms collect all that information.
So being able to translate, say, okay, well, he doesn't actually need all the information,
he just needs, you know, all the purchase information for everyone within the past year,
and then translate that to the data engineer so that they can go pull that data set
for you, rather than, you know, send them on a wild goose chase if the, you know, boss just
spoke directly to the data engineer and said, give me all this stuff. He just takes them at face
value. So that, you know, if you like talking to a lot of people from very different backgrounds,
great role for you. And one other thing to mention, you know, as they need to build all
these models, they need to be robust is they also need to maintain them. So they'll need to be,
you know, monitoring their data warehouse, optimizing their queries for maximum performance,
especially, you know, when you're working at large scale, even the small query optimizations
can really make a big impact on how fast your data pipelines are able to run and how fast you're
able to provide new reports on fresh data. And so to help ease this process as well, and also
make it easier for end users to understand, you know, what the data being presented is,
they have to maintain a really large emphasis on data documentation. So making sure, hey,
we're detailing how all these data models work, interpreting just a raw SQL script or a dbt model
into kind of human readable, you know, like a paragraph describing what it's actually doing.
Trans in the same with any other transformations, or, you know, any kind of reports,
making sure that's all extensively documented so that, you know, data doesn't get misinterpreted.
Or if, you know, someone leaves the company or someone, you know, someone needs a certain role,
you don't have to spend a ton of time retraining them to understand, you know, some spaghetti code
that your old data engineer wrote. So yeah, super interesting stuff there. And I know companies
actually where you can't actually get fired, and you just keep getting promoted and you'll match
any salary offer, because no one knows how to read your code or understands how it works. So
a little bit of a perverse incentive there. But you also will likely get fired before that
actually happens nowadays, because people are a little more aware. So now we've talked about,
you know, hey, what are the tools you're using? What is your day to day life look like,
I want to just kind of wrap it up with a little bit of a roadmap on how you can become a data
analytics engineer. So if this role really spoke to you, if you're really inspired by watching this
video, I want to kind of give you a little bit of a roadmap for how you might be able to become a
data analytics engineer yourself. And so this is just another graphic I've kind of found online
that I think, well, it's really ugly. And sorry to whoever created this. But yeah, I don't know,
maybe he's using Ms paint. Can't talk though, because I didn't create one myself. So I just
thought this was really accurate and useful, where this is just kind of a guideline, if you're
starting from scratch, you don't know what you need to know to actually become a analytics engineer,
I thought I would just kind of give you quick little guidelines, or just like the basic skills
you need to know, and maybe a suggested way, provided this guy of how to learn them, because
I think this learning path does kind of make sense in terms of, you know, continually building on top
of each concept. So number one, basic SQL, and I would also amend this to saying, you know, basic
Python, you're not going to get far. If you don't know Python, it's just it's even for interacting
with databases from external sources, like, you'll need to typically wrap a SQL query in Python,
if you're using within a script. So definitely need to have basic understanding of SQL and Python,
then you'll likely want to get into data modeling, understanding kind of, you know,
data warehousing concepts, getting hands on experience, you know, start a snowflake free
trial, run through some of the quick starts on there, figure out whatever databases the companies
you want to work for use, it's likely more public than you think. And then go try and use, you know,
go provision a free trial, almost every, you know, cloud service has a free trial that you can use,
because it's a great demand generation opportunity. And then once you've gotten kind of understanding
of, you know, you've got your basic SQL down, you've got your understanding how data warehouses
work, how to connect to them, how to manipulate data within them, that's where you can start
bringing in things like dbt, where, you know, you can understand, hey, you know, how do I structure
these SQL queries and create kind of scripts of SQL queries, create macros, do more efficient SQL
work, because that's really what dbt is, at the end of the day, it's, you know, scripts of SQL
queries that are run on a regular basis. And that's, you know, data model. So it's a data model
is just a set of transformations that you're doing in order on a regular basis on a set of
data. So, you know, this would be transforming a raw data file that you're getting every day from,
you know, point of sale system to make a human readable. And then you'll likely want to get some
knowledge of the cloud, learn some more Python for specifically for data. So getting really
experienced with, you know, someone like the matplotlib, or some of the different libraries
for Python, you know, obviously pandas, but other libraries that work with pandas, to do kind of
analytics on top of it. And also throughout this, if you can get practical experience, definitely
do it. But I know it's kind of hard to in that chicken to the egg problem of going from zero to
one, from no experience into analytics engineering job. So you might want to go into a data analyst
job role first, or straight into data engineering, maybe you're fully in one of those. But always try
to get any experience you can. And if you can't get experience, get a GitHub and start creating
some sick projects on there that you can use. And also in tandem with this, if you can do networking,
if you can, you know, obviously get out there, go to events, talk to other analytics engineers,
and see if you can get a job at their companies, that is always very helpful as well, obviously,
don't be too greedy, or, you know, needy. And, you know, this is just all kinds of stuff you'll
want to do as you're learning these skills. So learning how data is used by the business. So I
kind of gave you an intro there, but go deeper on, you know, maybe do a simulation where you're
trying to analyze some data about a company, you know, and then generate a dashboard, but do it in
a way where, hey, you're pulling some API every day, using dbt to model the transformations,
and then serving it up in Tableau, and storing it in a back end database. And so you kind of do that
as you learn that, learn some of those data engineering practices, learn how to create data
pipelines, get experience with things like airflow. So you can create, you know, data pipelines that
are scalable, rerunable. And then, you know, in tandem with that, learn some more advanced SQL
dbt, learning how to start to optimize your queries for analytics. And just keep learning.
There's always new services you can use, you know, there's always new platforms, there's always new
things coming out. So just constantly staying up to date. And, you know, a great way to get a job
is just to become the expert on some fancy new platform that no one else knows. And then you,
you know, find a couple companies that use that, and boom, you're in. So yeah, that is kind of all
I have for you today. So I hope this kind of helps you understand, you know, what analytics
engineer is and their role in, you know, shaping the data landscape for an organization, creating
ready, clean data for analysis. And, you know, just potentially being a really impactful person
on the team. So I hope this helps you if you are interested in becoming an analytics engineer.
If it did, please like subscribe. If it didn't, please let me know a video that you'd like to see
or another topic you'd like to see explored. And with that data guy out, peace.

