5 of Our Favourite Data Engineering Tools

If you are a loyal reader of our blog, you've probably already learned about the tools and good practices we've mentioned in our Data Engineering Fundamentals post. Those tips apply to anyone interested in starting a career in data: it doesn't matter if you want to be a Data Engineer, Data Scientist, Machine Learning Engineer, or something completely new and different!

However, if you've clicked on this post, chances are you're interested in Data Engineering — the craft behind every reliable data platform, every ML model in production, and every Gen AI application that actually works outside a demo.

A lot has changed since we first published this list in 2022. AI workloads went from "nice to have" to the reason most data platforms get built. But here's the thing: the fundamentals held up. The top data engineering tools we loved back then are still the backbone of the modern data stack — they've just gotten better. So we updated this post with what changed, what we'd pick today, and why these tools matter even more now that your pipelines feed AI agents and not just dashboards.

Without further ado, here are five data engineering tools our experts love using. We chose them because they allowed us to solve incredibly challenging problems, scale our solutions to terabytes of data, or just because they made our daily lives easier and better.

1. dbt

What is it?

Let's refresh the terms ETL and ELT that we previously covered in The Modern Data Stack. You might be familiar with Extract, Transform and Load pipelines, where you store what you've computed during the transform stage. ETLs are still relevant and useful, but modern data stacks have firmly settled on Extract, Load and Transform. Storage became a commodity thanks to cloud platforms, so it's cheap to load everything you extract and transform it afterwards — if only there were a quick, testable way to write those transformations in a language every engineer and analyst already knows. Enter dbt.

dbt (data build tool) is the T in ELT. It transforms data that's already been loaded using a trusty friend we all know too well: SQL, combined with Jinja templating. Anyone who knows SQL can build, test, and document data pipelines.

What changed since 2022: dbt grew up. The new dbt Fusion engine brought a ground-up rewrite focused on speed and state-aware orchestration, and dbt is now as common in Databricks lakehouse projects as it is in warehouses. If your semantic layer feeds an AI/BI tool or a Gen AI application, odds are dbt is somewhere in the middle.

Why we love dbt

Data engineering's goal is to make data available and useful to people. dbt lets us democratize data: everyone can build insights, metrics, and more. We also love how it brings software engineering best practices to analytics — testing, reusable operations, and documentation come built-in.

2. Great Expectations (now GX)

What is it?

Great Expectations — rebranded as GX with GX Core 1.0 — helps data teams eliminate pipeline debt through data testing, documentation, and profiling. But wait, didn't dbt do all that already? Yes! But just as dbt is an expertly crafted tool for transformations, GX is an expertly crafted tool for data validation. It integrates seamlessly with dbt and orchestrators like Airflow.

You validate data with an expectation — an assertion on your data. Use the pre-made expectations from the core library, use the ones created by the community, or write your own. It doesn't matter if you're running an ETL or an ELT pipeline.

Why it matters more in 2026: data quality stopped being a hygiene metric and became an AI reliability problem. When your pipelines feed Machine Learning models and Gen AI systems, bad data doesn't just make a dashboard look off — it makes an AI agent confidently wrong. Validation at every stage is how you go from reactive to proactive.

Why we love it

We've all been through data not being properly processed in a pipeline: missing data, stale data, metrics looking a little off. GX lets us know before it happens instead of paging us after. Adding new validations is easy, and connecting it to dbt sealed the deal.

3. Airbyte

What is it?

Living in the ELT era means storage and compute are accessible — but more data sources means more heavy lifting for us Data Engineers. Manually building and maintaining connectors to every new source is nobody's favourite job.

Airbyte to the rescue! It handles the Extraction and Loading stages of an ELT (Airbyte is an EL(T) tool — the Transform stage belongs to dbt). It provides a standardized way of extracting data thanks to its connector catalog: hundreds of connectors maintained by Airbyte and the community, plus the option to build your own.

What changed since 2022: back then we called Airbyte "an incredibly young but promising tool." It's not young anymore — it's a mature piece of the data integration landscape, and its connector catalog grew enormously. For a consultancy like ours that touches a different source system every week, that catalog is worth its weight in gold.

Why we love Airbyte

It's helped us connect to tons of different data sources easily, and the community remains excellent. When a client's data lives in fifteen SaaS tools and three databases, Airbyte is usually the fastest path to a unified data platform.

4. Terraform

What is it?

Wait, isn't Terraform a DevOps tool? Yes! But all of the tools we've mentioned have to run somewhere, right?

We use Terraform to define our infrastructure as code. No more hand-crafted instances where the correct recipe of configurations lives in one person's head. Infrastructure becomes versioned, reusable, and shared across people and projects. We pair it with CI/CD pipelines, Kubernetes, and Flux to achieve GitOps — yes, data engineering CI/CD is very much a thing, and it's how we keep platforms healthy long after launch day.

What changed since 2022: Terraform's license change spawned OpenTofu, the open-source fork now under the Linux Foundation. Both work great; which one we reach for depends on the client's stack and licensing preferences. The infrastructure-as-code discipline is what matters.

Why we love Terraform

At Muttdata we start new projects all the time. Terraform gives us a repeatable way to stand up new infrastructure and introduce changes safely — key for starting projects and for maintaining their health in the long run.

5. And last but certainly not least... Airflow!

Of course we were going to mention Airflow! We've been using Apache Airflow since before v1, and it has played a key role in productionizing data pipelines of all shapes and sizes.

What changed since 2022: Airflow 3 landed — the biggest release in the project's history, with a modernized UI, DAG versioning, and first-class support for the event-driven and ML/AI workloads that dominate today's platforms. Orchestration is also where AI shows up first in the stack: the same DAGs that used to end in a dashboard now end in a feature store, an ML training job, or a Gen AI pipeline serving AI agents in production.

We're long-time Astronomer partners, and these days you'll just as often find our Airflow DAGs orchestrating workloads on Databricks — where we're proud to be Databricks' LATAM Enterprise AI Partner of the Year. Different engines, same principle: reliable orchestration is the difference between an AI demo and an AI product.

Interested in using these tools? We're hiring!

If you've made it this far, it's safe to say you're interested in Data Engineering. If any of these tools caught your attention, make sure to apply! We are all Data Nerds at Muttdata, and we'd love to hear from you.

We take technical growth seriously. Here's why we think Muttdata is a great place to take the next step in your data engineering career:

• Once you join, you'll go through our guided technical onboarding (the Mutt Academy!), where you'll have time to learn and try out most of these tools — custom-fit to your experience and interests.
• Each week we have Data Office Hours, where we talk about cool data technologies and topics in a relaxed environment.
• Wanna grow your tech skills? We're AWS Advanced Consulting Partners and Databricks partners — we'll cover the cost of your certifications and help you prepare.

Come join us!

References & reading material

• dbt website
• Airbyte website
• GX (Great Expectations) website
• Terraform website
• OpenTofu website
• Flux website
• Apache Airflow website
• Astronomer website