New dbt feature in Matillion

The customers I work with who are on the journey to establishing a modern cloud data platform often have the question of which tool to use for transformations.

Often this boils down to a decision between Matillion or dbt. Both products are excellent and do what they do very well. While there’s some overlap between the two, there are some key differences.

You can think of Matillion as a like-for-like replacement for an ETL tool you may already have, or have used in the past. Informatica and SSIS spring to mind initially. The big advantage is that Matillion is cloud native, built to leverage the scale of modern, cloud-based data platforms. This makes it easier to work with files in cloud storage locations such as S3, as well as working with semi-structured data more efficiently.

dbt only handles the ‘T’ in ELT, or ETL. So there needs to be a consideration of how to orchestrate your data pipelines until it reaches your data platform. dbt doesn’t have a UI, but it is primarily built on SQL, meaning it doesn’t require any specialist skills — which opens the door to data democratisation. It also has the ability to easily integrate unit testing and generate documentation including lineage.

I was aware a few weeks ago that Matillion were due to release an integration with dbt. It’ll be interesting to see if this adds more complexity for customers when considering which tools to go with. My view is that this move by Matillion opens up new markets when dbt may already be the incumbent tool for those customers, rather than a new customer looking to chose both. Time will tell.

Regardless of this, I thought you should know about it and if you want to know more, grab a coffee and check this short video out which explains a little more!