Technical overview of what makes Snowflake the best data platform out there — Executive Summary If I had to summarize this article in just a few words… Snowflake is just a great productivity enabler. It increases our productive work, reduces the time waiting around, and makes performance equally available to everyone instead of limited shared asset. At the end of the day, Data Engineers and…

After years of using Snowflake, I was “forced” to use Redshift Serverless with proper big data. What did I find? — Introduction If you happen to follow my blog, you will have noticed how much I write about Snowflake. …

If you ever get fed up with queries that run for hours… this one’s for you! — I started tuning SQL queries very early in my career, from ETL scripts to business user queries, you name it. There was something highly rewarding about taking a SQL query runtime from 10 minutes to 30 seconds, and deep diving into how the query optimiser worked, understanding its core query…

A Distinct is just like a group by (*)… except when it’s not! — Out of all SQL commands, Distinct is one of the easiest to understand conceptually: for a given set of rows, return only the unique ones. Simple right? Think again… Though useful in many situations where true duplication exists (like messages in streams), it is very often misused because users don’t…

Joins are databases’ “bread and butter”, but can you actually have “too many” joins? TLDR: yes! — Joining tables is one of the most common things to do on a database. Sometimes though, you may need to outer join a long list of large tables which may lead to sub-optimal (slow) execution plans. A common example is where you need to join multiple large tables using common…

Every time I see a “DISTINCT” I ask the same question: Why?? — Humans are funny, we tend to do all sorts of assumptions. One assumption I see again and again, is the concept that “DISTINCT” is some kind of black magic keyword which resolves all your “duplication” issues. What if I told you you’re probably wrong? …

If you ever got fed up with queries that run for hours… this one is for you! — I started tuning SQL queries very early in my career, being that ETL scripts or business user queries. There was something highly rewarding about taking a SQL query runtime from 10 minutes to 30 seconds, and deep diving into how the database worked, understanding its core query planner behaviours and…

You probably use window functions already… but have you taken them this far? — Window functions are (in my opinion) one of the best inventions in SQL language. Why, might you ask? Using the SQL core language, without window functions, all you can do is compare columns with columns. Yes, it is true you can join tables to themselves, but sequencing rows is still…

Snowflake is just about to release something that looks like nothing, but is kind of awesome! Watch out for Bundle 2021_10. — Data lineage is a recurring theme in all my projects, and rightly so! When you’re building long and complex ELT transformations, you want to have a degree of documentation on it, and the teams will also need to understand the downstream impact of changes. The problem though, is that any…