Optimize Query Runtimes and Save Costs with Cloud Data Warehousing Platforms: Snowflake, BigQuery, and Redshift
Add Snowflake, BigQuery, and Redshift to your Knowledge
As a passionate Teradata expert, I have fine-tuned thousands of SQL queries throughout my professional journey. It was an enjoyable and fulfilling experience to witness how, with expertise, query runtimes could be optimized from hours to mere minutes or seconds. While end-users were pleased with the results, it was challenging to quantify the cost savings resulting from such significant accomplishments.
If you know Teradata, learning Redshift, Snowflake, BigQuery, etc. is ridiculously easy.
We are currently in the era of cloud computing, with the data warehousing market being dominated by the three major players: Google, Amazon, and Microsoft. The exciting narrative of "No more performance issues, just scalability" is somewhat accurate. The marketing was also truthful: "We require no tuning, indexes, or additional components. We scale, and that's it."
Is everything too good to be true? I would say yes — if finances are not a concern, then it may be plausible.
Want more practical data engineering analysis like this?
Join DWHPro Letters and get field-tested notes on Teradata, Snowflake, AI, migrations, performance, and enterprise data work. DWHPro Letters is free. Subscribe to get new issues by email.
Using Snowflake as an example, I admire its remarkable attributes, primarily its zero-copy cloning ability.
Get the next issue by email.
Disillusionment arises due to costs. Need faster query results? Scale up and pay. Want greater concurrency? Create a new warehouse and pay for it.
Conventional tuning methods like clustering or partitioning and search optimization services (referred to as indexes in Snowflake marketing) are once again pertinent. Notably, the latter has an extra charge, similar to the current pricing methods employed by car manufacturers that provide monthly rentals for parking heater functionalities.
As a performance tuner, it is clear that new opportunities are emerging regardless of personal opinions. For instance, reducing the query runtime from 30 to 1 minute can save a company a significant amount if Snowflake charges per minute.
I will continue supporting Teradata throughout my career while prioritizing other technologies such as Snowflake, Redshift, and BigQuery. These alternatives present profitable opportunities in the current market, and I am eager to take on migration and optimization projects.
Planning or surviving an enterprise data platform migration?
I write regularly about the performance, cost, architecture, and project mistakes that show up in real Teradata, Snowflake, Databricks, and enterprise data work.
Subscribe for free and keep launch access.
Written by Roland Wenzlofsky, founder of DWHPro and author of Teradata Query Performance Tuning. DWHPro has helped data warehouse practitioners for 15+ years.