I wish I had this article 2.5 years ago when I first started working with BigQuery. It would've prevented me from learning things the hard way and would've saved the company tens of thousands of $$ 🙄😅
Thank you so much for the article :) I was just curious when you said: "most of the calculations were On Demand", so why did you decide to use the on-demand model most of the time? Is there any obstacle with the BigQuery enterprise pricing model for your use cases?
Not really the pricing but the ease of development and probably lack of knowledge on best practices as it was mainly maintained by Analysts before I joined.
All queries were part of Looker and everytime someone ran a dashboard it would run the whole thing to fetch new data even persist in Looker doesn't help all the time. So pre computation helped as it served right away.
I wish I had this article 2.5 years ago when I first started working with BigQuery. It would've prevented me from learning things the hard way and would've saved the company tens of thousands of $$ 🙄😅
Thanks for the comment.
It actually happened the same time, 2021 October when I did the optimization, haha took a while to come up with this.
Seem like a lesson, the earlier you share the more people can benefit.
Thank you so much for the article :) I was just curious when you said: "most of the calculations were On Demand", so why did you decide to use the on-demand model most of the time? Is there any obstacle with the BigQuery enterprise pricing model for your use cases?
Not really the pricing but the ease of development and probably lack of knowledge on best practices as it was mainly maintained by Analysts before I joined.
All queries were part of Looker and everytime someone ran a dashboard it would run the whole thing to fetch new data even persist in Looker doesn't help all the time. So pre computation helped as it served right away.
Now imagine you’d be getting a cut of the savings. How many more opportunities for optimization people would suddenly spot :)