Why Google Cloud is the ideal platform for Block.one and other DLT companies
April 1, 2021
Analyze your GKE and GCE logging usage data easier with new dashboards
April 1, 2021
Why did I pick 10 GB?
One of the key ways that BI Engine speeds up queries on small tables is that it caches the tables in memory. It also does other speedups, but this is the main one. So, you want to provide it enough memory to comfortably hold the tables you will be querying. BI Engine will automatically manage the memory for you.
In my case, the Chicago taxicab query involves 3GB, so I used 10 GB. For the other two queries, 1 GB would have been enough. You can go up to 100 GB, something I’d recommend if you have many concurrent queries on small tables. You might be able to get away with less because BI Engine stores data compressed, but it’s better to provide some extra room.
You can turn on BI Engine from a script using the Reservations API. Note, however, that it takes a few minutes for the memory to become available, so this is something you should consider doing for a few hours at least, not on a per-query basis.
Turning on BI Engine
Here’s how the 3 queries compared with and without BI Engine:
