New this month: Data champions, a key analyst report & features you’re going to want to check out!

Thank you, thank you, thank you! We want to start off this month’s post with an extreme sense of gratitude for the work our customers and partners are doing to innovate with Google Cloud’s Smart Analytics Platform. Your success and enthusiasm was felt in many ways this past month. BigQuery Omni was shortlisted by CRN asone of the10 hottest Google Cloud Tools to watch in 2021 and Google BigQuery was named aLeader in The Forrester Wave™: Cloud Data Warehouse, Q1 2021 report.

We were also thrilled to see Forrester give BigQuery a score of 5 out of 5 across 19 different criteria, including Data Lake Integration, Data Ingestion, High Availability, Performance, and ML/Data Science features. We hope you’ll share this free report with your teams and colleagues!

If you can’t wait for May, we suggest joining us this week to learn about Rackspace’s data modernization journey at the company’s Strategies Series event. Juan Riojas, the firm’s Chief Data Officer, will explain how Rackspace leveraged Google Cloud solutions to improve customer experience, reduce churn, and save $1.2 million by consolidating 74 operational data sources, four data warehouses, one data lake, and 12 years of on-premises history.

Leaders come in all sizes

In last month’s data analytics post, you read about Verizon Media’s journey and how they picked BigQuery for scale, performance and cost. In March, we shared how companies like Veolia, Vodafone and PwC innovate with data using Google Cloud’s Smart Analytics Platform. 

We’re delighted to see companies of all sizes benefit from the value of our platform. Take a look at the story of leading decentralized video streaming platform Theta Labs, for instance, which required scalable, reliable, and secure infrastructure to reach more users and avoid hitting VM caps that previously caused issues. They turned to Google Cloud’s databases and analytics products, such as BigQuery, Dataflow, Pub/Sub and Firestore, saving hours or even days of engineering time. And they experienced a return on investment almost immediately—their migration took less than six months to complete. 

Another great example is Keap, a technology and services company that helps small businesses accelerate growth with an all-in-one customer relationship management (CRM), sales, and marketing automation solution. Rajesh Bhatia, the company’s Chief Technology Officer, wrote a great post about how moving to Google Cloud helped Keap overcome scalability issues and bring always-on, mission-critical services to its users while freeing up time and resources. The company assessed other public cloud providers but Google Cloud stood out for going above and beyond other providers in helping us identify our path to successful migration.

In addition to providing great details behind Keap’s technological modernization, Rajesh also gives us a peek into the cultural benefits experienced as part of its transformation. For instance, you’ll learn how their data team transformed from a cost center into a team focused on revenue-generating opportunities by providing embedded analytics to customers.

Keap helps small business customers think big and we are proud to be part of their journey!

Leaders come in all industries

We’re also proud to observe the rapid pace at which customers operate their digital transformation across many industries. For example, the telecommunication industry is experiencing ongoing, massive transformation. According to a new study by Analysys Mason, telecommunications information is growing at 20% CAGR, and network data traffic is expected to reach 13 zettabytes by 2025. During March, we learned how our partnership with Amdocs is helping telecommunications leaders use analytics and AI to make decisions in real time, whether to find anomalies, understand unseen correlations, or predict future trends.

We see the same thing happening in financial services, online business, and retailers, where businesses are built on reliable and trusted relationships. Quantiphi, an award-winning Google Cloud Premier Partner, has recognized the importance of helping their clients build and maintain real-time prediction models at scale to address business challenges like credit card fraud. Global losses from payment fraud have tripled in the past 10 years, and according to Merchant Savvy, payment fraud is expected to continue increasing with a projected cost of $40.62 billion in 2027—25% higher than in 2020.

That’s why we developed a smart analytics design pattern, together with Quantiphi, enabling companies to build a scalable real-time fraud detection solution in just one hour using serverless, no-ops products from Google Cloud. The solution leverages Dataflow’s real-time data processing capabilities to store transactions in Firestore, our flexible, scalable and serverless NoSQL cloud database. It also combines with BigQuery, our extensible and adaptable data warehouse, allowing you to leverage BigQuery ML’s built-in machine learning capabilities and the AI Platform’s real-time inference.

These resources are available to you and your team at no cost to set up fraud notifications. You can watch the tutorial below to learn how to build the solution as well as dashboards to monitor the performance of the entire fraud detection pipeline. 

You can also play with this interactive Data Studio dashboard to see the result of our analysis using the sample data used as part of this solution.

Finally, if you are currently a BigQuery reservation customer and looking to run real-time monitoring and troubleshooting of your BigQuery environments, you’ll be pleased to hear that our team just released Resource Charts for BigQuery Administrator. 

Resources Charts provide a native, out-of-the-box experience, making it easy to understand historical patterns across slot consumption, job concurrency, and job performance and take action to ensure your BigQuery environment continues running smoothly. And guess what? We also have public Data Studio dashboard templates available so you can see how this functionality might look at your company. 

Whether you’re in financial services, telecommunications, retail, manufacturing, or any other industry, we offer a plethora of free and self-serve solutions to help you tackle various other business issues beyond real-time credit card fraud detection. The most popular include predicting customer lifetime value, determining propensity to purchase, building product recommendation systems, solving anomaly detection, and demand forecasting.

Data Champions: Speed is the name of the game!

And if all of this is not cool enough, allow us to share our favorite story from this month! The Golden State Warriors (GSW) share how they used BigQuery to reduce their data integration times from multiple days to less than an hour, enabling analysts to explore more data, build accessible knowledge, and promote a more effective environment to support analytical ideation and hypothesis testing. 

Starting with data ingestion from an alternative cloud into BigQuery, the Golden State Warriors’ strategy team achieved phenomenal results with a combination of open source technology and Google Cloud’s fully-managed services:

• The team parallelized data ingestion at breakneck speed with Apache Beam, a parallel processing tool, and Cloud Dataflow, Google Cloud’s fully-managed service for stream and batch data processing.

• Using Cloud Composer, Google Cloud’s fully managed workflow orchestration tool built on Apache Airflow, the team built out fully integrated, continuously updating data pipelines. This allowed them to bring more than a dozen different data sources into their BigQuery data warehouse, while also building out long-term storage within Cloud Storage and logging exports with Cloud Pub/Sub.

Watch the video below to see key “Data Champions” in GSW, including Head Coach Steve Kerr, chime in on the importance of data and how their team uses intelligent technologies to better serve the needs of coaches, front office, staff, players and fans. 

We smiled when Mike Brown, assistant coach, looked at the camera and asked: “BigQuery, what is it?” and further answers his question by saying “it’s faster than a falcon”. The Golden State Warriors (GSW) is a great example of an organization that moves fast in an industry where speed is the name of the game!

It’s delightful to see the Golden State Warriors and many others validate our approach and our commitment to an open ecosystem. 

Other notable March features and announcements

We’re also excited to announce the release of an open source connector to read streams of messages from Pub/Sub Lite into Apache Spark. For those of you who aren’t familiar, Pub/Sub Lite is a scalable, managed messaging service for Spark users on Google Cloud Platform looking for an exceptionally low-cost ingestion solution. 

This connector allows you to use Pub/Sub Lite as a replayable source for Apache Spark’s Structured Streaming processing engine with exactly-once guarantees and ~100ms processing latencies. You can read more about it in our announcement blog here.

We also announced the general availability of Dataproc Metastore. A fully managed, serverless technical metadata repository based on the Apache Hive metastore. Enterprises building and migrating open source data lakes to Google Cloud now have a central and persistent metastore for their open source data analytics frameworks. This no hassle setup solution allows enterprises to migrate their open source metadata without having to worry about the overhead of setting up highly available architectures, backups, and performing maintenance tasks. 

Finally, if you’re wondering what Cloud Composer does and haven’t had time to research it, we recommend you watch the “Cloud Composer in a minute” video we just produced to guide you and your team.