google_bigquery_icon.png

Google BigQuery is a serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data. Use built-in ML/AI and BI for insights at scale. BigQuery's serverless architecture lets you use SQL queries to analyze your data. You can store and analyze your data within BigQuery or use BigQuery to assess your data where it lives. 

You can integrate with Google BigQuery using either of the following methods:

Integration via Amazon S3

You can use the Google BigQuery Connector for AWS Glue to migrate data cross-cloud from Google BigQuery to Amazon Simple Storage Service (Amazon S3). Once you have the data in Amazon S3, you can use Blueshift’s direct integration with S3 to import your data into Blueshift.

You can import the following data into Blueshift:

Prerequisites

Before you can start with the integration, you must complete the following steps:

  • Create an account in Google Cloud, specifically a service account that has permissions to Google BigQuery
  • Create an AWS Identity and Access Management (IAM) user with an access key and secret key to configure the AWS Command Line Interface (AWS CLI)
  • The IAM user in S3 must also have permissions to create an IAM role and policies.

Set up data migration from Google BigQuery to Amazon S3

Follow the instructions provided in the AWS documentation for Migrating data from Google BigQuery to Amazon S3 using AWS Glue custom connectors.

The data migration involves the following steps:

  1. Configure your Google Account.
  2. Create an IAM role for AWS Glue.
  3. Subscribe to Glue BigQuery Connector.
  4. Create the ETL job in AWS Glue Studio.
  5. Add AWS Glue trigger for a scheduled run.

You can now set up a task in Blueshift to import data from Amazon S3 into Blueshift. Follow the instructions in the Blueshift documentation to import data into Blueshift.

Integration via the Census platform

Census is a Reverse ETL platform that makes it easy to connect your data warehouse into sales, marketing, and other customer facing tools that drive your business. Using Census, you can easily copy data from Google BigQuery into Blueshift.

Using Census you can import the following data into Blueshift:

To integrate with Blueshift via Census, complete the following steps:

  1. Set up a data warehouse and data model in Census for the data that you want to import into Blueshift. For more information, see the Census documentation for Setting up a Google BigQuery connection.
  2. Set up a connection to Blueshift as the destination in Census.
  3. Create a Sync in Census with Google BigQuery as the source and Blueshift as the destination.

Integration via the RudderStack platform

RudderStack is an open-source Customer Data Platform (CDP), providing data pipelines that allow you to easily collect data from every application, website, and SaaS platform to activate in your warehouse and business tools. Using RudderStack, you can easily copy data from Google BigQuery into Blueshift.

Using RudderStack you can import the following data into Blueshift:

To integrate with Blueshift via RudderStack, complete the following steps:

  1. Set up Google BigQuery as a Reverse ETL source in RudderStack.
  2. Set up Blueshift as a destination in RudderStack.
  3. Create a Connection in Rudderstack with Google BigQuery as the source and Blueshift as the destination.
Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.