|
1 | 1 | # target-bigquery
|
2 |
| -Singer target for Google BigQuery |
| 2 | + |
| 3 | +A [Singer](https://singer.io) target that writes data to Google |
| 4 | +BigQuery. |
| 5 | + |
| 6 | +## How to use it |
| 7 | + |
| 8 | +`target-bigquery` works together with any other [Singer Tap] to move |
| 9 | +data from sources like [Braintree], [Freshdesk] and [Hubspot] to |
| 10 | +Google BigQuery. |
| 11 | + |
| 12 | +### Step 1: Activate the Google BigQuery API |
| 13 | + |
| 14 | + (originally found in the [Google API |
| 15 | + docs](https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/usage.html)) |
| 16 | + |
| 17 | + 1. Use [this |
| 18 | + wizard](https://console.developers.google.com/start/api?id=bigquery-json.googleapis.com) |
| 19 | + to create or select a project in the Google Developers Console and |
| 20 | + activate the BigQuery API. Click Continue, then Go to credentials. |
| 21 | + |
| 22 | + 1. On the **Add credentials to your project** page, click the |
| 23 | + **Cancel** button. |
| 24 | + |
| 25 | + 1. At the top of the page, select the **OAuth consent screen** |
| 26 | + tab. Select an **Email address**, enter a **Product name** if not |
| 27 | + already set, and click the **Save** button. |
| 28 | + |
| 29 | + 1. Select the **Credentials** tab, click the **Create credentials** |
| 30 | + button and select **OAuth client ID**. |
| 31 | + |
| 32 | + 1. Select the application type **Other**, enter the name "Singer |
| 33 | + BigQuery Target", and click the **Create** button. |
| 34 | + |
| 35 | + 1. Click **OK** to dismiss the resulting dialog. |
| 36 | + |
| 37 | + 1. Click the Download button to the right of the client ID. |
| 38 | + |
| 39 | + 1. Move this file to your working directory and rename it |
| 40 | + *client_secret.json*. |
| 41 | + |
| 42 | +### Step 2: Configure |
| 43 | + |
| 44 | +Create a file called `config.json` in your working directory, |
| 45 | +following [config.sample.json](config.sample.json). The required |
| 46 | + parameters are the project name `project_id`, the dataset name |
| 47 | +`dataset_id`, and table name `table_id`. |
| 48 | + |
| 49 | +### Step 3: Install and Run |
| 50 | + |
| 51 | +First, make sure Python 3 is installed on your system or follow these |
| 52 | +installation instructions for [Mac](python-mac) or |
| 53 | +[Ubuntu](python-ubuntu). |
| 54 | + |
| 55 | +`target-bigquery` can be run with any [Singer Tap], but we'll use |
| 56 | +[`tap-fixerio`][Fixerio] - which pulls currency exchange rate data |
| 57 | +from a public data set - as an example. |
| 58 | + |
| 59 | +These commands will install `tap-fixerio` and `target-bigquery` with |
| 60 | +pip and then run them together, piping the output of `tap-fixerio` to |
| 61 | +`target-bigquery`: |
| 62 | + |
| 63 | + |
| 64 | +```bash |
| 65 | +› pip install target-bigquery tap-fixerio |
| 66 | +› tap-fixerio | target-bigquery -c config.json |
| 67 | + INFO Replicating the latest exchange rate data from fixer.io |
| 68 | + INFO Tap exiting normally |
| 69 | +``` |
| 70 | + |
| 71 | +If you're using a different Tap, substitute `tap-fixerio` in the final |
| 72 | +command above to the command used to run your Tap. |
| 73 | + |
| 74 | +`target-bigquery` will attempt to open a new window or tab in your |
| 75 | +default browser. If this fails, copy the URL from the console and |
| 76 | +manually open it in your browser. |
| 77 | + |
| 78 | +If you are not already logged into your Google account, you will be |
| 79 | +prompted to log in. If you are logged into multiple Google accounts, |
| 80 | +you will be asked to select one account to use for the |
| 81 | +authorization. Click the **Accept** button to allow `target-bigquery` to |
| 82 | +access your Google BigQuery table. You can close the tab after the signup flow |
| 83 | +is complete. |
| 84 | + |
| 85 | +The data will be written to the table specified in your `config.json`. |
| 86 | + |
| 87 | +--- |
| 88 | + |
| 89 | +Copyright © 2018 Grayson Williams |
| 90 | + |
| 91 | +[Singer Tap]: https://singer.io |
| 92 | +[Braintree]: https://github.com/singer-io/tap-braintree |
| 93 | +[Freshdesk]: https://github.com/singer-io/tap-freshdesk |
| 94 | +[Hubspot]: https://github.com/singer-io/tap-hubspot |
| 95 | +[Fixerio]: https://github.com/singer-io/tap-fixerio |
| 96 | +[python-mac]: http://docs.python-guide.org/en/latest/starting/install3/osx/ |
| 97 | +[python-ubuntu]: https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-local-programming-environment-on-ubuntu-16-04 |
| 98 | + |
0 commit comments