Connecting Data Sources

​

How Orion connects

​

Choose your data source

BigQuery

Snowflake

Databricks

Looker

PostgreSQL

MySQL

Redshift

Amazon Athena

Microsoft Fabric

Delta Lake

MotherDuck

dbt

News API

Weather

​

BigQuery

1

Create a service account

1. Go to Google Cloud Console → IAM & Admin → Service Accounts
2. Click Create Service Account
3. Name: external-orion-data-access (or your preferred naming)
4. Description: “Service account for Orion data access”
5. Click Create and Continue
6. Skip role assignment for now → click Done

We’ll refer to this as the BigQuery Service Account going forward.

2

Enable service account impersonation

Orion queries BigQuery by impersonating the service account you just created. To allow that, grant Orion’s own service account the Service Account Token Creator role on it.Orion shows you the address to use. Go to Configuration → Data Sources, click Add Data Source, and choose BigQuery. The Orion Service Account sits under the Service Account Email field, with a button to copy it.Then, in Google Cloud Console:

1. Navigate to IAM & Admin → Service Accounts
2. Find the BigQuery Service Account you just created
3. Select the Principals With Access tab and click Grant Access
4. Add principal: the Orion Service Account address you copied
5. Assign role: Service Account Token Creator
6. Click Save

3

Create a scratch dataset and grant edit access

1. Navigate to BigQuery
2. Create a new dataset, a dedicated dataset reserved for Orion’s use (we recommend a descriptive name like orion_scratch)
3. Grant the BigQuery Service Account Data Editor access to this dataset

The scratch dataset must be in the same region (or multi-region) as the datasets you wish to query via Orion.

Orion uses this dataset to efficiently stream query results to a binary format optimized for quick analysis. Temp tables created here are automatically cleaned up.

4

Grant table-level access

In BigQuery, for each dataset or table you want to share:

1. Select the dataset → click Share Dataset
2. Add the BigQuery Service Account email
3. Assign role: BigQuery Data Viewer
4. Click Add, then Done

5

Grant project-level access

1. Navigate to IAM
2. Locate your BigQuery Service Account and click Edit
3. Assign these roles:
   • BigQuery Job User
   • BigQuery Connection User
   • BigQuery Read Session User
   • BigQuery Resource Viewer
4. Click Save

These roles let Orion run queries, use BigQuery connections, stream results, and read project query history. Query history powers Orion’s usage intelligence. BigQuery Resource Viewer provides bigquery.jobs.listAll, while BigQuery Job User provides bigquery.jobs.create.

Grant BigQuery Resource Viewer on every project whose query history Orion collects. A basic connection test can pass without this role because it does not read project-wide job history. Regular queries can therefore work while usage intelligence remains unavailable.

6

Enter the connection in Orion

In Orion, go to Configuration → Data Sources, click Add Data Source, and choose BigQuery. The form asks for exactly what the steps above produced:

Field	From
Project ID	The GCP project that runs Orion’s queries: where the service account, scratch dataset, and query jobs live
Service Account Email	The BigQuery Service Account from step 1
Scratch Dataset	The dataset from step 3
Data Project ID (optional)	The GCP project where your source datasets live, if it differs from Project ID. Leave blank when everything is in one project
Location	The GCP location of your datasets (defaults to the US multi-region)

The Orion Service Account shown on this form is Orion’s own address, the one you granted access to in step 2. You do not enter it anywhere. Your instance has its own address, so copy it from your own screen rather than from the picture above.No credentials need to be shared.

One project or two? Most customers keep the service account, scratch dataset, and source data in a single project and leave Data Project ID blank. Fill it in only when your datasets live in a different project from the one running Orion’s queries. See Querying datasets in another project.

​

Querying datasets in another project

• Project ID is the compute and billing project. The service account, the scratch dataset, and every query job live here. Grant BigQuery Job User, BigQuery Connection User, BigQuery Read Session User, and BigQuery Resource Viewer on this project (step 5), plus Data Editor on the scratch dataset (step 3).
• Data Project ID is where your source datasets live. Grant the service account BigQuery Data Viewer on those datasets (step 4).

​

Usage intelligence permissions

1. Grant the missing roles to the BigQuery Service Account shown in Orion.
2. In Orion, open the data source’s Refresh Data tab.
3. Select Full Resync to retry immediately. Orion will also retry during the next nightly refresh.

​

Per-user OAuth (optional)

• Scheduled work runs as its creator. A metric or workflow that someone schedules runs under that person’s connected account. Work with no individual owner uses the service account.
• If a user disconnects or loses BigQuery access, their scheduled work pauses until they reconnect. This is deliberate: Orion never falls back to broader access than the user has.

​

Snowflake

GRANT USAGE ON WAREHOUSE <w> TO ROLE <r>;
GRANT USAGE ON DATABASE <db> TO ROLE <r>;
GRANT USAGE ON SCHEMA <db>.<schema> TO ROLE <r>;
GRANT SELECT ON ALL TABLES IN SCHEMA <db>.<schema> TO ROLE <r>;

# Encrypted private key (an optional passphrase is recommended)
openssl genrsa 2048 | openssl pkcs8 -topk8 -v2 des3 -inform PEM -out rsa_key.p8
# Associated public key
openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub

CREATE USER orion_svc
  DEFAULT_ROLE = <r>
  MUST_CHANGE_PASSWORD = FALSE;

ALTER USER orion_svc SET RSA_PUBLIC_KEY='<contents of rsa_key.pub>';

​

Databricks

​

Looker

• Models (docs): can be scoped to necessary models via model sets; includes the explore permission
• see_system_activity, see_lookml, see_sql
• see_user_dashboards (or provide PDFs of sample dashboards instead)
• create_custom_fields: enables building custom fields via + Add
• login_special_email: only if using non-email (third-party) authentication

​

LookML augmentation (optional)

1

Open the Looker datasource

On the Configuration → Data Sources page, open the Looker datasource and select the Augmentation tab.

2

Start token creation

Click the Create token link to open GitHub.

3

Create a fine-grained PAT

On GitHub, begin creating a fine-grained personal access token.

4

Set the resource owner and expiration

• Resource owner: select the organization or account where the Looker GitHub repository lives.
• Expiration (TTL): set the token expiration. We recommend the longest your security policy allows; GitHub’s maximum for fine-grained tokens is 1 year (366 days).

When this token expires, Orion can no longer pull your LookML repo. Schema sync and the nightly enrichment that reads your LookML will fail with an authentication error, and any insights that depend on that business logic go stale until you issue a new token and update the connection. Set a reminder to rotate the token before it expires.

5

Select the repository

Choose the Looker repository from the dropdown.

6

Set permissions

Grant read-only access, nothing more:

• Contents: Read-only
• Metadata: Read-only

7

Create the token

Generate the token and copy it. You will not be able to view it again.

8

Save the configuration

Back in Orion, enter the repository URL, the PAT, and the rest of the configuration (see the Augmentation tab in Step 2), then save.

9

Grant the token's account access to the repository

The account that owns the token must have at least Read access to the LookML repo, or Orion authenticates but cannot see the repo and the connection fails with “repository not found.”A repository admin:

1. In GitHub, open the LookML repository → Settings → Collaborators and teams (under Access).
2. Under Manage access, click Add people (or Add teams if you manage access by team).
3. Enter the username of the account tied to the token, or the service account created for Orion.
4. Set the role to Read. No write or admin access is required.
5. Send the invitation.
6. The account owner accepts the invite. Until they do, access does not take effect and the connection will still fail.
7. Return to the Orion connection and reconnect.

Use a shared service account rather than an individual’s personal account. If a person leaves or loses access, a personal-account token breaks the connection.

10

Test the connection

On the Augmentation tab, click Test Connection. This verifies that Orion can reach the repository with the token and branch you configured, without running a full sync.

• Success: the connection is valid. Orion can authenticate and see the repo. Click Sync Now to pull the LookML files. When it finishes, Status shows Synced, along with the file count and the latest commit.
• Failure: read the error message:
  • “Repository or branch not found” usually means the token’s account lacks Read access to the repo (revisit Step 9), or the branch name is wrong. Leave Branch blank to use the repo default, or enter the correct branch.
  • “Authentication failed” means the token is invalid or expired. Generate a new one (Steps 3-7) and re-enter it.
  • “Access forbidden” means the token is missing a permission or, if your org enforces SSO, has not been SSO-authorized for the organization.

​

Migrating dashboards into Orion (optional)

1

Open the Migration tab

Go to Configuration → Data Sources, open the Looker data source, and select the Migration tab. Then click Migrate to Orion.

2

Pick the Target Database

Choose the Target Database, then click Choose Dashboards. The button stays disabled until a target is set and Looker reads as Connected.

3

Select the dashboards

Orion reads your Looker usage history, proposes dashboards to migrate, and groups them into projects. Each card shows the title, tile count, user count, and query count. Every dashboard starts selected: clear what you do not want, and confirm the selection.Paste a Looker dashboard URL into Add Dashboard by URL to include one the list missed.

​

PostgreSQL

GRANT CONNECT ON DATABASE <db> TO <user>;
GRANT USAGE ON SCHEMA <schema> TO <user>;
GRANT SELECT ON ALL TABLES IN SCHEMA <schema> TO <user>;
-- So future tables are visible automatically:
ALTER DEFAULT PRIVILEGES IN SCHEMA <schema> GRANT SELECT ON TABLES TO <user>;

CREATE USER orion_svc WITH PASSWORD '<secure_password>';

host    <database> orion_svc <ip_range> md5
hostssl <database> orion_svc <ip_range> md5   # SSL-only

​

MySQL

-- Read-only access to all current and future tables/views in <db>
GRANT SELECT, SHOW VIEW ON <db>.* TO '<user>'@'<host_or_%>';
FLUSH PRIVILEGES;

CREATE USER 'orion_svc'@'<host_or_%>' IDENTIFIED BY '<secure_password>';

[mysqld]
bind-address = 0.0.0.0
# For SSL:
require_secure_transport = ON
ssl-ca   = /path/to/ca.pem
ssl-cert = /path/to/server-cert.pem
ssl-key  = /path/to/server-key.pem

​

Redshift

CREATE USER orion PASSWORD '<choose-a-strong-password>';
-- For each schema you want Orion to access:
GRANT USAGE ON SCHEMA <schema_name> TO orion;
GRANT SELECT ON ALL TABLES IN SCHEMA <schema_name> TO orion;
-- So future tables are visible automatically:
ALTER DEFAULT PRIVILEGES IN SCHEMA <schema_name> GRANT SELECT ON TABLES TO orion;

​

Amazon Athena

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AthenaQueries",
      "Effect": "Allow",
      "Action": [
        "athena:StartQueryExecution",
        "athena:StopQueryExecution",
        "athena:GetQueryExecution",
        "athena:GetQueryResults",
        "athena:GetWorkGroup",
        "athena:ListWorkGroups",
        "athena:ListQueryExecutions",
        "athena:BatchGetQueryExecution"
      ],
      "Resource": "*"
    },
    {
      "Sid": "GlueCatalogReadOnly",
      "Effect": "Allow",
      "Action": [
        "glue:GetDatabase",
        "glue:GetDatabases",
        "glue:GetTable",
        "glue:GetTables",
        "glue:GetPartition",
        "glue:GetPartitions"
      ],
      "Resource": "*"
    },
    {
      "Sid": "S3ReadData",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": [
        "arn:aws:s3:::my-data-bucket",
        "arn:aws:s3:::my-data-bucket/*"
      ]
    },
    {
      "Sid": "S3ReadWriteResults",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": [
        "arn:aws:s3:::my-bucket",
        "arn:aws:s3:::my-bucket/athena-results/*"
      ]
    }
  ]
}

• IAM access key only. Cross-account IAM role assumption is a planned fast-follow; today the connection uses a long-lived access key, so rotate it on your normal cadence.
• Output location is mandatory. Athena cannot run a query without a place to write results, so the workgroup must enforce one or you must supply the S3 Output Location.
• dbt enrichment is supported on Athena connections (see dbt); semantic enrichment from a Looker connection can also be projected onto Athena schemas.

​

Microsoft Fabric

1. Create an App Registration in your Entra ID tenant (docs)
2. Note the Application (client) ID and Directory (tenant) ID
3. Create a Client Secret under Certificates & secrets (docs) and note the secret value (not the Secret ID). Recommended expiry: 12 months

1. Navigate to Fabric Admin Portal → Tenant settings → Developer settings
2. Enable Service principals can use Fabric APIs
3. Scope to a security group containing the Orion Service Principal (recommended), or enable for the entire organization

​

Delta Lake

​

MotherDuck

​

dbt

​

News API

​

Weather