Source
yaml
id: dbt-bigquery
namespace: company.team
tasks:
- id: git
type: io.kestra.plugin.core.flow.WorkingDirectory
tasks:
- id: clone_repository
type: io.kestra.plugin.git.Clone
url: https://github.com/kestra-io/dbt-example
branch: main
- id: dbt
type: io.kestra.plugin.dbt.cli.DbtCLI
inputFiles:
sa.json: "{{ secret('GCP_CREDS') }}"
taskRunner:
type: io.kestra.plugin.scripts.runner.docker.Docker
containerImage: ghcr.io/kestra-io/dbt-bigquery:latest
profiles: |
my_dbt_project:
outputs:
dev:
type: bigquery
dataset: your_big_query_dataset_name
project: your_big_query_project
keyfile: sa.json
location: EU
method: service-account
priority: interactive
threads: 16
timeout_seconds: 300
fixed_retries: 1
target: dev
commands:
- dbt deps
- dbt build
About this blueprint
dbt Git GCP
This flow:
- clones a dbt Git repository from GitHub,
- pulls a public container image with the required dependencies
- runs dbt CLI commands within a container.
This flow assumes that your Google Cloud Service Account JSON key file's
content is stored as a secret named GCP_CREDS.
Before you run this blueprint, simply copy-paste the contents of your
~/.dbt/profiles.yml into the profiles section and you're good to go!