CODEWITHSUNDEEP
dbt
Michel Hua
Michel Hua

Reputation: 1777

dbt two argument ref

How do I refer to relative path from view1 to view2 using ref('package_name', 'model_name') ?

/root_folder
   / project1
     /models
       view1.sql
     dbt_project.yml

   / project2
     /models
       view2.sql
     dbt_project.yml

There is no code example in the documentation.

Thank you.

Upvotes: 0

Views: 4551

Answers (2)

sgdata
sgdata

Reputation: 2763

To try and answer the question with a little more focus on your comment:

I have one folder for each dataset I have on bigquery. Can I write models for several datasets in one single folder?

Yes you can!

Quick note on terms from the dbt docs "BigQuery configurations" in case you are not using bigquery.

  • schema is interchangeable with the BigQuery concept dataset
  • database is interchangeable with the BigQuery concept of project

Here is how this works for me:

project-dir
     | analysis
     | data
     | macros
     | models
         |> sources
              - dataset1.yml
              - dataset2.yml
     | seed
     | dbt_project.yml
     | packages.yml

Where the contents of a dataset.yml is:

version: 2

sources:
  - name: fivetran_log
    database: my-bigquery-project-id
    loader: fivetran
    
    tables:
      - name: account
      - name: log
      - name: user

No references are required within the dbt_project.yml to utilize these sources immediately. Instead, you can reference this directly from models like:

select * 
from {{ source('fivetran_log', 'user') }}

That should allow you to have multiple dataset sources, but one single dbt project directory for all your views.

However, if the datasets you are referencing are within different bigquery regions or different billing projects, I believe you will run into some errors.

Appendix of related questions / resources across the dbt-verse:

Upvotes: 1

Claus Herther
Claus Herther

Reputation: 356

The only way for project2 to know about models in project1 is if project2 included project1 as a package in its packages.yml file. Then you could refer to view1 as ref('project1', 'view1') in project2.

You'll have to check the syntax here, but you could include project1 in project2 in packages.yml like so:

in project2\packages.yml:

packages:
    - local: ../project1

Needless to say, you'd save yourself a lot of headaches by simply not splitting projects. In most cases you shouldn't need to do that and simple folderization does most of what you might need.

Upvotes: 5

Related Questions