CODEWITHSUNDEEP
dbt
Teghan Nightengale
Teghan Nightengale

Reputation: 528

dbt pre-hooks cannot render/find macros?

A dbt_project.yml model pre-hook is failing to render macros. The following implementation:

# gold/dbt_project.yml

config-version: 2

...

models:
    gold:
        +pre-hooks: "{{ incremental_failsafe() }}"

And the following macro:

-- gold/macros/utils/incremental_failsafe.sql

{% macro incremental_failsafe() %}

{# /*
Used to wrap AND clauses that should only be run on dev/CI dbt runs.

For example, to limit the number of records scanned on an incremental
table. This macro will return True unless the target profile is 'prod':

`dbt run --target prod`
*/ #}

{% if target.name == "prod" %}
    {% set incremental_failsafe = False %}
{% else %}
    {% set incremental_failsafe = True %}
{% endif %}

{{ log("Running with incremental_failsafe: " ~ incremental_failsafe, info=True) }}
{{ return(incremental_failsafe) }}

{% endmacro %}

Results in the following:

(snowflake) Teghans-MacBook-Pro:gold tnightengale$ dbt compile
Running with dbt=0.17.2
Encountered an error:
Compilation Error
  Could not render {{ incremental_failsafe() }}: 'incremental_failsafe' is undefined

I know there was an issue with dbt 0.17.0 around pre-hooks as outlined in this slack exchange. I've tried using both +pre-hook: and pre-hook: specs in the yml as suggested in that conversation to no avail, despite the fact that I am running 0.17.2 (not 0.17.0). Any insight would be welcome!

EDIT: Solution

Truthfully, the answer is so dumb and obvious: the error resulted from having +pre-hooks: as opposed to +pre-hook:. However the suggested answer caused me to take a second look. I am marking it as correct because it gives a lot of helpful context on hooks in general. Cheers!

Upvotes: 3

Views: 7606

Answers (1)

sgdata
sgdata

Reputation: 2763

First impression - maybe you're mixing up where a model pre-hook for a particular model and a run pre-hook should go?

Edit: Correcting myself on the above - Asker is attempting to perform model config pre-hooks, not run pre-hooks. I now believe that you need to move your pre-hook to an equal, prior level as your namespace configuation. See below:

dbt_project.yml

# A run pre-hook that applies to all runs would go here:
on-run-start:
- "{{ example_pre_run_macro() }}"


models:
    # a pre-hooks to all models goes here *before* but equivalent depth to the namespace:
    +pre-hook: "{{ incremental_failsafe() }}"
 
    namespace:
        # Below configures models found in models/events/
        events:  
          +enabled: true
          +materialized: view
          # a pre-hook to a single model / model directory goes here:
          +pre-hook: "{{ example_pre_model_macro() }}"

A really great example of how all this can be setup is from GitLab's dbt repo: Gitlab dbt_project.yml

TLDR: Setup your dbt_project.yml like so

# gold/dbt_project.yml

config-version: 2

...

models:
    +pre-hook: "{{ incremental_failsafe() }}"
    gold:
        <models config here>

Upvotes: 1

Related Questions