CODEWITHSUNDEEP
graphqldagster
Atticus
Atticus

Reputation: 185

What is the value of $repositoryLocationName when running ExecutePipeline in Dagster's GraphQL API?

I am attempting to launch a Dagster pipeline run with the GraphQL API. I have Dagit running locally and a working pipeline that I can trigger via the playground.

However, I am now trying to trigger the pipeline via GraphQL Playground, available at /graphql.

I am using the following mutation:

mutation ExecutePipeline(
  $repositoryLocationName: String!
  $repositoryName: String!
  $pipelineName: String!
  $runConfigData: RunConfigData!
  $mode: String!
)

...and hence am providing the following query params:

{
  "repositoryName": "my_repo",
  "repositoryLocationName": <???>,
  "pipelineName": "my_pipeline",
  "mode": "dev",
  "runConfigData": {<MY_RUN_CONFIG>}
}

I am not sure what value repositoryLocationName should take? I have tried a few but receive the following error:

{
  "data": {
    "launchPipelineExecution": {
      "__typename": "PipelineNotFoundError"
    }
  }
}

This is the tutorial I am following.

Upvotes: 3

Views: 781

Answers (3)

Sairam Krish
Sairam Krish

Reputation: 11751

GraphQL API to fetch repository and location details :

query RepositoriesQuery {
  repositoriesOrError {
    ... on RepositoryConnection {
      nodes {
        name
        location {
          name
        }
      }
    }
  }
}

No parameters required for this. This should result in a response like this:

{
    "data": {
        "repositoriesOrError": {
            "nodes": [
                {
                    "name": "__repository__",
                    "location": {
                        "name": "data-pipeline-1"
                    }
                }
            ]
        }
    }
}

From above, we can form the repositoryName and locationName like :

{
  "repositoryName": "__repository__",
  "repositoryLocationName": "data-pipeline-1",
}

In the above example, it is the response from dagster cloud hosted instance. For dagster cloud hosted flow, repository is internal variable __repository__. For local or self hosted, we should observe the response and construct based on that dagster instance.

Upvotes: 0

g_uint
g_uint

Reputation: 2041

You can also find out using a GraphQL query. Starting from the example provided in the documentation, you just need to add

repositoryOrigin {
      repositoryLocationName
    }

resulting in

query PaginatedPipelineRuns {
  pipelineRunsOrError {
  __typename
     ... on PipelineRuns {
           results {
             runId
             pipelineName
             status
             runConfigYaml
             repositoryOrigin {
                repositoryLocationName
             }
          stats {
            ... on PipelineRunStatsSnapshot {
              startTime
              endTime
              stepsFailed
            }
          }
        }
      }
    }
  }

This will return the repository location name for any run that is returned. Trigger the pipeline that you want the location name for in the UI before querying and that run will be your first result.

Upvotes: 0

carte
carte

Reputation: 1083

Short answer:

Each repository lives inside a repository location. Dagster provides a default repository location name if you do not provide one yourself. To find the location name, you can click the repository picker in Dagit, and it'll be next to the repository name:

enter image description here

In this example, the repository name is toys_repository, and the location name is dagster_test.toys.repo

Longer answer:

A workspace (defined with your workspace.yaml) is a collection of repository locations.

There are currently three types of repository locations:

  • Python file
  • Python module
  • gRPC server

Each repository location can have multiple repositories. Once you define the location, Dagster is able to automatically go find all the repositories in that location. In the example above, I defined my workspace to have a single Python module repository location:

load_from:
  - python_module: dagster_test.toys.repo

Note that simply specified a module and did not specify a repository location name, so Dagster assigned a default repository location name.

If I wanted to specify a location name, I would do:

load_from:
- python_module:
    module_name: dagster_test.toys.repo
    location_name: "my_custom_location_name"

Similarly for a python file location:

load_from:
- python_file: repo.py

Or with a custom repository location name:

load_from:
- python_file:
    relative_path: repo.py
    location_name: "my_custom_location_name"

Upvotes: 1

Related Questions