YAML transform a task to another version

Question

With Microsoft breaking 2 tasks this week and breaking many pipelines until the fix was released, I am looking into workarounds. I already documented a few here.

One option I'm investigating is the use of a YAML transformation in a Pipeline Decorator.

I've crafted the following YAML transformation to try and replace the version of the npmAuthenticate@0 task to be exactly and always: 0.200.0:

- steps:
  - ${{ each step in job.steps }}:
    - ${{ if in(step.task.id, 'npmAuthenticate', 'ad884ca2-732e-4b85-b2d3-ed71bcbd2788')}}
      - ${{ each task in step }}:
        - ${{ each pair in task }}:
            ${{ if eq(pair.key, 'version') }}:
              ${{ pair.key }}: 0.200.0
            ${{ if ne(pair.key, 'version') }}: 
              ${{ pair.key }}: ${{ pair.value }}

But it's not working. Unfortunately I haven't found a way to locally debug these transformers, so the current process of trying out new versions takes forever.

My ask:

• Is this even possible from a pipeline decorator?
• What's wrong with the syntax above?
• Is there a way to debug these things locally?

Here's the Pipeline Decoractor Context debug information:

2023-01-13T09:38:23.6524788Z job={
2023-01-13T09:38:23.6525354Z   "steps": [

...

2023-01-13T09:38:23.6538541Z     {
2023-01-13T09:38:23.6539075Z       "task": {
2023-01-13T09:38:23.6539824Z         "id": "ad884ca2-732e-4b85-b2d3-ed71bcbd2788",
2023-01-13T09:38:23.6540412Z         "name": "npmAuthenticate",
2023-01-13T09:38:23.6541005Z         "version": "0.208.1"
2023-01-13T09:38:23.6542056Z       },
2023-01-13T09:38:23.6542581Z       "env": {},
2023-01-13T09:38:23.6543177Z       "inputs": {
2023-01-13T09:38:23.6545385Z         "workingFile": "npmrc",
2023-01-13T09:38:23.6545995Z         "customEndpoint": ""
2023-01-13T09:38:23.6546519Z       },
2023-01-13T09:38:23.6547100Z       "condition": "succeeded()",
2023-01-13T09:38:23.6547694Z       "continueOnError": false,
2023-01-13T09:38:23.6548265Z       "name": "npmAuthenticate",
2023-01-13T09:38:23.6548890Z       "displayName": "npm Authenticate npmrc",
2023-01-13T09:38:23.6549457Z       "enabled": true
2023-01-13T09:38:23.6549971Z     },

...

2023-01-13T09:38:23.6786723Z   ]
2023-01-13T09:38:23.6787219Z }
2023-01-13T09:38:23.7641252Z ##[section]Finishing: Pipeline decorator context (Windows)

No dice either:

- steps:
  - ${{ each task in job.steps.*.task }}:
    - ${{ if eq(task.id, 'ad884ca2-732e-4b85-b2d3-ed71bcbd2788')}}
      ${{ if eq(pair.key, 'version') }}:
        ${{ pair.key }}: 0.200.0
      ${{ if ne(pair.key, 'version') }}: 
        ${{ pair.key }}: ${{ pair.value }}

nor:

- steps:
  - ${{ each step in job.steps.* }}:
    - ${{ if eq(step.task.id, 'ad884ca2-732e-4b85-b2d3-ed71bcbd2788')}}:
      - ${{ each prop in step.task }}:
        ${{ if eq(prop.key, 'version') }}:
          ${{ prop.key }}: 0.200.0
        ${{ if ne(prop.key, 'version') }}: 
          ${{ prop.key }}: ${{ prop.value }}

The vss-extension.json:

{
    "manifestVersion": 1,
    "id": "replace-npmauthenticate-decorator",
    "publisher": "jessehouwing",
    "version": "1.0.0",
    "name": "replace-npmauthenticate-decorator",
    "description": "Azure DevOps Extension",
    "categories": [
        "Azure Pipelines"
    ],
    "icons": {
        "default": "logo.png"
    },
    "targets": [
        {
            "id": "Microsoft.VisualStudio.Services"
        }
    ],
    "contributions": [
        {
            "id": "replace-npmauthenticate-decorator",
            "type": "ms.azure-pipelines.pipeline-decorator",
            "targets": [
                "ms.azure-pipelines-agent-job.pre-job-tasks"
            ],
            "properties": {
                "template": "replace-npmauthenticate-decorator.yml"
            }
        }
    ],
    "files": [
        {
            "path": "replace-npmauthenticate-decorator.yml",
            "addressable": true,
            "contentType": "text/plain"
        }
    ]
}

Answer 1

You can see the error info, such as it is, for a pipeline decorator by downloading the logs for the run where it doesn't appear when it should. The name of the file appears to be 1_[job name].txt.

You will like see error like this, as the 'each' expression is sadly not supported:

2024-04-29T18:25:24.4030754Z   Begin evaluating template '<file name>.yaml@'
Evaluating: job['steps']
Result: Object
[error]<file name>.yaml@ (Line: 2, Col: 3):
Finished evaluating template '<file name>.yaml@'

I guess it makes sense as the pipeline parser is trying to figure out a binary to inject or not to inject decision, so a loop would mess with that, but it make doing evaluations against more than one part of the context a real challenge.

Answer 2

I've experimented with over 40 different versions of YAML transformations, and tried 8 suggestions by ChatGPT, but ultimately must conclude that this approach does not seem to be viable.

I've seen many different tools to validate YAML structures, but nothing that validates a transform template or allows me to run one with a given context.

Azure DevOps' feedback toward errors is about non-existent. If the YAML is syntactically correct it will say there is an error, but won't tell you what the error is, as such I must conclude that most of my 40 attempts are valid YAML, but that they don't work for some other undisclosed reason.

I've looked most of Azure DevOps Server in IlSpy to see if there is a place I could hook into to locally run the decorator, but so far no such luck, the code that interprets YAML is firmly embedded in the execution context of the pipeline and would require quite a bit if work to run locally out-of-server.

Answer 3

It is not possible to debug a decorator, since the debugging of agents is removed:

https://github.com/microsoft/azure-pipelines-agent/issues/2479

Alternatives are:

• split up and peal off the loop
• Use your own pre / post AzDo task instead