CODEWITHSUNDEEP
amazon-web-servicesaws-cdk
J.C Guzman
J.C Guzman

Reputation: 1334

AWS CDK pipeline python pass artifacts to next steps in cdk.stage

I am trying creating an AWS CodePipeline using AWS CDK in python.

cdk verson = 2.29.0

import aws_cdk as cdk
from aws_cdk.pipelines import CodePipeline, CodePipelineSource, ShellStep
from aws_cdk import (
    aws_codecommit,
    pipelines,
    aws_codepipeline_actions,
    aws_codepipeline,
    aws_codebuild as codebuild,
    aws_iam as iam
    )
from my_pipeline.my_pipeline_app_stage import MyPipelineAppStage
from constructs import Construct
        
        
class MyPipelineStack(cdk.Stack):

    def __init__(self, scope: Construct, construct_id: str, branch, **kwargs) -> None:
        super().__init__(scope, construct_id  ,**kwargs)
        
        
        repository = aws_codecommit.Repository.from_repository_name(self,"cdk_pipeline", repository_name="repository-name")
        
        pipeline_source = CodePipelineSource.code_commit(repository,"master")
        pipeline =  CodePipeline(self, "Pipeline",
                        self_mutation=False,
                        pipeline_name="cdk_pipeline",
                        synth=ShellStep("Synth", 
                            input=pipeline_source,
                            commands=["npm install -g aws-cdk", 
                                      "python -m pip install -r requirements.txt", 
                                    "cdk synth"],
                             
                        ),
                        
                        )

        shell_step1 = pipelines.ShellStep("Creating csv file", commands=["touch casa.csv"])
        shell_step2 = pipelines.ShellStep("Creating", commands=["touch cisa.csv"])
        shell_step3 = pipelines.ShellStep("printing", commands=["ls"])
        
        ordered_steps = pipelines.Step.sequence([shell_step1, shell_step2, shell_step3])
        
        
        app_stage = pipeline.add_stage(MyPipelineAppStage(self, "test",env=env_EU),
                                       pre=ordered_steps,
                                       )

I don't know how to pass the output created from shell_step1 to shell_step2 and to shell_step3.

if I try to add the parameter primary_output_directory into the step shells like this:

pipeline =  CodePipeline(self, "Pipeline",
                        self_mutation=False,
                        pipeline_name="cdk_pipeline",
                        synth=ShellStep("Synth", 
                            input=pipeline_source,
                            commands=["npm install -g aws-cdk", 
                                      "python -m pip install -r requirements.txt", 
                                    "cdk synth"],
                             primary_output_directory = "cdk.out"
                        ),
                        
                        )

        shell_step1 = pipelines.ShellStep("Creating csv file", commands=["touch casa.csv"], primary_output_directory = "cdk.out")
        shell_step2 = pipelines.ShellStep("Creating", commands=["touch cisa.csv"], primary_output_directory = "cdk.out")
        shell_step3 = pipelines.ShellStep("printing", commands=["ls"], primary_output_directory = "cdk.out")
        
        ordered_steps = pipelines.Step.sequence([shell_step1, shell_step2, shell_step3])
        
        
        app_stage = pipeline.add_stage(MyPipelineAppStage(self, "test",env=env_EU),
                                       pre=ordered_steps,
                                       )

I get this error

[Container] 2022/08/18 23:50:36 Phase context status code: CLIENT_ERROR Message: no matching base directory path found for cdk.out

I don't know if it is necessary or not, I guess that I need to pass artifacts from one step to another but I cannot find how to do it

Upvotes: 0

Views: 959

Answers (1)

gshpychka
gshpychka

Reputation: 11482

To use a ShellStep's output as an input for another ShellStep, pass it directly into the input prop of your ShellStep:

shell_step_1 = pipelines.ShellStep("Creating csv file", input=pipeline_source, commands=["touch casa.csv"], primary_output_directory = "cdk.out")
shell_step_2 = pipelines.ShellStep("Creating", commands=["touch cisa.csv"], input=shell_step_1, primary_output_directory = ".")

shell_step_2 will get the contents of cdk.out as input, so it will not have a cdk.out folder anymore.

This assumes that your source contains a cdk.out directory, which wouldn't be conventional. If you want to pass the output of the synth step to the Shell Step, you'll need to assign it to a variable and pass it as input:

synth_step = ShellStep(
    "Synth", 
    input=pipeline_source,
    commands=[
        "npm install -g aws-cdk", 
        "python -m pip install -r requirements.txt", 
        "cdk synth"
    ],   
)
shell_step_1 = pipelines.ShellStep(
    "Creating csv file",
    input=synth_step,
    commands=["touch casa.csv"],
    primary_output_directory="cdk.out"
)

It's worth noting, though, that a single shell step can have multiple shell commands - you don't have to create a separate one for each command.

Reference: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.pipelines.ShellStep.html#initializer

Upvotes: 1

Related Questions