CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-lambdaaws-cdk
axecopfire
axecopfire

Reputation: 652

How to upload a zip to S3 with CDK

I'm working on building a CDK library and am trying to upload a zip folder to S3 that I can then use for a Lambda deployment later. I've found a lot of direction online to use aws_s3_deployment.

The problem with that construct is that it loads the contents of a zip rather than a zip itself. I've tried to zip a zip inside a zip and that doesn't work. I've also tried to zip a folder and that doesn't work either. The behavior I see is that nothing shows up in S3 and there are no errors from CDK. Is there another way to load a zip to S3?

Upvotes: 6

Views: 8494

Answers (3)

Jonathan
Jonathan

Reputation: 61

You can now use BucketDeployment's attribute extract as seen here: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3_deployment.BucketDeployment.html

Unfortunately, it is not possible to define the name of the zip file. CDK will use its hashing functionality for the name, which means that when the file metadata changes, so does the hash. You can however use the BucketDeployment deployed bucket objectkey to refer to the file in the rest of your CDK code. You can use the object key using Fn.select from aws-cdk-libs .

Explained here: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3_deployment.BucketDeployment.html#deployedbucket

Here is an example on how I deployed and referenced the zip file:

  // directory to be zipped
   const srcCodeDir = './lib/infra/assets/src/example_dir';
   // target directory
   const trgBucketCodeDir = 'assets/src/target_dir';
  // zip directory, hashed name
   const CodeAsset = s3deploy.Source.asset(srcCodeDir);

   // Deploy as zip
    const zipBucketDeployment = new s3deploy.BucketDeployment(this, 'DeployModularizedCode', {
      sources: [CodeAsset],
      destinationBucket: SrcBucket,
      destinationKeyPrefix: trgBucketCodeDir,
      extract: false,
      prune: false
    });
    // hashed asset name
    const hash: string = Fn.select(0,zipBucketDeployment.objectKeys);
    // create string for hashed name
    const ZipDir = 'S3://'.concat(zipBucketDeployment.deployedBucket.bucketName, '/', trgBucketCodeDir, '/', hash);

Upvotes: 4

lmiguelmh
lmiguelmh

Reputation: 3202

In order to upload the zip file to a given bucket I ended up using BucketDeployment with a custom ILocalBundling. The custom bundler will compress the files and put them in an assets directory for CDK to upload. The important part is to set output_type=BundlingOutput.NOT_ARCHIVED, this way CDK will not try to unzip the file.

@implements(ILocalBundling)
class LocalBundling:
    @member(jsii_name="tryBundle")
    def try_bundle(self, output_dir: str, image: DockerImage,) -> bool:
        cwd = pathlib.Path.cwd()

        print(f"bundling to {output_dir}...")
        build_dir = f"{cwd}/directory/to"
        command = ["zip", "-r", f"{output_dir}/python.zip", f"zip"]
        print(command)
        output = subprocess.run(command, capture_output=True, check=True, cwd=build_dir)
        # print(output.stdout.decode("utf-8"))
        return True

local_bundling = LocalBundling()
s3_deployment.BucketDeployment(
    self,
    f"SomeIdForBucketDeployment",
    sources=[
        s3_deployment.Source.asset(
            "directory/to/zip",
            bundling=BundlingOptions(
                command=['none'],
                image=DockerImage.from_registry("lm"),
                local=local_bundling,
                output_type=BundlingOutput.NOT_ARCHIVED,
            ),
        )
    ],
    destination_bucket=some_bucket,
    destination_key_prefix=some_key_prefix,
)

Upvotes: 3

Maurice
Maurice

Reputation: 13127

What you're looking for is the aws-s3-assets module. It allows you to define either directories (which will be zipped) or regular files as assets that the CDK will upload to S3 for you. Using the attributes you can refer to the assets.

The documentation has this example for it:

import { Asset } from 'aws-cdk-lib/aws-s3-assets';

// Archived and uploaded to Amazon S3 as a .zip file
const directoryAsset = new Asset(this, "SampleZippedDirAsset", {
  path: path.join(__dirname, "sample-asset-directory")
});

// Uploaded to Amazon S3 as-is
const fileAsset = new Asset(this, 'SampleSingleFileAsset', {
  path: path.join(__dirname, 'file-asset.txt')
});

Upvotes: 5

Related Questions