Usage of AWS Lake Formation with CloudFormation

Question

I want to set up an additional security layer on top of my S3 / Glue Data Lake using Lake Formation. I want to do as much as possible via Infrastructure as Code, so naturally I looked into the documentation of the CloudFormation implementation of Lake Formation which is currently, frankly speaking, very useless.

I have a simple use case: Granting admin permission to one IAM-User on one bucket. Can someone help me out with an example or anything similar?

Answer 1

This is what I found out:

Setting a data lake location and granting data permissions to your data bases is currently possible. Unfortunately it seems like CloudFormation doesn't support Data locations yet. You will have to grant your IAM Role access to the S3 Bucket by hand in the AWS Console under Lake Formation -> Data locations. I will update the answer as soon as CloudFormation supports more.

This is the template that we are using at the moment:

DataBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    Properties:
      AccessControl: Private
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
      VersioningConfiguration:
        Status: Enabled
      LifecycleConfiguration:
        Rules:
          - Id: InfrequentAccessRule
            Status: Enabled
            Transitions:
              - TransitionInDays: 30
                StorageClass: INTELLIGENT_TIERING

GlueDatabase:
    Type: AWS::Glue::Database
    Properties:
      CatalogId: !Ref AWS::AccountId
      DatabaseInput:
        Name: !FindInMap [Environment, !Ref Environment, GlueDatabaseName]
        Description: !Sub Glue Database ${Environment}

GlueDataAccessRole:
    Type: AWS::IAM::Role
    Properties:
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: ''
            Effect: Allow
            Principal:
              Service: glue.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: AccessDataBucketPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - glue:*
                  - lakeformation:*
                Resource: '*'
              - Effect: Allow
                Action:
                  - s3:GetObject
                  - s3:PutObject
                  - s3:ListBucket
                  - s3:DeleteObject
                Resource:
                  - !Sub ${DataBucket.Arn}
                  - !Sub ${DataBucket.Arn}/*

 DataBucketLakeFormation:
    Type: AWS::LakeFormation::Resource
    Properties:
      ResourceArn: !GetAtt DataBucket.Arn
      UseServiceLinkedRole: true

DataLakeFormationPermission:
    Type: AWS::LakeFormation::Permissions
    Properties:
      DataLakePrincipal:
        DataLakePrincipalIdentifier: !GetAtt GlueDataAccessRole.Arn
      Permissions:
        - ALL
      Resource:
        DatabaseResource:
          Name: !Ref GlueDatabase
        DataLocationResource:
          S3Resource: !Ref DataBucket