AWS Cognito User Pool - SignInSuccesses CloudWatch Alarm and Metrics in CloudFormation

Question

I have been unsuccessfully trying to implement in CloudFormation what is described in the AWS documentation: Metrics for Amazon Cognito user pools [https://docs.aws.amazon.com/cognito/latest/developerguide/metrics-for-cognito-user-pools.html] for the Metric 'SignInSuccesses', specifically "To count the total number of failed user authentication requests, use the CloudWatch Math expression and subtract the Sum statistic from the Sample Count statistic."

I've made a bit of progress with resolving various errors with my template from the errors provided by CloudFormation; however, I seem to have hit a wall and have now been getting the generic "Invalid metrics list" which has been hard to diagnose how to move forward. I've also searched around a bunch to try and find other examples of Cognito Metrics/Alarms examples, as well as, have tried to achieve this manually through the console, both with no success.

My template so far:

  CognitoFailedSignInAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmName: !Sub Cognito-${ApplicationName}-FailedSignIn-Alarm
      AlarmDescription: Cognito UserPool Failed Sign In Attempts Alarm
      AlarmActions:
        - !Ref AlarmsTopic
      ComparisonOperator: GreaterThanOrEqualToThreshold
      EvaluationPeriods: 1
      TreatMissingData: ignore
      Threshold: !Ref AlarmThreshold
      Metrics:
        - Id: m1
          MetricStat:
            Metric:
              Dimensions:
                - Name: UserPool
                  Value: !Ref UserPoolId
                - Name: UserPoolClient
                  Value: !Ref UserPoolAppClientId
              MetricName: SignInSuccesses
              Namespace: AWS/Cognito
            Period: !Ref AlarmPeriod
            Stat: SampleCount
          ReturnData: False
        - Id: m2
          MetricStat:
            Metric:
              Dimensions:
                - Name: UserPool
                  Value: !Ref UserPoolId
                - Name: UserPoolClient
                  Value: !Ref UserPoolAppClientId
              MetricName: SignInSuccesses
              Namespace: AWS/Cognito
            Period: !Ref AlarmPeriod
            Stat: Sum
          ReturnData: False
        - Id: TotalMinusFailed
          Expression: m1-m2
          Label: FailedCount

Answer 1

My template does enable all the metrics, I wrote it last year but what I can find related, is this property on AWS::Cognito::UserPool

UserPoolAddOns: AdvancedSecurityMode: ENFORCED

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-cognito-userpool-userpooladdons.html

you can try "AUDIT" too

Answer 2

The generic CloudFormation error Invalid metrics list was masking two issues, a small formatting issue but more importantly the 'ReturnData: true' was missing on the Expression metric which is required.

The following deployed successfully and appears to work correctly. Hope that helps someone else out there!

Metrics:
  - Id: totalLogins
    MetricStat:
      Metric:
        Namespace: AWS/Cognito
        MetricName: SignInSuccesses
        Dimensions:
        - Name: UserPool
          Value: !Ref UserPoolId
        - Name: UserPoolClient
          Value: !Ref UserPoolAppClientId
      Period: !Ref AlarmPeriod
      Stat: SampleCount
    ReturnData: false
  - Id: successfulLogins
    MetricStat:
      Metric:
        Namespace: AWS/Cognito
        MetricName: SignInSuccesses
        Dimensions:
        - Name: UserPool
          Value: !Ref UserPoolId
        - Name: UserPoolClient
          Value: !Ref UserPoolAppClientId
      Period: !Ref AlarmPeriod
      Stat: Sum
    ReturnData: false
  - Id: e1
    Expression: totalLogins-successfulLogins
    Label: Failed Logins
    ReturnData: true