CODEWITHSUNDEEP
pythonamazon-rdsamazon-kinesis
Ekta Varshney
Ekta Varshney

Reputation: 441

Terraform: Validation error ... Member must satisfy regular expression pattern: arn:aws:iam::

I'm trying to stream rds through kinesis data stream but it is giving me this error:

botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the PutRecord operation: 1 validation error detected: Value 'arn:aws:kinesis:us-west-2:xxxxxxxxxx:stream/rds-temp-leads-stream' at 'streamName' failed to satisfy constraint: Member must satisfy regular expression pattern: [a-zA-Z0-9_.-]+

What can I do to fix this?


import json
import boto3
from datetime import datetime

from pymysqlreplication import BinLogStreamReader
from pymysqlreplication.row_event import (
  DeleteRowsEvent,
  UpdateRowsEvent,
  WriteRowsEvent,
)

class DateTimeEncoder(json.JSONEncoder):
    def default(self, o):
        if isinstance(o, datetime):
            return o.isoformat()

        return json.JSONEncoder.default(self, o)

def main():
  mysql = {
      "host": "",
      "port":,
      "user": "",
      "passwd": "",
      "db": ""}
  kinesis = boto3.client("kinesis", region_name = 'us-west-2')

  stream = BinLogStreamReader(
    connection_settings = mysql,
    server_id=100,
    blocking = True,
    log_file='mysql-bin.000003',
    resume_stream=True,
    only_events=[DeleteRowsEvent, WriteRowsEvent, UpdateRowsEvent]) 
  for binlogevent in stream:
    for row in binlogevent.rows:
      print row
      event = {"schema": binlogevent.schema,
      "table": binlogevent.table,
      "type": type(binlogevent).__name__,
      "row": row
      }

      kinesis.put_record(StreamName="jhgjh", Data=json.dumps(event, cls=DateTimeEncoder), PartitionKey="default")
      #print json.dumps(event)

if __name__ == "__main__":
   main()

Upvotes: 21

Views: 77577

Answers (3)

Cloud Cho
Cloud Cho

Reputation: 1774

For my case, I also had to remove sub folder in the address like this:

Incorrect

bucket='s3://image-video/image/'
photo='scene_academy.jpg'

client=boto3.client('rekognition')
response = client.detect_labels(Image={'S3Object':{'Bucket':bucket,'Name':photo}},
        MaxLabels=10)

Correct

bucket='image-video'
photo='image/scene_academy.jpg'

client=boto3.client('rekognition')
response = client.detect_labels(Image={'S3Object':{'Bucket':bucket,'Name':photo}},
        MaxLabels=10)

Runtime Environment: AWS S3 Bucket

Upvotes: 0

Sir Markpo
Sir Markpo

Reputation: 496

Based on the error:

1 validation error detected: Value 'arn:aws:kinesis:us-west-2:xxxxxxxxxx:stream/rds-temp-leads-stream' at 'streamName' failed to satisfy constraint: Member must satisfy regular expression pattern: [a-zA-Z0-9_.-]+

The highlighted value : arn:aws:kinesis:us-west-2:xxxxxxxxxx:stream/rds-temp-leads-stream doesn't match the regular expression [a-zA-Z0-9_.-]+ which causes the validation error.

This is because you do not need to add the full ARN and just add the name of your stream. i.e rds-temp-leads-stream

I'm not seeing it on the code but likely this line got the error:

kinesis.put_record(StreamName="jhgjh", Data=json.dumps(event, cls=DateTimeEncoder), PartitionKey="default")

Where "StreamName" should follow the same pattern as above. Reference can be found below: https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html

Upvotes: 1

Ekta Varshney
Ekta Varshney

Reputation: 441

remove 'arn:aws:kinesis:us-west-2:xxxxxxxxxx:stream/rds-temp-leads-stream' this from stream name. Just put the name of stream there like "rds-temp-leads-stream"

Upvotes: 17

Related Questions