CODEWITHSUNDEEP
pythonaws-glueopensearch
MiketheViking90
MiketheViking90

Reputation: 39

AWS Glue Writing to Opensearch

I have an Opensearch managed cluster, and I want to use AWS Glue to migrate hundreds of data in batch from Redshift to Opensearch.

I have followed these steps to

  1. create an AWS Glue connection to Opensearch, with appropriate IAM role (AmazonOpenSearchServiceFullAccess) and secrets (username and password)
  2. create ETL job using AWS Glue Visual ETL

My Opensearch cluster is on public network access. Despite this, I get the error

Error Category: UNCLASSIFIED_ERROR; Failed Line Number: 20; An error occurred while calling o134.getSink. Unable to execute HTTP request: Network is unreachable

The failing line of code is generated by my AWS Glue Visual ETL

AmazonOpenSearchService_node1733461823015 = 
  glueContext.write_dynamic_frame.from_options(
    frame=AmazonRedshift_node1733461813643, 
    connection_type="opensearch", 
    connection_options={"opensearch.resource": "taxi", "connectionName": "opensearch-connection"}, 
    transformation_ctx="AmazonOpenSearchService_node1733461823015"
)

Does anyone have any ideas how to move forward?

Upvotes: 0

Views: 128

Answers (0)

Related Questions