CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-glue
prakash
prakash

Reputation: 165

how to combine multiple s3 files into one using Glue

I need some help in combining multiple files in different company partition in S3 into one file with company name in the file as one of the column.

I am new and I am not able to find any information also I did spoke to support and they say it is not supported. But in DataStage it is a basic function to combin multiple files into one. Please throw some light Regards, Prakash

Upvotes: 4

Views: 19952

Answers (2)

Mandeep Singh
Mandeep Singh

Reputation: 185

If the Column names are same in the file and number of columns are also same, Glue will automatically combine them.

Make sure the files you want to combine are in same folder on s3 and your glue crawler is pointing to the folder.

Upvotes: 3

jarmod
jarmod

Reputation: 78563

Review the AWS Glue examples, particularly the Join and Rationalize Data in S3 example. It shows you how to use a Python script to do joins and filters with transforms.

Upvotes: 0

Related Questions