CODEWITHSUNDEEP
spring-bootamazon-s3apache-camel
Vinicius de Andrade
Vinicius de Andrade

Reputation: 27

Get S3 Objects With Apache Camel

I am trying to expose a rest endpoint with camel. It will show a json data which is inside some .json files stored in s3 bucket. Also, it will filter by a date range.

First, I got some s3 objects informations in my Camel routes. (I am using kotlin)

//expose the endpoint
        from("jetty:http://0.0.0.0:8080/getObjects")
                .routeId("list-objects-on-bucket")
                .to("aws-s3://[bucket-name]?amazonS3Client=#s3Client&operation=listObjects")
                .process(ListObjects())
                .to("direct:filter-list-from-s3")

then, I filter the data. (Till here everything is alright)

from("direct:filter-list-from-s3")
                .routeId("filter-list-from-s3")
                .process(FilterObjects())
                .to("log:info")

But in my FilterObject class I do not know how to download every files that matches (look the if statement) and pass it to the next route that will treat them

class SaoMateusFilterObjects : Processor {
    override fun process(exchange: Exchange?) {

        val start_date = exchange!!.getIn().getHeader("start_date") as String
        val end_date = exchange.getIn().getHeader("end_date") as String

        val formatter = DateTimeFormatter.ofPattern("dd-MM-yyyy")
        val start = LocalDate.parse(start_date).format(formatter)
        val end = LocalDate.parse(end_date).format(formatter)

        val objectsNames = exchange!!.getIn().body as LinkedList<String>
        for (objectName in objectsNames) {
            if(objectName.contains(start) && objectName.contains(end) && objectName.contains(".json")) {
                exchange.getIn() to "aws-s3://[bucket-name]?amazonS3Client=#s3Client&operation=getObject&fileName=$objectName"
            }
        }
    }

}

Some problems are:

1 - I want to read. By I think I can't use the from() method. Because it can be use just once. So, the to() method is used to read.

2 - exchange.getIn().to("[s3-uri]") maybe/must be converted in S3Object(). How??

Can Someone help me with this?

Thank you

Upvotes: 0

Views: 1848

Answers (1)

Sivaraj Velayutham
Sivaraj Velayutham

Reputation: 187

Instead of .to route, use .bean() and use the s3.getObject method to get the S3Object.

always Prefer using .bean() over .processor().

offical_s3_java_object operation sample.

Upvotes: 1

Related Questions