CODEWITHSUNDEEP
apache-camel
Maxime Laval
Maxime Laval

Reputation: 4378

Unmarshalling an xml file downloaded from Amazon S3 with Camel

I want to download xml files from an S3 bucket, then unmarshal them to insert the data in a database:

...
from("aws-s3://myBucket?accessKey=xxx&secretKey=yyy")
.to("file:D:/output?fileName=${in.header.CamelAwsS3Key}")
.unmarshal(new JaxbDataFormat("com.xxx"))
...

I am able to download the files when I don't try to unmarshal, but I get this error when I try to unmarshal: 

org.apache.camel.TypeConversionException:
Error during type conversion from type:
com.amazonaws.services.s3.model.S3ObjectInputStream to the required type:
javax.xml.stream.XMLStreamReader with value com.amazonaws.services.s3.model.S3ObjectInputStream@67b8328b due javax.xml.stream.XMLStreamException:
java.io.IOException: Attempted read on closed stream.

Since I am new to Camel there are maybe things I didn't understand yet...

  1. When I pipe endpoints, doesn't the current endpoint get the message the way the previous endpoint "modified" it? In my case it looks like the S3 stream is being marshalled, instead of the xml file newly created locally from the download, hence the error.
    My understanding is if I do .from().to().to(), the second .to() doesn't know what is coming from .from() so if my first .to() creates an xml file, the second .to() handles the message as an xml file. Am I wrong?
  2. Maybe I need to create 2 routes? I was able to do the other way around with only 1 route though, from database to file to S3.
  3. Do I need to write my own converter in that case?

Thanks!

Upvotes: 1

Views: 964

Answers (1)

Claus Ibsen
Claus Ibsen

Reputation: 55750

When you have the 2nd to Camel needs to read the stream again, but its closed. Its becomes sometimes streams are only readable once. And it was read in the first to, where you write it to a file.

So you can either enable stream caching to allow Camel to be able to re-read the stream again. See more in this FAQ and the links

You can also break it into 2 routes

from amq
to file

from file
unmarshal

Upvotes: 0

Related Questions