CODEWITHSUNDEEP
spring-cloud-dataflow
Venu Gopal
Venu Gopal

Reputation: 15

How to extend standard spring cloud data flow components and add our custom logic

I'm looking to develop a custom processor that leverages the functionality of s3-sink and then add some custom logic to it.

I could do it without using standard components. i.e. using S3 APIs to write files along with custom logic. I would like to know if there is any way I can leverage the functionality of standard component and extend it to add our custom logic.

Upvotes: 0

Views: 134

Answers (1)

onobc
onobc

Reputation: 621

If you want to invoke some logic before or after the S3Sink normal logic then you can use function composition and create a custom Sink application that imports the S3SinkConfiguration and defines a java.util.Function that is applied after the normal S3Sink logic. You will need to set the spring.cloud.function.definition property to s3Consumer|myCustomLogicFunctionName.

However, if you want to tweak what the S3Sink logic actually is (not before or after it executes), you will have to look into what it offers for extensibility/configurability. The S3Sink app looks like this:

@SpringBootApplication
@Import({ org.springframework.cloud.fn.consumer.s3.AwsS3ConsumerConfiguration.class })
public class S3SinkKafkaApplication {

    public static void main(String[] args) {
        SpringApplication.run(S3SinkKafkaApplication.class, args);
    }
}

As you can see its just a SpringBoot app that imports a single configuration class. All the logic/magic is in that configuration. It in turn leverages Spring Integration AWS S3MessageHandler.

What does the custom logic need to do? Maybe the underlying S3MessageHandler supports that via configuration/extension.

Upvotes: 1

Related Questions