CODEWITHSUNDEEP
hadoopspring-xd
Pushpender
Pushpender

Reputation: 11

spring xd Required module artifacts are either missing or invalid

I just started with spring xd and wanted to test out a simple hadoop job that I already have with single node config. I mostly followed (http://docs.spring.io/spring-xd/docs/1.2.1.RELEASE/reference/html)

the module jar file that I upload does not contain anything else just this file resources\config\spring-module.xml:

<!-- required since Hadoop Job is a class not an interface and we need to 
    use a Job with step scope to access #{jobParameters['...']} -->
<bean class="org.springframework.batch.core.scope.StepScope">
    <property name="proxyTargetClass" value="true" />
</bean>

<batch:job id="testetl">
    <batch:step id="testetl1">
        <batch:tasklet ref="testjob" />
    </batch:step>
</batch:job>

<hdp:jar-tasklet id="testjob"
    jar="/home/hduser/libs/abhishek.jar">

</hdp:jar-tasklet>

On xd shell:

xd:>module upload --type job --name test_job --file /home/hduser/springxd/test-job-module-0.0.1-SNAPSHOT.jar --force

Successfully uploaded module 'job:test_job'

xd:>job create --name test_elt --definition "test_job" --deploy

Successfully created and deployed job 'test_elt'

But on server instance getting below exception and unable to start the job.

I tried bit different xml also but same error.

Not sure what is going wrong?

2015-09-22T21:49:24+0530 1.2.1.RELEASE INFO DeploymentsPathChildrenCache-0 container.DeploymentListener - Deploying module [ModuleDescriptor@1280cb40 moduleName = 'test_job', moduleLabel = 'test_job', group = 'test_elt', sourceChannelName = [null], sinkChannelName = [null], index = 0, type = job, parameters = map[[empty]], children = list[[empty]]]
2015-09-22T21:49:24+0530 1.2.1.RELEASE ERROR DeploymentsPathChildrenCache-0 container.DeploymentListener - **Exception deploying module
java.lang.IllegalArgumentException: Required module artifacts are either missing or invalid. Unable to determine module type for module definition**: 'job:test_job'.
at org.springframework.util.Assert.notNull(Assert.java:112) ~[spring-core-4.1.7.RELEASE.jar:4.1.7.RELEASE]
at org.springframework.xd.module.core.ModuleFactory.createSimpleModule(ModuleFactory.java:122) ~[spring-xd-module-1.2.1.RELEASE.jar:1.2.1.RELEASE]
at org.springframework.xd.module.core.ModuleFactory.createAndConfigureModuleInstance(ModuleFactory.java:98) ~[spring-xd-module-1.2.1.RELEASE.jar:1.2.1.RELEASE]
at org.springframework.xd.module.core.ModuleFactory.createModule(ModuleFactory.java:81) ~[spring-xd-module-1.2.1.RELEASE.jar:1.2.1.RELEASE]
at org.springframework.xd.dirt.module.ModuleDeployer.createModule(ModuleDeployer.java:181) ~[spring-xd-dirt-1.2.1.RELEASE.jar:1.2.1.RELEASE]
at org.springframework.xd.dirt.server.container.DeploymentListener.deployModule(DeploymentListener.java:363) [spring-xd-dirt-1.2.1.RELEASE.jar:1.2.1.RELEASE]
at org.springframework.xd.dirt.server.container.DeploymentListener.deployJobModule(DeploymentListener.java:291) [spring-xd-dirt-1.2.1.RELEASE.jar:1.2.1.RELEASE]
at org.springframework.xd.dirt.server.container.DeploymentListener.onChildAdded(DeploymentListener.java:181) [spring-xd-dirt-1.2.1.RELEASE.jar:1.2.1.RELEASE]
at org.springframework.xd.dirt.server.container.DeploymentListener.childEvent(DeploymentListener.java:149) [spring-xd-dirt-1.2.1.RELEASE.jar:1.2.1.RELEASE]
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:509) [curator-recipes-2.6.0.jar:na]

Upvotes: 0

Views: 509

Answers (1)

ebottard
ebottard

Reputation: 1997

the module jar file that I upload does not contain anything else just this file resources\config\spring-module.xml:

To clarify, in the resulting .jar file, the xml file must be at 

config\spring-module.xml

and not at

resources\config\spring-module.xml

(This resources bit is part of the canonical maven location where source files are usually put, as in src/main/resources)

jar tvf your-module.jar should read config/spring-module.xml and not resources/...

Upvotes: 1

Related Questions