IvyIdea plugin - does not resolve dependencies

Question

I am using IntelliJ, added IvyIdea plugin and now trying to resolve dependencies for a project (Ant build) using IvyIdea but getting error about "UNRESOLVED DEPENDENCIES". I have downloaded and configured Ivy and Ant.

I can resolve and build the project using Ant command line but not via IDE. I can see the ~.ivy2\cache folder has got all the dependencies resolved which it is complaining about.

I have configured "Facets" for all modules to map to the correct Ivy settings and build properties files. Does anyone know if there is anything else I can try?

One thing I found now in the logs,

loadData of pentaho-kettle#kettle-core;7.1-SNAPSHOT of rootConf=default
    using default to resolve pentaho-kettle#kettle-core;7.1-SNAPSHOT
default: Checking cache for: dependency: pentaho-kettle#kettle-core;7.1-SNAPSHOT {default-ext=[default]}
don't use cache for pentaho-kettle#kettle-core;7.1-SNAPSHOT: changing=true
don't use cache for pentaho-kettle#kettle-core;7.1-SNAPSHOT: changing=true
    local (useCacheOnly) : no ivy file found for pentaho-kettle#kettle-core;7.1-SNAPSHOT
main: Checking cache for: dependency: pentaho-kettle#kettle-core;7.1-SNAPSHOT {default-ext=[default]}
don't use cache for pentaho-kettle#kettle-core;7.1-SNAPSHOT: changing=true
don't use cache for pentaho-kettle#kettle-core;7.1-SNAPSHOT: changing=true
    shared (useCacheOnly) : no ivy file found for pentaho-kettle#kettle-core;7.1-SNAPSHOT
don't use cache for pentaho-kettle#kettle-core;7.1-SNAPSHOT: changing=true
    public (useCacheOnly) : no ivy file found for pentaho-kettle#kettle-core;7.1-SNAPSHOT
WARN:   module not found: pentaho-kettle#kettle-core;7.1-SNAPSHOT
WARN: ==== local: tried
WARN: ==== shared: tried
WARN: ==== public: tried

I have made sure I am using correct Ivy settings and build properties but still it gives the same errors.

Answer 1

I also tried to work with Pentaho using IntelliJ. It is quite a job, because a lot of code is set up using Apache Ivy and Ant, and the rest is done with Maven.

What worked for me is the following..

Compiling in IntelliJ

• Make sure the normal build works from the command line.

• Install the latest IvyIdea plugin in IntelliJ

• Now open IntelliJ, and just open the pentaho-kettle directory without importing anything.

• Select File -> New -> Module from existing sources, then select the "assembly" directory.

• Let it import the sources, but deselect any libraries it finds. It should also automatically find the ivy.xml and add the IvyIdea facet.

• After the import click on "assembly" in the Project browser, then press F4 to go to the module settings.

• Click the "Sources" tab for the module and check if there is a directory called "res", "packages-res" or anything else nonstandard that means resources - EXCEPT build-res (ignore those). Click on those directories, then click "Mark as - resource" above the tree.

• Click the "IvyIdea" facet for the module, and configure the following:

  • select "Use module specific settings" (make it checked). Then select "Use your own", and select the ivysettings.xml file from the "assembly" directory.

  • Click the "properties" tab, then "Add file", and select the build.properties file from the assembly directory.

• Repeat this process for the other main modules, like core and engine and everything else where you can find a ivy.xml file. This is a lot of work, as there are many modules. You can use the following command (linux) to find all modules:

  find . -name 'ivy.xml'
  

• Once done let IvyIdea resolve all dependencies by clicking pentaho-kettle (the main directory) in the project explorer, then right-click -> IvyIdea -> resolve all dependencies. Make sure you have lots of coffee and lunch ready because it will take forever.

• Under plugins find the "pom.xml" file there. On the right side of IntelliJ open the "Maven Projects" view (or use View -> Tool Windows -> Maven projects).

• In that Maven view, click the green + to add a pom, and navigate to plugins/pom.xml. This lets IntelliJ import the rest of the plugins. This again takes a while.

• Now, click on the UI module in the Project Explorer, then press F4. Click the "dependencies" tab, then the green "+" on the right and select "Module dependency".

• In the list that follows: select everything, then press ok. This makes the ui module have a classpath that includes the plugins, so that spoon will find the thingies when starting.

After this IntelliJ should be able to build the modules added: press ctrl+f9 to check.

Running the compiled result

To get this thing to run is yet another bag of fun. I am running Linux, so the instructions here apply to that. YMMV on other operating systems or Windows.

• In the project explorer, find the "Spoon" class in the UI project (or press CTRL+N then Spoon).

• Find the main method and press the green button in the margin before it. This will try to run main, and will die horribly - but IntelliJ now has made a run configuration ;)

• Open the run configuration (Run -> Edit Configurations).

• Under "VM Options" add the following (click the little box to the right to get a big input field):

  -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2
  -Djava.library.path=./../libswt/linux/x86_64/
  -DKETTLE_HOME=
  -DKETTLE_REPOSITORY=
  -DKETTLE_USER=
  -DKETTLE_PASSWORD=
  -DKETTLE_PLUGIN_PACKAGES=
  -DKETTLE_LOG_SIZE_LIMIT=
  -DKETTLE_JNDI_ROOT=
  -DKETTLE_PLUGIN_CLASSES=org.pentaho.di.trans.steps.gpload.GPLoadMeta,org.pentaho.di.core.database.PALODatabaseMeta,org.pentaho.di.trans.steps.palo.cellinput.PaloCellInputMeta,org.pentaho.di.trans.steps.palo.celloutput.PaloCellOutputMeta,org.pentaho.di.trans.steps.palo.diminput.PaloDimInputMeta,org.pentaho.di.trans.steps.palo.dimoutput.PaloDimOutputMeta,org.pentaho.di.trans.steps.hl7input.HL7InputMeta,org.pentaho.di.job.entries.hl7mllpack.HL7MLLPAcknowledge,org.pentaho.di.job.entries.hl7mllpin.HL7MLLPInput,org.pentaho.di.starmodeler.StarModelerSpoonPlugin,org.pentaho.di.core.market.Market,org.pentaho.di.job.entries.palo.JobEntryCubeCreate.PaloCubeCreate,org.pentaho.di.job.entries.palo.JobEntryCubeDelete.PaloCubeDelete,org.pentaho.di.core.database.OpenERPDatabaseMeta,org.pentaho.di.trans.steps.openerp.objectinput.OpenERPObjectInputMeta,org.pentaho.di.trans.steps.openerp.objectoutput.OpenERPObjectOutputMeta,org.pentaho.di.trans.steps.openerp.objectdelete.OpenERPObjectDeleteMeta
  

• These parameters were stolen from spoon.sh, the script that usually starts Spoon. You might want to check that file if you still have trouble.

• Set the working directory to pentaho-kettle/assembly/package-res

• Add the following environment variables using the button after "Environment variables:

This should make it possible to run Spoon. You might get quite some messages in the console, and Spoon can take more than a minute to start (on my quite fast machine), so be patient ;)

Good luck ;)