CODEWITHSUNDEEP
eclipsepluginsweb-crawlerhbasenutch
Hafiz Muhammad Shafiq
Hafiz Muhammad Shafiq

Reputation: 8670

Create a Nutch 2.x plugin using eclipse

I have to write a plugin to parse crawled content vy Nutch 2.3.1. I have decided to use eclipse as Its better than simple editor. Now How can I create a plugin in eclipse and test it via some simple use case ?

Upvotes: 0

Views: 214

Answers (2)

user1264641
user1264641

Reputation: 139

You can use following steps to get the plugin working from Eclipse.

  1. Get Nutch source code.

    git clone https://github.com/apache/nutch.git

  2. Switch to 2.3.1 branch. If you want latest 2.x in development, you can use 2.x branch https://github.com/apache/nutch/tree/branch-2.3.1

  3. Import project in eclipse.

  4. Build for eclipse. It uses ant for build and has eclipse target.

    ant eclipse

  5. All available plugins in nutch are under src/plugins directory.

  6. You need a smilar structure for your new plugin so copy one of the existing plugin to new one.

    cp -r lib-http my-http

7 Now check the structure of plugin directory. It should be as below.

my-http/
├── build.xml
├── ivy.xml
├── plugin.xml
└── src
    ├── java
    └── test

  1. plugin.xml is the one which has definitions for extentions, extention-points, runtime libraries etc. You can view it in eclipse plugin editor and can do changes from there.

  2. Add proper implementation class and tests. Map that for extension in plugin.xml

  3. You have to change your build.xml and ivy.xml to add proper dependencies.

  4. You can override targets defined in src/plugin/build-plugin.xml in your build.xml. build-plugin.xml is being called by main build file src/build.xml for each plugin.

  5. You can test your plugin by using ant from the plugin directory. ant test.

You can also use eclipse to check JUint test results. Click on Test class and Run as JUnit Test

  1. Add plugin to deploy and test targets in src/plugin/build.xml. This file is used by the main build file

    <ant dir="my-http" target="deploy"/>

  2. add any required dependencies in build/ivy/ivy.xml

  3. Add plugin plugin.includes property in conf/nutch-site.xml

  4. Build nutch

    ant runtime

Now your plugin is set to run in local/distributed mode from runtime directory.

Upvotes: 2

Jorge Luis
Jorge Luis

Reputation: 3253

You can use any editor you want to write your code, as long as you generate a jar that you load in the Nutch plugin system with the right dependencies and configurations in the xml file everything should work. You can check https://wiki.apache.org/nutch/RunNutchInEclipse that contains detailed instructions to open and run within eclipse so debugging is easier, but its not required. Specially important is to run ant eclipse in your local copy of the project, so that you can open the Nutch entire source code in Eclipse, once this is done you can create your plugin file structure and start coding.

Hope it helps.

Upvotes: 0

Related Questions