CODEWITHSUNDEEP
elasticsearchelasticsearch-plugin
JChao
JChao

Reputation: 2319

Elasticsearch install custom plugins results in ERROR: `elasticsearch` directory is missing in the plugin zip

When I try to install the plugin I built by calling 

./elasticsearch-plugin install file:///fullpath/to/zipfile/custome_plugin.zip

It gives me this error:

ERROR: `elasticsearch` directory is missing in the plugin zip

I was reading some other similar posts and many tried to install a .jar instead of .zip. I did try installing .zip, but this same error still shows.

Additionally (might not be related to this issue), when I manually unzip the zip file and put it into plugin folder, doing elasticsearch-plugins list does list out the custom plugin. In this case the plugin is a custom analyzer, but somehow the mapping doesn't recognize the analyzer. Is it because I did not install it properly?

EDIT:

adding a little info, when I shut down the cluster and then restart after the plugin is manually unzipped and put into the plugin directory, the cluster wouldn't start. I got error that looks like 

[2017-08-31T11:15:52,668][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [] fatal error in thread [main], exiting
java.lang.NoClassDefFoundError: org/elasticsearch/index/analysis/AnalysisModule$AnalysisBinderProcessor
    at java.lang.Class.getDeclaredConstructors0(Native Method) ~[?:1.8.0_65]
    at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) ~[?:1.8.0_65]
    at java.lang.Class.getConstructor0(Class.java:3075) ~[?:1.8.0_65]
    at java.lang.Class.getConstructor(Class.java:1825) ~[?:1.8.0_65]
    at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:423) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:387) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:140) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.node.Node.<init>(Node.java:312) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.node.Node.<init>(Node.java:244) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:232) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:232) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.5.2.jar:5.5.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.5.2.jar:5.5.2]
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.index.analysis.AnalysisModule$AnalysisBinderProcessor
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_65]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_65]
    at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814) ~[?:1.8.0_65]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_65]
    ... 19 more

It seems like this Analysis

Is there some library I'm missing?

EDIT 2: After some more digging, I found this site

http://snacktrace.com/artifacts/org.elasticsearch/elasticsearch/1.7.3/org.elasticsearch.index.analysis.AnalysisModule$AnalysisBinderProcessor$AnalyzersBindings

Seems like this AnalyzersBindings is only for ES 1.x and 2.x (<2.4)? I need a way to replace this.

EDIT 3: As requested, here's how AnalyzerBinding is used originally

package org.elasticsearch.plugin.analysis.my.analyzer;

import org.elasticsearch.index.analysis.AnalysisModule;

public class AnalyzerBinderProcessor extends AnalysisModule.AnalysisBinderProcessor {

    @Override
    public void processAnalyzers(AnalyzersBindings analyzersBindings) {
        analyzersBindings.processAnalyzer("my_name", AnalyzerProvider.class);
    }

    @Override
    public void processTokenFilters(TokenFiltersBindings tokenFiltersBindings) {
    }
}

then in my AnalyzerPlugin.java I have

public class AnalyzerPlugin extends Plugin {
    //some other code here that's not related to Binder 

    public void onModule(AnalysisModule module) {
        module.addProcessor(new AnalyzerBinderProcessor());
    }
}

EDIT 4: this is neverending.......

if I keep the @Override, it throws

[ERROR] /opt/bg/analytics/src/java/src/analyzer5.0/src/main/java/org/elasticsearch/plugin/analysis/my_analyzer/AnalyzerPlugin.java:[16,5] method does not override or implement a method from a supertype

Also, facing an issue of 

[ERROR] /opt/bg/analytics/src/java/src/analyzer5.0/src/main/java/org/elasticsearch/plugin/analysis/my_analyzer/AnalyzerPlugin.java:[36,17] onModule(org.elasticsearch.indices.analysis.AnalysisModule) in org.elasticsearch.plugin.analysis.my_analyzer.AnalyzerPlugin cannot override onModule(org.elasticsearch.indices.analysis.AnalysisModule) in org.elasticsearch.plugins.Plugin

because overridden method is final

Is that an update from old Plugin to new? I'm so lost......

I feel like I need a thorough tutorial of how to create a plugin for custom analyzer, all I can find so far is this

http://david.pilato.fr/blog/2016/10/16/creating-a-plugin-for-elasticsearch-5-dot-0-using-maven-updated-for-ga/

which doesn't help much. Does anyone have a good link?

Upvotes: 0

Views: 4087

Answers (2)

btaek
btaek

Reputation: 572

Looks like you already have fixed your error.

However, I thought I would add my learning in case there are people who are getting ERROR: 'elasticsearch' directory is missing in the plugin zip becasue of the same mistake that I made.

As it is stated in Elasticsearch documentation (https://www.elastic.co/guide/en/elasticsearch/plugins/current/plugin-authors.html), the folder name of the plugin should be "elasticsearch" before the folder is compressed to a zip file.

So, regardless of the zip file's name like "elasticsearch-analysis-mecab-ko-5.5.3.0.zip", the plugin folder before compressed should be called "elasticsearch". If the folder name is not "elasticsearch", when you run elasticsearch-plugin install, your elasticsearch simply gives you ERROR: 'elasticsearch' directory is missing in the plugin zip

Thus, after you make or amend your custom plugin folder, compress it:

$ zip -r elasticsearch-analysis-mecab-ko-5.5.3.0.zip elasticsearch

then install the plugin

$ ./elasticsearch-plugin install file:///Users/btaek/Desktop/ElasticSearch/elasticsearch-analysis-mecab-ko-5.5.3.0.zip

Upvotes: 6

Mysterion
Mysterion

Reputation: 9320

Currently, in ES 5.x we have org.elasticsearch.plugins.AnalysisPlugin, which is the main point for creating custom analysis components. You need to adapt your plugin code, so you could register whatever custom analyzer/tokenizer/etc you have.

Example of the plugin, that should do the trick for you:

public class MyAnalyzerPlugin implements AnalysisPlugin {

@Override
public Map<String, AnalysisModule.AnalysisProvider<AnalyzerProvider<? extends Analyzer>>> getAnalyzers() {
    final Map<String, AnalysisModule.AnalysisProvider<AnalyzerProvider<? extends Analyzer>>> objectObjectHashMap = new HashMap<>();
    objectObjectHashMap.put("my_analyzer", new MyAnalyzerProviderFactory());
    return objectObjectHashMap;
}

class MyAnalyzerProviderFactory implements AnalysisModule.AnalysisProvider<AnalyzerProvider<?>> {

    private final MyAnalyzerProvider analyzerProvider;

    public MyAnalyzerProviderFactory() {
        analyzerProvider = new MyAnalyzerProvider(AnalyzerScope.INDICES);
    }

    public AnalyzerProvider<?> create(String name, Settings settings) {
        Version indexVersion = Version.indexCreated(settings);
        if (!Version.CURRENT.equals(indexVersion)) {
            PreBuiltAnalyzers preBuiltAnalyzers = PreBuiltAnalyzers.getOrDefault(name, null);
            if (preBuiltAnalyzers != null) {
                Analyzer analyzer = preBuiltAnalyzers.getAnalyzer(indexVersion);
                return new MyAnalyzerProvider(AnalyzerScope.INDICES);
            }
        }

        return analyzerProvider;
    }

    @Override
    public AnalyzerProvider<?> get(IndexSettings indexSettings, Environment environment, String name, Settings settings)
            throws IOException {
        return create(name, settings);
    }

    public Analyzer analyzer() {
        return analyzerProvider.get();
    }
}


class MyAnalyzerProvider implements AnalyzerProvider<StandardAnalyzer> {

    private final StandardAnalyzer analyzer;
    private final AnalyzerScope scope;

    public MyAnalyzerProvider(AnalyzerScope scope) {
        this.scope = scope;
        this.analyzer = new StandardAnalyzer();
    }

    @Override
    public String name() {
        return "my-standard-analyzer";
    }

    @Override
    public AnalyzerScope scope() {
        return scope;
    }

    @Override
    public StandardAnalyzer get() {
        return analyzer;
    }
}

Of course, you need to adapt your Analyzer as well, since Lucene version has been changed and some API may be deprecated or removed.

Upvotes: 1

Related Questions