CODEWITHSUNDEEP
javahadoop
Roman Nikitchenko
Roman Nikitchenko

Reputation: 13046

Global Hadoop configuration storage

Is there any way to set / get global Hadoop configuration object, something like pseudocode below? Of course I can create my own class with static methods which do what I need but its better if something like this could be found inside Hadoop Java API to not make dependencies more complex. By now I did not find anything usable. Any advice?

In some application level configurator.

Configuration conf = new Configuration();
conf.set(...);
<something>.SetGlobalConfig(conf);

In lower layer client code.

Configuration conf = <something>.GetGlobalConfig();
// ... something that needs configuration.

UPDATE 1: I know about Hadoop .xml configuration files and actually it's one of possible solutions but it's preferable to have all configuration done in code, without external files.

UPDATE 2: Based on points provided decision is to use 'usual' .xml configuration packaged together with job code. Client can 'tune' some parameters via command line parameters that keep hadoop-like semantic due to usage of the same Tool. To isolate rest of code from Hadoop configuration / user aspects even more application code requests configuration through special COnfigurator singletone.

Original question is considered solved though I don't mind against useful ideas.

Upvotes: 0

Views: 246

Answers (1)

Thomas Jungblut
Thomas Jungblut

Reputation: 20969

When you call

Configuration conf = new Configuration();

it will look into the classpath of your project for core-default.xml and core-site.xml.

And the core-site.xml is exactly where you want to put your "global" configurations into.

Upvotes: 1

Related Questions