CODEWITHSUNDEEP
hadoopsystem-properties
Judking
Judking

Reputation: 6381

Why System.setProperty cannot change the configuration attribute in Hadoop?

My environment is ubuntu12.04+eclipse3.3.0+hadoop0.20.2

When I test on the System.serProperty, which would change the configuration defined in xml file. But I don't get the same effect when I test it. Here is my code snippet:

//cofiguration class test
public static void test()   {
    Configuration conf = new Configuration();
    conf.addResource("raw/conf-1.xml");
    System.out.println(conf.get("z"));
    System.setProperty("z", "SystemProp_mz");
    System.out.println(conf.get("z"));
}

conf-1.xml is as follows:

<configuration>  
    <property>
        <name>z</name>
        <value>mz</value>
    </property>
</configuration>

the output is:

mz
mz

Could anyone give me some help? Thanks a lot!

Upvotes: 0

Views: 352

Answers (1)

Chris White
Chris White

Reputation: 30089

The Configuration object isn't linked to the System properties - if you want to change the value of z in the configuration, then use conf.set('z', 'SystemProp_mz') instead of System.setProperty(..)

Update

The Configuration object can use variable expansion as outlined in http://hadoop.apache.org/docs/current/api/org/apache/hadoop/conf/Configuration.html, but this requires you to have defined an entry as follows:

<configuration>  
  <property>
    <name>z</name>
    <value>${z}</value>
  </property>
</configuration>

If you have don't have the above entry, then just calling conf.get("z") will not resolve to system properties. The following unit test block demonstrates this:

@Test
public void testConfSystemProps() {
  System.setProperty("sysProp", "value");
  Configuration conf = new Configuration();
  conf.set("prop", "${sysProp}");

  Assert.assertNull(conf.get("sysProp"));
  Assert.assertEquals(conf.get("prop"), "value");
}

Upvotes: 3

Related Questions