CODEWITHSUNDEEP
javahadoopmapreduce
Venk K
Venk K

Reputation: 1177

Hadoop get relative path from absolute path and base path

I want to get the relative path from an absolute path, given the absolute base path. Is there any Hadoop Java API that does this?

For example, if my absolute HDFS path is abs_path = hdfs://name-node/level1/level2/level3 and my absolute base path is abs_base_path = hdfs://name-node/level1, I would like to extract the relative path from abs_path, which would be rel_path = level2/level3. I am familiar with using the path constructor to combine two paths.

For example, if I have the rel_path and abs_base_path, I can use one of the overloaded constructors in the Path class http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/Path to build abs_path but I cannot find an API to do the reverse.

Upvotes: 2

Views: 5736

Answers (2)

laughing_man
laughing_man

Reputation: 3978

This is actually done in FileOutputCommitter's source code. The relevant function is

   /**
   * Find the final name of a given output file, given the job output directory
   * and the work directory.
   * @param jobOutputDir the job's output directory
   * @param taskOutput the specific task output file
   * @param taskOutputPath the job's work directory
   * @return the final path for the specific output file
   * @throws IOException
   */
  private Path getFinalPath(Path jobOutputDir, Path taskOutput, 
                            Path taskOutputPath) throws IOException {
    URI taskOutputUri = taskOutput.toUri();
    URI relativePath = taskOutputPath.toUri().relativize(taskOutputUri);
    if (taskOutputUri == relativePath) {
      throw new IOException("Can not get the relative path: base = " + 
          taskOutputPath + " child = " + taskOutput);
    }
    if (relativePath.getPath().length() > 0) {
      return new Path(jobOutputDir, relativePath.getPath());
    } else {
      return jobOutputDir;
    }
  }

The idea is to create a URI for the base directory and then create a new Path for this new, relativized URI.

Hope that helps.

Upvotes: 6

timrs2998
timrs2998

Reputation: 1911

How about building a String while recursing with getParent() until the current path is equal to the base path? Here is a helper function that might do what you want. (I haven't tested it yet, but the idea might help)

private static String absolutePathToRelativeString(final Path path, final Path base) {
    final StringBuilder builder = new StringBuilder(path.toString().length());
    Path curPath = new Path(path);
    while (curPath != null && curPath.depth() != 0 && !curPath.equals(base)) {
       if (!curPath.equals(path)) {
          builder.append('/');
       }
       builder.insert(0, curPath.getName());
       curPath = curPath.getParent();
    }
    return builder.toString();
}

Upvotes: 0

Related Questions