No FileSystem for scheme: hdfs when building fat jar in Spark; works fine in Eclipse mars

Question

I get that error when I do a fat jar with my spark program and then I launch it using:

java -jar -myApp.jar [args]

When I run my app in Eclipse, it works perfectly well. Here is my POM:

 <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
   <maven.compiler.source>1.7</maven.compiler.source>
    <maven.compiler.target>1.7</maven.compiler.target>
  </properties>

  <dependencies>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming-kafka_2.10</artifactId>
    <version>1.5.0</version>
</dependency>
<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka_2.10</artifactId>
    <version>0.9.0.1</version>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming_2.10</artifactId>
    <version>1.6.1</version>
</dependency>
<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.6.2</version>
</dependency>
<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-hdfs</artifactId>
    <version>2.6.0</version>
</dependency>
 </dependencies>
 <build>
<plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>2.1</version>
                <executions>
                    <execution>
                    <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <source>1.7</source>
                            <target>1.7</target>
                            <transformers>
                                <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                                    <resource>reference.conf</resource>
                                </transformer>
                                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>gft.hbaseMedia.MainSparkCopyHDFS</mainClass>
                                </transformer>
                            </transformers>
                                <filters>
                                    <filter>
                                        <artifact>*:*</artifact>
                                        <excludes>  
                                            <exclude>META-INF/*.SF</exclude>
                                            <exclude>META-INF/*.DSA</exclude>
                                            <exclude>META-INF/*.RSA</exclude>
                                        </excludes>
                                    </filter>
                                </filters>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
    </plugins>
    <resources>
            <resource>
                <directory>resources</directory>
            </resource>
        </resources>
  </build>
</project>

The piece of code that gives problems is:

Configuration conf = new Configuration();
conf.set("fs.defaultFS", hdfsConfPath);
FileSystem fs = FileSystem.get(conf);

I know it's Maven that's screwing up things somehow, but I don't know how to solve this issue!

Answer 1

It looks like the answer @amitmah pointed me to was indeed the right one (talking about the second most voted one, the maven-shade transformer).
The first one gives a ClassNotFound Exception and the maven shade transformer one gives it too.
The solution for me was to use only what Sparks already has,
Apply the answer given in the comments under my question (for people who use Maven Shade),
delete the hdfs dependency, as it creates some kind of conflicts at runtime, so no error during the maven build.
Here are the transformers in the pom:

 <transformers>
         <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
         <resource>reference.conf</resource>
         </transformer>
         <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
         <mainClass>gft.hbaseMedia.MainSparkCopyHDFS</mainClass>
         </transformer>
         <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>