CODEWITHSUNDEEP
xmlloggingapache-sparklog4jasyncappender
Noam Shaish
Noam Shaish

Reputation: 1623

Configuring Spark logging with log4j.xml

I guesss its a silly question, but I couldnt find answer anywhere.

Can I configure logging in spark using log4j.xml?

In spark documentation its mentioned you can configure logging with log4j.properties, I wish to use log4j.xml for more advance log4j capabilities such as async appender. my job will run in cluster mode over yarn (CDH) scheduled with oozie. Im aware ill need in any solution to use --files

Upvotes: 2

Views: 2313

Answers (1)

Holden
Holden

Reputation: 7442

You can set the spark.executor.extraJavaOptions to add -Dlog4j.configuration=log4j.xml and include your log4j.xml file on the classpath of the workers (either bundling in the application jar or adding to the files included with --files).

Upvotes: 8

Related Questions