CODEWITHSUNDEEP
marklogicmlcpmarklogic-11
Ashish Patil
Ashish Patil

Reputation: 192

MLCP Logging Issues Post-Upgrade for MarkLogic, Gradle, and MLCP

We recently upgraded the following components:

Component Before After
MarkLogic Server 10.0-9.2 11.1.0
Gradle 5.2.1 6.4
mlcp/XCC 9.0.8/9.0.7 11.0.3

After the upgrade, our scheduled MLCP tasks are no longer logging details such as record counts. Instead, the logs show errors, although updates are still being successfully applied to the MarkLogic database.

Logs Before Upgrade:

Setting the GRADLE_USER_HOME to /ML-Config/Deployment-Workspace/D2C/Version-2.8-offline/build/gradle-home \n(needed for the creation of the offline repository)\n
Using offline plugin repository
Using offline repositories
For Jackson Kotlin classes support please add "com.fasterxml.jackson.module:jackson-module-kotlin" to the classpath
Using offline repositories
input proper collection null
input proper collecton null
input proper collecton null
:NeoLoad
mlcp arguments, excluding password: [IMPORT, -port, XXXX, -output_uri_prefix, /XXX/, -document_type, XML, -output_uri_replace, .*XXX/,'',\.xml,'', -output_collections, /type/XXX, -output_uri_suffix, .xml, -input_file_path, /mlcp-data/XXX/XXX/, -delimiter, ;, -input_file_type, documents, -output_permissions, rest-reader,read,rest-writer,update, -generate_uri, true, -host, XX.XX.XX.XX, -username, XXXXX]
[main] LocalJobRunner                 INFO  Content type: XML
[main] NativeCodeLoader               WARN  Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[main] ContentPump                    INFO  Job name: local_1784781227_1
[main] FileAndDirectoryInputFormat    INFO  Total input paths to process : 130
[Thread-6] LocalJobRunner                 INFO   completed 40%
[Thread-6] LocalJobRunner                 INFO   completed 50%
[Thread-6] LocalJobRunner                 INFO   completed 60%
[Thread-6] LocalJobRunner                 INFO   completed 70%
[Thread-6] LocalJobRunner                 INFO   completed 80%
[Thread-6] LocalJobRunner                 INFO   completed 90%
[Thread-6] LocalJobRunner                 INFO   completed 100%
[main] LocalJobRunner                 INFO  com.marklogic.mapreduce.MarkLogicCounter: 
[main] LocalJobRunner                 INFO  INPUT_RECORDS: 130
[main] LocalJobRunner                 INFO  OUTPUT_RECORDS: 130
[main] LocalJobRunner                 INFO  OUTPUT_RECORDS_COMMITTED: 130
[main] LocalJobRunner                 INFO  OUTPUT_RECORDS_FAILED: 0
[main] LocalJobRunner                 INFO  Total execution time: 192 sec

BUILD SUCCESSFUL in 3m 23s

Logs After Upgrade:

To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/6.4/userguide/gradle_daemon.html.
Daemon will be stopped at the end of the build stopping after processing
Using offline plugin repository

> Configure project :
Using offline repositories
Using offline repositories
Please update your dependency to directly use the correct version 'xml-apis:xml-apis:1.0.b2'.
Resolution will only pick dependencies of the relocated element.  Artifacts and other metadata will be ignored.
input proper collection null
input proper collecton null
input proper collecton null

> Task :NeoLoad
mlcp arguments, excluding known password arguments: [IMPORT, -port, XXXX, -output_uri_prefix, /XXX/, -document_type, XML, -output_uri_replace, .*XXX/,'',\.xml,'', -output_collections, /type/XXX, -output_uri_suffix, .xml, -input_file_path, /mlcp-data/XXX/XXX/, -delimiter, ;, -input_file_type, documents, -output_permissions, rest-reader,read,rest-writer,update, -generate_uri, true, -host, XX.XX.XX.XX, -username, XXXXX]
mlcp arguments, excluding known password arguments: [IMPORT, -port, 8000, -output_uri_prefix, /neo/, -document_type, XML, -output_uri_replace, .*NEO/,'',\.xml,'', -output_collections, /type/neo, -output_uri_suffix, .xml, -input_file_path, /mlcp-data/D2C/NEO, -delimiter, ;, -input_file_type, documents, -output_permissions, rest-reader,read,rest-writer,update, -generate_uri, true, -username, admin, -database, BackupDemo, -host, localhost]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/ML-Config/Deployment-Workspace/DHF5.8_D2C/design-to-cost/Version-2.8-offline/build/offline/maven-repo/ch/qos/logback/logback-classic/1.2.10/logback-classic-1.2.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/ML-Config/Deployment-Workspace/DHF5.8_D2C/design-to-cost/Version-2.8-offline/build/offline/maven-repo/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/ML-Config/Deployment-Workspace/DHF5.8_D2C/design-to-cost/Version-2.8-offline/build/offline/maven-repo/org/slf4j/slf4j-reload4j/1.7.36/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/ML-Config/Deployment-Workspace/DHF5.8_D2C/design-to-cost/Version-2.8-offline/build/offline/maven-repo/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See
http://www.slf4j.org/codes.html#multiple_bindings
for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
21:00:13.629 [main] WARN  o.a.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.4/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 3m 20s
1 actionable task: 1 executed

I need assistance in identifying the cause of the logging issue and how to resolve it. Any insights would be greatly appreciated.

Upvotes: 1

Views: 65

Answers (1)

rjrudin
rjrudin

Reputation: 2236

I just tested the config in the ml-gradle example project for MLCP - https://github.com/marklogic/ml-gradle/tree/master/examples/mlcp-project - and the logging shows up fine. However - if I bump MLCP to 11.2.0, the logging does not show up.

I then downloaded the MLCP 11.2.0 zip, and it is now using a log4j2.xml file. I added that to the lib directory in the ml-gradle example project, and the logging started showing up again.

I recommend bumping up your version of Gradle to at least the latest 7.x - Gradle 6.x is fairly old at this point, as Gradle 8.7 is available.

Upvotes: 1

Related Questions