Docker nexus3 repo failing to come up, Orient DB corrupted

Question

The nexus repository have been running in docker for years, never faced any issue. The server it's running on is a VM and using ceph cluster storage where the nexus data is being stored. Last week there was an RAM upgrade activity on the base machine it is on. Post the activity, the nexus setup is not coming up and throwing ERROR as given below: Docker images: sonatype/nexus3

2023-05-22 15:39:02,670+0000 INFO  [ForkJoinPool.commonPool-worker-1] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - Storage 'plocal:/nexus-data/db/config' is opened under OrientDB distribution : 2.2.36 (build d3beb772c02098ceaea89779a7afd4b7305d3788, branch 2.2.x)
2023-05-22 15:39:02,673+0000 ERROR [FelixStartLevel] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - Exception `01867043` in storage `plocal:/nexus-data/db/component`: 2.2.36 (build d3beb772c02098ceaea89779a7afd4b7305d3788, branch 2.2.x)
com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage '/nexus-data/db/component' with mode=rw
    DB name="component"
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:323)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
    at org.sonatype.nexus.orient.DatabaseManagerSupport.connect(DatabaseManagerSupport.java:178)
    at org.sonatype.nexus.orient.DatabaseManagerSupport.createInstance(DatabaseManagerSupport.java:312)
    at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
    at org.sonatype.nexus.orient.DatabaseManagerSupport.instance(DatabaseManagerSupport.java:289)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
    at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
    at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
    at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
    at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
    at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
    at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
    at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
    at org.sonatype.nexus.orient.restore.RestoreServiceImpl.doStart(RestoreServiceImpl.java:76)
    at org.sonatype.nexus.common.stateguard.StateGuardLifecycleSupport.start(StateGuardLifecycleSupport.java:69)
    at org.sonatype.nexus.orient.restore.RestoreServiceImpl$$EnhancerByGuice$$68d24e87.CGLIB$start$4(<generated>)
    at org.sonatype.nexus.orient.restore.RestoreServiceImpl$$EnhancerByGuice$$68d24e87$$FastClassByGuice$$690ee4fc.invoke(<generated>)
    at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
    at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:76)
    at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
    at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:193)
    at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:57)
    at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:78)
    at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:54)
    at org.sonatype.nexus.orient.restore.RestoreServiceImpl$$EnhancerByGuice$$68d24e87.start(<generated>)
    at org.sonatype.nexus.extender.NexusLifecycleManager.startComponent(NexusLifecycleManager.java:199)
    at org.sonatype.nexus.extender.NexusLifecycleManager.to(NexusLifecycleManager.java:111)
    at org.sonatype.nexus.extender.NexusContextListener.moveToPhase(NexusContextListener.java:321)
    at org.sonatype.nexus.extender.NexusContextListener.frameworkEvent(NexusContextListener.java:218)
    at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1431)
    at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
    at com.orientechnologies.orient.core.storage.cache.local.twoq.O2QCache.release(O2QCache.java:448)
    at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurableComponent.releasePage(ODurableComponent.java:166)
    at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.open(OPaginatedCluster.java:224)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.openClusters(OAbstractPaginatedStorage.java:387)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:290)
    ... 37 common frames omitted

After searching a while, got to know that the component database is corrupted, hence the issue. I have tried performing the action recommended on different forums (db backup is taken) but I can't connect to the component but can connect to other databases like security, config etc but not component. The error given is as below:

sh-4.4$ java -jar /opt/sonatype/nexus/lib/support/nexus-orient-console.jar

OrientDB console v.2.2.36 (build d3beb772c02098ceaea89779a7afd4b7305d3788, branch 2.2.x) https://www.orientdb.com
Type 'help' to display all the supported commands.

orientdb> connect plocal:/nexus-data/db/component admin admin


Connecting to database [plocal:/nexus-data/db/component] with user 'admin'...Exception `5D12A356` in storage `plocal:/nexus-data/db/component`: 2.2.36 (build d3beb772c02098ceaea89779a7afd4b7305d3788, branch 2.2.x)
com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage '/nexus-data/db/component' with mode=rw
    DB name="component"
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:323)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.connect(OConsoleDatabaseApp.java:234)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
    at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
    at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
    at org.sonatype.nexus.orient.console.Main.main(Main.java:63)
Caused by: java.lang.NullPointerException
    at com.orientechnologies.orient.core.storage.cache.local.twoq.O2QCache.release(O2QCache.java:448)
    at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurableComponent.releasePage(ODurableComponent.java:166)
    at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.open(OPaginatedCluster.java:224)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.openClusters(OAbstractPaginatedStorage.java:387)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:290)
    ... 11 more

Error: com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage '/nexus-data/db/component' with mode=rw
    DB name="component"

Error: java.lang.NullPointerException

Could anyone please help me recover this database somehow and fix this nexus issue? Meantime, we've setup another new instance so that things keep on running. These are the question where people faced the same issue.

https://issues.sonatype.org/browse/NEXUS-18036 https://mariosk.wordpress.com/2019/09/07/how-a-nexus-repository-manager-corruption-led-to-a-mini-odyssey/ https://technotes.adelerhof.eu/nexus/nexus_database_restore/