Gorgonzola
Gorgonzola

Reputation: 401

After fresh install of a lambda stack, ai-benchmark is not working

I did a fresh Lambda Stack install. After I tested that tensorflow-gpu was indeed working, I tried running ai-benchmark (https://pypi.org/project/ai-benchmark/) but it is not working.

The output is the following:

Python 3.8.5 (default, Jul 28 2020, 12:59:40) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ai_benchmark import AIBenchmark
2020-11-25 15:25:07.660243: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-25 15:25:07.663124: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
Invalid MIT-MAGIC-COOKIE-1 key>>> results = AIBenchmark().run()

>>   AI-Benchmark-v.0.1.2   
>>   Let the AI Games begin..

2020-11-25 15:25:24.179548: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 3693055000 Hz
2020-11-25 15:25:24.180403: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1a945d0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-11-25 15:25:24.180460: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-11-25 15:25:24.184604: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2020-11-25 15:25:24.261100: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.261618: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xe68480 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-11-25 15:25:24.261652: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1080, Compute Capability 6.1
2020-11-25 15:25:24.261992: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.262970: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce GTX 1080 computeCapability: 6.1
coreClock: 1.7335GHz coreCount: 20 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 298.32GiB/s
2020-11-25 15:25:24.263022: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-25 15:25:24.266092: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-25 15:25:24.267408: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2020-11-25 15:25:24.267768: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2020-11-25 15:25:24.271212: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2020-11-25 15:25:24.272057: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2020-11-25 15:25:24.272208: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-25 15:25:24.272364: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.273190: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.273858: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-11-25 15:25:24.590979: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-25 15:25:24.591018: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 
2020-11-25 15:25:24.591023: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N 
2020-11-25 15:25:24.591243: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.591711: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.592105: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/device:GPU:0 with 7018 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1)
2020-11-25 15:25:24.592836: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.593239: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce GTX 1080 computeCapability: 6.1
coreClock: 1.7335GHz coreCount: 20 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 298.32GiB/s
2020-11-25 15:25:24.593258: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-25 15:25:24.593294: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-25 15:25:24.593306: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2020-11-25 15:25:24.593316: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2020-11-25 15:25:24.593327: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2020-11-25 15:25:24.593337: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2020-11-25 15:25:24.593347: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-25 15:25:24.593398: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.593806: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.594170: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-11-25 15:25:24.594193: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-25 15:25:24.594199: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 
2020-11-25 15:25:24.594203: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N 
2020-11-25 15:25:24.594272: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.594681: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.595055: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/device:GPU:0 with 7018 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1)
2020-11-25 15:25:24.595330: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.595714: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce GTX 1080 computeCapability: 6.1
coreClock: 1.7335GHz coreCount: 20 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 298.32GiB/s
2020-11-25 15:25:24.595730: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-25 15:25:24.595756: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-25 15:25:24.595766: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2020-11-25 15:25:24.595777: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2020-11-25 15:25:24.595787: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2020-11-25 15:25:24.595796: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2020-11-25 15:25:24.595806: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-25 15:25:24.595853: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.596259: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.596622: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-11-25 15:25:24.596641: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-25 15:25:24.596646: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 
2020-11-25 15:25:24.596651: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N 
2020-11-25 15:25:24.596716: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.597125: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:24.597497: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/device:GPU:0 with 7018 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1)
*  TF Version: 2.3.1
*  Platform: Linux-5.4.0-54-generic-x86_64-with-glibc2.29
*  CPU: N/A
*  CPU RAM: 16 GB
*  GPU/0: GeForce GTX 1080
*  GPU RAM: 6.9 GB
*  CUDA Version: 11.1
*  CUDA Build: V11.1.74

The benchmark is running...
The tests might take up to 20 minutes
Please dont interrupt the script

1/19. MobileNet-V2

2020-11-25 15:25:31.191492: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:31.192235: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce GTX 1080 computeCapability: 6.1
coreClock: 1.7335GHz coreCount: 20 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 298.32GiB/s
2020-11-25 15:25:31.192286: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-25 15:25:31.192354: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-25 15:25:31.192387: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2020-11-25 15:25:31.192418: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2020-11-25 15:25:31.192448: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2020-11-25 15:25:31.192474: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2020-11-25 15:25:31.192500: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-25 15:25:31.192644: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:31.193372: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:31.194012: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-11-25 15:25:31.194810: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:31.195522: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce GTX 1080 computeCapability: 6.1
coreClock: 1.7335GHz coreCount: 20 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 298.32GiB/s
2020-11-25 15:25:31.195556: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-25 15:25:31.195604: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-25 15:25:31.195632: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2020-11-25 15:25:31.195658: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2020-11-25 15:25:31.195681: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2020-11-25 15:25:31.195700: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2020-11-25 15:25:31.195724: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-25 15:25:31.195822: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:31.196510: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:31.197127: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-11-25 15:25:31.197169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-25 15:25:31.197181: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 
2020-11-25 15:25:31.197193: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N 
2020-11-25 15:25:31.197330: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:31.198053: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-25 15:25:31.198703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7018 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1)
2020-11-25 15:25:32.271855: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-25 15:25:32.835532: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
1.1 - inference | batch=50, size=224x224: 57.9 ± 1.5 ms
Could not load library libcudnn_cnn_train.so.8. Error: libcudnn_ops_train.so.8: cannot open shared object file: No such file or directory
Please make sure libcudnn_cnn_train.so.8 is in your library path!
[MINI:25659] *** Process received signal ***
[MINI:25659] Signal: Aborted (6)
[MINI:25659] Signal code:  (-6)
[MINI:25659] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x46210)[0x7fd617cd0210]
[MINI:25659] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7fd617cd018b]
[MINI:25659] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7fd617caf859]
[MINI:25659] [ 3] /usr/lib/python3/dist-packages/tensorflow/python/../libcudnn.so.8(cudnnGetConvolutionBackwardFilterWorkspaceSize+0xc9)[0x7fd5a4089959]
[MINI:25659] [ 4] /usr/lib/python3/dist-packages/tensorflow/python/../libtensorflow_framework.so.2(cudnnGetConvolutionBackwardFilterWorkspaceSize+0x5f)[0x7fd5da35f9cf]
[MINI:25659] [ 5] /usr/lib/python3/dist-packages/tensorflow/python/../libtensorflow_framework.so.2(+0x1516c4d)[0x7fd5da340c4d]
[MINI:25659] [ 6] /usr/lib/python3/dist-packages/tensorflow/python/../libtensorflow_framework.so.2(_ZN15stream_executor3gpu12CudnnSupport23DoPrepareForConvolutionENS_3dnn15ConvolutionKindENS2_8DataTypeEPNS_6StreamERKNS2_15BatchDescriptorENS_16DeviceMemoryBaseERKNS2_16FilterDescriptorESA_S9_SA_RKNS2_21ConvolutionDescriptorERKNS2_15AlgorithmConfigEPNS_16ScratchAllocatorEPNS2_13AlgorithmDescEPNS_12DeviceMemoryIhEE+0xfc9)[0x7fd5da34bdd9]
[MINI:25659] [ 7] /usr/lib/python3/dist-packages/tensorflow/python/_pywrap_tensorflow_internal.cpython-38-x86_64-linux-gnu.so(_ZN15stream_executor6Stream39ThenConvolveBackwardFilterWithAlgorithmERKNS_3dnn15BatchDescriptorERKNS_12DeviceMemoryIfEES4_S6_RKNS1_21ConvolutionDescriptorERKNS1_16FilterDescriptorEPS6_PNS_16ScratchAllocatorERKNS1_15AlgorithmConfigEPNS1_13ProfileResultE+0xa4b)[0x7fd5f1a781ab]
[MINI:25659] [ 8] /usr/lib/python3/dist-packages/tensorflow/python/_pywrap_tensorflow_internal.cpython-38-x86_64-linux-gnu.so(_ZN10tensorflow28LaunchConv2DBackpropFilterOpIN5Eigen9GpuDeviceEfEclEPNS_15OpKernelContextEbbRKNS_6TensorES8_iiiiRKNS_7PaddingERKSt6vectorIxSaIxEEPS6_NS_12TensorFormatE+0x1a18)[0x7fd5edc5e4b8]
[MINI:25659] [ 9] /usr/lib/python3/dist-packages/tensorflow/python/_pywrap_tensorflow_internal.cpython-38-x86_64-linux-gnu.so(_ZN10tensorflow22Conv2DBackpropFilterOpIN5Eigen9GpuDeviceEfE7ComputeEPNS_15OpKernelContextE+0x23b)[0x7fd5edc5f8bb]
[MINI:25659] [10] /usr/lib/python3/dist-packages/tensorflow/python/../libtensorflow_framework.so.2(_ZN10tensorflow13BaseGPUDevice7ComputeEPNS_8OpKernelEPNS_15OpKernelContextE+0x245)[0x7fd5d9e47675]
[MINI:25659] [11] /usr/lib/python3/dist-packages/tensorflow/python/../libtensorflow_framework.so.2(+0x10e2307)[0x7fd5d9f0c307]
[MINI:25659] [12] /usr/lib/python3/dist-packages/tensorflow/python/../libtensorflow_framework.so.2(+0x10e3c13)[0x7fd5d9f0dc13]
[MINI:25659] [13] /usr/lib/python3/dist-packages/tensorflow/python/../libtensorflow_framework.so.2(_ZN5Eigen15ThreadPoolTemplIN10tensorflow6thread16EigenEnvironmentEE10WorkerLoopEi+0x2a5)[0x7fd5d9facf75]
[MINI:25659] [14] /usr/lib/python3/dist-packages/tensorflow/python/../libtensorflow_framework.so.2(_ZNSt17_Function_handlerIFvvEZN10tensorflow6thread16EigenEnvironment12CreateThreadESt8functionIS0_EEUlvE_E9_M_invokeERKSt9_Any_data+0x47)[0x7fd5d9fa9f57]
[MINI:25659] [15] /usr/lib/python3/dist-packages/tensorflow/python/../libtensorflow_framework.so.2(+0x1171cef)[0x7fd5d9f9bcef]
[MINI:25659] [16] /lib/x86_64-linux-gnu/libpthread.so.0(+0x9609)[0x7fd617c70609]
[MINI:25659] [17] /lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7fd617dac293]
[MINI:25659] *** End of error message ***
Aborted (core dumped)

Can someone point me in the right direction? Many thanks.

Please forgive the following, I am told my post is mostly code so I should increase my text.

"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."

Upvotes: 2

Views: 827

Answers (1)

Theodore Popp
Theodore Popp

Reputation: 856

Could not load library libcudnn_cnn_train.so.8. Error: libcudnn_ops_train.so.8: cannot open shared object file: No such file or directory is the important line.

First, you need to ensure that you have the relevant libcudnn file on your computer (using find or locate). If you don't you need to download them.

Then you need to ensure that your PATH/LD_LIBRARY_PATH variables point to the directories that contain those files.

This discussion here is from 2 years ago and thus mentions different files, but the rest of the discussion should give examples to base your fix on: https://github.com/tensorflow/tensorflow/issues/20271

Upvotes: 1

Related Questions