CODEWITHSUNDEEP
pythononnxruntime
Ari
Ari

Reputation: 583

Onnx Runtime Adding Multiple Initializers in Python

When trying to prepare the session options for onnx runtime I receive a onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException when trying to add more than one initializer at a time.

See code


import onnxruntime
import numpy as np

params = {"fc1.weight": [3, 4], "fc1.bias": [3]}
options = onnxruntime.SessionOptions()
ortvalue_initializers = []
for p, dim in list(params.items()):
    ortvalue_initializers.append(onnxruntime.OrtValue.ortvalue_from_numpy(
        np.float32(np.random.rand(*dim))
    ))
    options.add_initializer(p, ortvalue_initializers[-1])

net_session = onnxruntime.InferenceSession(
    'net1.onnx',
    sess_options=options,
    providers=["CPUExecutionProvider"])

input = np.random.random((1,4)).astype('f')
output = net_session.run(["output"], {"input": input})
print(output)

Upvotes: 1

Views: 1212

Answers (1)

Ari
Ari

Reputation: 583

Thanks to Pranav Sharma for this solution.

In the code above the second initializer overwrites the first one as you're using the same variable; the first one gets garbage collected and hence the pointer is not valid by the time session is constructed. For this API to work you need to keep the initializers around until you're done with the session. Since you're supplying a memory ptr directly to ORT, it relies on its validity during the Run calls.

Following works.


import onnxruntime
import numpy as np

params = {"fc1.weight": [3, 4], "fc1.bias": [3]}
options = onnxruntime.SessionOptions()
ortvalue_initializers = []
for p, dim in list(params.items()):
    ortvalue_initializers.append(onnxruntime.OrtValue.ortvalue_from_numpy(
        np.float32(np.random.rand(*dim))
    ))
    options.add_initializer(p, ortvalue_initializers[-1])

net_session = onnxruntime.InferenceSession(
    'net1.onnx',
    sess_options=options,
    providers=["CPUExecutionProvider"])

input = np.random.random((1,4)).astype('f')
output = net_session.run(["output"], {"input": input})
print(output)

Upvotes: 4

Related Questions