CODEWITHSUNDEEP
pysparkjohnsnowlabs-spark-nlpnlu
RodrigoJonas
RodrigoJonas

Reputation: 1

How to load spark nlu model from pre-loaded file?

So, I'm using a pyspark nlu model and the time it takes to load the model (model.load('model.name')) is really long and I'd like to store the loading locally in my PC so I can access it without going through the whole .load() process.

I've tried:

import nlu
import os
from urllib.request import pathname2url

if len(os.listdir(path)) == 0:
    print('empty')
    model = nlu.load('pt.embed.BR_BERTo')
    model.save(pathname2url(path))
else:
    print('not empty')
    model = nlu.load_nlu_pipe_from_hdd(path)

And I was expecting that load_nlu_pipe_from_hdd() would load the stored model but it doesn't seem to have any results and I can't find documentation about it.

Upvotes: 0

Views: 18

Answers (0)

Related Questions