CODEWITHSUNDEEP
tensorflowgoogle-colaboratorybert-language-modeltensorflow-hub
jjjayn
jjjayn

Reputation: 555

How to use a custom model with Tensorflow Hub?

My goal is to test out Google's BERT algorithm in Google Colab.

I'd like to use a pre-trained custom model for Finnish (https://github.com/TurkuNLP/FinBERT). The model can not be found on TFHub library. I have not found a way to load model with Tensorflow Hub.

Is there a neat way to load and use a custom model with Tensorflow Hub?

Upvotes: 0

Views: 484

Answers (1)

arnoegw
arnoegw

Reputation: 1238

Fundamentally: yes. Everyone can create the kind of models that TF Hub hosts, and I hope authors of interesting models do consider that.

That said, a sophisticated model like BERT requires a bit of attention to export it with all bells and whistles, so it helps to have some tooling to build on. The BERT reference implementation for TF2 at https://github.com/tensorflow/models/tree/master/official/nlp/bert comes with an open-sourced export_tfhub.py script, and anyone can use that to export custom BERT instances created from that code base.

However, I understand from https://github.com/TurkuNLP/FinBERT/blob/master/nlpl_tutorial/training_bert.md#general-info that you are using Nvidia's fork of the original TF1 implementation of BERT. There are Hub modules created from the original research code, but the tooling to that end has not been open-sourced, and Nvidia doesn't seem to have added their own either.

If that's not changing, you'll probably have to resort to doing things the pedestrian way and get acquainted with their codebase and load their checkpoints into it.

Upvotes: 2

Related Questions