Reputation: 63
I want to understand if the training duration is decided by the document number? Say 100 documents cost 4 hours, and 200 documents will need 8 hours? I'm not seeing such linear relationship in my practice.
Upvotes: 0
Views: 933
Reputation: 63
Currently training time does not directly depend on dataset size or labels. Our training in the backend has quite a few components. Some of them are dependent on the dataset size and some are not. The overall net result however is that the longest pole determines the final training time and that's usually not directly dependent on the size of the dataset.
In one example, we are seeing 3.5 hours training spent for a public Kaggle dataset HappyDB, which contains about 12,000 items.
Upvotes: 1