Reputation: 381
I have hosted a model on hugging face. While using the inference API on the model card UI page. the model is generating text with default values. I wish to change max_new_tokens to 250 from default value of 100. How to do this?
using requests
we can do this but I wanted to have the changes implemented on the hugging face UI.
import requests
API_ENDPOINT = "https://api-inference.huggingface.co/models/user/model_id"
API_TOKEN = "hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
data = {
"inputs": "कड़ी मेहनत के महत्व पर एक निबंध लिखें",
"parameters": {
"temperature": 0.2,
"top_k": 500,
"top_p": 0.9,
"max_new_tokens": 250,
"repetition_penalty": 5,
"do_sample": True,
"num_return_sequences": 1
}
}
response = requests.post(API_ENDPOINT, json=data, headers={"Authorization": f"Bearer {API_TOKEN}"})
if response.status_code == 200:
output = response.json()
generated_text = output[0]["generated_text"]
print(generated_text)
else:
print("Error:", response.text)
Upvotes: 0
Views: 85