Vinay Sharma
Vinay Sharma

Reputation: 381

How to modify the inference API parameters on model card page

I have hosted a model on hugging face. While using the inference API on the model card UI page. the model is generating text with default values. I wish to change max_new_tokens to 250 from default value of 100. How to do this?

enter image description here

using requests we can do this but I wanted to have the changes implemented on the hugging face UI.

import requests

API_ENDPOINT = "https://api-inference.huggingface.co/models/user/model_id"
API_TOKEN = "hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

data = {
    "inputs": "कड़ी मेहनत के महत्व पर एक निबंध लिखें",
    "parameters": {
        "temperature": 0.2,
        "top_k": 500,
        "top_p": 0.9,
        "max_new_tokens": 250,
        "repetition_penalty": 5,
        "do_sample": True,
        "num_return_sequences": 1
    }
}

response = requests.post(API_ENDPOINT, json=data, headers={"Authorization": f"Bearer {API_TOKEN}"})

if response.status_code == 200:
    output = response.json()
    generated_text = output[0]["generated_text"]
    print(generated_text)
else:
    print("Error:", response.text)

Upvotes: 0

Views: 85

Answers (0)

Related Questions