Llama 3 deployed on Vertex AI generating completions than fulfilling prompt

Question

I have deployed a Llama 3 70b chat from Vertex AI's Model Garden.

When making prompts, it seems like the model is trying to complete the prompt rather than trying to answer it, e.g. with JSON request of the below params:

Prompt:

Classify the below as fruit or vegetable:

Examples:
Apple - fruit
Spinach - vegetable

Classify these:
Brinjal
Orange
Lettuce
Corn

Params

temperature: 0
max_tokens: 1024
stop_token_ids": [128009]

Response

Prompt:
Classify the below as fruit or vegetable:

Examples:
Apple - fruit
Spinach - vegetable

Classify these:
Brinjal
Orange
Lettuce
Corn
Output:
Pineapple
Grapes<|eot_id|>

Am I missing some parameters to the model deployment to get the model to respond to the prompt as instructions?

Answer 1

Add parameter : stop_sequences=["<|start_header_id|>", "<|end_header_id|>", "<|eot_id|>", "<|reserved_special_token"]

Use llama3 prompt technique for better response https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/