CODEWITHSUNDEEP
pythonc#artificial-intelligencegpt-3large-language-model
user10176535
user10176535

Reputation:

Alpaca Large Language Model from Python script

I was able to install Alpaca under Linux and start and use it interactivelly via the corresponding ./chat command.

However, I would like to run it not in interactive mode but from a Python (Jupyter) script with the prompt as string parameter. Also, it should be possible to call the model several times without needing to reload it each time.

I already wrote a Python script that works technically:

import subprocess

# start the Alpaca model as a subprocess 
alpaca_process = subprocess.Popen(["./chat"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)

# send an initial newline to the subprocess to ensure it's ready to receive input 
alpaca_process.stdin.write("\n") 
alpaca_process.stdin.flush()

def alpaca_predict(prompt):
    # send the prompt to Alpaca and get the output
    alpaca_process.stdin.write(prompt + "\n")
    alpaca_process.stdin.flush()
    output = alpaca_process.stdout.readline().strip()
    return output

# test the function 
prompts = ["Hello", "What is the meaning of life?", "Tell me a joke", "Goodbye"] 
for prompt in prompts:
    response = alpaca_predict(prompt)
    print(f"Prompt: {prompt} - Response: {response}")

It works technically now, but unfortunately the model produces only nonsense like this:

Prompt: Hello - Response: 
Prompt: What is the meaning of life? - Response: > The following are some of the most popular programming languages used in web development today, ranked by market share (source Stack Overflow): 1) JavaScript; 2) Python; 3) Java/Javascript hybrid language such as Node.js and AngularJS; 4) PHP; 5) Ruby on Rails
Prompt: Tell me a joke - Response: 
Prompt: Goodbye - Response: ## Instruction: Create a list of the most popular programming languages used in web development today, ranked by market share (source Stack Overflow).

How to fix this?

Upvotes: 2

Views: 1645

Answers (2)

Satheesh Gorla
Satheesh Gorla

Reputation: 1

make sure your passing the prompt in a particular manner.prompt_1 = "### Instruction: write the product description based on attributes=[Fit-Type : Slim Fit, color : Blue, Neck : Round Neck, Pattern : Printed ,Sleeve Styling : Regular Sleeves, Material: Pure Cotton].don't include other text\n\n### Response: "

Upvotes: 0

Ivan Viti
Ivan Viti

Reputation: 271

I think the problem is your original \n: alpaca_process.stdin.write("\n") it is causing alpaca to think that you want a response to nothing basically. If you comment that out it seems to work

Upvotes: 0

Related Questions