StackOverflow Questions for Tag: llamacpp

khalidwalamri

Reputation: 21

Running Local LLMs in Production and handling multiple requests

flaskllama-indexllama-cpp-pythonllamacppgemma

Score: 2

Answers: 1

arkuzo

Reputation: 41

How to quantize a HF safetensors model and save it to llama.cpp GGUF format with less than q8_0 quantization?

large-language-modelhuggingfacequantizationllamacpp

Score: 2

Answers: 1

sandeep

Reputation: 163

while pip install llama-cpp-python getting error on windows pc

pythonllamallama-cpp-pythonllamacpp

Score: 0

Answers: 1

evashort

Reputation: 1

Generating an n-gram dataset based on an LLM

nlplarge-language-modeln-gramllama-cpp-pythonllamacpp

Score: 0

Answers: 0

reach

Reputation: 21

I have problem using n_gpu_layers in llama_cpp Llama function

large-language-modelllama-cpp-pythonllamacpp

Score: 1

Answers: 0

caveman

Reputation: 444

How to use `llama-cpp-python` to output list of candidate tokens and their probabilities?

large-language-modelllama-cpp-pythonllamacpp

Score: 1

Answers: 1

ekcrisp

Reputation: 1911

How do you enable runtime-repack in llama cpp python?

pythonpython-3.xpipllama-cpp-pythonllamacpp

Score: 0

Answers: 0

Programmer.zip

Reputation: 809

Detecting GPU availability in llama-cpp-python

pythongpullama-cpp-pythonllamacpp

Score: 2

Answers: 2

Charles Ju

Reputation: 1279

nearest_int and make_qx_quants from llama.cpp

llamacpp

Score: 0

Answers: 0

Maxl Gemeinderat

Reputation: 555

CMAKE in requirements.txt file: Install llama-cpp-python for Mac

pythonmacosllama-cpp-pythonllamacpp

Score: 0

Answers: 2

Knox

Reputation: 21

Wheel Build error while installing LLama-CPP-Python on python

pythoncmakellama-cpp-pythonllamacpp

Score: 0

Answers: 0

Owen Zhang

Reputation: 23

Issue building llama.cpp with hipBLAS backend. error: call to undeclared function '_mm256_dpbusd_epi32'

amdllamallamacppamd-rocmhip

Score: 0

Answers: 0

VanechikSpace

Reputation: 555

Is llama able to choose tools automatically?

llamallamacpp

Score: 1

Answers: 0

Monet Geoffrey

Reputation: 11

How to use llm models downloaded with ollama with llama.cpp?

ollamallamacpp

Score: 1

Answers: 1

VanechikSpace

Reputation: 555

Why does llamacpp add a phrase to my prompt and print it out in the output?

large-language-modelhuggingfacellama-cpp-pythonllamacpp

Score: 0

Answers: 0

Jengi829

Reputation: 1

Unable to set top_k value in Llama cpp Python server

python-3.xopenai-apillamallama-cpp-pythonllamacpp

Score: 0

Answers: 0

Krish Parikh

Reputation: 31

Unable to make llama.cpp on M1 Mac

apple-m1large-language-modelllamallama-cpp-pythonllamacpp

Score: 1

Answers: 1

bibiibibin

Reputation: 1

Does langchain with llama-cpp-python fail to work with very long prompts?

langchainollamallama-cpp-pythonllamacppllama3

Score: 0

Answers: 1

ritwikv

Reputation: 1

Defining Agent in LLamaIndex and Mistral 7B is throwing Attribute error

large-language-modelllamallama-indexllama-cpp-pythonllamacpp

Score: 0

Answers: 0

6core

Reputation: 1

Persist VectorStoreIndex (LlamaIndex) locally

large-language-modelllama-indexllamacpp

Score: 0

Answers: 2

PreviousPage 1Next

StackOverflow Questions for Tag: llamacpp

Running Local LLMs in Production and handling multiple requests

How to quantize a HF safetensors model and save it to llama.cpp GGUF format with less than q8_0 quantization?

while pip install llama-cpp-python getting error on windows pc

Generating an n-gram dataset based on an LLM

I have problem using n_gpu_layers in llama_cpp Llama function

How to use `llama-cpp-python` to output list of candidate tokens and their probabilities?

How do you enable runtime-repack in llama cpp python?

Detecting GPU availability in llama-cpp-python

nearest_int and make_qx_quants from llama.cpp

CMAKE in requirements.txt file: Install llama-cpp-python for Mac

Wheel Build error while installing LLama-CPP-Python on python

Issue building llama.cpp with hipBLAS backend. error: call to undeclared function &#39;_mm256_dpbusd_epi32&#39;

Is llama able to choose tools automatically?

How to use llm models downloaded with ollama with llama.cpp?

Why does llamacpp add a phrase to my prompt and print it out in the output?

Unable to set top_k value in Llama cpp Python server

Unable to make llama.cpp on M1 Mac

Does langchain with llama-cpp-python fail to work with very long prompts?

Defining Agent in LLamaIndex and Mistral 7B is throwing Attribute error

Persist VectorStoreIndex (LlamaIndex) locally

Issue building llama.cpp with hipBLAS backend. error: call to undeclared function '_mm256_dpbusd_epi32'