Ollama

Installing Ollama

MacOS:

brew install ollama

Linux:

curl -fsSL https://ollama.com/install.sh | sh

Install official models

ollama pull <modelname>

Official models

Then use run use the model

Installing custom huggingface models in ollama

First find a .gguf model file and download it.
Then create a modelfile (the filename does not matter)

FROM /path/to/model.gguf

TEMPLATE """<|im_start|>system
<|im_end|>
<|im_start|>user
<|im_end|>
<|im_start|>assistant
"""

SYSTEM """"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

Then run the following command to install the model:

ollama create PreferedModelName -f /path/to/modelfile