Ollama: Run, build, and share LLMs

Ollama is a tool for running large language models on macOS, especially on Apple Silicon devices. It simplifies the process of installing and running Llama2, a powerful and versatile language model that can handle various natural language tasks. With Ollama, users can easily create and deploy applications that use Llama2 without worrying about compatibility issues or performance bottlenecks.

To run and chat with the Meta’s new model, Llama 2:

ollama run llama2

Model library

ollama contains a library of open-source models:

Examples

Run the model

ollama run llama2
>>> hi
Hello! How can I help you today?

Create a custom model

Pull a base model:

ollama pull llama2

Create the Modelfile:

FROM llama2

# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1

# set the system prompt
SYSTEM """
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
"""

Then, build and run the model:

ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.

Check the examples directory for more examples

Pull a model from the registry

ollama pull orca

Listing local models

ollama list

Model packages

Overview

A Modelfile is a file that specifies the data, configuration, and model weights for Ollama bundles. Ollama bundles are packages that contain everything needed to run a machine-learning model.

Build

go build .

To run it, you have to start the server:

./ollama serve &

Finally, run the model!

./ollama run llama2

REST API

POST /api/generate

Generate text from a model.

curl -X POST http://localhost:11434/api/generate -d '{"model": "llama2", "prompt":"Why is the sky blue?"}'