Ollama is a tool for running large language models on macOS, especially on Apple Silicon devices. It simplifies the process of installing and running Llama2, a powerful and versatile language model that can handle various natural language tasks. With Ollama, users can easily create and deploy applications that use Llama2 without worrying about compatibility issues or performance bottlenecks.
To run and chat with the Meta’s new model, Llama 2:
ollama run llama2
Model library
ollama contains a library of open-source models:
| Model | Parameters | Size | Download |
|---|---|---|---|
| Llama2 | 7B | 3.8GB | ollama pull llama2 |
| Llama2 13B | 13B | 7.3GB | ollama pull llama2:13b |
| Orca Mini | 3B | 1.9GB | ollama pull orca |
| Vicuna | 7B | 3.8GB | ollama pull vicuna |
| Nous-Hermes | 13B | 7.3GB | ollama pull nous-hermes |
| Wizard Vicuna Uncensored | 13B | 7.3GB | ollama pull wizard-vicuna |
The 3B models require a minimum of 8 GB of RAM, the 7B models need at least 16 GB of RAM, and the 13B models demand 32 GB of RAM or more.
Examples
Run the model
ollama run llama2
>>> hi
Hello! How can I help you today?
Create a custom model
Pull a base model:
ollama pull llama2
Create the Modelfile:
FROM llama2
# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
# set the system prompt
SYSTEM """
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
"""
Then, build and run the model:
ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
Check the examples directory for more examples
Pull a model from the registry
ollama pull orca
Listing local models
ollama list
Model packages
Overview
A Modelfile is a file that specifies the data, configuration, and model weights for Ollama bundles. Ollama bundles are packages that contain everything needed to run a machine-learning model.
Build
go build .
To run it, you have to start the server:
./ollama serve &
Finally, run the model!
./ollama run llama2
REST API
POST /api/generate
Generate text from a model.
curl -X POST http://localhost:11434/api/generate -d '{"model": "llama2", "prompt":"Why is the sky blue?"}'
0 Comments