GitHub - getumbrel/llama-gpt: A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
A self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. 100% private, with no data leaving your device.
New: Support for Code Llama models and Nvidia GPUs.
umbrel.com (we're hiring) ยป
Contents
- Demo
- Supported Models
- How to install
- OpenAI-compatible API
- Benchmarks
- Roadmap and contributing
- Acknowledgements
Demo
LlamaGPT.mp4Supported models
Currently, LlamaGPT supports the following models. Support for running custom models is on the roadmap.
| Model name | Model size | Model download size | Memory required |
|---|---|---|---|
| Nous Hermes Llama 2 7B Chat (GGML q4_0) | 7B | 3.79GB | 6.29GB |
| Nous Hermes Llama 2 13B Chat (GGML q4_0) | 13B | 7.32GB | 9.82GB |
| Nous Hermes Llama 2 70B Chat (GGML q4_0) | 70B | 38.87GB | 41.37GB |
| Code Llama 7B Chat (GGUF Q4_K_M) | 7B | 4.24GB | 6.74GB |
| Code Llama 13B Chat (GGUF Q4_K_M) | 13B | 8.06GB | 10.56GB |
| Phind Code Llama 34B Chat (GGUF Q4_K_M) | 34B | 20.22GB | 22.72GB |
How to install
Install LlamaGPT on your umbrelOS home server
Running LlamaGPT on an umbrelOS home server is one click. Simply install it from the Umbrel App Store.
Install LlamaGPT on M1/M2 Mac
Make sure your have Docker and Xcode installed.
Then, clone this repo and cd into it: