Skip to main content

Llama.cpp

The llama provider is compatible with the HTTP server bundled with llama.cpp.

You can use it by specifying llama as the provider.

Supported environment variables:

  • LLAMA_BASE_URL - scheme, host name, and port (defaults to http://localhost:8080)