llama-cpp-agent

About

Not Longer Maintained. Use instead ToolAgents or other Python agentic frameworks.

The llama-cpp-agent framework is a tool designed to simplify interactions with Large Language Models (LLMs). It provides an interface for chatting with LLMs, executing function calls, generating structured output, performing retrieval augmented generation, and processing text using agentic chains with tools.

The framework uses guided sampling to constrain the model output to the user defined structures. This way also models not fine-tuned to do function calling and JSON output will be able to do it.

The framework is compatible with the llama.cpp server, llama-cpp-python and its server, and with TGI and vllm…