Provider Integrations#

This page provides an overview of all available provider integrations in the Serapeum framework. Providers implement the core abstractions to work with different LLM backends and services.

Overview#

Serapeum uses a provider-based organization where each provider package contains all features that provider offers (LLM, embeddings, and any provider-specific capabilities). This keeps related code together and makes it easy to install only the providers you need.

Available Providers#

• Ollama - Local LLM inference with Ollama server
• OpenAI - OpenAI API integration (coming soon)
• Azure OpenAI - Azure OpenAI Service integration (in development)

Provider Architecture#

All providers follow the same architectural pattern:

graph TB
    subgraph "Core Abstractions"
        A[BaseLLM Protocol]
        B[BaseEmbedding Protocol]
        C[FunctionCallingLLM]
    end

    subgraph "Provider Package"
        D[ProviderLLM]
        E[ProviderEmbedding]
        F[Shared Client]
        G[Provider Errors]
    end

    C --> A
    D --> C
    E --> B
    D --> F
    E --> F
    F --> G

Key Components:

1. LLM Implementation: Inherits from FunctionCallingLLM to provide chat, completion, tool calling, and structured outputs
2. Embedding Implementation: Implements BaseEmbedding protocol for text and query embeddings
3. Shared Client: HTTP client and configuration shared across LLM and embedding classes
4. Provider Errors: Custom exceptions for provider-specific error handling

Provider Comparison#

Installation#

Each provider is distributed as a separate package. Install only the providers you need:

# Ollama provider
pip install serapeum-ollama

# OpenAI provider (when available)
pip install serapeum-openai

# Azure OpenAI provider (when available)
pip install serapeum-azure-openai

# Install multiple providers
pip install serapeum-ollama serapeum-openai

All provider packages depend on serapeum-core, which will be installed automatically.

Quick Comparison#

Ollama#

Read Ollama Documentation →

OpenAI (Coming Soon)#

Azure OpenAI (Coming Soon)#

Adding New Providers#

Want to integrate a new LLM provider (OpenAI, Anthropic, Cohere, etc.) into Serapeum?

We've created a comprehensive guide that walks you through every step of implementing a new provider integration, from directory structure to testing and documentation.

📖 Read the Complete Provider Implementation Guide →

What You'll Learn#

• Directory Structure: How to organize your provider package
• LLM Implementation: Step-by-step guide to implementing the LLM class
• Embeddings: How to add embedding support (if applicable)
• Testing: Writing comprehensive tests with proper markers
• Documentation: Creating user-facing documentation
• Best Practices: Common pitfalls and how to avoid them

Quick Overview#

All providers follow the same pattern:

1. Inherit from Core Classes: FunctionCallingLLM for LLMs, BaseEmbedding for embeddings
2. Implement Required Methods: Chat, completion, streaming (sync & async)
3. Add to Workspace: Configure in pyproject.toml
4. Write Tests: Unit tests and e2e tests with markers
5. Document: README, usage examples, and docs page

Reference Implementation#

The Ollama provider serves as a complete reference showing:

• Full LLM implementation with streaming and async support
• Embedding implementation with batching
• Shared client and error handling
• Comprehensive test suite
• Complete documentation

Development Checklist#

Use this checklist when implementing a new provider:

• [ ] Inherits from FunctionCallingLLM for LLM
• [ ] Implements BaseEmbedding for embeddings (if applicable)
• [ ] Supports sync, async, and streaming operations
• [ ] Includes comprehensive unit tests (≥95% coverage)
• [ ] Includes e2e tests with appropriate markers
• [ ] Has README with examples and configuration
• [ ] Has documentation page in docs/overview/providers/
• [ ] Exports public API in __init__.py
• [ ] Added to workspace in root pyproject.toml
• [ ] Added to provider comparison table above
• [ ] Follows code style and type annotations

Get Started: Read the Full Implementation Guide →

Support#

For provider-specific issues:

• Ollama: Ollama GitHub
• Serapeum Integration: Serapeum Issues

For general framework questions, see the Contributing Guide.

Next Steps#

Adding New Providers#

To add a new provider integration, see the Provider Integrations Guide for detailed instructions and the provider architecture pattern.