Skip to content

Models internals

This page lists a few topics that detail some implementation choices made in the BaseAIModel and its related classes.

See also

More on models:

More on messages:

Conversion Pipelines

The workflow for any model provider (OpenAI, Anthropic, Google, ...) follows a pipeline with two operations:

  1. Convert the input (prompt, tools, etc.) to the provider's native format
  2. Convert the provider's response back to Conatus's standardized output.

As a diagram, it would look like this:

AIPrompt
   |
   |  (convert_{messages, tools, etc.}_to_ai_model_format)
   v
provider payload   -->  [call the API]
   v
provider-native response
   |
   |  (convert_provider_response_to_ai_response)
   v
AIResponse

This includes:

  • Detecting the provider and model variant (see model‑name retrieval logic)
  • Building the provider-native payload (conversion functions for messages, tools, schemas, etc.)
  • Executing the API call (with logging/printing)
  • Converting the provider's response back into standardized AIResponse

Each provider (OpenAI, Anthropic, Google) has a family of conversion functions, designed for clarity, composability, and easy streaming.

Message conversion classes

OpenAI 

from conatus.models.open_ai.cc_conversion import OpenAIChatCompletionConverters

# Example: Convert a Conatus Assistant message to OpenAI's format
# for the Chat Completions API
from conatus.models.inputs_outputs.messages import AssistantAIMessage
msg = AssistantAIMessage.from_text("Give me a Python example.")
openai_msg = OpenAIChatCompletionConverters.assistant_message_to_openai_assistant_message(msg)
assert openai_msg["role"] == "assistant"

Anthropic 

from conatus.models.anthropic.conversion import AnthropicConverters
from conatus.models.inputs_outputs.messages import UserAIMessage

user_msg = UserAIMessage(content="Summarize this paragraph.")
anthropic_msg = AnthropicConverters.ai_message_to_anthropic_message(user_msg)
assert anthropic_msg["role"] == "user"

Google (Gemini) 

from conatus.models.google.conversion import GoogleConverters
from conatus.models.inputs_outputs.messages import AssistantAIMessage

msg = AssistantAIMessage.from_text("Explain the concept of entropy.")
g_content = GoogleConverters.assistant_ai_message_to_google_msg(msg)
assert g_content.role == "model"

Common structure

Each provider has a family of conversion classes, but they share a similar structure.

  • Each class bundles static functions for:
    • Message conversion
    • Tools conversion
    • System message conversion
    • Output schema conversion (for structured output)
    • Streaming/incomplete chunks conversion
  • BaseAIModel calls these as hooks via its core logic (convert_messages_to_ai_model_format , convert_tools_to_ai_model_format, etc.). Each provider is responsible for implementing these hooks.

The message conversion flow in BaseAIModel.prepare_call_args

The method (prepare_call_args) is at the heart of every model implementation: all provider-specific payloads are derived from this standardized step.

What it does

  1. Merges configs: Combines self.model_config with any overrides passed as user_provided_config
  2. Sets fields: Handles computer use mode, previous messages IDs (for stateful APIs), etc.
  3. Calls hooks: Invokes the following overridable methods:
  4. Returns: An instance of AIModelCallArgs (or a subclass, per model) with all relevant fields

This means that if you want to add a new provider, you only need to implement the conversion hooks. For more information on how do implement a new provider, see How-to: Add a new AI provider.

Example: How this works in-situ (runnable) 

import os
from conatus import AIPrompt
from conatus.models.open_ai import OpenAIModel
from conatus.models.base import BaseAIModel, ModelConfig

os.environ["DUMMY_API_KEY"] = "dummy"

class DummyModel(OpenAIModel): # (1)!
    api_key_env_variable = "DUMMY_API_KEY"
    provider = "dummy"

    def default_client(self, model_config, api_key, **kwargs):
        return None

    def default_config(self):
        return ModelConfig()

    def convert_system_message_to_ai_model_format(self, system_message, config):
        return f"SYSTEM: {system_message.content}"

    def convert_messages_to_ai_model_format(self, prompt, config, only_new_messages=False):
        return [f"{m.role.upper()}: {getattr(m, 'content', '')}" for m in prompt.messages]

    def convert_tools_to_ai_model_format(self, prompt, config):
        return []

    def convert_output_schema_to_ai_model_format(self, prompt, config):
        return super().convert_output_schema_to_ai_model_format(prompt, config)

prompt = AIPrompt(
    user="What's 2 + 2",
    system="You are a helpful assistant.",
    output_schema=int,
)
m = DummyModel()
args = m.prepare_call_args(prompt)
assert args.system_message == "SYSTEM: You are a helpful assistant."
assert args.messages == ["USER: What's 2 + 2"]
assert args.output_schema["format"]["schema"]["properties"] == {
    "item": {"type": "number"}
}
  1. We subclass OpenAIModel to avoid having to implement the abstract methods of BaseAIModel: call, acall, call_stream, acall_stream.

Pricing

Conatus tracks per-model pricing rules in a registry. Its goal is to return accurate cost estimates given CompletionUsage; this relies on knowing the pricing model and price per token for each model.

Because different models have different pricing models, we need to be able to look up the right pricing strategy for a given model.

Pricing strategies and the registry

  • The PricingStrategy ABC defines each strategy as .cost(usage) -> float
  • There are strategies for:
  • The global registry (REGISTRY) is a singleton mapping model_name to PricingStrategy.
  • The registry is populated at import time (from conatus.models.prices).

Example:

from conatus.models.inputs_outputs.usage import CompletionUsage
from conatus.models.prices import calculate_cost

usage = CompletionUsage(model_name="openai:gpt-4o", prompt_tokens=2000, completion_tokens=500)
cost = calculate_cost(usage)
assert cost > 0

How cost lookups work

  • calculate_cost normalizes a model_name (applies mapping for "unclean" names), retrieves the proper pricing strategy, and invokes .cost(usage)
  • If the model is unknown or the cost can't be calculated, returns -1

Where to add/update prices

  • To change prices, update the relevant REGISTRY.register() calls in conatus.models.prices
  • If introducing a new model/provider, register default prices before releasing to users.

This structure is subject to change

For now, we rely on a centralized registry, but we might move to a more decentralized structure in the future, where each provider can declare its names, pricing, etc. within their own package.

Model name retrieval and normalization logic

Conatus provides helpers to resolve fuzzy or shorthand model names to canonical names. This allows users to type "gpt-4o", "claude-3-7" or "gemini-2.5-pro" and always get the right model class and settings.

Main utilities 

from conatus.models.retrieval import get_model_from_name
model = get_model_from_name("gpt-4o")
assert model.provider == "openai"

Model types

For some providers like OpenAI, defining a single default model might not be adequate. OpenAI, for instance, have models that are optimized for different tasks (chat, code, etc.) and have different pricing.

To support this, we use the ModelType enum to specify the type of model (e.g. chat, execution, reasoning, etc.). You can use this enum to select the right model variant for your use case.

In turn, AI providers can subclass the default_model_name method to return the right model name for a given model type. See the OpenAI model implementation for an example.

Printing-mixin

The AIModelPrintingMixin class manages terminal output for streaming and non-streaming model calls.

There are three different modes:

  • "normal": Print progress indicator (indicates the number of chunks received)
  • "preview": Print a real-time partial response, clearing+updating as new chunks are received (default for most models)
  • "silent": No output to the terminal. This can be useful in CI environments.

Additional notes and tips

  • If you're adding a new provider, follow add_ai_provider.md and base your design on OpenAIModel, AnthropicAIModel, or GoogleAIModel.
  • Use the conversion classes as adapters; never mutate prompts in-place.
  • Pricing always relies on accurate model_name in CompletionUsage; if your provider has variable pricing, prefer a custom PricingStrategy class.
  • Always be sure to close/init your client in __del__ to avoid resource leaks.

Reference: Key APIs