Function Calling Across AI Providers in 2026: A Practical API Guide

Function Calling Across AI Providers in 2026: A Practical API Guide#

Function calling lets an AI model request a typed operation—such as searching a database, creating a ticket, or charging an account—without directly receiving access to your infrastructure. The hard part is portability: providers use different message formats, tool schemas, streaming events, and finish reasons. This guide presents a provider-neutral design that keeps your application stable while you change models.

What Is This Approach?#

A native SDK is convenient when one provider is permanent, but an OpenAI-compatible gateway reduces migration work. Anthropic commonly represents tool use as content blocks, Gemini uses function declarations and parts, and many open models follow OpenAI-style tools. Normalize all of them into an internal object: name, arguments, call_id, result, and status.

How to Implement It#

import json
from openai import OpenAI

client = OpenAI(base_url="https://ai.crazyrouter.us/v1", api_key="YOUR_KEY")
tools = [{"type":"function","function":{"name":"get_weather","description":"Get current weather","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}]

response = client.chat.completions.create(
    model="gpt-5-mini", messages=[{"role":"user","content":"Weather in Tokyo?"}], tools=tools
)
call = response.choices[0].message.tool_calls[0]
args = json.loads(call.function.arguments)
print(call.id, call.function.name, args)

const result = await fetch('https://ai.crazyrouter.us/v1/chat/completions', {
  method: 'POST', headers: {'Authorization': `Bearer ${process.env.AI_KEY}`, 'Content-Type':'application/json'},
  body: JSON.stringify({model:'claude-sonnet-4-5', messages, tools})
});

Pricing Breakdown#

Prices and model availability change over time, so verify the current provider and Crazyrouter pricing before making a production forecast. Calculate effective cost per successful task, not only cost per token.

Production Checklist#

• Define a timeout and a bounded retry policy.
• Validate structured outputs and tool arguments outside the model.
• Record model, version, request ID, latency, token usage, and cost.
• Add tenant quotas, circuit breakers, and a human review path for high-impact actions.
• Keep prompts, schemas, and evaluation cases versioned.

A Practical Rollout Plan#

Start with a thin vertical slice instead of abstracting every provider feature on day one. Pick one user journey, one primary model, and one fallback. Capture a small set of representative requests, including short prompts, long context, malformed input, empty results, and adversarial instructions. This gives you a baseline before optimization changes the behavior you are measuring.

Next, put policy decisions outside the model. Authentication, tenant permissions, tool authorization, spending limits, and data retention should be enforced by application code. The model may suggest an action, but it should not decide whether the current user is allowed to perform it. This separation makes security reviews and incident investigations much easier.

For reliability, make every request observable without storing raw sensitive content by default. A useful trace records a request ID, tenant ID, route, model version, latency, token counts, retry count, and final outcome. Hash prompts or store redacted summaries when full content is not required. Add dashboards for successful-task cost, p95 latency, schema-validation failures, and fallback frequency.

Finally, review the route on a schedule. Model providers change pricing, limits, and behavior. Re-run the evaluation set after provider updates, prompt edits, or routing changes. Keep a rollback route available and communicate limits honestly to users. This operating discipline usually saves more money than prematurely optimizing a few cents of token cost.

Frequently Asked Questions#

What is function calling?#

It is a structured protocol where a model emits a tool name and validated arguments for your application to execute.

Is function calling portable?#

The concepts are portable, but message and streaming formats differ. Use an internal normalized representation.

Should tools execute automatically?#

Only after schema validation, authorization, rate limiting, and an audit decision.

Can Crazyrouter route tool-calling requests?#

Yes. Use its OpenAI-compatible endpoint and verify the selected model’s tool-calling behavior in your tests.

Example Decision Matrix#

Before choosing an implementation, write down the workload’s quality bar, latency target, data sensitivity, and monthly volume. A customer-support draft may tolerate a fast small model and a short cache TTL; a financial reconciliation workflow may need deterministic tool calls, a stronger model, and human approval. This simple matrix prevents teams from choosing a model based only on a leaderboard or a single impressive demo.

For staged delivery, begin in shadow mode: send a small percentage of sanitized production-shaped traffic to the candidate route while keeping the existing response visible to users. Compare quality and latency, but also inspect failure categories and escalation frequency. Once the candidate is stable, expose it to a limited tenant cohort with a rollback flag. This turns provider changes into reversible deployments rather than risky migrations.

Summary#

Treat tool calls as untrusted input, not executable instructions. Keep schemas small, validate arguments with JSON Schema or Pydantic, add idempotency keys, and log every call. A compatible gateway such as Crazyrouter can give your application one endpoint while you compare models and prices.