What does OpenAI-compatible actually mean?
An OpenAI-compatible gateway accepts some OpenAI-style requests and returns a response your client can consume. Compatibility can vary by operation: a working chat request does not establish support for streaming, tool calls or structured output.
Model identity is a separate question. A provider-supplied model label or a model's answer to ‘what model are you?’ does not independently authenticate the backend. Behavioral verification compares observations with a selected official reference; it is not cryptographic attestation.
Before you test: collect the exact configuration
Use the endpoint and model identifier that your application actually uses. A provider's marketing name, dashboard URL and request values can differ. Do not change several fields at once while diagnosing a failure.
- Base URL: copy the documented API root. Keep /v1 or another prefix only when required; do not paste the complete chat/completions operation or a dashboard page.
- Model ID: copy the exact request identifier, including aliases and version suffixes. Confirm that your account can access it.
- Protocol: choose the documented wire format. A Claude-branded model served through an OpenAI-compatible gateway still uses the OpenAI request protocol.
- API key: use a key for this provider, endpoint and account with sufficient quota. Prefer a restricted key where supported. Verification requests may incur provider charges.
- Official reference: choose the supported model you intend to compare against. The provider's model ID and the official reference are separate selections.
Run a behavioral verification in ModelVerify
Open the verification form, enter your provider configuration and review the selected official reference. The form lists the supported references; a model being available through a gateway does not mean a suitable official fingerprint is available.
Submit one run and review its report when it completes. Keep the model identifier, reference, date and non-secret configuration with your notes. Never include an API key in screenshots, shared reports or support posts. Respect the tool's rate limits rather than repeatedly submitting the same request.
If the API returns 401, 403 or 404
Read the response together with the provider's documentation. A 401 often involves credentials; a 403 can involve model access, region or account restrictions; a 404 can involve the API route or model availability. Gateways do not all use status codes identically.
A handshake failure means behavioral verification could not proceed. First resolve the connection issue using the provider's documented minimal request in your own environment. It is not evidence that the advertised model is fake.
How to interpret match, mismatch and uncertainty
A match means observed behavior agrees with the selected reference under the current checks. It does not guarantee that every future request will use the same backend, or that the provider will perform well on every task.
A mismatch means the observed behavior deviates from the reference. Check the intended model version and gateway configuration before drawing an identity conclusion. Routing, caching and version differences can affect responses; a mismatch does not establish intentional substitution or fraud.
An uncertain or incomplete result leaves the question unresolved. Latency and stability provide operational context, but response speed alone cannot identify a model.
Evaluate the gateway on your own workload
Combine identity evidence with tests that reflect your application: instruction following, tool calls, output validation, failure handling, latency and cost. Use non-sensitive representative examples and a consistent configuration when comparing providers.
An official benchmark score is a capability reference, not a guarantee about a reseller or your production workload. Review observations over time before committing spend, and document uncertainty instead of turning one result into a universal provider rating.