Skip to main content

OpenAI API routing guide: fixed egress, concurrency and timeout tests

API calls need a different network setup than the web app: stable egress, reliable concurrent connections and long-lived sessions. Learn what to check and how developers can choose a route.

For OpenAI API calls, check egress stability first, then concurrency and timeouts, and only then download bandwidth. A working webpage doesn't prove that your terminal, container or background job uses the same route. Fast speed tests don't guarantee a complete streaming response.

Choosing a route: For development and debugging, prioritize routes with an egress location you can verify and that work in your actual runtime environment. Monitor egress changes, connection failures and interrupted responses in production. This guide provides test methods and criteria, not an unverified route ranking or a guarantee of access to any service.

Stable egress: Confirm where requests originate

Stable egress means the public IP address seen by the destination remains consistent. Selecting the same node in a client only prevents manual switching; if the node uses an egress pool, load balancing or failover, its public IP may still change. A “fixed node” is not the same as a “dedicated static IP”; the node name alone doesn't confirm that capability.

Stable egress helps reproduce issues, review access logs and maintain egress IP allowlists. It does not grant permission to access a service. Before using the OpenAI API, check the countries and regions currently supported by the service, account eligibility and applicable usage rules. An IP lookup page alone cannot confirm availability.

Run tests in the environment that actually sends requests. A browser may use a proxy extension, a terminal may read environment variables, and a container may have its own network. A browser's reported egress doesn't replace checks in the deployment environment. Check egress after reconnecting the client, while a task is running and after a network change to assess stability.

VPNiy's regional node list is not a description of a static IP product. See the route list to review available options. If your workload requires a fixed egress IP, confirm that capability through the contact channel first—don't assume it's available.

Comparing direct, relayed and dedicated routes

Direct, relayed and IEPL describe how network paths are organized. Shadowsocks, VMess, Trojan, VLESS, Hysteria2 and TUIC are proxy protocols or related transport options. These refer to different layers. A protocol name alone doesn't prove stable egress, higher concurrency or access to a destination service.

Route selection criteria, not a ranking of measured performance
Type Route characteristics What to test for API calls What this does—and doesn't—tell you
Direct The client connects directly to an overseas proxy node Cross-border public network variability, handshake time and sustained responses Fewer hops don't necessarily mean less congestion
Relayed Traffic passes through an entry node before reaching the egress Entry-node congestion, relay path and final egress stability An extra hop isn't necessarily slower or faster
IEPL dedicated route An international Ethernet private line segment is used in the service path Coverage of the dedicated segment, shared load and the path from egress to the destination A dedicated segment doesn't mean the entire end-to-end path is exclusive

If the main issue is cross-border public network variability, consider relayed and dedicated routes. If the issue is rate limiting by the destination service, changing transport paths usually won't help. Even with a stable entry and cross-border segment, the final path from egress to OpenAI, account quotas and server load can affect results.

Choose a protocol based first on client compatibility and actual network conditions. Hysteria2 and TUIC use QUIC-based transport and depend on UDP reachability; other options also depend on their specific transport configuration. Don't assume TCP or UDP is inherently better for API calls—test with the same workload.

Concurrency testing: distinguish route issues from API rate limits

Concurrency refers to requests or active connections at the same time, not the number of calls submitted per unit of time. Streaming generation keeps connections open; request uploads, connection-pool waits and model generation also extend connection time. Testing only short requests can overstate real-world capacity.

Keep the model, request content, output constraints, runtime environment and client version fixed, then increase concurrency gradually. Set concurrency levels based on workload needs and account limits, not an arbitrary marketing figure. For each run, record actual egress, active requests, successful completions and error types so results from different conditions aren't mixed.

  • ✅ Single-request baseline: Confirm authentication, request format, target model and route all work.
  • ✅ Increase concurrency gradually: Keep request content consistent and record connection-pool wait time, time to first valid output and whether each request completes.
  • ✅ Retest at different times: Repeat the same requests during typical usage hours and keep a record of local network and egress details.
  • ✅ Classify errors: Track parsing failures, connection failures, TLS errors, rate limits and interrupted responses separately.
  • ❌ Don't count a request that succeeded after automatic retries as a request that never failed.

For a clear rate-limit response, first check account quotas, model limits and any retry guidance in the response. If requests are queued locally before being sent, review the SDK connection-pool size and task scheduling. If connections frequently drop after being established, compare results across route-switching tests to see whether the network may be involved.

Don't respond to an error with unlimited retries. Retries increase concurrency pressure and may incur additional charges. If a request has reached the server, a client-side timeout doesn't mean server-side processing has stopped. Whether a retry is safe depends on the endpoint's semantics and your application's deduplication design.

Timeout comparison: Measure first output and full completion

A timeout can occur at different stages. DNS resolution, connection setup, the TLS handshake, connection-pool waiting, data reads and the overall task may each have their own limits. Increasing the total timeout only changes how long you wait; it won't fix unreachable egress or a proxy that actively closes the connection.

For streaming calls, track each stage separately: time to response headers, time to first valid output, long pauses between events and whether the expected completion event arrives. Receiving headers doesn't mean the model has produced output, and receiving partial text doesn't mean the task completed successfully.

SSE is a mechanism for continuously sending events over HTTP, not a WebSocket. Parse events according to the format used by the endpoint; don't treat each network read as a complete event. Network chunks, text characters and event boundaries don't always align, and parsing errors can look like “output suddenly stopped.”

Check limits across the entire application chain: the SDK, application service, reverse proxy and gateway may all set read timeouts or buffering. If the server keeps generating but the browser receives content in batches, check buffering first instead of assuming the route is stalling.

Set read timeouts based on how long the workload can reasonably wait, and configure cancellation and an overall task deadline. Test both short requests and requests with long outputs, recording the stage where interruptions occur. Pinpointing the failure helps determine whether to adjust the network, connection pool or application configuration.

Proxy and subscription setup: Route requests through the right path

A subscription link provides nodes and related settings to compatible clients. It is not an OpenAI API endpoint or an API key. Get the subscription from your account dashboard, import it into a client that supports the relevant format and protocols, then select a route and start the proxy. Subscription links contain sensitive connection details; don't put them in public repositories, support-ticket screenshots or shared logs.

System proxy settings on Windows and macOS aren't necessarily used by every command-line program. Linux background services, containers and remote development environments also need separate configuration. Connections from an iOS client affect only traffic handled by that device; they can't configure egress for a remote server.

  1. Check the client's local HTTP or SOCKS proxy endpoint and confirm that it's listening.
  2. Check how your SDK configures proxies; use the relevant proxy environment variables only if the SDK supports them.
  3. Check the destination request path in the same process environment, and note that exclusion rules may bypass the proxy.
  4. For container workloads, confirm that the proxy address is reachable from inside the container; the container's loopback address usually isn't the host machine.
  5. Keep HTTPS certificate verification enabled, then test an actual API call and confirm the stream completes.

The command below is for comparison testing with an existing configuration. Before running it, set the API key, proxy address and timeout values that fit your local environment and workload budget. request.json should contain a valid request body and a model you're authorized to use. This isn't a preset test result and won't select a route automatically.

curl --no-buffer --show-error \
  --proxy "${HTTPS_PROXY:?Set the proxy address first}" \
  --connect-timeout "${CONNECT_TIMEOUT:?Set the connection timeout first}" \
  --max-time "${REQUEST_TIMEOUT:?Set the overall task deadline first}" \
  --header "Authorization: Bearer ${OPENAI_API_KEY:?Set the API key first}" \
  --header "Content-Type: application/json" \
  --data-binary @request.json \
  https://api.openai.com/v1/responses

For streaming tests, enable the relevant streaming option in the request body and process output according to the event protocol. The overall task deadline in the command may terminate a response that is still being generated; mark such cases separately. Don't enable debug output that could expose authentication details, and never enter your API key on a third-party speed-test site.

DNS and routing rules: Rule out issues that only look like route problems

Routing rules determine whether a request goes through a proxy or connects directly. API domains, authentication pages and regular websites may match different rules, so “the website works, but the SDK fails” is entirely possible. Check the client's connection log or rule-match details to confirm the request took the expected path.

Where DNS is handled depends on the proxy mode and client configuration. Some modes resolve names locally; others send them to a remote resolver. A DNS leak generally means queries didn't follow the expected resolution path; it's not the same as an API connection failure. Check the resolution policy, system resolver and actual traffic coverage rather than randomly changing DNS addresses.

Don't disable certificate verification as a troubleshooting step. HTTPS encryption and certificate validation protect the connection between your application and its destination; a proxy tunnel doesn't replace that verification. If a managed certificate or network gateway is used on a corporate device, ask the appropriate administrator to check the trust-chain configuration.

Developer picks: choose a route based on where failures occur

For local debugging, start with a route that's easy to configure and whose egress is easy to verify. Use a successful request and a completed stream as your baseline. For batch jobs, watch connection-pool waits and API rate limits, and use controlled concurrency with capped backoff retries. For user-facing services, build cancellation propagation, timeout budgets and recovery from failures into the application design.

When comparing routes, use the same request set, runtime environment and error categories. Record whether egress changed, whether requests completed and where failures occurred; that's more useful than keeping only a bandwidth screenshot. Results apply only to the conditions at the time of testing and shouldn't be presented as a long-term availability guarantee.

Final selection criteria: The egress location complies with the destination service's rules, the calling process uses the intended proxy, requests complete reliably at the required concurrency, and streaming responses finish correctly. Once these conditions are met, compare cost and operational convenience. Don't use a protocol name, a dedicated-route label or download speed as a substitute for API testing.

VPNiy doesn't require an email address; create an account with a username and password. Before testing routes, read the getting started guide for client and subscription setup. For connection, routing or DNS issues, follow the troubleshooting guide. Confirm the destination service's permissions, quotas and regional rules separately.

Start Free