How I centralized API connections to call two LLMs with the same code
Recently, agent-related workloads have begun using different models depending on the use case—for example, GPT-5.6 Sol for agents and Claude Opus 5.5 for loading long documents. Both have become indispensable to day-to-day development workflows, but trying to call both from the same project led to less obvious costs piling up more than I had expected. This article is a memo from when I tidied up the connection setup.
There were three challenges.
The first was accounts and billing. Since registration, top-ups, and usage details are handled separately for each service, you end up managing multiple accounts and payment methods, while accounting also has to reconcile each service separately. The smaller the team, the more noticeable this overhead becomes.
The second was differences in interfaces. Request formats, parameter names, how streaming responses are returned, and error-code definitions differ from model to model, so using them together requires a custom adapter layer. The more models you add, the more combinations you have to handle, and each time you replace one, you tend to have to make the same fixes again.
The third was a single point of failure. If you pin the upstream to just one, there is nowhere to switch to when you encounter rate limits or outages, and the app simply comes to a halt. Because such problems are difficult to reproduce during development, isolating the cause also takes time.
What I did was implement the adapter layer just once.
I set up a unified endpoint in the OpenAI-compatible format, pointed the code's base_url to that address, and replaced the key with the issued API key. As a result, GPT-5.6 Sol, Claude Opus 5.5, and other major models can all be called in the same way, using the calling code originally written for the OpenAI SDK. The business logic does not need to be rewritten, and model switching is consolidated into a single configuration value. Token usage statistics and details are also brought together on one screen, making it much easier to check billing.
It also integrated well with the existing toolchain.
When building agents with LangChain or LlamaIndex, the underlying model is often specified at initialization, so it can be replaced simply by changing one configuration location. With Cursor or VS Code, you can enter the API address and key in the settings and continue using code generation and refactoring directly in the editor. Function calling / tool calling also remains OpenAI-compatible, so there was no need to rewrite the tool layer in existing agent implementations. If a project is already running on the OpenAI SDK, the migration burden tends to be smaller.
Billing is pay-as-you-go, with no minimum spend. Because you are charged only for the tokens you use, there is no need to deposit funds in advance based on the top-up tier.When multiple models are consolidated under a single account, you also spend less time checking the balance of each service. If you are still experimenting with different combinations of models, this may seem like a minor benefit, but it does help.
The main concern in production is connection stability. We distribute upstream traffic across multiple connections and a keep pool, routing traffic through another connection whenever one becomes unstable. There are now fewer cases where users notice interruptions on the application side. Monitoring screens for success rate, latency, and usage also make troubleshooting easier when things are not going well.
In summary, by pushing model-specific differences, billing, and connection fluctuations down to the connection layer, the application side only needs to focus on business logic. We have been freed from having to add adapter code every time we add a model. I hope this serves as a useful reference for others facing the same issue. I would also like to hear in the comments how others handle it, such as “This is what we do.”
Please note that FLYSTAR, the unified API service used in this evaluation, is a service whose operation I am personally involved in (I am explicitly stating this so that it does not become so-called stealth marketing). Details and links are available in my profile.