Design Flaws: LLM API's OpenAI node tethered to OpenAI provider & Hard-coded Model Names in code.

[JamesClarke7283]

Expected Behavior

1. Ability to use any suitable compatible OpenAI endpoints (from any provider).

2. New OpenAI model comes out, Can choose from dropdown.

Actual Behavior

1. Nodes tethered to 'api.openai.com' API Endpoint.

2. Model not listed as model names are hardcoded and require maintainer to add each time(also won't support private finetunes or otherwise unlisted OpenAI models).

Steps to Reproduce

Workflow N/A

Relevant code snippet here:
https://github.com/comfyanonymous/ComfyUI/blob/c9e1821a7b49bb58f18f114336bae911160ac69d/comfy_api_nodes/nodes_openai.py#L74C1-L82C34

Debug Logs

❯ comfy --workspace=comfyui launch

Launching ComfyUI from: REDACTED/PATH/FOR/PRIVACY

[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2025-05-27 15:56:27.979
** Platform: Linux
** Python version: 3.12.10 (main, Apr  9 2025, 04:03:51) [Clang 20.1.0 ]
** Python executable: /REDACTED/PATH/FOR/PRIVACY/python3
** ComfyUI Path: /REDACTED/PATH/FOR/PRIVACY/comfyui
** ComfyUI Base Folder Path: /REDACTED/PATH/FOR/PRIVACY/comfyui
** User directory: /REDACTED/PATH/FOR/PRIVACY/comfyui/user
** ComfyUI-Manager config path: /REDACTED/PATH/FOR/PRIVACY/ComfyUI-Manager/config.ini
** Log path: /REDACTED/PATH/FOR/PRIVACY/comfyui/user/comfyui.log

Prestartup times for custom nodes:
   0.0 seconds: /REDACTED/PATH/FOR/PRIVACY/comfyui/custom_nodes/rgthree-comfy
   1.5 seconds:/REDACTED/PATH/FOR/PRIVACY/comfyui/custom_nodes/ComfyUI-Manager

Checkpoint files will always be loaded safely.
Total VRAM 11882 MB, total RAM 64083 MB
pytorch version: 2.7.0+cu128
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4070 : cudaMallocAsync
Using pytorch attention
Python version: 3.12.10 (main, TIMESTAMP:REDACTED:FOR_PRIVACY) [Clang 20.1.0 ]
ComfyUI version: 0.3.37
ComfyUI frontend version: 1.20.6
---
etc... (only software version info may be useful here)

Other

Since OpenAI API is a standardised API spec adopted by most commercial providers and inference engines, it should be possible to change the 'OPENAI_BASE_URL' to any provider (in principal).

For the workflow template interface, I realise the "API" suffix in the Workflow Templates usually means third party providers, one can make a generic "LLM" or the more descriptive and precise "Text Generation" tab.

Side Note: (Personally i would prefer if open weight and proprietary API's should be presented in a more unified way, otherwise you double up on any modality/task type), but that is more of a cosmetic issue at time of writing

There are only temporary compatibility issues (such as the reliance on newer: responses API over chat completions), the inference engines are implementing this newer API (we can expect any API shift will be re-implemented, due to it being essential to commercial providers interoperability requirements).

See here for Open PR's:
ollama/ollama#10316

Proposal

1. Add a "Base Endpoint URL" field or similar to all OpenAI nodes.

2. Call the /show endpoint to retrieve capability information (such as vision & tools) instead of manually embedding fixed enum values (this can remove the maintainers burden of needing to manually update SupportedOpenAIModel for it to show up, and enable people to use their private gpt4o finetunes for example), also if there is no API query to check the user is authorised to use say "o1-pro" due to plan limits(lower tier user), the model may show, but they may get a error when they try and run it.

3. Interface wise, add the Model Agnostic "OpenAI Compatible Endpoint" template. (we can supersede the provider specific template in "LLM API", or add a identical root tab like: "Text Generation" to build on later with different examples of integrating Text Generation models in a workflow)

Side Note

I can also say the same for commercial image gen. Many Commercial Image gen providers have OpenAI API support, additionally, projects like LocalAI (which aim to make a complete libre implementation of OpenAI's API spec, but for local models) would benefit.

Design Flaw vs Bug

There is 1 design flaw (tether to OpenAI) because once one has text generation support for OpenAI, at the technical level, it should work for any compatible provider.

I realise you have Gemini API integration, i don't know how that is implemented(weather using the genai sdk through the standard OpenAI compatible), but i thought i would put this here (it seems they are fully compatible with OpenAI spec, both for Google AI Studio and Vertex AI):
https://ai.google.dev/gemini-api/docs/openai

Additionally, providers like OpenRouter provides a unified interface across LLM providers, which further evidences the demand for this "provider agnostic" approach.

And 1 bug: Hardcoded Model Names requiring manual changes each time for model support (maintainability burden & practical limitation, i.e a maintainer could forget a less known model in the list but its in demand for certain environments).

Since the bug and design flaw can be resolved with the same minor changes, i thought i would put them both in this issue, instead of creating a separate "feature" issue, if dynamic model detection happens, going from there to model agnostic support is just a Input Parameter and binding away, regardless of if popular providers like "ollama" support the new OpenAI API version yet.