From d2e1acbaf49c798b6595deeef5ab6632d4a3be55 Mon Sep 17 00:00:00 2001 From: nonononull Date: Fri, 26 Jun 2026 11:24:00 +0800 Subject: [PATCH] Add strict 502 mode and upstream retry handling for streaming responses --- README.md | 524 ++-- config.example.json | 24 +- err.md | 296 ++- gateway.mjs | 2878 +++++++++++----------- scripts/admin-lib.mjs | 1240 +++++----- scripts/install-for-current-provider.ps1 | 218 +- scripts/test-gateway-e2e.mjs | 634 ++--- 7 files changed, 2997 insertions(+), 2817 deletions(-) diff --git a/README.md b/README.md index 5fcd760..c13ef24 100644 --- a/README.md +++ b/README.md @@ -1,263 +1,265 @@ -# Codex Retry Gateway - -一个不依赖 `cc-switch` 路由模式的独立本地网关。 - -目标： - -- 保持 Codex 继续使用现有 `auth.json` -- 只把 `config.toml` 的当前 provider `base_url` 改成本地网关 +# Codex Retry Gateway + +一个不依赖 `cc-switch` 路由模式的独立本地网关。 + +目标： + +- 保持 Codex 继续使用现有 `auth.json` +- 只把 `config.toml` 的当前 provider `base_url` 改成本地网关 - 非流式命中 `reasoning_tokens = 516` 时返回 `502` -- 流式命中时直接断开连接，让 Codex 自行重试 -- 默认同时拦截 root 路径和 `/v1` 路径： - - `/responses` - - `/chat/completions` - - `/v1/responses` - - `/v1/chat/completions` - -限制： - -- 这个网关不负责 `Responses` 和 `Chat Completions` 协议互转 -- 如果你的上游本身不支持 Codex 当前使用的协议，这个网关不会替你补齐转换能力 - -## 默认路径 - -Windows: - -- Codex 配置：`%USERPROFILE%\.codex\config.toml` -- Gateway 状态目录：`%USERPROFILE%\.codex-retry-gateway` - -macOS / Linux: - -- Codex 配置：`~/.codex/config.toml` -- Gateway 状态目录：`~/.codex-retry-gateway` - -## 当前版本说明 - -- 这是一个可独立发布、独立运行的仓库 -- 默认监听地址是 `http://127.0.0.1:4610` -- 默认示例上游见 `config.example.json` -- 实际运行时配置会写到当前用户目录下的 gateway 状态目录 - -## 一键启动并打开管理页 - -在仓库根目录执行： - -Windows: - -```powershell -powershell -ExecutionPolicy Bypass -File .\scripts\launch-ui.ps1 -``` - -macOS / Linux: - -```bash -bash ./scripts/launch-ui.sh -``` - -这个脚本是默认入口，执行后会自动完成： - -- 第一次运行时： - - 备份当前用户目录下的 Codex `config.toml` - - 生成当前用户目录下的 gateway `config.json` - - 启动本地 gateway - - 把当前 `model_provider` 对应的 `base_url` 改到本地 gateway -- 之后再次运行时： - - 自动复用现有安装状态 - - 自动重启或拉起 gateway - - 自动再次打开管理页 - -默认会打开： - -```text -http://127.0.0.1:4610/__codex_retry_gateway/ui -``` - -如果你只想启动、不自动开浏览器： - -```powershell -powershell -ExecutionPolicy Bypass -File .\scripts\launch-ui.ps1 -NoOpen -``` - -```bash -bash ./scripts/launch-ui.sh --no-open -``` - -常用参数： - -- Windows 参数： - - `-CodexConfigPath` - - `-StateRoot` - - `-ListenHost` - - `-ListenPort` - - `-NoOpen` -- macOS / Linux 参数： - - `--codex-config-path` - - `--state-root` - - `--listen-host` - - `--listen-port` - - `--no-open` - -macOS / Linux 说明： - -- 需要 `bash` -- 需要 `Node.js 18+` -- Unix 入口会调用跨平台 `node` 管理核心，不依赖 PowerShell -- 推荐显式使用 `bash ...sh` -- 这样即使目录是从 Windows 或压缩包复制过来、没有可执行位，也能直接运行 - -## 手工安装入口 - -如果你明确只想做脚本级安装，不想自动打开 UI，也可以直接执行： - -Windows: - -```powershell -powershell -ExecutionPolicy Bypass -File .\scripts\install-for-current-provider.ps1 -``` - -macOS / Linux: - -```bash -bash ./scripts/install-for-current-provider.sh -``` - -## 如何恢复 - -Windows: - -```powershell -powershell -ExecutionPolicy Bypass -File .\scripts\restore-codex-config.ps1 -``` - -macOS / Linux: - -```bash -bash ./scripts/restore-codex-config.sh -``` - -这个脚本会： - -- 停掉本地 gateway -- 用最近一次备份恢复当前用户目录下的 Codex `config.toml` -- 删除当前安装状态文件 - -## 管理页面 - -页面入口： - -```text -http://127.0.0.1:4610/__codex_retry_gateway/ui -``` - -页面里可以直接做这几件事： - -- 看当前监听地址、真实上游、当前 provider、当前 Codex base URL -- 看本次 gateway 启动以来的实时统计 - - 代理请求总数 - - 被检查响应总数 - - `516` 命中次数 - - `516` 占比 -- 改 `reasoning_equals` -- 改 `endpoints` -- 改 `non_stream_status_code` -- 开关 `log_match` -- 动态查看当前 gateway 的实时日志 -- 一键恢复 Codex 原设置 - -说明： - -- 页面保存配置后会立即热生效，不需要重启 gateway -- 页面点“恢复 Codex 原设置并关闭网关”后，当前页面会失联，这是预期行为 -- 日常恢复优先用 UI；`restore-codex-config.ps1` 作为脚本级应急回滚入口保留 -- UI 恢复不会再额外拉起恢复子进程，而是由当前 gateway 直接完成恢复并退出 -- 统计口径默认按“本次 gateway 启动以来”累计 -- `516` 占比 = `reasoning_tokens = 516` 的响应次数 / 被检查响应总数 - -## 如何调整拦截条件 - -编辑： - -```text -Windows: %USERPROFILE%\.codex-retry-gateway\config\config.json -macOS / Linux: ~/.codex-retry-gateway/config/config.json -``` - -常用字段： - -- `reasoning_equals` - - 例如 `[516]` -- `endpoints` - - 默认包含 root 与 `/v1` 两套路径 -- `non_stream_status_code` - - 默认 `502` +- 流式命中时默认先缓存并判断；一旦命中 `516`，统一返回 `502` +- 默认同时拦截 root 路径和 `/v1` 路径： + - `/responses` + - `/chat/completions` + - `/v1/responses` + - `/v1/chat/completions` + +限制： + +- 这个网关不负责 `Responses` 和 `Chat Completions` 协议互转 +- 如果你的上游本身不支持 Codex 当前使用的协议，这个网关不会替你补齐转换能力 + +## 默认路径 + +Windows: + +- Codex 配置：`%USERPROFILE%\.codex\config.toml` +- Gateway 状态目录：`%USERPROFILE%\.codex-retry-gateway` + +macOS / Linux: + +- Codex 配置：`~/.codex/config.toml` +- Gateway 状态目录：`~/.codex-retry-gateway` + +## 当前版本说明 + +- 这是一个可独立发布、独立运行的仓库 +- 默认监听地址是 `http://127.0.0.1:4610` +- 默认示例上游见 `config.example.json` +- 实际运行时配置会写到当前用户目录下的 gateway 状态目录 + +## 一键启动并打开管理页 + +在仓库根目录执行： + +Windows: + +```powershell +powershell -ExecutionPolicy Bypass -File .\scripts\launch-ui.ps1 +``` + +macOS / Linux: + +```bash +bash ./scripts/launch-ui.sh +``` + +这个脚本是默认入口，执行后会自动完成： + +- 第一次运行时： + - 备份当前用户目录下的 Codex `config.toml` + - 生成当前用户目录下的 gateway `config.json` + - 启动本地 gateway + - 把当前 `model_provider` 对应的 `base_url` 改到本地 gateway +- 之后再次运行时： + - 自动复用现有安装状态 + - 自动重启或拉起 gateway + - 自动再次打开管理页 + +默认会打开： + +```text +http://127.0.0.1:4610/__codex_retry_gateway/ui +``` + +如果你只想启动、不自动开浏览器： + +```powershell +powershell -ExecutionPolicy Bypass -File .\scripts\launch-ui.ps1 -NoOpen +``` + +```bash +bash ./scripts/launch-ui.sh --no-open +``` + +常用参数： + +- Windows 参数： + - `-CodexConfigPath` + - `-StateRoot` + - `-ListenHost` + - `-ListenPort` + - `-NoOpen` +- macOS / Linux 参数： + - `--codex-config-path` + - `--state-root` + - `--listen-host` + - `--listen-port` + - `--no-open` + +macOS / Linux 说明： + +- 需要 `bash` +- 需要 `Node.js 18+` +- Unix 入口会调用跨平台 `node` 管理核心，不依赖 PowerShell +- 推荐显式使用 `bash ...sh` +- 这样即使目录是从 Windows 或压缩包复制过来、没有可执行位，也能直接运行 + +## 手工安装入口 + +如果你明确只想做脚本级安装，不想自动打开 UI，也可以直接执行： + +Windows: + +```powershell +powershell -ExecutionPolicy Bypass -File .\scripts\install-for-current-provider.ps1 +``` + +macOS / Linux: + +```bash +bash ./scripts/install-for-current-provider.sh +``` + +## 如何恢复 + +Windows: + +```powershell +powershell -ExecutionPolicy Bypass -File .\scripts\restore-codex-config.ps1 +``` + +macOS / Linux: + +```bash +bash ./scripts/restore-codex-config.sh +``` + +这个脚本会： + +- 停掉本地 gateway +- 用最近一次备份恢复当前用户目录下的 Codex `config.toml` +- 删除当前安装状态文件 + +## 管理页面 + +页面入口： + +```text +http://127.0.0.1:4610/__codex_retry_gateway/ui +``` + +页面里可以直接做这几件事： + +- 看当前监听地址、真实上游、当前 provider、当前 Codex base URL +- 看本次 gateway 启动以来的实时统计 + - 代理请求总数 + - 被检查响应总数 + - `516` 命中次数 + - `516` 占比 +- 改 `reasoning_equals` +- 改 `endpoints` +- 改 `non_stream_status_code` +- 开关 `log_match` +- 动态查看当前 gateway 的实时日志 +- 一键恢复 Codex 原设置 + +说明： + +- 页面保存配置后会立即热生效，不需要重启 gateway +- 页面点“恢复 Codex 原设置并关闭网关”后，当前页面会失联，这是预期行为 +- 日常恢复优先用 UI；`restore-codex-config.ps1` 作为脚本级应急回滚入口保留 +- UI 恢复不会再额外拉起恢复子进程，而是由当前 gateway 直接完成恢复并退出 +- 统计口径默认按“本次 gateway 启动以来”累计 +- `516` 占比 = `reasoning_tokens = 516` 的响应次数 / 被检查响应总数 + +## 如何调整拦截条件 + +编辑： + +```text +Windows: %USERPROFILE%\.codex-retry-gateway\config\config.json +macOS / Linux: ~/.codex-retry-gateway/config/config.json +``` + +常用字段： + +- `reasoning_equals` + - 例如 `[516]` +- `endpoints` + - 默认包含 root 与 `/v1` 两套路径 +- `non_stream_status_code` + - 默认 `502` - `stream_action` - - 默认 `disconnect` -- `log_match` - - 是否记录命中日志 - -改完后重启： - -```powershell -powershell -ExecutionPolicy Bypass -File .\scripts\start-gateway.ps1 -RestartIfRunning -``` - -```bash -bash ./scripts/start-gateway.sh --restart-if-running -``` - -如果你已经打开管理页，优先直接在页面里改，通常不需要手改 `config.json`。 - -## 其他机器如何应用 - -在其他 Windows 机器上： - -1. 复制整个仓库目录 -2. 确保本机有 `Node.js 18+` -3. 不需要安装 `cc-switch`，也不需要使用 `cc-switch` 路由模式 -4. 在仓库根目录执行 `powershell -ExecutionPolicy Bypass -File .\scripts\launch-ui.ps1` -5. 如需回滚，优先在 UI 里点“恢复 Codex 原设置并关闭网关”；脚本级回滚仍可执行 `powershell -ExecutionPolicy Bypass -File .\scripts\restore-codex-config.ps1` - -在其他 macOS / Linux 机器上： - -1. 复制整个仓库目录 -2. 确保本机有 `bash` -3. 确保本机有 `Node.js 18+` -4. 不需要安装 `cc-switch`，也不需要使用 `cc-switch` 路由模式 -5. 在仓库根目录执行 `bash ./scripts/launch-ui.sh` -6. 如需回滚，优先在 UI 里点“恢复 Codex 原设置并关闭网关”；脚本级回滚仍可执行 `bash ./scripts/restore-codex-config.sh` - -运行时状态默认写到当前用户目录： - -```text -Windows: %USERPROFILE%\.codex-retry-gateway -macOS / Linux: ~/.codex-retry-gateway -``` - -## 已验证事项 - -- `test-gateway-e2e.ps1` - - 已通过 - - 验证 `/responses`、`/chat/completions`、`/v1/responses`、`/v1/chat/completions` -- `test-install-restore.ps1` - - 已通过 - - 验证安装、透传、UI 页面、热更新配置、实时日志、516 统计、恢复闭环 -- `test-launch-ui.ps1` - - 已通过 - - 验证首次一键启动自动安装、再次启动自动复用、UI 可访问、默认 516 拦截仍生效 -- `test-launch-ui-unix.ps1` - - 已通过 - - 在当前 Windows 主机的 Bash 环境里验证 Unix `.sh` 入口能完成启动、透传、恢复闭环 -- `bash ./scripts/launch-ui.sh --no-open` - - 已通过 - - 当前机器实测返回 `mode=reuse` - - 后续 `GET /__codex_retry_gateway/health`、`GET /__codex_retry_gateway/ui`、`GET /v1/models` 都返回 `200` -- `codex exec` - - 已通过 - - 在 Bash 默认入口重新拉起 gateway 后，当前机器再次返回 `OK` -- 当前实机验证示例 - - `GET http://127.0.0.1:4610/__codex_retry_gateway/health` 已通过 - - `GET http://127.0.0.1:4610/v1/models` 已通过，并成功透传到配置里的真实上游 - - `GET http://127.0.0.1:4610/__codex_retry_gateway/ui` 已实际打开并确认页面内容 -- `codex exec` 历史现象 - - gateway 关闭时，真实报错地址为 `http://127.0.0.1:4610/responses` - - gateway 恢复后，`codex exec` 已再次成功返回 `OK` + - 默认 `strict_502` + - `strict_502`：先缓存整个流，命中 `516` 时统一返回 `502` + - `disconnect`：兼容旧行为；若命中发生在已透传 chunk 之后，则直接断开连接 +- `log_match` + - 是否记录命中日志 + +改完后重启： + +```powershell +powershell -ExecutionPolicy Bypass -File .\scripts\start-gateway.ps1 -RestartIfRunning +``` + +```bash +bash ./scripts/start-gateway.sh --restart-if-running +``` + +如果你已经打开管理页，优先直接在页面里改，通常不需要手改 `config.json`。 + +## 其他机器如何应用 + +在其他 Windows 机器上： + +1. 复制整个仓库目录 +2. 确保本机有 `Node.js 18+` +3. 不需要安装 `cc-switch`，也不需要使用 `cc-switch` 路由模式 +4. 在仓库根目录执行 `powershell -ExecutionPolicy Bypass -File .\scripts\launch-ui.ps1` +5. 如需回滚，优先在 UI 里点“恢复 Codex 原设置并关闭网关”；脚本级回滚仍可执行 `powershell -ExecutionPolicy Bypass -File .\scripts\restore-codex-config.ps1` + +在其他 macOS / Linux 机器上： + +1. 复制整个仓库目录 +2. 确保本机有 `bash` +3. 确保本机有 `Node.js 18+` +4. 不需要安装 `cc-switch`，也不需要使用 `cc-switch` 路由模式 +5. 在仓库根目录执行 `bash ./scripts/launch-ui.sh` +6. 如需回滚，优先在 UI 里点“恢复 Codex 原设置并关闭网关”；脚本级回滚仍可执行 `bash ./scripts/restore-codex-config.sh` + +运行时状态默认写到当前用户目录： + +```text +Windows: %USERPROFILE%\.codex-retry-gateway +macOS / Linux: ~/.codex-retry-gateway +``` + +## 已验证事项 + +- `test-gateway-e2e.ps1` + - 已通过 + - 验证 `/responses`、`/chat/completions`、`/v1/responses`、`/v1/chat/completions` +- `test-install-restore.ps1` + - 已通过 + - 验证安装、透传、UI 页面、热更新配置、实时日志、516 统计、恢复闭环 +- `test-launch-ui.ps1` + - 已通过 + - 验证首次一键启动自动安装、再次启动自动复用、UI 可访问、默认 516 拦截仍生效 +- `test-launch-ui-unix.ps1` + - 已通过 + - 在当前 Windows 主机的 Bash 环境里验证 Unix `.sh` 入口能完成启动、透传、恢复闭环 +- `bash ./scripts/launch-ui.sh --no-open` + - 已通过 + - 当前机器实测返回 `mode=reuse` + - 后续 `GET /__codex_retry_gateway/health`、`GET /__codex_retry_gateway/ui`、`GET /v1/models` 都返回 `200` +- `codex exec` + - 已通过 + - 在 Bash 默认入口重新拉起 gateway 后，当前机器再次返回 `OK` +- 当前实机验证示例 + - `GET http://127.0.0.1:4610/__codex_retry_gateway/health` 已通过 + - `GET http://127.0.0.1:4610/v1/models` 已通过，并成功透传到配置里的真实上游 + - `GET http://127.0.0.1:4610/__codex_retry_gateway/ui` 已实际打开并确认页面内容 +- `codex exec` 历史现象 + - gateway 关闭时，真实报错地址为 `http://127.0.0.1:4610/responses` + - gateway 恢复后，`codex exec` 已再次成功返回 `OK` diff --git a/config.example.json b/config.example.json index e36ce18..ec0e9df 100644 --- a/config.example.json +++ b/config.example.json @@ -1,12 +1,12 @@ -{ - "listen_host": "127.0.0.1", - "listen_port": 4610, - "upstream_base_url": "https://api.openai.com", - "request_body_limit_bytes": 10485760, - "endpoints": ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"], - "reasoning_equals": [516], - "non_stream_status_code": 502, - "stream_action": "disconnect", - "log_match": true, - "health_path": "/__codex_retry_gateway/health" -} +{ + "listen_host": "127.0.0.1", + "listen_port": 4610, + "upstream_base_url": "https://api.openai.com", + "request_body_limit_bytes": 10485760, + "endpoints": ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"], + "reasoning_equals": [516], + "non_stream_status_code": 502, + "stream_action": "strict_502", + "log_match": true, + "health_path": "/__codex_retry_gateway/health" +} diff --git a/err.md b/err.md index 6428cc6..ddc3c5f 100644 --- a/err.md +++ b/err.md @@ -1,106 +1,106 @@ -# err.md - -## 2026-06-26 独立 Codex Retry Gateway - -### 设计边界 - -- 只解决 Codex 已可访问上游时的 `reasoning_tokens = 516` 重试问题 -- 不替代 `cc-switch` 的协议路由转换 +# err.md + +## 2026-06-26 独立 Codex Retry Gateway + +### 设计边界 + +- 只解决 Codex 已可访问上游时的 `reasoning_tokens = 516` 重试问题 +- 不替代 `cc-switch` 的协议路由转换 - 流式场景默认策略是： - - 先实时透传 + - 先缓存上游流 - 一旦检测到命中 `516` - - 直接断开连接 - -### 当前已知限制 - -- 如果上游只支持 Chat Completions、而 Codex 当前链路需要 Responses 协议转换，这个项目不处理该转换 -- 这个项目依赖 Codex / Codex Desktop 自身的自动重试能力 - -### 本次已确认并修复的问题 - -1. `gateway.mjs` 非流式透传发头顺序错误 - - 现象：`ERR_HTTP_HEADERS_SENT` - - 根因：`writeHead()` 在 `copyHeadersToClient()` 之前调用 - - 结果：正常 `128` 响应也会被打断 - -2. PowerShell 脚本在 `powershell.exe` 下的解析兼容性 - - 现象：脚本乱码并伴随解析异常 - - 根因：新脚本初版包含中文运行时字符串，且 `param(...)` 不在文件最前 - - 处理：运行时输出改成 ASCII，并把 `param(...)` 提前到文件顶部 - -3. `stop-gateway.ps1` 与 PowerShell 内置只读变量 `$PID` 冲突 - - 现象：安装脚本在重启 gateway 时失败 - - 处理：改用 `$gatewayPid` - -4. `start-gateway.ps1` 启动 Node 时路径带空格 - - 现象：gateway 进程启动后立刻退出 - - 根因：`Start-Process` 参数未显式带引号 - - 处理：改为手工拼带引号的 `ArgumentList` - -5. PowerShell 单元素数组落盘时被拆成标量 - - 现象：`reasoning_equals` 被写成 `516`，不是 `[516]` - - 处理：在公共归一化函数里强制返回数组 - -6. 旧脏配置迁移后出现嵌套/拼接 endpoints - - 现象：`endpoints` 可能变成嵌套数组，或出现一条用空格拼接的脏字符串 - - 处理：安装脚本合并 endpoints 时做递归拍平和空白拆分 - -7. 真实 Codex 客户端请求路径不是 `/v1/responses` - - 现象：`codex exec` 在 gateway 关闭时真实报错地址是 `http://127.0.0.1:4610/responses` - - 结论：默认配置必须同时覆盖： - - `/responses` - - `/chat/completions` - - `/v1/responses` - - `/v1/chat/completions` - -8. UI 恢复动作最初采用“子进程拉起 restore 脚本”方案 - - 现象：浏览器拿到 `202`，但临时 `config.toml`、`state.json`、`gateway.pid` 都没有变化 - - 根因：恢复动作通过 detached 子进程接力时，链路可靠性不足，实际没有把恢复流程真正执行完 - - 处理：改为当前 gateway 进程直接复制备份、清理状态并自我退出 - -9. 新增内嵌 UI 管理页 - - 入口：`/__codex_retry_gateway/ui` - - 能力： - - 查看当前接管状态 - - 热更新 `reasoning_equals` - - 热更新 `endpoints` - - 热更新 `non_stream_status_code` - - 开关 `log_match` - - 一键恢复 Codex 原设置 - -10. 用户不接受 `cc-switch` 路由模式，且不希望手工改设置 - - 现象：仅有安装脚本和 UI 还不够，首次接管、再次拉起、重新打开 UI 仍需要手工串命令 - - 处理：新增 `launch-ui.ps1` - - 结果： - - 首次运行自动安装并打开 UI - - 再次运行自动复用 `state.json + config.json` 并重启 gateway - - 平时规则调整和恢复统一回到 UI 内完成 - -11. UI 需要动态显示实时日志、`516` 次数和占比 - - 现象：原 UI 只能改配置，看不到运行中的命中趋势 - - 处理： - - 在 `gateway.mjs` 内增加运行期统计 - - 增加日志接口 - - UI 轮询显示“被检查响应总数 / 516 命中次数 / 516 占比 / 实时日志” - - 统计口径： - - 按本次 gateway 启动以来累计 - - `516` 占比 = `reasoning_tokens = 516` 的响应次数 / 被检查响应总数 - -12. macOS / Linux 不能直接使用现有 PowerShell 管理脚本 - - 现象：`launch-ui.ps1`、`restore-codex-config.ps1` 等入口绑定了 PowerShell 和 Windows 进程控制 - - 处理： - - 新增跨平台 `node` 管理核心 - - 新增 `.sh` 包装入口： - - `launch-ui.sh` - - `restore-codex-config.sh` - - `install-for-current-provider.sh` - - `start-gateway.sh` - - `stop-gateway.sh` - - 结果： - - Windows 继续走 `.ps1` - - macOS / Linux 直接走 `.sh` - - UI、状态文件、gateway 主逻辑保持同一套 - + - 统一返回 `502` + +### 当前已知限制 + +- 如果上游只支持 Chat Completions、而 Codex 当前链路需要 Responses 协议转换，这个项目不处理该转换 +- 这个项目依赖 Codex / Codex Desktop 自身的自动重试能力 + +### 本次已确认并修复的问题 + +1. `gateway.mjs` 非流式透传发头顺序错误 + - 现象：`ERR_HTTP_HEADERS_SENT` + - 根因：`writeHead()` 在 `copyHeadersToClient()` 之前调用 + - 结果：正常 `128` 响应也会被打断 + +2. PowerShell 脚本在 `powershell.exe` 下的解析兼容性 + - 现象：脚本乱码并伴随解析异常 + - 根因：新脚本初版包含中文运行时字符串，且 `param(...)` 不在文件最前 + - 处理：运行时输出改成 ASCII，并把 `param(...)` 提前到文件顶部 + +3. `stop-gateway.ps1` 与 PowerShell 内置只读变量 `$PID` 冲突 + - 现象：安装脚本在重启 gateway 时失败 + - 处理：改用 `$gatewayPid` + +4. `start-gateway.ps1` 启动 Node 时路径带空格 + - 现象：gateway 进程启动后立刻退出 + - 根因：`Start-Process` 参数未显式带引号 + - 处理：改为手工拼带引号的 `ArgumentList` + +5. PowerShell 单元素数组落盘时被拆成标量 + - 现象：`reasoning_equals` 被写成 `516`，不是 `[516]` + - 处理：在公共归一化函数里强制返回数组 + +6. 旧脏配置迁移后出现嵌套/拼接 endpoints + - 现象：`endpoints` 可能变成嵌套数组，或出现一条用空格拼接的脏字符串 + - 处理：安装脚本合并 endpoints 时做递归拍平和空白拆分 + +7. 真实 Codex 客户端请求路径不是 `/v1/responses` + - 现象：`codex exec` 在 gateway 关闭时真实报错地址是 `http://127.0.0.1:4610/responses` + - 结论：默认配置必须同时覆盖： + - `/responses` + - `/chat/completions` + - `/v1/responses` + - `/v1/chat/completions` + +8. UI 恢复动作最初采用“子进程拉起 restore 脚本”方案 + - 现象：浏览器拿到 `202`，但临时 `config.toml`、`state.json`、`gateway.pid` 都没有变化 + - 根因：恢复动作通过 detached 子进程接力时，链路可靠性不足，实际没有把恢复流程真正执行完 + - 处理：改为当前 gateway 进程直接复制备份、清理状态并自我退出 + +9. 新增内嵌 UI 管理页 + - 入口：`/__codex_retry_gateway/ui` + - 能力： + - 查看当前接管状态 + - 热更新 `reasoning_equals` + - 热更新 `endpoints` + - 热更新 `non_stream_status_code` + - 开关 `log_match` + - 一键恢复 Codex 原设置 + +10. 用户不接受 `cc-switch` 路由模式，且不希望手工改设置 + - 现象：仅有安装脚本和 UI 还不够，首次接管、再次拉起、重新打开 UI 仍需要手工串命令 + - 处理：新增 `launch-ui.ps1` + - 结果： + - 首次运行自动安装并打开 UI + - 再次运行自动复用 `state.json + config.json` 并重启 gateway + - 平时规则调整和恢复统一回到 UI 内完成 + +11. UI 需要动态显示实时日志、`516` 次数和占比 + - 现象：原 UI 只能改配置，看不到运行中的命中趋势 + - 处理： + - 在 `gateway.mjs` 内增加运行期统计 + - 增加日志接口 + - UI 轮询显示“被检查响应总数 / 516 命中次数 / 516 占比 / 实时日志” + - 统计口径： + - 按本次 gateway 启动以来累计 + - `516` 占比 = `reasoning_tokens = 516` 的响应次数 / 被检查响应总数 + +12. macOS / Linux 不能直接使用现有 PowerShell 管理脚本 + - 现象：`launch-ui.ps1`、`restore-codex-config.ps1` 等入口绑定了 PowerShell 和 Windows 进程控制 + - 处理： + - 新增跨平台 `node` 管理核心 + - 新增 `.sh` 包装入口： + - `launch-ui.sh` + - `restore-codex-config.sh` + - `install-for-current-provider.sh` + - `start-gateway.sh` + - `stop-gateway.sh` + - 结果： + - Windows 继续走 `.ps1` + - macOS / Linux 直接走 `.sh` + - UI、状态文件、gateway 主逻辑保持同一套 + 13. Windows 主机上模拟 Unix shell 入口时存在路径与 Node 版本兼容问题 - 现象： - Bash 入口最初找不到脚本路径 @@ -111,37 +111,63 @@ - `.sh` 优先选择 `node.exe` - 在 WSL / Bash 场景下把路径参数转换回 Windows 路径后再交给 `node.exe` -### 2026-06-26 实测证据 - -- 假上游 E2E - - `test-gateway-e2e.ps1` 通过 - - 已验证 root 路径和 `/v1` 路径都能区分 `516` 与 `128` -- 安装/恢复闭环 - - `test-install-restore.ps1` 通过 - - 已验证 UI 页面、状态接口、日志接口、516 统计、热更新配置、UI 恢复闭环 -- 一键启动入口 - - `test-launch-ui.ps1` 通过 - - 已验证首次启动自动安装、再次启动自动复用、UI 页面可达、默认 `516 -> 502` 规则仍生效 -- Unix shell 入口 - - `test-launch-ui-unix.ps1` 通过 - - 已验证 `.sh` 入口能完成启动、透传、恢复闭环 -- Bash 默认入口实机验证 - - `bash ./scripts/launch-ui.sh --no-open` 通过 - - 输出 `mode=reuse` - - `GET /__codex_retry_gateway/health` 返回 `200` - - `GET /__codex_retry_gateway/ui` 返回 `200` - - `GET /v1/models` 返回 `200`，并继续透传到真实上游 -- Bash 入口后的 `codex exec` 实机验证 - - 命令退出码 `0` - - 最后一条消息文件返回 `OK` -- 当前真实 provider - - 当前 Codex 配置里的 `base_url` 已可切到 `http://127.0.0.1:4610` - - 当前 gateway 运行配置里的 `upstream_base_url` 会指向用户自己的真实上游 - - `GET /__codex_retry_gateway/health` 返回 `ok=true` - - `GET /v1/models` 已经经本地 gateway 成功透传到真实上游 - - `GET /__codex_retry_gateway/ui` 已实机打开，页面显示当前 upstream、provider、config 路径和 516 规则 -- 真实 `codex exec` - - gateway 停止时，CLI 真实提示： - - `url: http://127.0.0.1:4610/responses` - - 并自动进入 `Reconnecting...` - - gateway 恢复后，`codex exec` 在临时目录再次成功返回 `OK` +14. 上游流式连接中途终止时被误记为网关错误，首次瞬断也缺少最小重试 + - 现象： + - 日志出现： + - `TypeError: terminated` + - `TypeError: fetch failed` + - 其中一部分来自上游 SSE 中途断流，另一部分来自上游首次连接瞬时失败 + - 根因： + - `handleStreaming()` 直接把 `reader.read()` 抛出的 `AbortError` / `TypeError: terminated` 冒到统一错误处理 + - `proxyRequest()` 对上游 `fetch()` 没有做一次轻量重试，首个瞬断会直接返回 `502` + - 处理： + - 新增预期流终止识别： + - `AbortError` + - `TypeError: terminated` + - 这两类在流式处理中按“连接已结束”收口，不再记 `[error]` + - 新增上游 `fetch failed` 的一次自动重试 + - 新增严格 `502` 流式模式： + - 默认不再抢先透传 `200` 头和首个 chunk + - 先缓存流，再根据 `reasoning_tokens` 决定透传或返回 `502` + - 验证： + - `scripts/test-gateway-e2e.mjs` + - 新增 `/responses` 流式覆盖 + - 新增“上游半路断流不刷 error 日志”断言 + - 新增“首次 fetch failed 后第二次成功恢复”断言 + - 新增“流式 `516` 统一返回 `502`，不再先透传半截 chunk”断言 + - `scripts/test-install-restore.mjs` 继续通过 + +### 2026-06-26 实测证据 + +- 假上游 E2E + - `test-gateway-e2e.ps1` 通过 + - 已验证 root 路径和 `/v1` 路径都能区分 `516` 与 `128` +- 安装/恢复闭环 + - `test-install-restore.ps1` 通过 + - 已验证 UI 页面、状态接口、日志接口、516 统计、热更新配置、UI 恢复闭环 +- 一键启动入口 + - `test-launch-ui.ps1` 通过 + - 已验证首次启动自动安装、再次启动自动复用、UI 页面可达、默认 `516 -> 502` 规则仍生效 +- Unix shell 入口 + - `test-launch-ui-unix.ps1` 通过 + - 已验证 `.sh` 入口能完成启动、透传、恢复闭环 +- Bash 默认入口实机验证 + - `bash ./scripts/launch-ui.sh --no-open` 通过 + - 输出 `mode=reuse` + - `GET /__codex_retry_gateway/health` 返回 `200` + - `GET /__codex_retry_gateway/ui` 返回 `200` + - `GET /v1/models` 返回 `200`，并继续透传到真实上游 +- Bash 入口后的 `codex exec` 实机验证 + - 命令退出码 `0` + - 最后一条消息文件返回 `OK` +- 当前真实 provider + - 当前 Codex 配置里的 `base_url` 已可切到 `http://127.0.0.1:4610` + - 当前 gateway 运行配置里的 `upstream_base_url` 会指向用户自己的真实上游 + - `GET /__codex_retry_gateway/health` 返回 `ok=true` + - `GET /v1/models` 已经经本地 gateway 成功透传到真实上游 + - `GET /__codex_retry_gateway/ui` 已实机打开，页面显示当前 upstream、provider、config 路径和 516 规则 +- 真实 `codex exec` + - gateway 停止时，CLI 真实提示： + - `url: http://127.0.0.1:4610/responses` + - 并自动进入 `Reconnecting...` + - gateway 恢复后，`codex exec` 在临时目录再次成功返回 `OK` diff --git a/gateway.mjs b/gateway.mjs index 6badeb9..b4aa5fa 100644 --- a/gateway.mjs +++ b/gateway.mjs @@ -1,1288 +1,1332 @@ -#!/usr/bin/env node - -import http from "node:http"; -import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises"; -import fs from "node:fs"; -import path from "node:path"; -import { TextDecoder } from "node:util"; -import { fileURLToPath } from "node:url"; - -const __filename = fileURLToPath(import.meta.url); -const __dirname = path.dirname(__filename); -const ADMIN_BASE_PATH = "/__codex_retry_gateway"; -const UI_PATH = `${ADMIN_BASE_PATH}/ui`; -const STATUS_API_PATH = `${ADMIN_BASE_PATH}/api/status`; -const CONFIG_API_PATH = `${ADMIN_BASE_PATH}/api/config`; -const LOGS_API_PATH = `${ADMIN_BASE_PATH}/api/logs`; -const RESTORE_API_PATH = `${ADMIN_BASE_PATH}/api/restore`; - -const DEFAULT_CONFIG = { - listen_host: "127.0.0.1", - listen_port: 4610, - upstream_base_url: "", - request_body_limit_bytes: 10 * 1024 * 1024, - endpoints: ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"], - reasoning_equals: [516], - non_stream_status_code: 502, - stream_action: "disconnect", - log_match: true, - health_path: "/__codex_retry_gateway/health", -}; - -const REASONING_POINTERS = [ - "/usage/output_tokens_details/reasoning_tokens", - "/usage/completion_tokens_details/reasoning_tokens", - "/response/usage/output_tokens_details/reasoning_tokens", - "/response/usage/completion_tokens_details/reasoning_tokens", -]; - -function parseArgs(argv) { - const args = { config: null, log: null }; - for (let i = 2; i < argv.length; i += 1) { - const current = argv[i]; - if (current === "--config") { - args.config = argv[i + 1]; - i += 1; - } else if (current === "--log") { - args.log = argv[i + 1]; - i += 1; - } else if (current === "--help" || current === "-h") { - printHelp(); - process.exit(0); - } - } - return args; -} - -function printHelp() { - process.stdout.write( - [ - "用法:", - " node gateway.mjs --config [--log ]", - "", - "说明:", - " 独立 Codex 本地重试网关。", - " 非流式命中 reasoning_tokens=516 时返回 502。", - " 流式命中时默认直接断开连接，交给 Codex 自身重试。", - "", - ].join("\n"), - ); -} - -function normalizePath(inputPath) { - const [withoutQuery] = `${inputPath || "/"}`.split("?"); - const trimmed = withoutQuery.length > 1 ? withoutQuery.replace(/\/+$/, "") : withoutQuery; - return trimmed || "/"; -} - -function flattenValues(value) { - if (Array.isArray(value)) { - return value.flatMap((item) => flattenValues(item)); - } - return [value]; -} - -function isJsonContentType(contentType) { - return `${contentType || ""}`.toLowerCase().includes("application/json"); -} - -function isSseContentType(contentType) { - return `${contentType || ""}`.toLowerCase().includes("text/event-stream"); -} - -function jsonPointerGet(value, pointer) { - if (!pointer.startsWith("/")) { - return undefined; - } - return pointer - .slice(1) - .split("/") - .map((segment) => segment.replace(/~1/g, "/").replace(/~0/g, "~")) - .reduce((current, segment) => { - if (current === null || current === undefined) { - return undefined; - } - return current[segment]; - }, value); -} - -function extractReasoningTokens(payload) { - for (const pointer of REASONING_POINTERS) { - const raw = jsonPointerGet(payload, pointer); - if (Number.isInteger(raw)) { - return raw; - } - } - return null; -} - -function normalizeIntegerList(values, fallback = []) { - const source = values === undefined || values === null ? fallback : values; - const normalized = flattenValues(source) - .flatMap((value) => { - if (typeof value === "string") { - return value.split(/[\s,]+/).filter(Boolean); - } - return [value]; - }) - .map((value) => Number.parseInt(`${value}`, 10)) - .filter((value) => Number.isInteger(value)); - - return [...new Set(normalized)]; -} - -function normalizeStringList(values, fallback = []) { - const source = values === undefined || values === null ? fallback : values; - const normalized = flattenValues(source) - .flatMap((value) => `${value ?? ""}`.split(/[\s,]+/)) - .map((value) => value.trim()) - .filter(Boolean); - - return [...new Set(normalized)]; -} - +#!/usr/bin/env node + +import http from "node:http"; +import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises"; +import fs from "node:fs"; +import path from "node:path"; +import { TextDecoder } from "node:util"; +import { fileURLToPath } from "node:url"; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = path.dirname(__filename); +const ADMIN_BASE_PATH = "/__codex_retry_gateway"; +const UI_PATH = `${ADMIN_BASE_PATH}/ui`; +const STATUS_API_PATH = `${ADMIN_BASE_PATH}/api/status`; +const CONFIG_API_PATH = `${ADMIN_BASE_PATH}/api/config`; +const LOGS_API_PATH = `${ADMIN_BASE_PATH}/api/logs`; +const RESTORE_API_PATH = `${ADMIN_BASE_PATH}/api/restore`; + +const DEFAULT_CONFIG = { + listen_host: "127.0.0.1", + listen_port: 4610, + upstream_base_url: "", + request_body_limit_bytes: 10 * 1024 * 1024, + endpoints: ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"], + reasoning_equals: [516], + non_stream_status_code: 502, + stream_action: "strict_502", + log_match: true, + health_path: "/__codex_retry_gateway/health", +}; + +const REASONING_POINTERS = [ + "/usage/output_tokens_details/reasoning_tokens", + "/usage/completion_tokens_details/reasoning_tokens", + "/response/usage/output_tokens_details/reasoning_tokens", + "/response/usage/completion_tokens_details/reasoning_tokens", +]; + +function parseArgs(argv) { + const args = { config: null, log: null }; + for (let i = 2; i < argv.length; i += 1) { + const current = argv[i]; + if (current === "--config") { + args.config = argv[i + 1]; + i += 1; + } else if (current === "--log") { + args.log = argv[i + 1]; + i += 1; + } else if (current === "--help" || current === "-h") { + printHelp(); + process.exit(0); + } + } + return args; +} + +function printHelp() { + process.stdout.write( + [ + "用法:", + " node gateway.mjs --config [--log ]", + "", + "说明:", + " 独立 Codex 本地重试网关。", + " 非流式命中 reasoning_tokens=516 时返回 502。", + " 流式命中时默认缓存并返回 502，避免半截流返回。", + "", + ].join("\n"), + ); +} + +function normalizePath(inputPath) { + const [withoutQuery] = `${inputPath || "/"}`.split("?"); + const trimmed = withoutQuery.length > 1 ? withoutQuery.replace(/\/+$/, "") : withoutQuery; + return trimmed || "/"; +} + +function flattenValues(value) { + if (Array.isArray(value)) { + return value.flatMap((item) => flattenValues(item)); + } + return [value]; +} + +function isJsonContentType(contentType) { + return `${contentType || ""}`.toLowerCase().includes("application/json"); +} + +function isSseContentType(contentType) { + return `${contentType || ""}`.toLowerCase().includes("text/event-stream"); +} + +function jsonPointerGet(value, pointer) { + if (!pointer.startsWith("/")) { + return undefined; + } + return pointer + .slice(1) + .split("/") + .map((segment) => segment.replace(/~1/g, "/").replace(/~0/g, "~")) + .reduce((current, segment) => { + if (current === null || current === undefined) { + return undefined; + } + return current[segment]; + }, value); +} + +function extractReasoningTokens(payload) { + for (const pointer of REASONING_POINTERS) { + const raw = jsonPointerGet(payload, pointer); + if (Number.isInteger(raw)) { + return raw; + } + } + return null; +} + +function normalizeIntegerList(values, fallback = []) { + const source = values === undefined || values === null ? fallback : values; + const normalized = flattenValues(source) + .flatMap((value) => { + if (typeof value === "string") { + return value.split(/[\s,]+/).filter(Boolean); + } + return [value]; + }) + .map((value) => Number.parseInt(`${value}`, 10)) + .filter((value) => Number.isInteger(value)); + + return [...new Set(normalized)]; +} + +function normalizeStringList(values, fallback = []) { + const source = values === undefined || values === null ? fallback : values; + const normalized = flattenValues(source) + .flatMap((value) => `${value ?? ""}`.split(/[\s,]+/)) + .map((value) => value.trim()) + .filter(Boolean); + + return [...new Set(normalized)]; +} + function buildBlockedBody(pathname, reasoning, statusCode) { return JSON.stringify({ error: { message: `codex retry gateway blocked suspicious reasoning response on ${pathname}`, - type: "codex_retry_gateway", - code: "reasoning_guard_triggered", - reasoning_tokens: reasoning, - status_code: statusCode, + type: "codex_retry_gateway", + code: "reasoning_guard_triggered", + reasoning_tokens: reasoning, + status_code: statusCode, }, }); } -function createMonitor() { - return { - started_at: new Date().toISOString(), - next_log_seq: 1, - log_entries: [], - total_proxy_request_count: 0, - inspected_response_count: 0, - matched_response_count: 0, - observed_reasoning_counts: {}, - }; -} - -function createMonitorRecorder(monitor) { - return (message) => { - const entry = { - seq: monitor.next_log_seq, - at: new Date().toISOString(), +function buildGatewayErrorBody(message) { + return JSON.stringify({ + error: { message, - }; - monitor.next_log_seq += 1; - monitor.log_entries.push(entry); - return entry; - }; -} - -function createLogger(logPath, recordEntry) { - if (!logPath) { - return (message) => { - const entry = recordEntry ? recordEntry(message) : { at: new Date().toISOString(), message }; - process.stdout.write(`${entry.at} ${entry.message}\n`); - }; - } - - const stream = fs.createWriteStream(logPath, { flags: "a" }); - return (message) => { - const entry = recordEntry ? recordEntry(message) : { at: new Date().toISOString(), message }; - const line = `${entry.at} ${entry.message}\n`; - stream.write(line); - process.stdout.write(line); - }; -} - -function incrementReasoningCount(counter, reasoning) { - if (!Number.isInteger(reasoning)) { - return; - } - const key = `${reasoning}`; - counter[key] = (counter[key] || 0) + 1; -} - -function recordInspectedResponse(monitor, reasoning, matched) { - monitor.inspected_response_count += 1; - incrementReasoningCount(monitor.observed_reasoning_counts, reasoning); - if (matched) { - monitor.matched_response_count += 1; - } -} - -function buildMetricsSnapshot(monitor) { - const reasoning516Count = monitor.observed_reasoning_counts["516"] || 0; - const inspectedResponseCount = monitor.inspected_response_count; - return { - started_at: monitor.started_at, - total_proxy_request_count: monitor.total_proxy_request_count, - inspected_response_count: inspectedResponseCount, - matched_response_count: monitor.matched_response_count, - reasoning_516_count: reasoning516Count, - reasoning_516_ratio: - inspectedResponseCount === 0 ? 0 : reasoning516Count / inspectedResponseCount, - observed_reasoning_counts: { ...monitor.observed_reasoning_counts }, - }; -} - -function buildLogsSnapshot(monitor, sinceSeq = null) { - const entries = Number.isInteger(sinceSeq) - ? monitor.log_entries.filter((entry) => entry.seq > sinceSeq) - : monitor.log_entries; - - return { - total_entries: monitor.log_entries.length, - latest_seq: monitor.next_log_seq - 1, - entries, - }; -} - -async function loadConfig(configPath) { - const content = await readFile(configPath, "utf8"); - const loaded = JSON.parse(content); - const config = { ...DEFAULT_CONFIG, ...loaded }; - config.endpoints = normalizeStringList(config.endpoints, DEFAULT_CONFIG.endpoints).map(normalizePath); - config.reasoning_equals = normalizeIntegerList( - config.reasoning_equals, - DEFAULT_CONFIG.reasoning_equals, - ); - if (!config.upstream_base_url) { - throw new Error("配置缺少 upstream_base_url"); - } - return config; -} - -function buildRuntimePaths(configPath, logPath) { - const configDirectory = path.dirname(configPath); - const stateRoot = path.dirname(configDirectory); - return { - stateRoot, - statePath: path.join(stateRoot, "state.json"), - pidPath: path.join(stateRoot, "gateway.pid"), - configPath, - logPath, - }; -} - -async function readOptionalJson(jsonPath) { - try { - const content = await readFile(jsonPath, "utf8"); - return JSON.parse(content); - } catch { - return null; - } -} - -async function writeConfig(configPath, config) { - await writeFile(configPath, `${JSON.stringify(config, null, 2)}\n`, "utf8"); -} - -function extractProviderBaseUrl(content, providerName) { - if (!content || !providerName) { - return null; - } - - const sectionPattern = new RegExp( - String.raw`^\[model_providers\.${providerName.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}\]\s*$[\s\S]*?(?=^\[|\Z)`, - "m", - ); - const sectionMatch = content.match(sectionPattern); - if (!sectionMatch) { - return null; - } - - const baseUrlMatch = sectionMatch[0].match(/^\s*base_url\s*=\s*"([^"]+)"\s*$/m); - return baseUrlMatch ? baseUrlMatch[1] : null; -} - -async function readRuntimeState(runtime) { - const state = await readOptionalJson(runtime.paths.statePath); - if (!state) { - return null; - } - - let codexCurrentBaseUrl = null; - if (state.codex_config_path && state.provider_name) { - try { - const codexConfig = await readFile(state.codex_config_path, "utf8"); - codexCurrentBaseUrl = extractProviderBaseUrl(codexConfig, state.provider_name); - } catch { - codexCurrentBaseUrl = null; - } - } - - return { - ...state, - codex_current_base_url: codexCurrentBaseUrl, - }; -} - -async function restoreRuntimeState(runtime, state) { - const backupPath = state?.latest_backup_path; - const codexConfigPath = state?.codex_config_path; - - if (!backupPath || !fs.existsSync(backupPath)) { - throw new Error(`未找到可恢复备份: ${backupPath || "unknown"}`); - } - if (!codexConfigPath) { - throw new Error("安装状态里缺少 codex_config_path"); - } - - await copyFile(backupPath, codexConfigPath); - await Promise.all([ - rm(runtime.paths.statePath, { force: true }), - rm(runtime.paths.pidPath, { force: true }), - ]); -} - -function jsonResponse(res, statusCode, payload, headers = {}) { - res.writeHead(statusCode, { - "content-type": "application/json; charset=utf-8", - ...headers, + type: "codex_retry_gateway_error", + code: "gateway_error", + }, }); - res.end(JSON.stringify(payload)); } - -function htmlResponse(res, html) { - res.writeHead(200, { "content-type": "text/html; charset=utf-8" }); - res.end(html); -} - -function buildEditableConfig(currentConfig, payload) { - const nextReasoning = normalizeIntegerList(payload.reasoning_equals, currentConfig.reasoning_equals); - const nextEndpoints = normalizeStringList(payload.endpoints, currentConfig.endpoints).map(normalizePath); - const nextStatusCode = - payload.non_stream_status_code === undefined - ? currentConfig.non_stream_status_code - : Number.parseInt(`${payload.non_stream_status_code}`, 10); - - if (nextReasoning.length === 0) { - throw new Error("reasoning_equals 不能为空"); - } - if (nextEndpoints.length === 0) { - throw new Error("endpoints 不能为空"); - } - if (!Number.isInteger(nextStatusCode) || nextStatusCode < 100 || nextStatusCode > 599) { - throw new Error("non_stream_status_code 必须是 100-599 的整数"); - } - - return { - ...currentConfig, - reasoning_equals: nextReasoning, - endpoints: nextEndpoints, - non_stream_status_code: nextStatusCode, - log_match: payload.log_match === undefined ? currentConfig.log_match : Boolean(payload.log_match), - }; -} - -function buildManagementHtml() { - const uiConfig = { - statusPath: STATUS_API_PATH, - configPath: CONFIG_API_PATH, - logsPath: LOGS_API_PATH, - restorePath: RESTORE_API_PATH, - }; - - return ` - - - - - Codex Retry Gateway - - - -
-
-
本地管理页
-

Codex Retry Gateway

-

- 这个页面直接挂在正在运行的 gateway 上。你可以在这里查看当前接管状态、修改 516 拦截条件，并一键恢复 Codex 原设置。 -

-
- -
-
-
-

运行状态

-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
0
-
0
-
0
-
0.00%
-
0
-
-

- 如果“当前 Codex Base URL”已经是本机监听地址，就说明当前 Codex 已经被这个 gateway 接管。统计口径按本次 gateway 启动以来累计。 -

-
-
- -
-
-

拦截规则

-
-
- - -
多个值用英文逗号或空格分隔。
-
- -
- - -
每行一个路径。默认建议同时保留 root 与 /v1 两套路径。
-
- -
- - -
- -
- - -
- -
- - -
-
-
-

- 点击“恢复”后，gateway 会停掉，所以这个页面会失联。这是预期行为，不是报错。 -

-
-
- -
-
-

实时日志

-

正在读取日志...

- 
正在读取日志...
-
-
-
-
- - - -`; -} - -async function handleManagementRequest(runtime, req, res, requestUrl) { - const pathname = normalizePath(requestUrl.pathname); - - if (pathname === UI_PATH) { - htmlResponse(res, buildManagementHtml()); - return true; - } - - if (pathname === STATUS_API_PATH && req.method === "GET") { - const state = await readRuntimeState(runtime); - jsonResponse(res, 200, { - ok: true, - listen: `${runtime.config.listen_host}:${runtime.config.listen_port}`, - config: runtime.config, - state, - paths: { - config_path: runtime.configPath, - state_path: runtime.paths.statePath, - state_root: runtime.paths.stateRoot, - log_path: runtime.logPath, - }, - metrics: buildMetricsSnapshot(runtime.monitor), - }); - return true; - } - - if (pathname === LOGS_API_PATH && req.method === "GET") { - const sinceSeqRaw = requestUrl.searchParams.get("since_seq"); - const sinceSeq = sinceSeqRaw === null ? null : Number.parseInt(sinceSeqRaw, 10); - jsonResponse(res, 200, { - ok: true, - ...buildLogsSnapshot(runtime.monitor, Number.isInteger(sinceSeq) ? sinceSeq : null), - }); - return true; - } - - if (pathname === CONFIG_API_PATH && req.method === "POST") { - const body = await readRequestBody(req, runtime.config.request_body_limit_bytes); - const payload = parseJsonSafely(body); - if (!payload) { - jsonResponse(res, 400, { - error: { - message: "配置保存请求必须是有效 JSON", - code: "invalid_json", - }, - }); - return true; - } - - const nextConfig = buildEditableConfig(runtime.config, payload); - await writeConfig(runtime.configPath, nextConfig); - runtime.config = nextConfig; - runtime.logger( - `[config] updated reasoning_equals=${nextConfig.reasoning_equals.join(",")} endpoints=${nextConfig.endpoints.join(",")}`, - ); - const state = await readRuntimeState(runtime); - jsonResponse(res, 200, { - ok: true, - message: "配置已保存并立即生效", - config: runtime.config, - state, - paths: { - config_path: runtime.configPath, - state_path: runtime.paths.statePath, - state_root: runtime.paths.stateRoot, - log_path: runtime.logPath, - }, - metrics: buildMetricsSnapshot(runtime.monitor), - }); - return true; - } - - if (pathname === RESTORE_API_PATH && req.method === "POST") { - const state = await readRuntimeState(runtime); - if (!state) { - jsonResponse(res, 409, { - error: { - message: "当前未检测到安装状态，无法恢复 Codex 原设置", - code: "state_not_found", - }, - }); - return true; - } - - await restoreRuntimeState(runtime, state); - runtime.logger(`[restore] restored via UI state_root=${runtime.paths.stateRoot}`); - jsonResponse(res, 202, { - ok: true, - message: "原设置已恢复，gateway 即将关闭", - }); - res.on("finish", () => { - const exitTimer = setTimeout(() => { - if (runtime.server) { - runtime.server.close(() => { - process.exit(0); - }); - } else { - process.exit(0); - } - - const hardExitTimer = setTimeout(() => { - process.exit(0); - }, 600); - hardExitTimer.unref(); - }, 120); - exitTimer.unref(); - }); - return true; - } - - return false; -} - -function buildUpstreamUrl(baseUrl, requestUrl) { - const upstream = new URL(baseUrl); - const normalizedBasePath = upstream.pathname.endsWith("/") - ? upstream.pathname.slice(0, -1) - : upstream.pathname; - const incomingPath = requestUrl.pathname; - - let finalPath = incomingPath; - if (normalizedBasePath && normalizedBasePath !== "/") { - if (incomingPath.startsWith(`${normalizedBasePath}/`) || incomingPath === normalizedBasePath) { - finalPath = incomingPath; - } else if (normalizedBasePath.endsWith("/v1") && incomingPath.startsWith("/v1/")) { - finalPath = `${normalizedBasePath}${incomingPath.slice(3)}`; - } else { - finalPath = `${normalizedBasePath}${incomingPath}`; - } - } - - upstream.pathname = finalPath; - upstream.search = requestUrl.search; - return upstream.toString(); -} - -function cloneHeadersForUpstream(headers) { - const outgoing = new Headers(); - for (const [key, value] of Object.entries(headers)) { - if (value === undefined) { - continue; - } - const lowerKey = key.toLowerCase(); - if ( - lowerKey === "host" || - lowerKey === "content-length" || - lowerKey === "connection" || - lowerKey === "transfer-encoding" - ) { - continue; - } - if (Array.isArray(value)) { - for (const item of value) { - outgoing.append(key, item); - } - } else { - outgoing.set(key, value); - } - } - return outgoing; -} - -function copyHeadersToClient(sourceHeaders, target) { - for (const [key, value] of sourceHeaders.entries()) { - const lowerKey = key.toLowerCase(); - if ( - lowerKey === "content-length" || - lowerKey === "transfer-encoding" || - lowerKey === "content-encoding" || - lowerKey === "connection" - ) { - continue; - } - target.setHeader(key, value); - } -} - -async function readRequestBody(req, limitBytes) { - const chunks = []; - let total = 0; - for await (const chunk of req) { - total += chunk.length; - if (total > limitBytes) { - throw new Error(`请求体超过限制: ${limitBytes} bytes`); - } - chunks.push(chunk); - } - return Buffer.concat(chunks); -} - -function parseJsonSafely(buffer) { - try { - return JSON.parse(buffer.toString("utf8")); - } catch { - return null; - } -} - -function matchPath(config, pathname) { - return config.endpoints.includes(normalizePath(pathname)); -} - + +function createMonitor() { + return { + started_at: new Date().toISOString(), + next_log_seq: 1, + log_entries: [], + total_proxy_request_count: 0, + inspected_response_count: 0, + matched_response_count: 0, + observed_reasoning_counts: {}, + }; +} + +function createMonitorRecorder(monitor) { + return (message) => { + const entry = { + seq: monitor.next_log_seq, + at: new Date().toISOString(), + message, + }; + monitor.next_log_seq += 1; + monitor.log_entries.push(entry); + return entry; + }; +} + +function createLogger(logPath, recordEntry) { + if (!logPath) { + return (message) => { + const entry = recordEntry ? recordEntry(message) : { at: new Date().toISOString(), message }; + process.stdout.write(`${entry.at} ${entry.message}\n`); + }; + } + + const stream = fs.createWriteStream(logPath, { flags: "a" }); + return (message) => { + const entry = recordEntry ? recordEntry(message) : { at: new Date().toISOString(), message }; + const line = `${entry.at} ${entry.message}\n`; + stream.write(line); + process.stdout.write(line); + }; +} + +function incrementReasoningCount(counter, reasoning) { + if (!Number.isInteger(reasoning)) { + return; + } + const key = `${reasoning}`; + counter[key] = (counter[key] || 0) + 1; +} + +function recordInspectedResponse(monitor, reasoning, matched) { + monitor.inspected_response_count += 1; + incrementReasoningCount(monitor.observed_reasoning_counts, reasoning); + if (matched) { + monitor.matched_response_count += 1; + } +} + +function buildMetricsSnapshot(monitor) { + const reasoning516Count = monitor.observed_reasoning_counts["516"] || 0; + const inspectedResponseCount = monitor.inspected_response_count; + return { + started_at: monitor.started_at, + total_proxy_request_count: monitor.total_proxy_request_count, + inspected_response_count: inspectedResponseCount, + matched_response_count: monitor.matched_response_count, + reasoning_516_count: reasoning516Count, + reasoning_516_ratio: + inspectedResponseCount === 0 ? 0 : reasoning516Count / inspectedResponseCount, + observed_reasoning_counts: { ...monitor.observed_reasoning_counts }, + }; +} + +function buildLogsSnapshot(monitor, sinceSeq = null) { + const entries = Number.isInteger(sinceSeq) + ? monitor.log_entries.filter((entry) => entry.seq > sinceSeq) + : monitor.log_entries; + + return { + total_entries: monitor.log_entries.length, + latest_seq: monitor.next_log_seq - 1, + entries, + }; +} + +async function loadConfig(configPath) { + const content = await readFile(configPath, "utf8"); + const loaded = JSON.parse(content); + const config = { ...DEFAULT_CONFIG, ...loaded }; + config.endpoints = normalizeStringList(config.endpoints, DEFAULT_CONFIG.endpoints).map(normalizePath); + config.reasoning_equals = normalizeIntegerList( + config.reasoning_equals, + DEFAULT_CONFIG.reasoning_equals, + ); + if (!config.upstream_base_url) { + throw new Error("配置缺少 upstream_base_url"); + } + return config; +} + +function buildRuntimePaths(configPath, logPath) { + const configDirectory = path.dirname(configPath); + const stateRoot = path.dirname(configDirectory); + return { + stateRoot, + statePath: path.join(stateRoot, "state.json"), + pidPath: path.join(stateRoot, "gateway.pid"), + configPath, + logPath, + }; +} + +async function readOptionalJson(jsonPath) { + try { + const content = await readFile(jsonPath, "utf8"); + return JSON.parse(content); + } catch { + return null; + } +} + +async function writeConfig(configPath, config) { + await writeFile(configPath, `${JSON.stringify(config, null, 2)}\n`, "utf8"); +} + +function extractProviderBaseUrl(content, providerName) { + if (!content || !providerName) { + return null; + } + + const sectionPattern = new RegExp( + String.raw`^\[model_providers\.${providerName.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}\]\s*$[\s\S]*?(?=^\[|\Z)`, + "m", + ); + const sectionMatch = content.match(sectionPattern); + if (!sectionMatch) { + return null; + } + + const baseUrlMatch = sectionMatch[0].match(/^\s*base_url\s*=\s*"([^"]+)"\s*$/m); + return baseUrlMatch ? baseUrlMatch[1] : null; +} + +async function readRuntimeState(runtime) { + const state = await readOptionalJson(runtime.paths.statePath); + if (!state) { + return null; + } + + let codexCurrentBaseUrl = null; + if (state.codex_config_path && state.provider_name) { + try { + const codexConfig = await readFile(state.codex_config_path, "utf8"); + codexCurrentBaseUrl = extractProviderBaseUrl(codexConfig, state.provider_name); + } catch { + codexCurrentBaseUrl = null; + } + } + + return { + ...state, + codex_current_base_url: codexCurrentBaseUrl, + }; +} + +async function restoreRuntimeState(runtime, state) { + const backupPath = state?.latest_backup_path; + const codexConfigPath = state?.codex_config_path; + + if (!backupPath || !fs.existsSync(backupPath)) { + throw new Error(`未找到可恢复备份: ${backupPath || "unknown"}`); + } + if (!codexConfigPath) { + throw new Error("安装状态里缺少 codex_config_path"); + } + + await copyFile(backupPath, codexConfigPath); + await Promise.all([ + rm(runtime.paths.statePath, { force: true }), + rm(runtime.paths.pidPath, { force: true }), + ]); +} + +function jsonResponse(res, statusCode, payload, headers = {}) { + res.writeHead(statusCode, { + "content-type": "application/json; charset=utf-8", + ...headers, + }); + res.end(JSON.stringify(payload)); +} + +function htmlResponse(res, html) { + res.writeHead(200, { "content-type": "text/html; charset=utf-8" }); + res.end(html); +} + +function buildEditableConfig(currentConfig, payload) { + const nextReasoning = normalizeIntegerList(payload.reasoning_equals, currentConfig.reasoning_equals); + const nextEndpoints = normalizeStringList(payload.endpoints, currentConfig.endpoints).map(normalizePath); + const nextStatusCode = + payload.non_stream_status_code === undefined + ? currentConfig.non_stream_status_code + : Number.parseInt(`${payload.non_stream_status_code}`, 10); + + if (nextReasoning.length === 0) { + throw new Error("reasoning_equals 不能为空"); + } + if (nextEndpoints.length === 0) { + throw new Error("endpoints 不能为空"); + } + if (!Number.isInteger(nextStatusCode) || nextStatusCode < 100 || nextStatusCode > 599) { + throw new Error("non_stream_status_code 必须是 100-599 的整数"); + } + + return { + ...currentConfig, + reasoning_equals: nextReasoning, + endpoints: nextEndpoints, + non_stream_status_code: nextStatusCode, + log_match: payload.log_match === undefined ? currentConfig.log_match : Boolean(payload.log_match), + }; +} + +function buildManagementHtml() { + const uiConfig = { + statusPath: STATUS_API_PATH, + configPath: CONFIG_API_PATH, + logsPath: LOGS_API_PATH, + restorePath: RESTORE_API_PATH, + }; + + return ` + + + + + Codex Retry Gateway + + + +
+
+
本地管理页
+

Codex Retry Gateway

+

+ 这个页面直接挂在正在运行的 gateway 上。你可以在这里查看当前接管状态、修改 516 拦截条件，并一键恢复 Codex 原设置。 +

+
+ +
+
+
+

运行状态

+
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
0
+
0
+
0
+
0.00%
+
0
+
+

+ 如果“当前 Codex Base URL”已经是本机监听地址，就说明当前 Codex 已经被这个 gateway 接管。统计口径按本次 gateway 启动以来累计。 +

+
+
+ +
+
+

拦截规则

+
+
+ + +
多个值用英文逗号或空格分隔。
+
+ +
+ + +
每行一个路径。默认建议同时保留 root 与 /v1 两套路径。
+
+ +
+ + +
+ +
+ + +
+ +
+ + +
+
+
+

+ 点击“恢复”后，gateway 会停掉，所以这个页面会失联。这是预期行为，不是报错。 +

+
+
+ +
+
+

实时日志

+

正在读取日志...

+ 
正在读取日志...
+
+
+
+
+ + + +`; +} + +async function handleManagementRequest(runtime, req, res, requestUrl) { + const pathname = normalizePath(requestUrl.pathname); + + if (pathname === UI_PATH) { + htmlResponse(res, buildManagementHtml()); + return true; + } + + if (pathname === STATUS_API_PATH && req.method === "GET") { + const state = await readRuntimeState(runtime); + jsonResponse(res, 200, { + ok: true, + listen: `${runtime.config.listen_host}:${runtime.config.listen_port}`, + config: runtime.config, + state, + paths: { + config_path: runtime.configPath, + state_path: runtime.paths.statePath, + state_root: runtime.paths.stateRoot, + log_path: runtime.logPath, + }, + metrics: buildMetricsSnapshot(runtime.monitor), + }); + return true; + } + + if (pathname === LOGS_API_PATH && req.method === "GET") { + const sinceSeqRaw = requestUrl.searchParams.get("since_seq"); + const sinceSeq = sinceSeqRaw === null ? null : Number.parseInt(sinceSeqRaw, 10); + jsonResponse(res, 200, { + ok: true, + ...buildLogsSnapshot(runtime.monitor, Number.isInteger(sinceSeq) ? sinceSeq : null), + }); + return true; + } + + if (pathname === CONFIG_API_PATH && req.method === "POST") { + const body = await readRequestBody(req, runtime.config.request_body_limit_bytes); + const payload = parseJsonSafely(body); + if (!payload) { + jsonResponse(res, 400, { + error: { + message: "配置保存请求必须是有效 JSON", + code: "invalid_json", + }, + }); + return true; + } + + const nextConfig = buildEditableConfig(runtime.config, payload); + await writeConfig(runtime.configPath, nextConfig); + runtime.config = nextConfig; + runtime.logger( + `[config] updated reasoning_equals=${nextConfig.reasoning_equals.join(",")} endpoints=${nextConfig.endpoints.join(",")}`, + ); + const state = await readRuntimeState(runtime); + jsonResponse(res, 200, { + ok: true, + message: "配置已保存并立即生效", + config: runtime.config, + state, + paths: { + config_path: runtime.configPath, + state_path: runtime.paths.statePath, + state_root: runtime.paths.stateRoot, + log_path: runtime.logPath, + }, + metrics: buildMetricsSnapshot(runtime.monitor), + }); + return true; + } + + if (pathname === RESTORE_API_PATH && req.method === "POST") { + const state = await readRuntimeState(runtime); + if (!state) { + jsonResponse(res, 409, { + error: { + message: "当前未检测到安装状态，无法恢复 Codex 原设置", + code: "state_not_found", + }, + }); + return true; + } + + await restoreRuntimeState(runtime, state); + runtime.logger(`[restore] restored via UI state_root=${runtime.paths.stateRoot}`); + jsonResponse(res, 202, { + ok: true, + message: "原设置已恢复，gateway 即将关闭", + }); + res.on("finish", () => { + const exitTimer = setTimeout(() => { + if (runtime.server) { + runtime.server.close(() => { + process.exit(0); + }); + } else { + process.exit(0); + } + + const hardExitTimer = setTimeout(() => { + process.exit(0); + }, 600); + hardExitTimer.unref(); + }, 120); + exitTimer.unref(); + }); + return true; + } + + return false; +} + +function buildUpstreamUrl(baseUrl, requestUrl) { + const upstream = new URL(baseUrl); + const normalizedBasePath = upstream.pathname.endsWith("/") + ? upstream.pathname.slice(0, -1) + : upstream.pathname; + const incomingPath = requestUrl.pathname; + + let finalPath = incomingPath; + if (normalizedBasePath && normalizedBasePath !== "/") { + if (incomingPath.startsWith(`${normalizedBasePath}/`) || incomingPath === normalizedBasePath) { + finalPath = incomingPath; + } else if (normalizedBasePath.endsWith("/v1") && incomingPath.startsWith("/v1/")) { + finalPath = `${normalizedBasePath}${incomingPath.slice(3)}`; + } else { + finalPath = `${normalizedBasePath}${incomingPath}`; + } + } + + upstream.pathname = finalPath; + upstream.search = requestUrl.search; + return upstream.toString(); +} + +function cloneHeadersForUpstream(headers) { + const outgoing = new Headers(); + for (const [key, value] of Object.entries(headers)) { + if (value === undefined) { + continue; + } + const lowerKey = key.toLowerCase(); + if ( + lowerKey === "host" || + lowerKey === "content-length" || + lowerKey === "connection" || + lowerKey === "transfer-encoding" + ) { + continue; + } + if (Array.isArray(value)) { + for (const item of value) { + outgoing.append(key, item); + } + } else { + outgoing.set(key, value); + } + } + return outgoing; +} + +function copyHeadersToClient(sourceHeaders, target) { + for (const [key, value] of sourceHeaders.entries()) { + const lowerKey = key.toLowerCase(); + if ( + lowerKey === "content-length" || + lowerKey === "transfer-encoding" || + lowerKey === "content-encoding" || + lowerKey === "connection" + ) { + continue; + } + target.setHeader(key, value); + } +} + +async function readRequestBody(req, limitBytes) { + const chunks = []; + let total = 0; + for await (const chunk of req) { + total += chunk.length; + if (total > limitBytes) { + throw new Error(`请求体超过限制: ${limitBytes} bytes`); + } + chunks.push(chunk); + } + return Buffer.concat(chunks); +} + +function parseJsonSafely(buffer) { + try { + return JSON.parse(buffer.toString("utf8")); + } catch { + return null; + } +} + +function matchPath(config, pathname) { + return config.endpoints.includes(normalizePath(pathname)); +} + function reasoningMatched(config, reasoning) { return reasoning !== null && config.reasoning_equals.includes(reasoning); } -function inspectSseChunk(state, chunk) { - const decoded = state.decoder.decode(chunk, { stream: true }); - state.buffer += decoded; +function isExpectedStreamTermination(error) { + if (!error) { + return false; + } + if (error.name === "AbortError") { + return true; + } + return error instanceof TypeError && error.message === "terminated"; +} - const blocks = state.buffer.split(/\r?\n\r?\n/); - state.buffer = blocks.pop() ?? ""; +function isRetryableUpstreamFetchError(error) { + if (!error) { + return false; + } + return error instanceof TypeError && error.message === "fetch failed"; +} - for (const block of blocks) { - const lines = block - .split(/\r?\n/) - .map((line) => line.trimEnd()) - .filter(Boolean); - const dataLines = lines - .filter((line) => line.startsWith("data:")) - .map((line) => line.replace(/^data:\s?/, "")); +async function fetchUpstreamWithRetry(upstreamUrl, init, logger) { + const maxAttempts = 2; + let lastError = null; - if (dataLines.length === 0) { - continue; - } - const payloadText = dataLines.join("\n"); - if (payloadText === "[DONE]") { - continue; - } + for (let attempt = 1; attempt <= maxAttempts; attempt += 1) { try { - const parsed = JSON.parse(payloadText); - const reasoning = extractReasoningTokens(parsed); - if (reasoning !== null) { - return reasoning; + return await fetch(upstreamUrl, init); + } catch (error) { + lastError = error; + if (!isRetryableUpstreamFetchError(error) || attempt === maxAttempts) { + break; } - } catch { - // ignore malformed SSE payloads + logger?.(`[retry] upstream fetch failed attempt=${attempt} url=${upstreamUrl}`); } } - return null; + + throw lastError; } - -async function handleNonStreaming({ - config, - logger, - monitor, - pathname, - upstreamResponse, - res, -}) { - const bodyBuffer = Buffer.from(await upstreamResponse.arrayBuffer()); - const parsed = isJsonContentType(upstreamResponse.headers.get("content-type")) - ? parseJsonSafely(bodyBuffer) - : null; - const reasoning = parsed ? extractReasoningTokens(parsed) : null; - const matched = reasoningMatched(config, reasoning); - - recordInspectedResponse(monitor, reasoning, matched); - - if (matched) { - if (config.log_match) { - logger( - `[match] non-stream path=${pathname} reasoning_tokens=${reasoning} action=status_${config.non_stream_status_code}`, - ); - } - const blockedBody = buildBlockedBody(pathname, reasoning, config.non_stream_status_code); - res.writeHead(config.non_stream_status_code, { - "content-type": "application/json; charset=utf-8", - "x-codex-retry-gateway-reason": "reasoning-guard-triggered", - }); - res.end(blockedBody); - return; - } - - copyHeadersToClient(upstreamResponse.headers, res); - res.writeHead(upstreamResponse.status); - res.end(bodyBuffer); -} - + +function inspectSseChunk(state, chunk) { + const decoded = state.decoder.decode(chunk, { stream: true }); + state.buffer += decoded; + + const blocks = state.buffer.split(/\r?\n\r?\n/); + state.buffer = blocks.pop() ?? ""; + + for (const block of blocks) { + const lines = block + .split(/\r?\n/) + .map((line) => line.trimEnd()) + .filter(Boolean); + const dataLines = lines + .filter((line) => line.startsWith("data:")) + .map((line) => line.replace(/^data:\s?/, "")); + + if (dataLines.length === 0) { + continue; + } + const payloadText = dataLines.join("\n"); + if (payloadText === "[DONE]") { + continue; + } + try { + const parsed = JSON.parse(payloadText); + const reasoning = extractReasoningTokens(parsed); + if (reasoning !== null) { + return reasoning; + } + } catch { + // ignore malformed SSE payloads + } + } + return null; +} + +async function handleNonStreaming({ + config, + logger, + monitor, + pathname, + upstreamResponse, + res, +}) { + const bodyBuffer = Buffer.from(await upstreamResponse.arrayBuffer()); + const parsed = isJsonContentType(upstreamResponse.headers.get("content-type")) + ? parseJsonSafely(bodyBuffer) + : null; + const reasoning = parsed ? extractReasoningTokens(parsed) : null; + const matched = reasoningMatched(config, reasoning); + + recordInspectedResponse(monitor, reasoning, matched); + + if (matched) { + if (config.log_match) { + logger( + `[match] non-stream path=${pathname} reasoning_tokens=${reasoning} action=status_${config.non_stream_status_code}`, + ); + } + const blockedBody = buildBlockedBody(pathname, reasoning, config.non_stream_status_code); + res.writeHead(config.non_stream_status_code, { + "content-type": "application/json; charset=utf-8", + "x-codex-retry-gateway-reason": "reasoning-guard-triggered", + }); + res.end(blockedBody); + return; + } + + copyHeadersToClient(upstreamResponse.headers, res); + res.writeHead(upstreamResponse.status); + res.end(bodyBuffer); +} + async function handleStreaming({ config, logger, monitor, - pathname, + pathname, upstreamResponse, res, abortController, }) { - copyHeadersToClient(upstreamResponse.headers, res); - res.writeHead(upstreamResponse.status); - + const strict502Mode = config.stream_action !== "disconnect"; const reader = upstreamResponse.body.getReader(); const sseState = { decoder: new TextDecoder("utf8"), @@ -1291,175 +1335,211 @@ async function handleStreaming({ let wroteAnyChunk = false; let observedReasoning = null; + const bufferedChunks = []; + + if (!strict502Mode) { + copyHeadersToClient(upstreamResponse.headers, res); + res.writeHead(upstreamResponse.status); + } + while (true) { - const { done, value } = await reader.read(); + let readResult; + try { + readResult = await reader.read(); + } catch (error) { + if (isExpectedStreamTermination(error)) { + recordInspectedResponse(monitor, observedReasoning, false); + if (strict502Mode) { + logger?.(`[stream] upstream terminated before completion path=${pathname} action=status_502`); + res.writeHead(502, { "content-type": "application/json; charset=utf-8" }); + res.end(buildGatewayErrorBody("upstream stream terminated before completion")); + } else { + res.end(); + } + return; + } + throw error; + } + + const { done, value } = readResult; if (done) { recordInspectedResponse(monitor, observedReasoning, false); - res.end(); + if (strict502Mode) { + copyHeadersToClient(upstreamResponse.headers, res); + res.writeHead(upstreamResponse.status); + res.end(Buffer.concat(bufferedChunks)); + } else { + res.end(); + } return; } + const chunkBuffer = Buffer.from(value); const reasoning = inspectSseChunk(sseState, value); if (Number.isInteger(reasoning)) { observedReasoning = reasoning; } if (reasoningMatched(config, reasoning)) { - recordInspectedResponse(monitor, reasoning, true); - if (config.log_match) { - logger( - `[match] stream path=${pathname} reasoning_tokens=${reasoning} action=${config.stream_action}`, - ); + recordInspectedResponse(monitor, reasoning, true); + if (config.log_match) { + logger( + `[match] stream path=${pathname} reasoning_tokens=${reasoning} action=${config.stream_action}`, + ); } - if (!wroteAnyChunk) { + if (strict502Mode || !wroteAnyChunk) { + abortController.abort(); + reader.cancel().catch(() => {}); const blockedBody = buildBlockedBody(pathname, reasoning, config.non_stream_status_code); - if (!res.headersSent) { - res.writeHead(config.non_stream_status_code, { - "content-type": "application/json; charset=utf-8", - "x-codex-retry-gateway-reason": "reasoning-guard-triggered", - }); - } + res.writeHead(config.non_stream_status_code, { + "content-type": "application/json; charset=utf-8", + "x-codex-retry-gateway-reason": "reasoning-guard-triggered", + }); res.end(blockedBody); } else { abortController.abort(); reader.cancel().catch(() => {}); res.socket?.destroy(); - } - return; + } + return; } - wroteAnyChunk = true; - res.write(Buffer.from(value)); + if (strict502Mode) { + bufferedChunks.push(chunkBuffer); + } else { + wroteAnyChunk = true; + res.write(chunkBuffer); + } } } - -async function proxyRequest(runtime, req, res) { - const { logger } = runtime; - const config = runtime.config; - const incomingUrl = new URL(req.url, `http://${req.headers.host || "127.0.0.1"}`); - const pathname = normalizePath(incomingUrl.pathname); - - if (pathname === config.health_path) { - res.writeHead(200, { "content-type": "application/json; charset=utf-8" }); - res.end( - JSON.stringify({ - ok: true, - listen: `${config.listen_host}:${config.listen_port}`, - upstream_base_url: config.upstream_base_url, - ui_path: UI_PATH, - }), - ); - return; - } - - if (await handleManagementRequest(runtime, req, res, incomingUrl)) { - return; - } - - runtime.monitor.total_proxy_request_count += 1; - - const requestBody = await readRequestBody(req, config.request_body_limit_bytes); - const requestJson = isJsonContentType(req.headers["content-type"]) - ? parseJsonSafely(requestBody) - : null; - const requestIsStream = Boolean(requestJson?.stream); - - const upstreamUrl = buildUpstreamUrl(config.upstream_base_url, incomingUrl); - const abortController = new AbortController(); - - const upstreamResponse = await fetch(upstreamUrl, { + +async function proxyRequest(runtime, req, res) { + const { logger } = runtime; + const config = runtime.config; + const incomingUrl = new URL(req.url, `http://${req.headers.host || "127.0.0.1"}`); + const pathname = normalizePath(incomingUrl.pathname); + + if (pathname === config.health_path) { + res.writeHead(200, { "content-type": "application/json; charset=utf-8" }); + res.end( + JSON.stringify({ + ok: true, + listen: `${config.listen_host}:${config.listen_port}`, + upstream_base_url: config.upstream_base_url, + ui_path: UI_PATH, + }), + ); + return; + } + + if (await handleManagementRequest(runtime, req, res, incomingUrl)) { + return; + } + + runtime.monitor.total_proxy_request_count += 1; + + const requestBody = await readRequestBody(req, config.request_body_limit_bytes); + const requestJson = isJsonContentType(req.headers["content-type"]) + ? parseJsonSafely(requestBody) + : null; + const requestIsStream = Boolean(requestJson?.stream); + + const upstreamUrl = buildUpstreamUrl(config.upstream_base_url, incomingUrl); + const abortController = new AbortController(); + + const upstreamResponse = await fetchUpstreamWithRetry(upstreamUrl, { method: req.method, headers: cloneHeadersForUpstream(req.headers), body: requestBody.length > 0 ? requestBody : undefined, signal: abortController.signal, - }); - - const shouldInspect = matchPath(config, pathname); - const responseIsStream = - requestIsStream || isSseContentType(upstreamResponse.headers.get("content-type")); - - if (!shouldInspect) { - copyHeadersToClient(upstreamResponse.headers, res); - res.writeHead(upstreamResponse.status); - const body = Buffer.from(await upstreamResponse.arrayBuffer()); - res.end(body); - return; - } - - if (responseIsStream) { - await handleStreaming({ - config, - logger, - monitor: runtime.monitor, - pathname, - upstreamResponse, - res, - abortController, - }); - return; - } - - await handleNonStreaming({ - config, - logger, - monitor: runtime.monitor, - pathname, - upstreamResponse, - res, - }); -} - -async function main() { - const args = parseArgs(process.argv); - const configPath = args.config || path.join(__dirname, "config.json"); - const config = await loadConfig(configPath); - const monitor = createMonitor(); - - if (args.log) { - await mkdir(path.dirname(args.log), { recursive: true }); - } - const logger = createLogger(args.log, createMonitorRecorder(monitor)); - const runtime = { - config, - configPath, - logPath: args.log || null, - logger, - monitor, - paths: buildRuntimePaths(configPath, args.log || null), - server: null, - }; - - const server = http.createServer(async (req, res) => { - try { - await proxyRequest(runtime, req, res); - } catch (error) { - logger(`[error] ${error?.stack || error}`); - if (!res.headersSent) { - res.writeHead(502, { "content-type": "application/json; charset=utf-8" }); - res.end( - JSON.stringify({ - error: { - message: `${error?.message || error}`, - type: "codex_retry_gateway_error", - code: "gateway_error", - }, - }), - ); - } else { - res.socket?.destroy(); - } - } - }); - runtime.server = server; - - server.listen(config.listen_port, config.listen_host, () => { - logger( - `[start] codex retry gateway listening on http://${config.listen_host}:${config.listen_port} -> ${config.upstream_base_url}`, - ); - }); -} - -main().catch((error) => { - process.stderr.write(`${error?.stack || error}\n`); - process.exit(1); -}); + }, logger); + + const shouldInspect = matchPath(config, pathname); + const responseIsStream = + requestIsStream || isSseContentType(upstreamResponse.headers.get("content-type")); + + if (!shouldInspect) { + copyHeadersToClient(upstreamResponse.headers, res); + res.writeHead(upstreamResponse.status); + const body = Buffer.from(await upstreamResponse.arrayBuffer()); + res.end(body); + return; + } + + if (responseIsStream) { + await handleStreaming({ + config, + logger, + monitor: runtime.monitor, + pathname, + upstreamResponse, + res, + abortController, + }); + return; + } + + await handleNonStreaming({ + config, + logger, + monitor: runtime.monitor, + pathname, + upstreamResponse, + res, + }); +} + +async function main() { + const args = parseArgs(process.argv); + const configPath = args.config || path.join(__dirname, "config.json"); + const config = await loadConfig(configPath); + const monitor = createMonitor(); + + if (args.log) { + await mkdir(path.dirname(args.log), { recursive: true }); + } + const logger = createLogger(args.log, createMonitorRecorder(monitor)); + const runtime = { + config, + configPath, + logPath: args.log || null, + logger, + monitor, + paths: buildRuntimePaths(configPath, args.log || null), + server: null, + }; + + const server = http.createServer(async (req, res) => { + try { + await proxyRequest(runtime, req, res); + } catch (error) { + logger(`[error] ${error?.stack || error}`); + if (!res.headersSent) { + res.writeHead(502, { "content-type": "application/json; charset=utf-8" }); + res.end( + JSON.stringify({ + error: { + message: `${error?.message || error}`, + type: "codex_retry_gateway_error", + code: "gateway_error", + }, + }), + ); + } else { + res.socket?.destroy(); + } + } + }); + runtime.server = server; + + server.listen(config.listen_port, config.listen_host, () => { + logger( + `[start] codex retry gateway listening on http://${config.listen_host}:${config.listen_port} -> ${config.upstream_base_url}`, + ); + }); +} + +main().catch((error) => { + process.stderr.write(`${error?.stack || error}\n`); + process.exit(1); +}); diff --git a/scripts/admin-lib.mjs b/scripts/admin-lib.mjs index b7cb502..30ae29b 100644 --- a/scripts/admin-lib.mjs +++ b/scripts/admin-lib.mjs @@ -1,620 +1,620 @@ -#!/usr/bin/env node - -import { spawn } from "node:child_process"; -import fs from "node:fs"; -import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises"; -import os from "node:os"; -import path from "node:path"; - -export const DEFAULT_STATE_ROOT = path.join(os.homedir(), ".codex-retry-gateway"); -export const DEFAULT_CODEX_CONFIG_PATH = path.join(os.homedir(), ".codex", "config.toml"); -export const DEFAULT_LISTEN_HOST = "127.0.0.1"; -export const DEFAULT_LISTEN_PORT = 4610; -export const DEFAULT_HEALTH_PATH = "/__codex_retry_gateway/health"; - -function escapeRegExp(value) { - return `${value}`.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); -} - -export function parseOptions(argv, { booleanFlags = [] } = {}) { - const options = { _: [] }; - const booleanSet = new Set(booleanFlags); - - for (let index = 2; index < argv.length; index += 1) { - const current = argv[index]; - if (!current.startsWith("--")) { - options._.push(current); - continue; - } - - const flagName = current.slice(2); - const optionKey = flagName.replace(/-([a-z])/g, (_, char) => char.toUpperCase()); - if (booleanSet.has(flagName)) { - options[optionKey] = true; - continue; - } - - const nextValue = argv[index + 1]; - if (nextValue === undefined) { - throw new Error(`Missing value for --${flagName}`); - } - options[optionKey] = nextValue; - index += 1; - } - - return options; -} - -export function getGatewayRoot() { - return path.resolve(import.meta.dirname, ".."); -} - -export function getGatewayStatePaths(stateRoot = DEFAULT_STATE_ROOT) { - return { - stateRoot, - configDir: path.join(stateRoot, "config"), - logDir: path.join(stateRoot, "logs"), - backupDir: path.join(stateRoot, "backups"), - configPath: path.join(stateRoot, "config", "config.json"), - logPath: path.join(stateRoot, "logs", "gateway.log"), - statePath: path.join(stateRoot, "state.json"), - pidPath: path.join(stateRoot, "gateway.pid"), - }; -} - -export function getGatewayBaseUrl(listenHost, listenPort) { - return `http://${listenHost}:${listenPort}`; -} - -export function getGatewayBaseUrlFromConfig(gatewayConfig) { - if (!gatewayConfig) { - return null; - } - if (!gatewayConfig.listen_host || gatewayConfig.listen_port === undefined || gatewayConfig.listen_port === null) { - return null; - } - return getGatewayBaseUrl(`${gatewayConfig.listen_host}`, Number.parseInt(`${gatewayConfig.listen_port}`, 10)); -} - -export async function ensureDirectory(targetPath) { - await mkdir(targetPath, { recursive: true }); -} - -export async function writeUtf8File(targetPath, content) { - const parent = path.dirname(targetPath); - if (parent && parent !== ".") { - await ensureDirectory(parent); - } - await writeFile(targetPath, content, "utf8"); -} - -export async function readJsonFile(filePath) { - if (!fs.existsSync(filePath)) { - return null; - } - const raw = await readFile(filePath, "utf8"); - if (!raw.trim()) { - return null; - } - return JSON.parse(raw); -} - -export async function writeJsonFile(filePath, value) { - await writeUtf8File(filePath, `${JSON.stringify(value, null, 2)}\n`); -} - -export async function getCodexProviderContext(codexConfigPath) { - const content = await readFile(codexConfigPath, "utf8"); - const providerMatch = content.match(/^\s*model_provider\s*=\s*"([^"]+)"\s*$/m); - if (!providerMatch) { - throw new Error(`model_provider was not found in ${codexConfigPath}`); - } - - const providerName = providerMatch[1]; - const sectionHeaderRegex = new RegExp(`^\\[model_providers\\.${escapeRegExp(providerName)}\\]\\s*$`, "m"); - const sectionHeaderMatch = sectionHeaderRegex.exec(content); - if (!sectionHeaderMatch) { - throw new Error(`[model_providers.${providerName}] was not found in ${codexConfigPath}`); - } - - const sectionIndex = sectionHeaderMatch.index; - const headerEndIndex = sectionIndex + sectionHeaderMatch[0].length; - const remainder = content.slice(headerEndIndex); - const nextSectionMatch = /^\[.*$/m.exec(remainder); - const sectionEndIndex = nextSectionMatch ? headerEndIndex + nextSectionMatch.index : content.length; - const sectionText = content.slice(sectionIndex, sectionEndIndex); - const baseUrlMatch = sectionText.match(/^\s*base_url\s*=\s*"([^"]+)"\s*$/m); - if (!baseUrlMatch) { - throw new Error(`base_url was not found in [model_providers.${providerName}]`); - } - - return { - content, - providerName, - sectionText, - sectionIndex, - sectionLength: sectionText.length, - currentBaseUrl: baseUrlMatch[1], - baseUrlLineText: baseUrlMatch[0], - }; -} - -export async function setCodexProviderBaseUrl({ codexConfigPath, providerName, newBaseUrl }) { - const context = await getCodexProviderContext(codexConfigPath); - if (context.providerName !== providerName) { - throw new Error(`model_provider changed unexpectedly: expected ${providerName}, actual ${context.providerName}`); - } - - let replaced = false; - const updatedSection = context.sectionText.replace( - /^(\s*base_url\s*=\s*")([^"]*)("\s*)$/m, - (_, prefix, __existing, suffix) => { - replaced = true; - return `${prefix}${newBaseUrl}${suffix}`; - }, - ); - if (!replaced) { - throw new Error(`base_url was not found in [model_providers.${providerName}]`); - } - - const updatedContent = - context.content.slice(0, context.sectionIndex) + - updatedSection + - context.content.slice(context.sectionIndex + context.sectionLength); - - await writeUtf8File(codexConfigPath, updatedContent); -} - -export function normalizeIntArray(values, fallback = [516]) { - const source = values === undefined || values === null ? fallback : values; - const queue = Array.isArray(source) ? source.flat(Infinity) : [source]; - const normalized = queue - .map((value) => (typeof value === "string" ? value.split(/[\s,]+/).filter(Boolean) : [value])) - .flat() - .map((value) => Number.parseInt(`${value}`, 10)) - .filter((value) => Number.isInteger(value)); - - return normalized.length > 0 ? [...new Set(normalized)] : [...fallback]; -} - -export function normalizeStringArray(values, fallback = []) { - const source = values === undefined || values === null ? fallback : values; - const queue = Array.isArray(source) ? source.flat(Infinity) : [source]; - const normalized = queue - .flatMap((value) => `${value ?? ""}`.split(/[\s,]+/)) - .map((value) => value.trim()) - .filter(Boolean); - - return normalized.length > 0 ? [...new Set(normalized)] : [...fallback]; -} - -export function isProcessAlive(processId) { - try { - process.kill(processId, 0); - return true; - } catch { - return false; - } -} - -export async function waitGatewayHealth({ - listenHost, - listenPort, - healthPath, - timeoutSeconds = 10, -}) { - const deadline = Date.now() + timeoutSeconds * 1000; - const healthUrl = `${getGatewayBaseUrl(listenHost, listenPort)}${healthPath}`; - - while (Date.now() < deadline) { - try { - const response = await fetch(healthUrl, { signal: AbortSignal.timeout(2000) }); - if (response.status === 200) { - return response; - } - } catch { - // ignore and retry - } - await new Promise((resolve) => setTimeout(resolve, 200)); - } - - throw new Error(`Gateway health check timed out: ${healthUrl}`); -} - -async function readTail(filePath, lineCount = 20) { - if (!fs.existsSync(filePath)) { - return ""; - } - const raw = await readFile(filePath, "utf8"); - return raw.split(/\r?\n/).slice(-lineCount).join("\n").trim(); -} - -function openUrl(url) { - let command; - let args; - if (process.platform === "win32") { - command = "cmd"; - args = ["/c", "start", "", url]; - } else if (process.platform === "darwin") { - command = "open"; - args = [url]; - } else { - command = "xdg-open"; - args = [url]; - } - - const child = spawn(command, args, { - detached: true, - stdio: "ignore", - windowsHide: true, - }); - child.unref(); -} - -export async function stopGateway({ stateRoot = DEFAULT_STATE_ROOT, quiet = false }) { - const paths = getGatewayStatePaths(stateRoot); - if (!fs.existsSync(paths.pidPath)) { - return quiet ? null : "No running gateway PID file was found."; - } - - const pidRaw = (await readFile(paths.pidPath, "utf8")).trim(); - if (!pidRaw) { - await rm(paths.pidPath, { force: true }); - return quiet ? null : "Gateway PID file was empty and has been removed."; - } - - const gatewayPid = Number.parseInt(pidRaw, 10); - if (Number.isInteger(gatewayPid) && isProcessAlive(gatewayPid)) { - try { - process.kill(gatewayPid); - } catch { - // ignore first failure - } - - const deadline = Date.now() + 3000; - while (Date.now() < deadline && isProcessAlive(gatewayPid)) { - await new Promise((resolve) => setTimeout(resolve, 100)); - } - - if (isProcessAlive(gatewayPid)) { - try { - process.kill(gatewayPid, "SIGKILL"); - } catch { - // ignore hard kill failure - } - } - } - - await rm(paths.pidPath, { force: true }); - return quiet ? null : `Gateway stopped. PID=${gatewayPid}`; -} - -export async function startGateway({ - stateRoot = DEFAULT_STATE_ROOT, - configPath, - logPath, - restartIfRunning = false, -}) { - const paths = getGatewayStatePaths(stateRoot); - const effectiveConfigPath = configPath || paths.configPath; - const effectiveLogPath = logPath || paths.logPath; - - if (!fs.existsSync(effectiveConfigPath)) { - throw new Error(`Gateway config file was not found: ${effectiveConfigPath}`); - } - - await ensureDirectory(path.dirname(effectiveLogPath)); - - if (fs.existsSync(paths.pidPath)) { - const existingPidRaw = (await readFile(paths.pidPath, "utf8")).trim(); - if (existingPidRaw) { - const existingPid = Number.parseInt(existingPidRaw, 10); - if (Number.isInteger(existingPid) && isProcessAlive(existingPid)) { - if (restartIfRunning) { - await stopGateway({ stateRoot, quiet: true }); - } else { - return `Gateway is already running. PID=${existingPid}`; - } - } else { - await rm(paths.pidPath, { force: true }); - } - } - } - - const gatewayConfig = await readJsonFile(effectiveConfigPath); - if (!gatewayConfig) { - throw new Error(`Gateway config file could not be read: ${effectiveConfigPath}`); - } - - const gatewayRoot = getGatewayRoot(); - const gatewayEntry = path.join(gatewayRoot, "gateway.mjs"); - if (!fs.existsSync(gatewayEntry)) { - throw new Error(`Gateway entry file was not found: ${gatewayEntry}`); - } - - const child = spawn(process.execPath, [gatewayEntry, "--config", effectiveConfigPath, "--log", effectiveLogPath], { - cwd: gatewayRoot, - detached: true, - stdio: "ignore", - windowsHide: true, - }); - child.unref(); - - await writeUtf8File(paths.pidPath, `${child.pid}`); - - await new Promise((resolve) => setTimeout(resolve, 300)); - if (!isProcessAlive(child.pid)) { - const logTail = await readTail(effectiveLogPath, 20); - throw new Error(`Gateway exited right after startup. PID=${child.pid}\n${logTail}`); - } - - await waitGatewayHealth({ - listenHost: `${gatewayConfig.listen_host}`, - listenPort: Number.parseInt(`${gatewayConfig.listen_port}`, 10), - healthPath: `${gatewayConfig.health_path || DEFAULT_HEALTH_PATH}`, - }); - - return `Gateway started. PID=${child.pid}. Listen=${getGatewayBaseUrl(gatewayConfig.listen_host, gatewayConfig.listen_port)}`; -} - -export async function installForCurrentProvider({ - codexConfigPath = DEFAULT_CODEX_CONFIG_PATH, - stateRoot = DEFAULT_STATE_ROOT, - listenHost = DEFAULT_LISTEN_HOST, - listenPort = DEFAULT_LISTEN_PORT, -}) { - const paths = getGatewayStatePaths(stateRoot); - await ensureDirectory(paths.stateRoot); - await ensureDirectory(paths.configDir); - await ensureDirectory(paths.logDir); - await ensureDirectory(paths.backupDir); - - if (!fs.existsSync(codexConfigPath)) { - throw new Error(`Codex config file was not found: ${codexConfigPath}`); - } - - const providerContext = await getCodexProviderContext(codexConfigPath); - const localGatewayBaseUrl = getGatewayBaseUrl(listenHost, listenPort); - const existingState = await readJsonFile(paths.statePath); - - let originalBaseUrl = providerContext.currentBaseUrl; - if (providerContext.currentBaseUrl === localGatewayBaseUrl) { - if (!existingState?.original_base_url) { - throw new Error("Provider already points to the local gateway, but original_base_url is missing from state."); - } - originalBaseUrl = `${existingState.original_base_url}`; - } - - if (originalBaseUrl === localGatewayBaseUrl) { - throw new Error("A real upstream_base_url could not be determined."); - } - - const backupPath = path.join(paths.backupDir, `config-${new Date().toISOString().replace(/[:.]/g, "").replace("T", "-").slice(0, 15)}.toml`); - await copyFile(codexConfigPath, backupPath); - - const existingGatewayConfig = await readJsonFile(paths.configPath); - const defaultEndpoints = ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"]; - const mergedEndpoints = []; - for (const endpoint of [ - ...normalizeStringArray(existingGatewayConfig?.endpoints, []), - ...defaultEndpoints, - ]) { - if (!mergedEndpoints.includes(endpoint)) { - mergedEndpoints.push(endpoint); - } - } - - const gatewayConfig = { - listen_host: listenHost, - listen_port: listenPort, - upstream_base_url: originalBaseUrl, - request_body_limit_bytes: - existingGatewayConfig?.request_body_limit_bytes === undefined || existingGatewayConfig?.request_body_limit_bytes === null - ? 10485760 - : Number.parseInt(`${existingGatewayConfig.request_body_limit_bytes}`, 10), - endpoints: mergedEndpoints, - reasoning_equals: normalizeIntArray(existingGatewayConfig?.reasoning_equals, [516]), - non_stream_status_code: - existingGatewayConfig?.non_stream_status_code === undefined || existingGatewayConfig?.non_stream_status_code === null - ? 502 - : Number.parseInt(`${existingGatewayConfig.non_stream_status_code}`, 10), - stream_action: existingGatewayConfig?.stream_action || "disconnect", - log_match: existingGatewayConfig?.log_match === undefined ? true : Boolean(existingGatewayConfig.log_match), - health_path: existingGatewayConfig?.health_path || DEFAULT_HEALTH_PATH, - }; - - const previousConfigContent = await readFile(codexConfigPath, "utf8"); - - try { - await writeJsonFile(paths.configPath, gatewayConfig); - await setCodexProviderBaseUrl({ - codexConfigPath, - providerName: providerContext.providerName, - newBaseUrl: localGatewayBaseUrl, - }); - - await startGateway({ - stateRoot, - configPath: paths.configPath, - logPath: paths.logPath, - restartIfRunning: true, - }); - - const state = { - installed_at: new Date().toISOString(), - codex_config_path: codexConfigPath, - provider_name: providerContext.providerName, - original_base_url: originalBaseUrl, - gateway_base_url: localGatewayBaseUrl, - gateway_config_path: paths.configPath, - gateway_log_path: paths.logPath, - gateway_pid_path: paths.pidPath, - latest_backup_path: backupPath, - state_root: paths.stateRoot, - }; - await writeJsonFile(paths.statePath, state); - - return { - provider: providerContext.providerName, - upstream: originalBaseUrl, - gateway: localGatewayBaseUrl, - configPath: paths.configPath, - backupPath, - }; - } catch (error) { - await writeUtf8File(codexConfigPath, previousConfigContent); - await stopGateway({ stateRoot, quiet: true }); - throw error; - } -} - -export async function restoreCodexConfig({ - stateRoot = DEFAULT_STATE_ROOT, - codexConfigPath = DEFAULT_CODEX_CONFIG_PATH, -}) { - const paths = getGatewayStatePaths(stateRoot); - const state = await readJsonFile(paths.statePath); - if (!state) { - throw new Error(`Install state file was not found: ${paths.statePath}`); - } - - const backupPath = `${state.latest_backup_path || ""}`; - if (!backupPath || !fs.existsSync(backupPath)) { - throw new Error(`A restorable backup file was not found: ${backupPath}`); - } - - await stopGateway({ stateRoot, quiet: true }); - await copyFile(backupPath, codexConfigPath); - await rm(paths.statePath, { force: true }); - - return { - configPath: codexConfigPath, - restoredFrom: backupPath, - }; -} - -export async function launchUi({ - codexConfigPath = DEFAULT_CODEX_CONFIG_PATH, - stateRoot = DEFAULT_STATE_ROOT, - listenHost = DEFAULT_LISTEN_HOST, - listenPort = DEFAULT_LISTEN_PORT, - noOpen = false, -}) { - const paths = getGatewayStatePaths(stateRoot); - await ensureDirectory(paths.stateRoot); - await ensureDirectory(paths.configDir); - await ensureDirectory(paths.logDir); - await ensureDirectory(paths.backupDir); - - if (!fs.existsSync(codexConfigPath)) { - throw new Error(`Codex config file was not found: ${codexConfigPath}`); - } - - const providerContext = await getCodexProviderContext(codexConfigPath); - const currentBaseUrl = `${providerContext.currentBaseUrl}`; - const requestedGatewayBaseUrl = getGatewayBaseUrl(listenHost, listenPort); - const existingState = await readJsonFile(paths.statePath); - const existingGatewayConfig = await readJsonFile(paths.configPath); - const stateGatewayBaseUrl = existingState?.gateway_base_url ? `${existingState.gateway_base_url}` : null; - const configGatewayBaseUrl = getGatewayBaseUrlFromConfig(existingGatewayConfig); - const managedGatewayBaseUrls = [requestedGatewayBaseUrl]; - for (const candidate of [stateGatewayBaseUrl, configGatewayBaseUrl]) { - if (candidate && !managedGatewayBaseUrls.includes(candidate)) { - managedGatewayBaseUrls.push(candidate); - } - } - - const originalBaseUrl = - existingState?.original_base_url - ? `${existingState.original_base_url}` - : existingGatewayConfig?.upstream_base_url - ? `${existingGatewayConfig.upstream_base_url}` - : null; - - const canReuseExistingInstall = - existingGatewayConfig && - originalBaseUrl && - managedGatewayBaseUrls.includes(currentBaseUrl); - - let mode = "install"; - if (!canReuseExistingInstall) { - await installForCurrentProvider({ - codexConfigPath, - stateRoot, - listenHost, - listenPort, - }); - } else { - mode = "reuse"; - const previousCodexConfigContent = await readFile(codexConfigPath, "utf8"); - const previousGatewayConfigContent = fs.existsSync(paths.configPath) - ? await readFile(paths.configPath, "utf8") - : null; - const previousStateContent = fs.existsSync(paths.statePath) - ? await readFile(paths.statePath, "utf8") - : null; - - try { - existingGatewayConfig.listen_host = listenHost; - existingGatewayConfig.listen_port = listenPort; - if (!existingGatewayConfig.health_path) { - existingGatewayConfig.health_path = DEFAULT_HEALTH_PATH; - } - await writeJsonFile(paths.configPath, existingGatewayConfig); - - if (currentBaseUrl !== requestedGatewayBaseUrl) { - await setCodexProviderBaseUrl({ - codexConfigPath, - providerName: providerContext.providerName, - newBaseUrl: requestedGatewayBaseUrl, - }); - } - - await startGateway({ - stateRoot, - configPath: paths.configPath, - logPath: paths.logPath, - restartIfRunning: true, - }); - - const statePayload = { - installed_at: existingState?.installed_at ? `${existingState.installed_at}` : new Date().toISOString(), - last_started_at: new Date().toISOString(), - codex_config_path: codexConfigPath, - provider_name: providerContext.providerName, - original_base_url: originalBaseUrl, - gateway_base_url: requestedGatewayBaseUrl, - gateway_config_path: paths.configPath, - gateway_log_path: paths.logPath, - gateway_pid_path: paths.pidPath, - latest_backup_path: existingState?.latest_backup_path ? `${existingState.latest_backup_path}` : "", - state_root: paths.stateRoot, - }; - await writeJsonFile(paths.statePath, statePayload); - } catch (error) { - await writeUtf8File(codexConfigPath, previousCodexConfigContent); - if (previousGatewayConfigContent !== null) { - await writeUtf8File(paths.configPath, previousGatewayConfigContent); - } - if (previousStateContent !== null) { - await writeUtf8File(paths.statePath, previousStateContent); - } - await stopGateway({ stateRoot, quiet: true }); - throw error; - } - } - - const effectiveGatewayConfig = await readJsonFile(paths.configPath); - const effectiveGatewayBaseUrl = getGatewayBaseUrlFromConfig(effectiveGatewayConfig) || requestedGatewayBaseUrl; - const uiUrl = `${effectiveGatewayBaseUrl}/__codex_retry_gateway/ui`; - - if (!noOpen) { - openUrl(uiUrl); - } - - return { - mode, - uiUrl, - gatewayBaseUrl: effectiveGatewayBaseUrl, - }; -} +#!/usr/bin/env node + +import { spawn } from "node:child_process"; +import fs from "node:fs"; +import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; + +export const DEFAULT_STATE_ROOT = path.join(os.homedir(), ".codex-retry-gateway"); +export const DEFAULT_CODEX_CONFIG_PATH = path.join(os.homedir(), ".codex", "config.toml"); +export const DEFAULT_LISTEN_HOST = "127.0.0.1"; +export const DEFAULT_LISTEN_PORT = 4610; +export const DEFAULT_HEALTH_PATH = "/__codex_retry_gateway/health"; + +function escapeRegExp(value) { + return `${value}`.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); +} + +export function parseOptions(argv, { booleanFlags = [] } = {}) { + const options = { _: [] }; + const booleanSet = new Set(booleanFlags); + + for (let index = 2; index < argv.length; index += 1) { + const current = argv[index]; + if (!current.startsWith("--")) { + options._.push(current); + continue; + } + + const flagName = current.slice(2); + const optionKey = flagName.replace(/-([a-z])/g, (_, char) => char.toUpperCase()); + if (booleanSet.has(flagName)) { + options[optionKey] = true; + continue; + } + + const nextValue = argv[index + 1]; + if (nextValue === undefined) { + throw new Error(`Missing value for --${flagName}`); + } + options[optionKey] = nextValue; + index += 1; + } + + return options; +} + +export function getGatewayRoot() { + return path.resolve(import.meta.dirname, ".."); +} + +export function getGatewayStatePaths(stateRoot = DEFAULT_STATE_ROOT) { + return { + stateRoot, + configDir: path.join(stateRoot, "config"), + logDir: path.join(stateRoot, "logs"), + backupDir: path.join(stateRoot, "backups"), + configPath: path.join(stateRoot, "config", "config.json"), + logPath: path.join(stateRoot, "logs", "gateway.log"), + statePath: path.join(stateRoot, "state.json"), + pidPath: path.join(stateRoot, "gateway.pid"), + }; +} + +export function getGatewayBaseUrl(listenHost, listenPort) { + return `http://${listenHost}:${listenPort}`; +} + +export function getGatewayBaseUrlFromConfig(gatewayConfig) { + if (!gatewayConfig) { + return null; + } + if (!gatewayConfig.listen_host || gatewayConfig.listen_port === undefined || gatewayConfig.listen_port === null) { + return null; + } + return getGatewayBaseUrl(`${gatewayConfig.listen_host}`, Number.parseInt(`${gatewayConfig.listen_port}`, 10)); +} + +export async function ensureDirectory(targetPath) { + await mkdir(targetPath, { recursive: true }); +} + +export async function writeUtf8File(targetPath, content) { + const parent = path.dirname(targetPath); + if (parent && parent !== ".") { + await ensureDirectory(parent); + } + await writeFile(targetPath, content, "utf8"); +} + +export async function readJsonFile(filePath) { + if (!fs.existsSync(filePath)) { + return null; + } + const raw = await readFile(filePath, "utf8"); + if (!raw.trim()) { + return null; + } + return JSON.parse(raw); +} + +export async function writeJsonFile(filePath, value) { + await writeUtf8File(filePath, `${JSON.stringify(value, null, 2)}\n`); +} + +export async function getCodexProviderContext(codexConfigPath) { + const content = await readFile(codexConfigPath, "utf8"); + const providerMatch = content.match(/^\s*model_provider\s*=\s*"([^"]+)"\s*$/m); + if (!providerMatch) { + throw new Error(`model_provider was not found in ${codexConfigPath}`); + } + + const providerName = providerMatch[1]; + const sectionHeaderRegex = new RegExp(`^\\[model_providers\\.${escapeRegExp(providerName)}\\]\\s*$`, "m"); + const sectionHeaderMatch = sectionHeaderRegex.exec(content); + if (!sectionHeaderMatch) { + throw new Error(`[model_providers.${providerName}] was not found in ${codexConfigPath}`); + } + + const sectionIndex = sectionHeaderMatch.index; + const headerEndIndex = sectionIndex + sectionHeaderMatch[0].length; + const remainder = content.slice(headerEndIndex); + const nextSectionMatch = /^\[.*$/m.exec(remainder); + const sectionEndIndex = nextSectionMatch ? headerEndIndex + nextSectionMatch.index : content.length; + const sectionText = content.slice(sectionIndex, sectionEndIndex); + const baseUrlMatch = sectionText.match(/^\s*base_url\s*=\s*"([^"]+)"\s*$/m); + if (!baseUrlMatch) { + throw new Error(`base_url was not found in [model_providers.${providerName}]`); + } + + return { + content, + providerName, + sectionText, + sectionIndex, + sectionLength: sectionText.length, + currentBaseUrl: baseUrlMatch[1], + baseUrlLineText: baseUrlMatch[0], + }; +} + +export async function setCodexProviderBaseUrl({ codexConfigPath, providerName, newBaseUrl }) { + const context = await getCodexProviderContext(codexConfigPath); + if (context.providerName !== providerName) { + throw new Error(`model_provider changed unexpectedly: expected ${providerName}, actual ${context.providerName}`); + } + + let replaced = false; + const updatedSection = context.sectionText.replace( + /^(\s*base_url\s*=\s*")([^"]*)("\s*)$/m, + (_, prefix, __existing, suffix) => { + replaced = true; + return `${prefix}${newBaseUrl}${suffix}`; + }, + ); + if (!replaced) { + throw new Error(`base_url was not found in [model_providers.${providerName}]`); + } + + const updatedContent = + context.content.slice(0, context.sectionIndex) + + updatedSection + + context.content.slice(context.sectionIndex + context.sectionLength); + + await writeUtf8File(codexConfigPath, updatedContent); +} + +export function normalizeIntArray(values, fallback = [516]) { + const source = values === undefined || values === null ? fallback : values; + const queue = Array.isArray(source) ? source.flat(Infinity) : [source]; + const normalized = queue + .map((value) => (typeof value === "string" ? value.split(/[\s,]+/).filter(Boolean) : [value])) + .flat() + .map((value) => Number.parseInt(`${value}`, 10)) + .filter((value) => Number.isInteger(value)); + + return normalized.length > 0 ? [...new Set(normalized)] : [...fallback]; +} + +export function normalizeStringArray(values, fallback = []) { + const source = values === undefined || values === null ? fallback : values; + const queue = Array.isArray(source) ? source.flat(Infinity) : [source]; + const normalized = queue + .flatMap((value) => `${value ?? ""}`.split(/[\s,]+/)) + .map((value) => value.trim()) + .filter(Boolean); + + return normalized.length > 0 ? [...new Set(normalized)] : [...fallback]; +} + +export function isProcessAlive(processId) { + try { + process.kill(processId, 0); + return true; + } catch { + return false; + } +} + +export async function waitGatewayHealth({ + listenHost, + listenPort, + healthPath, + timeoutSeconds = 10, +}) { + const deadline = Date.now() + timeoutSeconds * 1000; + const healthUrl = `${getGatewayBaseUrl(listenHost, listenPort)}${healthPath}`; + + while (Date.now() < deadline) { + try { + const response = await fetch(healthUrl, { signal: AbortSignal.timeout(2000) }); + if (response.status === 200) { + return response; + } + } catch { + // ignore and retry + } + await new Promise((resolve) => setTimeout(resolve, 200)); + } + + throw new Error(`Gateway health check timed out: ${healthUrl}`); +} + +async function readTail(filePath, lineCount = 20) { + if (!fs.existsSync(filePath)) { + return ""; + } + const raw = await readFile(filePath, "utf8"); + return raw.split(/\r?\n/).slice(-lineCount).join("\n").trim(); +} + +function openUrl(url) { + let command; + let args; + if (process.platform === "win32") { + command = "cmd"; + args = ["/c", "start", "", url]; + } else if (process.platform === "darwin") { + command = "open"; + args = [url]; + } else { + command = "xdg-open"; + args = [url]; + } + + const child = spawn(command, args, { + detached: true, + stdio: "ignore", + windowsHide: true, + }); + child.unref(); +} + +export async function stopGateway({ stateRoot = DEFAULT_STATE_ROOT, quiet = false }) { + const paths = getGatewayStatePaths(stateRoot); + if (!fs.existsSync(paths.pidPath)) { + return quiet ? null : "No running gateway PID file was found."; + } + + const pidRaw = (await readFile(paths.pidPath, "utf8")).trim(); + if (!pidRaw) { + await rm(paths.pidPath, { force: true }); + return quiet ? null : "Gateway PID file was empty and has been removed."; + } + + const gatewayPid = Number.parseInt(pidRaw, 10); + if (Number.isInteger(gatewayPid) && isProcessAlive(gatewayPid)) { + try { + process.kill(gatewayPid); + } catch { + // ignore first failure + } + + const deadline = Date.now() + 3000; + while (Date.now() < deadline && isProcessAlive(gatewayPid)) { + await new Promise((resolve) => setTimeout(resolve, 100)); + } + + if (isProcessAlive(gatewayPid)) { + try { + process.kill(gatewayPid, "SIGKILL"); + } catch { + // ignore hard kill failure + } + } + } + + await rm(paths.pidPath, { force: true }); + return quiet ? null : `Gateway stopped. PID=${gatewayPid}`; +} + +export async function startGateway({ + stateRoot = DEFAULT_STATE_ROOT, + configPath, + logPath, + restartIfRunning = false, +}) { + const paths = getGatewayStatePaths(stateRoot); + const effectiveConfigPath = configPath || paths.configPath; + const effectiveLogPath = logPath || paths.logPath; + + if (!fs.existsSync(effectiveConfigPath)) { + throw new Error(`Gateway config file was not found: ${effectiveConfigPath}`); + } + + await ensureDirectory(path.dirname(effectiveLogPath)); + + if (fs.existsSync(paths.pidPath)) { + const existingPidRaw = (await readFile(paths.pidPath, "utf8")).trim(); + if (existingPidRaw) { + const existingPid = Number.parseInt(existingPidRaw, 10); + if (Number.isInteger(existingPid) && isProcessAlive(existingPid)) { + if (restartIfRunning) { + await stopGateway({ stateRoot, quiet: true }); + } else { + return `Gateway is already running. PID=${existingPid}`; + } + } else { + await rm(paths.pidPath, { force: true }); + } + } + } + + const gatewayConfig = await readJsonFile(effectiveConfigPath); + if (!gatewayConfig) { + throw new Error(`Gateway config file could not be read: ${effectiveConfigPath}`); + } + + const gatewayRoot = getGatewayRoot(); + const gatewayEntry = path.join(gatewayRoot, "gateway.mjs"); + if (!fs.existsSync(gatewayEntry)) { + throw new Error(`Gateway entry file was not found: ${gatewayEntry}`); + } + + const child = spawn(process.execPath, [gatewayEntry, "--config", effectiveConfigPath, "--log", effectiveLogPath], { + cwd: gatewayRoot, + detached: true, + stdio: "ignore", + windowsHide: true, + }); + child.unref(); + + await writeUtf8File(paths.pidPath, `${child.pid}`); + + await new Promise((resolve) => setTimeout(resolve, 300)); + if (!isProcessAlive(child.pid)) { + const logTail = await readTail(effectiveLogPath, 20); + throw new Error(`Gateway exited right after startup. PID=${child.pid}\n${logTail}`); + } + + await waitGatewayHealth({ + listenHost: `${gatewayConfig.listen_host}`, + listenPort: Number.parseInt(`${gatewayConfig.listen_port}`, 10), + healthPath: `${gatewayConfig.health_path || DEFAULT_HEALTH_PATH}`, + }); + + return `Gateway started. PID=${child.pid}. Listen=${getGatewayBaseUrl(gatewayConfig.listen_host, gatewayConfig.listen_port)}`; +} + +export async function installForCurrentProvider({ + codexConfigPath = DEFAULT_CODEX_CONFIG_PATH, + stateRoot = DEFAULT_STATE_ROOT, + listenHost = DEFAULT_LISTEN_HOST, + listenPort = DEFAULT_LISTEN_PORT, +}) { + const paths = getGatewayStatePaths(stateRoot); + await ensureDirectory(paths.stateRoot); + await ensureDirectory(paths.configDir); + await ensureDirectory(paths.logDir); + await ensureDirectory(paths.backupDir); + + if (!fs.existsSync(codexConfigPath)) { + throw new Error(`Codex config file was not found: ${codexConfigPath}`); + } + + const providerContext = await getCodexProviderContext(codexConfigPath); + const localGatewayBaseUrl = getGatewayBaseUrl(listenHost, listenPort); + const existingState = await readJsonFile(paths.statePath); + + let originalBaseUrl = providerContext.currentBaseUrl; + if (providerContext.currentBaseUrl === localGatewayBaseUrl) { + if (!existingState?.original_base_url) { + throw new Error("Provider already points to the local gateway, but original_base_url is missing from state."); + } + originalBaseUrl = `${existingState.original_base_url}`; + } + + if (originalBaseUrl === localGatewayBaseUrl) { + throw new Error("A real upstream_base_url could not be determined."); + } + + const backupPath = path.join(paths.backupDir, `config-${new Date().toISOString().replace(/[:.]/g, "").replace("T", "-").slice(0, 15)}.toml`); + await copyFile(codexConfigPath, backupPath); + + const existingGatewayConfig = await readJsonFile(paths.configPath); + const defaultEndpoints = ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"]; + const mergedEndpoints = []; + for (const endpoint of [ + ...normalizeStringArray(existingGatewayConfig?.endpoints, []), + ...defaultEndpoints, + ]) { + if (!mergedEndpoints.includes(endpoint)) { + mergedEndpoints.push(endpoint); + } + } + + const gatewayConfig = { + listen_host: listenHost, + listen_port: listenPort, + upstream_base_url: originalBaseUrl, + request_body_limit_bytes: + existingGatewayConfig?.request_body_limit_bytes === undefined || existingGatewayConfig?.request_body_limit_bytes === null + ? 10485760 + : Number.parseInt(`${existingGatewayConfig.request_body_limit_bytes}`, 10), + endpoints: mergedEndpoints, + reasoning_equals: normalizeIntArray(existingGatewayConfig?.reasoning_equals, [516]), + non_stream_status_code: + existingGatewayConfig?.non_stream_status_code === undefined || existingGatewayConfig?.non_stream_status_code === null + ? 502 + : Number.parseInt(`${existingGatewayConfig.non_stream_status_code}`, 10), + stream_action: existingGatewayConfig?.stream_action || "strict_502", + log_match: existingGatewayConfig?.log_match === undefined ? true : Boolean(existingGatewayConfig.log_match), + health_path: existingGatewayConfig?.health_path || DEFAULT_HEALTH_PATH, + }; + + const previousConfigContent = await readFile(codexConfigPath, "utf8"); + + try { + await writeJsonFile(paths.configPath, gatewayConfig); + await setCodexProviderBaseUrl({ + codexConfigPath, + providerName: providerContext.providerName, + newBaseUrl: localGatewayBaseUrl, + }); + + await startGateway({ + stateRoot, + configPath: paths.configPath, + logPath: paths.logPath, + restartIfRunning: true, + }); + + const state = { + installed_at: new Date().toISOString(), + codex_config_path: codexConfigPath, + provider_name: providerContext.providerName, + original_base_url: originalBaseUrl, + gateway_base_url: localGatewayBaseUrl, + gateway_config_path: paths.configPath, + gateway_log_path: paths.logPath, + gateway_pid_path: paths.pidPath, + latest_backup_path: backupPath, + state_root: paths.stateRoot, + }; + await writeJsonFile(paths.statePath, state); + + return { + provider: providerContext.providerName, + upstream: originalBaseUrl, + gateway: localGatewayBaseUrl, + configPath: paths.configPath, + backupPath, + }; + } catch (error) { + await writeUtf8File(codexConfigPath, previousConfigContent); + await stopGateway({ stateRoot, quiet: true }); + throw error; + } +} + +export async function restoreCodexConfig({ + stateRoot = DEFAULT_STATE_ROOT, + codexConfigPath = DEFAULT_CODEX_CONFIG_PATH, +}) { + const paths = getGatewayStatePaths(stateRoot); + const state = await readJsonFile(paths.statePath); + if (!state) { + throw new Error(`Install state file was not found: ${paths.statePath}`); + } + + const backupPath = `${state.latest_backup_path || ""}`; + if (!backupPath || !fs.existsSync(backupPath)) { + throw new Error(`A restorable backup file was not found: ${backupPath}`); + } + + await stopGateway({ stateRoot, quiet: true }); + await copyFile(backupPath, codexConfigPath); + await rm(paths.statePath, { force: true }); + + return { + configPath: codexConfigPath, + restoredFrom: backupPath, + }; +} + +export async function launchUi({ + codexConfigPath = DEFAULT_CODEX_CONFIG_PATH, + stateRoot = DEFAULT_STATE_ROOT, + listenHost = DEFAULT_LISTEN_HOST, + listenPort = DEFAULT_LISTEN_PORT, + noOpen = false, +}) { + const paths = getGatewayStatePaths(stateRoot); + await ensureDirectory(paths.stateRoot); + await ensureDirectory(paths.configDir); + await ensureDirectory(paths.logDir); + await ensureDirectory(paths.backupDir); + + if (!fs.existsSync(codexConfigPath)) { + throw new Error(`Codex config file was not found: ${codexConfigPath}`); + } + + const providerContext = await getCodexProviderContext(codexConfigPath); + const currentBaseUrl = `${providerContext.currentBaseUrl}`; + const requestedGatewayBaseUrl = getGatewayBaseUrl(listenHost, listenPort); + const existingState = await readJsonFile(paths.statePath); + const existingGatewayConfig = await readJsonFile(paths.configPath); + const stateGatewayBaseUrl = existingState?.gateway_base_url ? `${existingState.gateway_base_url}` : null; + const configGatewayBaseUrl = getGatewayBaseUrlFromConfig(existingGatewayConfig); + const managedGatewayBaseUrls = [requestedGatewayBaseUrl]; + for (const candidate of [stateGatewayBaseUrl, configGatewayBaseUrl]) { + if (candidate && !managedGatewayBaseUrls.includes(candidate)) { + managedGatewayBaseUrls.push(candidate); + } + } + + const originalBaseUrl = + existingState?.original_base_url + ? `${existingState.original_base_url}` + : existingGatewayConfig?.upstream_base_url + ? `${existingGatewayConfig.upstream_base_url}` + : null; + + const canReuseExistingInstall = + existingGatewayConfig && + originalBaseUrl && + managedGatewayBaseUrls.includes(currentBaseUrl); + + let mode = "install"; + if (!canReuseExistingInstall) { + await installForCurrentProvider({ + codexConfigPath, + stateRoot, + listenHost, + listenPort, + }); + } else { + mode = "reuse"; + const previousCodexConfigContent = await readFile(codexConfigPath, "utf8"); + const previousGatewayConfigContent = fs.existsSync(paths.configPath) + ? await readFile(paths.configPath, "utf8") + : null; + const previousStateContent = fs.existsSync(paths.statePath) + ? await readFile(paths.statePath, "utf8") + : null; + + try { + existingGatewayConfig.listen_host = listenHost; + existingGatewayConfig.listen_port = listenPort; + if (!existingGatewayConfig.health_path) { + existingGatewayConfig.health_path = DEFAULT_HEALTH_PATH; + } + await writeJsonFile(paths.configPath, existingGatewayConfig); + + if (currentBaseUrl !== requestedGatewayBaseUrl) { + await setCodexProviderBaseUrl({ + codexConfigPath, + providerName: providerContext.providerName, + newBaseUrl: requestedGatewayBaseUrl, + }); + } + + await startGateway({ + stateRoot, + configPath: paths.configPath, + logPath: paths.logPath, + restartIfRunning: true, + }); + + const statePayload = { + installed_at: existingState?.installed_at ? `${existingState.installed_at}` : new Date().toISOString(), + last_started_at: new Date().toISOString(), + codex_config_path: codexConfigPath, + provider_name: providerContext.providerName, + original_base_url: originalBaseUrl, + gateway_base_url: requestedGatewayBaseUrl, + gateway_config_path: paths.configPath, + gateway_log_path: paths.logPath, + gateway_pid_path: paths.pidPath, + latest_backup_path: existingState?.latest_backup_path ? `${existingState.latest_backup_path}` : "", + state_root: paths.stateRoot, + }; + await writeJsonFile(paths.statePath, statePayload); + } catch (error) { + await writeUtf8File(codexConfigPath, previousCodexConfigContent); + if (previousGatewayConfigContent !== null) { + await writeUtf8File(paths.configPath, previousGatewayConfigContent); + } + if (previousStateContent !== null) { + await writeUtf8File(paths.statePath, previousStateContent); + } + await stopGateway({ stateRoot, quiet: true }); + throw error; + } + } + + const effectiveGatewayConfig = await readJsonFile(paths.configPath); + const effectiveGatewayBaseUrl = getGatewayBaseUrlFromConfig(effectiveGatewayConfig) || requestedGatewayBaseUrl; + const uiUrl = `${effectiveGatewayBaseUrl}/__codex_retry_gateway/ui`; + + if (!noOpen) { + openUrl(uiUrl); + } + + return { + mode, + uiUrl, + gatewayBaseUrl: effectiveGatewayBaseUrl, + }; +} diff --git a/scripts/install-for-current-provider.ps1 b/scripts/install-for-current-provider.ps1 index d14aec0..bdb589d 100644 --- a/scripts/install-for-current-provider.ps1 +++ b/scripts/install-for-current-provider.ps1 @@ -1,109 +1,109 @@ -param( - [string]$CodexConfigPath = "$HOME\.codex\config.toml", - [string]$StateRoot = "$HOME\.codex-retry-gateway", - [string]$ListenHost = "127.0.0.1", - [int]$ListenPort = 4610 -) - -$ErrorActionPreference = "Stop" -Set-StrictMode -Version Latest - -. (Join-Path $PSScriptRoot "common.ps1") - -$paths = Get-GatewayStatePaths -StateRoot $StateRoot -Ensure-Directory -Path $paths.StateRoot -Ensure-Directory -Path $paths.ConfigDir -Ensure-Directory -Path $paths.LogDir -Ensure-Directory -Path $paths.BackupDir - -if (-not (Test-Path -LiteralPath $CodexConfigPath)) { - throw "Codex config file was not found: $CodexConfigPath" -} - -$providerContext = Get-CodexProviderContext -CodexConfigPath $CodexConfigPath -$localGatewayBaseUrl = "http://{0}:{1}" -f $ListenHost, $ListenPort -$existingState = Read-JsonFile -Path $paths.StatePath - -$originalBaseUrl = $providerContext.CurrentBaseUrl -if ($providerContext.CurrentBaseUrl -eq $localGatewayBaseUrl) { - if ($null -eq $existingState -or [string]::IsNullOrWhiteSpace([string]$existingState.original_base_url)) { - throw "Provider already points to the local gateway, but original_base_url is missing from state." - } - $originalBaseUrl = [string]$existingState.original_base_url -} - -if ($originalBaseUrl -eq $localGatewayBaseUrl) { - throw "A real upstream_base_url could not be determined." -} - -$backupPath = Join-Path $paths.BackupDir ("config-" + (Get-Date -Format "yyyyMMdd-HHmmss") + ".toml") -Copy-Item -LiteralPath $CodexConfigPath -Destination $backupPath -Force - -$existingGatewayConfig = Read-JsonFile -Path $paths.ConfigPath -$defaultEndpoints = @("/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions") -$mergedEndpoints = @() -foreach ($endpoint in @( - $(if ($existingGatewayConfig) { Normalize-StringArray -Values $existingGatewayConfig.endpoints -Default @() } else { @() }) + - $defaultEndpoints -)) { - if ([string]::IsNullOrWhiteSpace([string]$endpoint)) { - continue - } - if ($mergedEndpoints -notcontains [string]$endpoint) { - $mergedEndpoints += [string]$endpoint - } -} - -$gatewayConfig = [ordered]@{ - listen_host = $ListenHost - listen_port = $ListenPort - upstream_base_url = $originalBaseUrl - request_body_limit_bytes = if ($existingGatewayConfig -and $null -ne $existingGatewayConfig.request_body_limit_bytes) { [int]$existingGatewayConfig.request_body_limit_bytes } else { 10485760 } - endpoints = @($mergedEndpoints) - reasoning_equals = Normalize-IntArray -Values $(if ($existingGatewayConfig) { $existingGatewayConfig.reasoning_equals } else { $null }) -Default @(516) - non_stream_status_code = if ($existingGatewayConfig -and $null -ne $existingGatewayConfig.non_stream_status_code) { [int]$existingGatewayConfig.non_stream_status_code } else { 502 } - stream_action = if ($existingGatewayConfig -and -not [string]::IsNullOrWhiteSpace([string]$existingGatewayConfig.stream_action)) { [string]$existingGatewayConfig.stream_action } else { "disconnect" } - log_match = if ($existingGatewayConfig -and $null -ne $existingGatewayConfig.log_match) { [bool]$existingGatewayConfig.log_match } else { $true } - health_path = if ($existingGatewayConfig -and -not [string]::IsNullOrWhiteSpace([string]$existingGatewayConfig.health_path)) { [string]$existingGatewayConfig.health_path } else { "/__codex_retry_gateway/health" } -} - -$previousConfigContent = Get-Content -LiteralPath $CodexConfigPath -Raw - -try { - Write-JsonFile -Path $paths.ConfigPath -Value $gatewayConfig - Set-CodexProviderBaseUrl ` - -CodexConfigPath $CodexConfigPath ` - -ProviderName $providerContext.ProviderName ` - -NewBaseUrl $localGatewayBaseUrl - - & (Join-Path $PSScriptRoot "start-gateway.ps1") ` - -StateRoot $StateRoot ` - -ConfigPath $paths.ConfigPath ` - -LogPath $paths.LogPath ` - -RestartIfRunning - - $state = [ordered]@{ - installed_at = (Get-Date).ToString("o") - codex_config_path = $CodexConfigPath - provider_name = $providerContext.ProviderName - original_base_url = $originalBaseUrl - gateway_base_url = $localGatewayBaseUrl - gateway_config_path = $paths.ConfigPath - gateway_log_path = $paths.LogPath - gateway_pid_path = $paths.PidPath - latest_backup_path = $backupPath - state_root = $paths.StateRoot - } - Write-JsonFile -Path $paths.StatePath -Value $state - - Write-Output "Installed Codex Retry Gateway" - Write-Output "provider=$($providerContext.ProviderName)" - Write-Output "upstream=$originalBaseUrl" - Write-Output "gateway=$localGatewayBaseUrl" - Write-Output "config=$($paths.ConfigPath)" - Write-Output "backup=$backupPath" -} catch { - Write-Utf8NoBomFile -Path $CodexConfigPath -Content $previousConfigContent - & (Join-Path $PSScriptRoot "stop-gateway.ps1") -StateRoot $StateRoot -Quiet - throw -} +param( + [string]$CodexConfigPath = "$HOME\.codex\config.toml", + [string]$StateRoot = "$HOME\.codex-retry-gateway", + [string]$ListenHost = "127.0.0.1", + [int]$ListenPort = 4610 +) + +$ErrorActionPreference = "Stop" +Set-StrictMode -Version Latest + +. (Join-Path $PSScriptRoot "common.ps1") + +$paths = Get-GatewayStatePaths -StateRoot $StateRoot +Ensure-Directory -Path $paths.StateRoot +Ensure-Directory -Path $paths.ConfigDir +Ensure-Directory -Path $paths.LogDir +Ensure-Directory -Path $paths.BackupDir + +if (-not (Test-Path -LiteralPath $CodexConfigPath)) { + throw "Codex config file was not found: $CodexConfigPath" +} + +$providerContext = Get-CodexProviderContext -CodexConfigPath $CodexConfigPath +$localGatewayBaseUrl = "http://{0}:{1}" -f $ListenHost, $ListenPort +$existingState = Read-JsonFile -Path $paths.StatePath + +$originalBaseUrl = $providerContext.CurrentBaseUrl +if ($providerContext.CurrentBaseUrl -eq $localGatewayBaseUrl) { + if ($null -eq $existingState -or [string]::IsNullOrWhiteSpace([string]$existingState.original_base_url)) { + throw "Provider already points to the local gateway, but original_base_url is missing from state." + } + $originalBaseUrl = [string]$existingState.original_base_url +} + +if ($originalBaseUrl -eq $localGatewayBaseUrl) { + throw "A real upstream_base_url could not be determined." +} + +$backupPath = Join-Path $paths.BackupDir ("config-" + (Get-Date -Format "yyyyMMdd-HHmmss") + ".toml") +Copy-Item -LiteralPath $CodexConfigPath -Destination $backupPath -Force + +$existingGatewayConfig = Read-JsonFile -Path $paths.ConfigPath +$defaultEndpoints = @("/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions") +$mergedEndpoints = @() +foreach ($endpoint in @( + $(if ($existingGatewayConfig) { Normalize-StringArray -Values $existingGatewayConfig.endpoints -Default @() } else { @() }) + + $defaultEndpoints +)) { + if ([string]::IsNullOrWhiteSpace([string]$endpoint)) { + continue + } + if ($mergedEndpoints -notcontains [string]$endpoint) { + $mergedEndpoints += [string]$endpoint + } +} + +$gatewayConfig = [ordered]@{ + listen_host = $ListenHost + listen_port = $ListenPort + upstream_base_url = $originalBaseUrl + request_body_limit_bytes = if ($existingGatewayConfig -and $null -ne $existingGatewayConfig.request_body_limit_bytes) { [int]$existingGatewayConfig.request_body_limit_bytes } else { 10485760 } + endpoints = @($mergedEndpoints) + reasoning_equals = Normalize-IntArray -Values $(if ($existingGatewayConfig) { $existingGatewayConfig.reasoning_equals } else { $null }) -Default @(516) + non_stream_status_code = if ($existingGatewayConfig -and $null -ne $existingGatewayConfig.non_stream_status_code) { [int]$existingGatewayConfig.non_stream_status_code } else { 502 } + stream_action = if ($existingGatewayConfig -and -not [string]::IsNullOrWhiteSpace([string]$existingGatewayConfig.stream_action)) { [string]$existingGatewayConfig.stream_action } else { "strict_502" } + log_match = if ($existingGatewayConfig -and $null -ne $existingGatewayConfig.log_match) { [bool]$existingGatewayConfig.log_match } else { $true } + health_path = if ($existingGatewayConfig -and -not [string]::IsNullOrWhiteSpace([string]$existingGatewayConfig.health_path)) { [string]$existingGatewayConfig.health_path } else { "/__codex_retry_gateway/health" } +} + +$previousConfigContent = Get-Content -LiteralPath $CodexConfigPath -Raw + +try { + Write-JsonFile -Path $paths.ConfigPath -Value $gatewayConfig + Set-CodexProviderBaseUrl ` + -CodexConfigPath $CodexConfigPath ` + -ProviderName $providerContext.ProviderName ` + -NewBaseUrl $localGatewayBaseUrl + + & (Join-Path $PSScriptRoot "start-gateway.ps1") ` + -StateRoot $StateRoot ` + -ConfigPath $paths.ConfigPath ` + -LogPath $paths.LogPath ` + -RestartIfRunning + + $state = [ordered]@{ + installed_at = (Get-Date).ToString("o") + codex_config_path = $CodexConfigPath + provider_name = $providerContext.ProviderName + original_base_url = $originalBaseUrl + gateway_base_url = $localGatewayBaseUrl + gateway_config_path = $paths.ConfigPath + gateway_log_path = $paths.LogPath + gateway_pid_path = $paths.PidPath + latest_backup_path = $backupPath + state_root = $paths.StateRoot + } + Write-JsonFile -Path $paths.StatePath -Value $state + + Write-Output "Installed Codex Retry Gateway" + Write-Output "provider=$($providerContext.ProviderName)" + Write-Output "upstream=$originalBaseUrl" + Write-Output "gateway=$localGatewayBaseUrl" + Write-Output "config=$($paths.ConfigPath)" + Write-Output "backup=$backupPath" +} catch { + Write-Utf8NoBomFile -Path $CodexConfigPath -Content $previousConfigContent + & (Join-Path $PSScriptRoot "stop-gateway.ps1") -StateRoot $StateRoot -Quiet + throw +} diff --git a/scripts/test-gateway-e2e.mjs b/scripts/test-gateway-e2e.mjs index de8058e..7992f07 100644 --- a/scripts/test-gateway-e2e.mjs +++ b/scripts/test-gateway-e2e.mjs @@ -1,316 +1,388 @@ -#!/usr/bin/env node - -import http from "node:http"; -import net from "node:net"; -import { once } from "node:events"; -import { spawn } from "node:child_process"; -import { mkdtemp, rm, writeFile } from "node:fs/promises"; -import os from "node:os"; -import path from "node:path"; - -const gatewayRoot = path.resolve(import.meta.dirname, ".."); -const gatewayEntry = path.join(gatewayRoot, "gateway.mjs"); - -function assert(condition, message) { - if (!condition) { - throw new Error(message); - } -} - -async function getFreePort() { - const server = net.createServer(); - server.listen(0, "127.0.0.1"); - await once(server, "listening"); - const address = server.address(); - const port = address && typeof address === "object" ? address.port : null; - server.close(); - await once(server, "close"); - if (!port) { - throw new Error("无法分配空闲端口"); - } - return port; -} - -function createJsonResponse(res, statusCode, body, extraHeaders = {}) { - res.writeHead(statusCode, { - "content-type": "application/json; charset=utf-8", - ...extraHeaders, - }); - res.end(JSON.stringify(body)); -} - +#!/usr/bin/env node + +import http from "node:http"; +import net from "node:net"; +import { once } from "node:events"; +import { spawn } from "node:child_process"; +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; + +const gatewayRoot = path.resolve(import.meta.dirname, ".."); +const gatewayEntry = path.join(gatewayRoot, "gateway.mjs"); + +function assert(condition, message) { + if (!condition) { + throw new Error(message); + } +} + +async function getFreePort() { + const server = net.createServer(); + server.listen(0, "127.0.0.1"); + await once(server, "listening"); + const address = server.address(); + const port = address && typeof address === "object" ? address.port : null; + server.close(); + await once(server, "close"); + if (!port) { + throw new Error("无法分配空闲端口"); + } + return port; +} + +function createJsonResponse(res, statusCode, body, extraHeaders = {}) { + res.writeHead(statusCode, { + "content-type": "application/json; charset=utf-8", + ...extraHeaders, + }); + res.end(JSON.stringify(body)); +} + function createSseResponse(res, chunks) { - res.writeHead(200, { - "content-type": "text/event-stream; charset=utf-8", - "cache-control": "no-cache", - connection: "keep-alive", - "x-upstream-test": "sse", - }); - - let index = 0; - const timer = setInterval(() => { - if (index >= chunks.length) { - clearInterval(timer); - res.end(); - return; - } - res.write(chunks[index]); - index += 1; - }, 20); - + res.writeHead(200, { + "content-type": "text/event-stream; charset=utf-8", + "cache-control": "no-cache", + connection: "keep-alive", + "x-upstream-test": "sse", + }); + + let index = 0; + const timer = setInterval(() => { + if (index >= chunks.length) { + clearInterval(timer); + res.end(); + return; + } + res.write(chunks[index]); + index += 1; + }, 20); + res.on("close", () => { clearInterval(timer); }); } +function createTerminatedSseResponse(res, chunks, destroyDelayMs = 20) { + res.writeHead(200, { + "content-type": "text/event-stream; charset=utf-8", + "cache-control": "no-cache", + connection: "keep-alive", + "x-upstream-test": "sse-terminated", + }); + + for (const chunk of chunks) { + res.write(chunk); + } + + setTimeout(() => { + res.socket?.destroy(); + }, destroyDelayMs); +} + function startFakeUpstream(port) { + const failBeforeResponseCounts = new Map(); const server = http.createServer((req, res) => { - const responsePaths = new Set(["/responses", "/v1/responses"]); - const chatCompletionPaths = new Set(["/chat/completions", "/v1/chat/completions"]); - - if (req.method === "GET" && req.url === "/v1/models") { - createJsonResponse( - res, - 200, - { - object: "list", - data: [{ id: "fake-model" }], - }, - { "x-upstream-test": "models-ok" }, - ); - return; - } - - if (req.method === "POST" && responsePaths.has(req.url)) { - let body = ""; - req.setEncoding("utf8"); - req.on("data", (chunk) => { - body += chunk; - }); + const responsePaths = new Set(["/responses", "/v1/responses"]); + const chatCompletionPaths = new Set(["/chat/completions", "/v1/chat/completions"]); + + if (req.method === "GET" && req.url === "/v1/models") { + createJsonResponse( + res, + 200, + { + object: "list", + data: [{ id: "fake-model" }], + }, + { "x-upstream-test": "models-ok" }, + ); + return; + } + + if (req.method === "POST" && responsePaths.has(req.url)) { + let body = ""; + req.setEncoding("utf8"); + req.on("data", (chunk) => { + body += chunk; + }); req.on("end", () => { const parsed = JSON.parse(body || "{}"); const reasoning = parsed.test_reasoning_tokens ?? 128; + if (parsed.test_fail_before_response_once) { + const failKey = `${req.url}:fail-before-response-once`; + const failCount = (failBeforeResponseCounts.get(failKey) || 0) + 1; + failBeforeResponseCounts.set(failKey, failCount); + if (failCount === 1) { + res.socket?.destroy(); + return; + } + } + if (parsed.test_force_terminate) { + createTerminatedSseResponse(res, [ + 'data: {"type":"response.output_text.delta","delta":"hello"}\n\n', + ]); + return; + } + if (parsed.stream) { + createSseResponse(res, [ + 'data: {"type":"response.output_text.delta","delta":"hello"}\n\n', + `data: {"response":{"usage":{"output_tokens_details":{"reasoning_tokens":${reasoning}}}}}\n\n`, + "data: [DONE]\n\n", + ]); + return; + } createJsonResponse( res, 200, { id: "resp_test", + retry_attempt: parsed.test_fail_before_response_once + ? failBeforeResponseCounts.get(`${req.url}:fail-before-response-once`) || 0 + : 0, usage: { output_tokens_details: { reasoning_tokens: reasoning, }, }, - }, - { "x-upstream-test": `responses-${reasoning}` }, - ); - }); - return; - } - - if (req.method === "POST" && chatCompletionPaths.has(req.url)) { - let body = ""; - req.setEncoding("utf8"); - req.on("data", (chunk) => { - body += chunk; - }); - req.on("end", () => { - const parsed = JSON.parse(body || "{}"); - const reasoning = parsed.test_reasoning_tokens ?? 128; - if (reasoning === 516) { - createSseResponse(res, [ - 'data: {"id":"chunk-1","choices":[{"delta":{"content":"hello"}}]}\n\n', - 'data: {"usage":{"completion_tokens_details":{"reasoning_tokens":516}}}\n\n', - "data: [DONE]\n\n", - ]); - return; - } - - createSseResponse(res, [ - 'data: {"id":"chunk-1","choices":[{"delta":{"content":"hello"}}]}\n\n', - 'data: {"usage":{"completion_tokens_details":{"reasoning_tokens":128}}}\n\n', - "data: [DONE]\n\n", - ]); - }); - return; - } - - createJsonResponse(res, 404, { error: "not found" }); - }); - - return new Promise((resolve, reject) => { - server.once("error", reject); - server.listen(port, "127.0.0.1", () => resolve(server)); - }); -} - -async function waitForHealth(url, timeoutMs = 5000) { - const startedAt = Date.now(); - while (Date.now() - startedAt < timeoutMs) { - try { - const response = await fetch(url); - if (response.ok) { - return; - } - } catch { - // ignore startup race - } - await new Promise((resolve) => setTimeout(resolve, 100)); - } - throw new Error(`等待网关健康检查超时: ${url}`); -} - -function startGateway(configPath, logPath) { - const child = spawn(process.execPath, [gatewayEntry, "--config", configPath, "--log", logPath], { - cwd: gatewayRoot, - stdio: ["ignore", "pipe", "pipe"], - }); - - let stdout = ""; - let stderr = ""; - child.stdout.on("data", (chunk) => { - stdout += chunk.toString(); - }); - child.stderr.on("data", (chunk) => { - stderr += chunk.toString(); - }); - - return { - child, - getOutput() { - return { stdout, stderr }; - }, - }; -} - -async function readSseUntilClose(url, requestBody) { - const response = await fetch(url, { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify(requestBody), - }); - - const reader = response.body.getReader(); - const decoder = new TextDecoder("utf8"); - let text = ""; - let closedByError = false; - - while (true) { - try { - const { done, value } = await reader.read(); - if (done) { - break; - } - text += decoder.decode(value, { stream: true }); - } catch (error) { - closedByError = true; - text += `\n[[reader-error:${error?.name || "unknown"}]]`; - break; - } - } - - text += decoder.decode(); - return { - status: response.status, - headers: response.headers, - text, - closedByError, - }; -} - -async function run() { - const tempRoot = await mkdtemp(path.join(os.tmpdir(), "codex-retry-gateway-")); - const upstreamPort = await getFreePort(); - const gatewayPort = await getFreePort(); - const configPath = path.join(tempRoot, "config.json"); - const logPath = path.join(tempRoot, "gateway.log"); - + }, + { "x-upstream-test": `responses-${reasoning}` }, + ); + }); + return; + } + + if (req.method === "POST" && chatCompletionPaths.has(req.url)) { + let body = ""; + req.setEncoding("utf8"); + req.on("data", (chunk) => { + body += chunk; + }); + req.on("end", () => { + const parsed = JSON.parse(body || "{}"); + const reasoning = parsed.test_reasoning_tokens ?? 128; + if (reasoning === 516) { + createSseResponse(res, [ + 'data: {"id":"chunk-1","choices":[{"delta":{"content":"hello"}}]}\n\n', + 'data: {"usage":{"completion_tokens_details":{"reasoning_tokens":516}}}\n\n', + "data: [DONE]\n\n", + ]); + return; + } + + createSseResponse(res, [ + 'data: {"id":"chunk-1","choices":[{"delta":{"content":"hello"}}]}\n\n', + 'data: {"usage":{"completion_tokens_details":{"reasoning_tokens":128}}}\n\n', + "data: [DONE]\n\n", + ]); + }); + return; + } + + createJsonResponse(res, 404, { error: "not found" }); + }); + + return new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(port, "127.0.0.1", () => resolve(server)); + }); +} + +async function waitForHealth(url, timeoutMs = 5000) { + const startedAt = Date.now(); + while (Date.now() - startedAt < timeoutMs) { + try { + const response = await fetch(url); + if (response.ok) { + return; + } + } catch { + // ignore startup race + } + await new Promise((resolve) => setTimeout(resolve, 100)); + } + throw new Error(`等待网关健康检查超时: ${url}`); +} + +function startGateway(configPath, logPath) { + const child = spawn(process.execPath, [gatewayEntry, "--config", configPath, "--log", logPath], { + cwd: gatewayRoot, + stdio: ["ignore", "pipe", "pipe"], + }); + + let stdout = ""; + let stderr = ""; + child.stdout.on("data", (chunk) => { + stdout += chunk.toString(); + }); + child.stderr.on("data", (chunk) => { + stderr += chunk.toString(); + }); + + return { + child, + getOutput() { + return { stdout, stderr }; + }, + }; +} + +async function readSseUntilClose(url, requestBody) { + const response = await fetch(url, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(requestBody), + }); + + const reader = response.body.getReader(); + const decoder = new TextDecoder("utf8"); + let text = ""; + let closedByError = false; + + while (true) { + try { + const { done, value } = await reader.read(); + if (done) { + break; + } + text += decoder.decode(value, { stream: true }); + } catch (error) { + closedByError = true; + text += `\n[[reader-error:${error?.name || "unknown"}]]`; + break; + } + } + + text += decoder.decode(); + return { + status: response.status, + headers: response.headers, + text, + closedByError, + }; +} + +async function run() { + const tempRoot = await mkdtemp(path.join(os.tmpdir(), "codex-retry-gateway-")); + const upstreamPort = await getFreePort(); + const gatewayPort = await getFreePort(); + const configPath = path.join(tempRoot, "config.json"); + const logPath = path.join(tempRoot, "gateway.log"); + const config = { - listen_host: "127.0.0.1", - listen_port: gatewayPort, - upstream_base_url: `http://127.0.0.1:${upstreamPort}`, - request_body_limit_bytes: 10 * 1024 * 1024, - endpoints: ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"], - reasoning_equals: [516], - non_stream_status_code: 502, - stream_action: "disconnect", - log_match: true, - health_path: "/__codex_retry_gateway/health", - }; + listen_host: "127.0.0.1", + listen_port: gatewayPort, + upstream_base_url: `http://127.0.0.1:${upstreamPort}`, + request_body_limit_bytes: 10 * 1024 * 1024, + endpoints: ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"], + reasoning_equals: [516], + non_stream_status_code: 502, + stream_action: "strict_502", + log_match: true, + health_path: "/__codex_retry_gateway/health", + }; + + await writeFile(configPath, JSON.stringify(config, null, 2), "utf8"); + + const upstream = await startFakeUpstream(upstreamPort); + const gateway = startGateway(configPath, logPath); + + try { + await waitForHealth(`http://127.0.0.1:${gatewayPort}${config.health_path}`); + + const modelsResponse = await fetch(`http://127.0.0.1:${gatewayPort}/v1/models`); + assert(modelsResponse.status === 200, `/v1/models 透传状态异常: ${modelsResponse.status}`); + assert( + modelsResponse.headers.get("x-upstream-test") === "models-ok", + "/v1/models 未保留上游头", + ); + + for (const responsePath of ["/responses", "/v1/responses"]) { + const blockedResponse = await fetch(`http://127.0.0.1:${gatewayPort}${responsePath}`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ test_reasoning_tokens: 516 }), + }); + const blockedBody = await blockedResponse.json(); + assert(blockedResponse.status === 502, `${responsePath} 516 未返回 502: ${blockedResponse.status}`); + assert( + blockedBody?.error?.code === "reasoning_guard_triggered", + `${responsePath} 516 返回体不正确`, + ); + + const okResponse = await fetch(`http://127.0.0.1:${gatewayPort}${responsePath}`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ test_reasoning_tokens: 128 }), + }); + const okBody = await okResponse.json(); + assert(okResponse.status === 200, `${responsePath} 128 透传状态异常: ${okResponse.status}`); + assert(okResponse.headers.get("x-upstream-test") === "responses-128", `${responsePath} 128 未保留头`); + assert( + okBody?.usage?.output_tokens_details?.reasoning_tokens === 128, + `${responsePath} 128 返回体异常`, + ); + } + + const recoveredResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ test_fail_before_response_once: true }), + }); + const recoveredBody = await recoveredResponse.json(); + assert(recoveredResponse.status === 200, `首次 fetch failed 后未自动恢复: ${recoveredResponse.status}`); + assert(recoveredBody?.retry_attempt === 2, "首次 fetch failed 后未命中第二次上游请求"); - await writeFile(configPath, JSON.stringify(config, null, 2), "utf8"); - - const upstream = await startFakeUpstream(upstreamPort); - const gateway = startGateway(configPath, logPath); - - try { - await waitForHealth(`http://127.0.0.1:${gatewayPort}${config.health_path}`); - - const modelsResponse = await fetch(`http://127.0.0.1:${gatewayPort}/v1/models`); - assert(modelsResponse.status === 200, `/v1/models 透传状态异常: ${modelsResponse.status}`); - assert( - modelsResponse.headers.get("x-upstream-test") === "models-ok", - "/v1/models 未保留上游头", - ); - - for (const responsePath of ["/responses", "/v1/responses"]) { - const blockedResponse = await fetch(`http://127.0.0.1:${gatewayPort}${responsePath}`, { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify({ test_reasoning_tokens: 516 }), - }); - const blockedBody = await blockedResponse.json(); - assert(blockedResponse.status === 502, `${responsePath} 516 未返回 502: ${blockedResponse.status}`); - assert( - blockedBody?.error?.code === "reasoning_guard_triggered", - `${responsePath} 516 返回体不正确`, - ); - - const okResponse = await fetch(`http://127.0.0.1:${gatewayPort}${responsePath}`, { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify({ test_reasoning_tokens: 128 }), - }); - const okBody = await okResponse.json(); - assert(okResponse.status === 200, `${responsePath} 128 透传状态异常: ${okResponse.status}`); - assert(okResponse.headers.get("x-upstream-test") === "responses-128", `${responsePath} 128 未保留头`); - assert( - okBody?.usage?.output_tokens_details?.reasoning_tokens === 128, - `${responsePath} 128 返回体异常`, - ); - } - - for (const streamPath of ["/chat/completions", "/v1/chat/completions"]) { + for (const streamPath of [ + "/responses", + "/v1/responses", + "/chat/completions", + "/v1/chat/completions", + ]) { const blockedStream = await readSseUntilClose( `http://127.0.0.1:${gatewayPort}${streamPath}`, { stream: true, test_reasoning_tokens: 516 }, ); - assert(blockedStream.status === 200, `${streamPath} 516 首状态异常: ${blockedStream.status}`); - assert(blockedStream.text.includes('"content":"hello"'), `${streamPath} 流式 516 未先透传正常 chunk`); - assert(!blockedStream.text.includes("[DONE]"), `${streamPath} 流式 516 不应完整结束`); + assert(blockedStream.status === 502, `${streamPath} 516 未返回 502: ${blockedStream.status}`); + assert(!blockedStream.text.includes("hello"), `${streamPath} 严格 502 模式不应先透传正常 chunk`); + assert(!blockedStream.text.includes("[DONE]"), `${streamPath} 严格 502 模式不应回放 DONE`); + const blockedStreamBody = JSON.parse(blockedStream.text); assert( - blockedStream.closedByError || blockedStream.text.includes("[[reader-error:"), - `${streamPath} 流式 516 未表现为中途断开`, - ); - - const okStream = await readSseUntilClose( - `http://127.0.0.1:${gatewayPort}${streamPath}`, - { stream: true, test_reasoning_tokens: 128 }, + blockedStreamBody?.error?.code === "reasoning_guard_triggered", + `${streamPath} 流式 516 返回体不正确`, ); + + const okStream = await readSseUntilClose( + `http://127.0.0.1:${gatewayPort}${streamPath}`, + { stream: true, test_reasoning_tokens: 128 }, + ); assert(okStream.status === 200, `${streamPath} 128 首状态异常: ${okStream.status}`); assert(okStream.text.includes("[DONE]"), `${streamPath} 流式 128 未完整结束`); assert(!okStream.closedByError, `${streamPath} 流式 128 不应异常断开`); } - process.stdout.write("PASS codex-retry-gateway e2e\n"); - } finally { - gateway.child.kill(); - upstream.close(); - await once(upstream, "close"); - await rm(tempRoot, { recursive: true, force: true }); - } -} + const terminatedStream = await readSseUntilClose( + `http://127.0.0.1:${gatewayPort}/responses`, + { stream: true, test_force_terminate: true }, + ); + assert(terminatedStream.status === 502, `/responses 上游半路断流未返回 502: ${terminatedStream.status}`); -run().catch((error) => { - process.stderr.write(`${error?.stack || error}\n`); - process.exit(1); -}); + await new Promise((resolve) => setTimeout(resolve, 120)); + const logText = await readFile(logPath, "utf8"); + assert( + !logText.includes("[error] TypeError: terminated"), + "上游半路断流后不应记录 terminated error 日志", + ); + + process.stdout.write("PASS codex-retry-gateway e2e\n"); + } finally { + gateway.child.kill(); + upstream.close(); + await once(upstream, "close"); + await rm(tempRoot, { recursive: true, force: true }); + } +} + +run().catch((error) => { + process.stderr.write(`${error?.stack || error}\n`); + process.exit(1); +});