Compare commits
13
Commits
778068af42
...
a642022066
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a642022066 | ||
|
|
ec1f8ceb16 | ||
|
|
43097a22f1 | ||
|
|
1d7016bf65 | ||
|
|
ab0572baf6 | ||
|
|
2db86643c8 | ||
|
|
be22c84497 | ||
|
|
f1a755eebe | ||
|
|
8c21691478 | ||
|
|
57c7264c7b | ||
|
|
94bece3cf2 | ||
|
|
5f6fb3504d | ||
|
|
283c14b6b4 |
@@ -9,6 +9,8 @@ tg群:https://t.me/AI_INPUT_IM
|
||||
- 保持 Codex 继续使用现有 `auth.json`
|
||||
- 只把 `config.toml` 的当前 provider `base_url` 改成本地网关
|
||||
- 非流式命中 `reasoning_tokens = 516` 时返回 `502`
|
||||
- 上游若返回明确的容量错误(默认匹配错误文案 `Selected model is at capacity. Please try a different model.`,以及 `stream disconnected before completion: Concurrency limit exceeded for account, please retry later`),也会自动重试;重试耗尽后转成本地 `502`
|
||||
- 除了 `429/503` JSON 错误响应,也会识别 `200` 但返回体本质是错误、以及流式失败事件里携带同样文案的情况
|
||||
- 流式命中时默认先缓存并判断;一旦命中 `516`,统一返回 `502`
|
||||
- 默认同时拦截 root 路径和 `/v1` 路径:
|
||||
- `/responses`
|
||||
@@ -139,6 +141,8 @@ profile env 默认放在:
|
||||
- `CODEX_RETRY_GATEWAY_LISTEN_PORT`
|
||||
- `CODEX_RETRY_GATEWAY_UPSTREAM_BASE_URL`
|
||||
- `CODEX_RETRY_GATEWAY_REASONING_EQUALS`
|
||||
- `CODEX_RETRY_GATEWAY_RETRYABLE_STATUS_CODES`
|
||||
- `CODEX_RETRY_GATEWAY_RETRYABLE_ERROR_MESSAGES`
|
||||
- `CODEX_RETRY_GATEWAY_UPSTREAM_AUTH_MODE`
|
||||
- `CODEX_RETRY_GATEWAY_UPSTREAM_AUTH_ENV`
|
||||
- `CODEX_RETRY_GATEWAY_UPSTREAM_AUTH_FILE`
|
||||
@@ -159,6 +163,12 @@ profile env 默认放在:
|
||||
node ./scripts/run-profile.mjs default
|
||||
```
|
||||
|
||||
如果已经由 state 记录了当前活跃 profile,也可以直接不传 profile 参数:
|
||||
|
||||
```bash
|
||||
node ./scripts/run-profile.mjs --prefer-state-profile
|
||||
```
|
||||
|
||||
## 如何恢复
|
||||
|
||||
Windows:
|
||||
@@ -199,7 +209,7 @@ gateway 运行时只负责 API 与静态文件服务,不再把复杂 UI 硬写
|
||||
页面里可以直接做这几件事:
|
||||
|
||||
- 看当前监听地址、真实上游、当前 provider、当前 Codex base URL
|
||||
- 看本次 gateway 启动以来的实时统计
|
||||
- 看基于持久请求记录累计的实时统计
|
||||
- 代理请求总数
|
||||
- 被检查响应总数
|
||||
- 累计 input / output / total / reasoning tokens
|
||||
@@ -207,12 +217,14 @@ gateway 运行时只负责 API 与静态文件服务,不再把复杂 UI 硬写
|
||||
- `516` 占比
|
||||
- 看最近请求记录
|
||||
- 请求时间戳、首字耗时、总耗时、请求体大小、路径、模型、状态码
|
||||
- 相同重发请求会带相同的 `request_id`
|
||||
- `usage` 中的 input / output / total / reasoning tokens
|
||||
- 管理 profiles
|
||||
- 新建 / 编辑 profile env
|
||||
- 切换 provider `base_url`
|
||||
- 切换 `passthrough` / `manual_bearer` / `fixed_bearer` / `auth_json` 认证模式
|
||||
- 改 `reasoning_equals`
|
||||
- 改 capacity error 的匹配状态码和错误文案
|
||||
- 改 `endpoints`
|
||||
- 改 `non_stream_status_code`
|
||||
- 开关 `log_match`
|
||||
@@ -225,11 +237,11 @@ gateway 运行时只负责 API 与静态文件服务,不再把复杂 UI 硬写
|
||||
- 页面点“恢复 Codex 原设置并关闭网关”后,当前页面会失联,这是预期行为
|
||||
- 日常恢复优先用 UI;`restore-codex-config.ps1` 作为脚本级应急回滚入口保留
|
||||
- UI 恢复不会再额外拉起恢复子进程,而是由当前 gateway 直接完成恢复并退出
|
||||
- 统计口径默认按“本次 gateway 启动以来”累计
|
||||
- 统计口径默认按持久请求记录累计,重启后会从 `requests.sqlite` / `requests.jsonl` 回灌
|
||||
- `516` 占比 = `reasoning_tokens = 516` 的响应次数 / 被检查响应总数
|
||||
- 请求历史只记录元数据、请求体字节数和 token usage,不保存请求正文或响应正文;默认展示最近 200 条
|
||||
- gateway 日志持久化到 `~/.codex-retry-gateway/logs/gateway.log`
|
||||
- 请求记录持久化到 `~/.codex-retry-gateway/logs/requests.jsonl`,重启后仍可用于 UI 请求页和 token totals
|
||||
- 请求记录持久化到 `~/.codex-retry-gateway/logs/requests.jsonl` 与 `~/.codex-retry-gateway/logs/requests.sqlite`,重启后会用于 UI 请求页以及 overview 的累计 token / reasoning 统计
|
||||
- `manual_bearer` 的手动 token/password 只写入系统 secret 文件,API/UI 不读回明文;profile env 只保存 secret 文件路径
|
||||
- 其他 profile env 不保存明文 `sk-...`;固定密钥请使用 env/file 引用,或用 `auth_json` 指向 `auth.json` 字段名
|
||||
|
||||
@@ -246,6 +258,12 @@ macOS / Linux: ~/.codex-retry-gateway/config/config.json
|
||||
|
||||
- `reasoning_equals`
|
||||
- 例如 `[516]`
|
||||
- `retryable_status_codes`
|
||||
- 默认 `[429, 503]`
|
||||
- `retryable_error_messages`
|
||||
- 默认包含 `Selected model is at capacity. Please try a different model.`
|
||||
- 也默认包含 `stream disconnected before completion: Concurrency limit exceeded for account, please retry later`
|
||||
- 只要上游 JSON 错误里包含这些文案之一,gateway 就会把上游错误翻成本地 `non_stream_status_code`
|
||||
- `endpoints`
|
||||
- 默认包含 root 与 `/v1` 两套路径
|
||||
- `non_stream_status_code`
|
||||
|
||||
@@ -81,7 +81,7 @@ http://127.0.0.1:4610/__codex_retry_gateway/ui
|
||||
页面支持:
|
||||
|
||||
- 查看当前接管状态
|
||||
- 查看本次启动以来的实时日志
|
||||
- 查看跨重启保留的请求累计统计,以及当前进程的实时日志
|
||||
- 查看 `516` 命中次数与 `516` 占比
|
||||
- 热更新 `reasoning_equals` / `endpoints` / `non_stream_status_code` / `log_match`
|
||||
- 一键恢复 Codex 原设置并关闭 gateway
|
||||
|
||||
@@ -12,6 +12,11 @@
|
||||
"request_history_limit": 200,
|
||||
"endpoints": ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"],
|
||||
"reasoning_equals": [516],
|
||||
"retryable_status_codes": [429, 503],
|
||||
"retryable_error_messages": [
|
||||
"Selected model is at capacity. Please try a different model.",
|
||||
"stream disconnected before completion: Concurrency limit exceeded for account, please retry later"
|
||||
],
|
||||
"non_stream_status_code": 502,
|
||||
"stream_action": "strict_502",
|
||||
"log_match": true,
|
||||
|
||||
+1103
-121
File diff suppressed because it is too large
Load Diff
+32
-16
@@ -177,16 +177,27 @@ export function normalizeIntArray(values, fallback = [516]) {
|
||||
return normalized.length > 0 ? [...new Set(normalized)] : [...fallback];
|
||||
}
|
||||
|
||||
export function normalizeStringArray(values, fallback = []) {
|
||||
const source = values === undefined || values === null ? fallback : values;
|
||||
const queue = Array.isArray(source) ? source.flat(Infinity) : [source];
|
||||
const normalized = queue
|
||||
export function normalizeStringArray(values, fallback = []) {
|
||||
const source = values === undefined || values === null ? fallback : values;
|
||||
const queue = Array.isArray(source) ? source.flat(Infinity) : [source];
|
||||
const normalized = queue
|
||||
.flatMap((value) => `${value ?? ""}`.split(/[\s,]+/))
|
||||
.map((value) => value.trim())
|
||||
.filter(Boolean);
|
||||
|
||||
return normalized.length > 0 ? [...new Set(normalized)] : [...fallback];
|
||||
}
|
||||
|
||||
return normalized.length > 0 ? [...new Set(normalized)] : [...fallback];
|
||||
}
|
||||
|
||||
export function normalizePhraseArray(values, fallback = []) {
|
||||
const source = values === undefined || values === null ? fallback : values;
|
||||
const queue = Array.isArray(source) ? source.flat(Infinity) : [source];
|
||||
const normalized = queue
|
||||
.flatMap((value) => (typeof value === "string" ? value.split(/\r?\n/) : [value]))
|
||||
.map((value) => `${value ?? ""}`.trim())
|
||||
.filter(Boolean);
|
||||
|
||||
return normalized.length > 0 ? [...new Set(normalized)] : [...fallback];
|
||||
}
|
||||
|
||||
export function isProcessAlive(processId) {
|
||||
try {
|
||||
@@ -404,19 +415,24 @@ export async function installForCurrentProvider({
|
||||
}
|
||||
}
|
||||
|
||||
const gatewayConfig = {
|
||||
listen_host: listenHost,
|
||||
listen_port: listenPort,
|
||||
upstream_base_url: originalBaseUrl,
|
||||
const gatewayConfig = {
|
||||
listen_host: listenHost,
|
||||
listen_port: listenPort,
|
||||
upstream_base_url: originalBaseUrl,
|
||||
request_body_limit_bytes:
|
||||
existingGatewayConfig?.request_body_limit_bytes === undefined || existingGatewayConfig?.request_body_limit_bytes === null
|
||||
? 10485760
|
||||
: Number.parseInt(`${existingGatewayConfig.request_body_limit_bytes}`, 10),
|
||||
endpoints: mergedEndpoints,
|
||||
reasoning_equals: normalizeIntArray(existingGatewayConfig?.reasoning_equals, [516]),
|
||||
non_stream_status_code:
|
||||
existingGatewayConfig?.non_stream_status_code === undefined || existingGatewayConfig?.non_stream_status_code === null
|
||||
? 502
|
||||
endpoints: mergedEndpoints,
|
||||
reasoning_equals: normalizeIntArray(existingGatewayConfig?.reasoning_equals, [516]),
|
||||
retryable_status_codes: normalizeIntArray(existingGatewayConfig?.retryable_status_codes, [429, 503]),
|
||||
retryable_error_messages: normalizePhraseArray(existingGatewayConfig?.retryable_error_messages, [
|
||||
"Selected model is at capacity. Please try a different model.",
|
||||
"stream disconnected before completion: Concurrency limit exceeded for account, please retry later",
|
||||
]),
|
||||
non_stream_status_code:
|
||||
existingGatewayConfig?.non_stream_status_code === undefined || existingGatewayConfig?.non_stream_status_code === null
|
||||
? 502
|
||||
: Number.parseInt(`${existingGatewayConfig.non_stream_status_code}`, 10),
|
||||
stream_action: existingGatewayConfig?.stream_action || "strict_502",
|
||||
log_match: existingGatewayConfig?.log_match === undefined ? true : Boolean(existingGatewayConfig.log_match),
|
||||
|
||||
+28
-2
@@ -17,6 +17,7 @@ import {
|
||||
getGatewayBaseUrl,
|
||||
getGatewayStatePaths,
|
||||
normalizeIntArray,
|
||||
normalizePhraseArray,
|
||||
normalizeStringArray,
|
||||
parseOptions,
|
||||
readJsonFile,
|
||||
@@ -54,7 +55,15 @@ function parseEnvFile(content) {
|
||||
|
||||
function getProfileName(options) {
|
||||
const positional = options._?.[0];
|
||||
return `${options.profile || positional || process.env.CODEX_RETRY_GATEWAY_PROFILE || "default"}`;
|
||||
return `${options.profile || positional || process.env.CODEX_RETRY_GATEWAY_PROFILE || ""}`.trim();
|
||||
}
|
||||
|
||||
function resolveDefaultRequestedProfileName(existingState) {
|
||||
const stateProfileName = `${existingState?.profile_name || ""}`.trim();
|
||||
if (stateProfileName) {
|
||||
return stateProfileName;
|
||||
}
|
||||
return process.env.CODEX_RETRY_GATEWAY_DEFAULT_PROFILE || "default";
|
||||
}
|
||||
|
||||
function resolvePreferredProfileName({ requestedProfileName, existingState, preferStateProfile, profilesDir }) {
|
||||
@@ -185,6 +194,23 @@ function buildProfileConfig({ profileName, profileEnv, existingGatewayConfig, pr
|
||||
profileEnv.CODEX_RETRY_GATEWAY_REASONING_EQUALS || existingGatewayConfig?.reasoning_equals,
|
||||
[516],
|
||||
),
|
||||
retryable_status_codes: normalizeIntArray(
|
||||
profileEnv.CODEX_RETRY_GATEWAY_RETRYABLE_STATUS_CODES || existingGatewayConfig?.retryable_status_codes,
|
||||
[429, 503],
|
||||
),
|
||||
retryable_error_messages: normalizePhraseArray(
|
||||
profileEnv.CODEX_RETRY_GATEWAY_RETRYABLE_ERROR_MESSAGES || existingGatewayConfig?.retryable_error_messages,
|
||||
[
|
||||
"Selected model is at capacity. Please try a different model.",
|
||||
"stream disconnected before completion: Concurrency limit exceeded for account, please retry later",
|
||||
],
|
||||
),
|
||||
upstream_fetch_retry_attempts: profileEnv.CODEX_RETRY_GATEWAY_UPSTREAM_FETCH_RETRY_ATTEMPTS
|
||||
? Number.parseInt(`${profileEnv.CODEX_RETRY_GATEWAY_UPSTREAM_FETCH_RETRY_ATTEMPTS}`, 10)
|
||||
: Number.parseInt(`${existingGatewayConfig?.upstream_fetch_retry_attempts || 5}`, 10),
|
||||
upstream_fetch_retry_backoff_ms: profileEnv.CODEX_RETRY_GATEWAY_UPSTREAM_FETCH_RETRY_BACKOFF_MS
|
||||
? Number.parseInt(`${profileEnv.CODEX_RETRY_GATEWAY_UPSTREAM_FETCH_RETRY_BACKOFF_MS}`, 10)
|
||||
: Number.parseInt(`${existingGatewayConfig?.upstream_fetch_retry_backoff_ms || 350}`, 10),
|
||||
non_stream_status_code: profileEnv.CODEX_RETRY_GATEWAY_NON_STREAM_STATUS_CODE
|
||||
? Number.parseInt(`${profileEnv.CODEX_RETRY_GATEWAY_NON_STREAM_STATUS_CODE}`, 10)
|
||||
: Number.parseInt(`${existingGatewayConfig?.non_stream_status_code || 502}`, 10),
|
||||
@@ -231,7 +257,6 @@ async function ensureCodexPointsToGateway({ paths, codexConfigPath, providerCont
|
||||
|
||||
async function main() {
|
||||
const options = parseOptions(process.argv, { booleanFlags: ["no-codex-config-update", "prefer-state-profile"] });
|
||||
const requestedProfileName = getProfileName(options);
|
||||
const profilesDir = options.profilesDir || DEFAULT_PROFILES_DIR;
|
||||
const stateRoot = options.stateRoot || process.env.CODEX_RETRY_GATEWAY_STATE_ROOT || DEFAULT_STATE_ROOT;
|
||||
const codexConfigPath =
|
||||
@@ -246,6 +271,7 @@ async function main() {
|
||||
await ensureDirectory(paths.backupDir);
|
||||
|
||||
const existingState = await readJsonFile(paths.statePath);
|
||||
const requestedProfileName = getProfileName(options) || resolveDefaultRequestedProfileName(existingState);
|
||||
const profileName = resolvePreferredProfileName({
|
||||
requestedProfileName,
|
||||
existingState,
|
||||
|
||||
+445
-30
@@ -39,24 +39,24 @@ function createJsonResponse(res, statusCode, body, extraHeaders = {}) {
|
||||
res.end(JSON.stringify(body));
|
||||
}
|
||||
|
||||
function createSseResponse(res, chunks) {
|
||||
res.writeHead(200, {
|
||||
"content-type": "text/event-stream; charset=utf-8",
|
||||
"cache-control": "no-cache",
|
||||
connection: "keep-alive",
|
||||
function createSseResponse(res, chunks, intervalMs = 20) {
|
||||
res.writeHead(200, {
|
||||
"content-type": "text/event-stream; charset=utf-8",
|
||||
"cache-control": "no-cache",
|
||||
connection: "keep-alive",
|
||||
"x-upstream-test": "sse",
|
||||
});
|
||||
|
||||
let index = 0;
|
||||
const timer = setInterval(() => {
|
||||
if (index >= chunks.length) {
|
||||
clearInterval(timer);
|
||||
res.end();
|
||||
return;
|
||||
}
|
||||
res.write(chunks[index]);
|
||||
index += 1;
|
||||
}, 20);
|
||||
if (index >= chunks.length) {
|
||||
clearInterval(timer);
|
||||
res.end();
|
||||
return;
|
||||
}
|
||||
res.write(chunks[index]);
|
||||
index += 1;
|
||||
}, intervalMs);
|
||||
|
||||
res.on("close", () => {
|
||||
clearInterval(timer);
|
||||
@@ -80,8 +80,48 @@ function createTerminatedSseResponse(res, chunks, destroyDelayMs = 20) {
|
||||
}, destroyDelayMs);
|
||||
}
|
||||
|
||||
function buildCapacityStreamPayload(message, shape = "default") {
|
||||
if (shape === "response_failed") {
|
||||
return {
|
||||
type: "response.failed",
|
||||
response: {
|
||||
status: "failed",
|
||||
error: {
|
||||
message,
|
||||
type: "server_error",
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
return {
|
||||
error: {
|
||||
message,
|
||||
type: "server_error",
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function createCapacityErrorSseResponse(
|
||||
res,
|
||||
message = "Selected model is at capacity. Please try a different model.",
|
||||
intervalMs = 20,
|
||||
options = {},
|
||||
) {
|
||||
const eventName = options.eventName || "error";
|
||||
const payload = options.payload || buildCapacityStreamPayload(message, options.payloadShape || "default");
|
||||
createSseResponse(
|
||||
res,
|
||||
[
|
||||
`event: ${eventName}\n`,
|
||||
`data: ${JSON.stringify(payload)}\n\n`,
|
||||
],
|
||||
intervalMs,
|
||||
);
|
||||
}
|
||||
|
||||
function startFakeUpstream(port) {
|
||||
const failBeforeResponseCounts = new Map();
|
||||
const capacityBeforeSuccessCounts = new Map();
|
||||
const server = http.createServer((req, res) => {
|
||||
const responsePaths = new Set(["/responses", "/v1/responses"]);
|
||||
const chatCompletionPaths = new Set(["/chat/completions", "/v1/chat/completions"]);
|
||||
@@ -123,12 +163,79 @@ function startFakeUpstream(port) {
|
||||
]);
|
||||
return;
|
||||
}
|
||||
if (parsed.test_capacity_error) {
|
||||
const capacityMessage = parsed.test_capacity_message || "Selected model is at capacity. Please try a different model.";
|
||||
createJsonResponse(
|
||||
res,
|
||||
parsed.test_capacity_status ?? 503,
|
||||
{
|
||||
error: {
|
||||
message: capacityMessage,
|
||||
type: "server_error",
|
||||
},
|
||||
},
|
||||
{ "x-upstream-test": `capacity-error-${parsed.test_capacity_status ?? 503}` },
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (parsed.test_capacity_before_success_times) {
|
||||
const capacityKey = [
|
||||
req.url,
|
||||
"capacity-before-success",
|
||||
parsed.test_capacity_before_success_times,
|
||||
parsed.stream ? "stream" : "non-stream",
|
||||
parsed.test_capacity_status ?? "default",
|
||||
parsed.test_capacity_stream_event_name || "",
|
||||
parsed.test_capacity_stream_payload_shape || "",
|
||||
].join(":");
|
||||
const capacityCount = (capacityBeforeSuccessCounts.get(capacityKey) || 0) + 1;
|
||||
capacityBeforeSuccessCounts.set(capacityKey, capacityCount);
|
||||
if (capacityCount <= parsed.test_capacity_before_success_times) {
|
||||
const capacityMessage = parsed.test_capacity_message || "Selected model is at capacity. Please try a different model.";
|
||||
if (parsed.stream) {
|
||||
createCapacityErrorSseResponse(
|
||||
res,
|
||||
capacityMessage,
|
||||
20,
|
||||
{
|
||||
eventName: parsed.test_capacity_stream_event_name || "error",
|
||||
payloadShape: parsed.test_capacity_stream_payload_shape || "default",
|
||||
},
|
||||
);
|
||||
return;
|
||||
}
|
||||
createJsonResponse(
|
||||
res,
|
||||
parsed.test_capacity_status ?? 503,
|
||||
{
|
||||
error: {
|
||||
message: capacityMessage,
|
||||
type: "server_error",
|
||||
},
|
||||
},
|
||||
{ "x-upstream-test": `capacity-error-${parsed.test_capacity_status ?? 503}` },
|
||||
);
|
||||
return;
|
||||
}
|
||||
}
|
||||
if (parsed.stream && parsed.test_capacity_error) {
|
||||
createCapacityErrorSseResponse(
|
||||
res,
|
||||
parsed.test_capacity_message || "Selected model is at capacity. Please try a different model.",
|
||||
20,
|
||||
{
|
||||
eventName: parsed.test_capacity_stream_event_name || "error",
|
||||
payloadShape: parsed.test_capacity_stream_payload_shape || "default",
|
||||
},
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (parsed.stream) {
|
||||
createSseResponse(res, [
|
||||
'data: {"type":"response.output_text.delta","delta":"hello"}\n\n',
|
||||
`data: ${JSON.stringify({ type: "response.output_text.delta", delta: "hello", response_id: "resp_stream", thread_id: parsed.thread_id || "thread_stream" })}\n\n`,
|
||||
`data: {"response":{"usage":{"output_tokens_details":{"reasoning_tokens":${reasoning}}}}}\n\n`,
|
||||
"data: [DONE]\n\n",
|
||||
]);
|
||||
], parsed.test_stream_chunk_delay_ms ?? 20);
|
||||
return;
|
||||
}
|
||||
createJsonResponse(
|
||||
@@ -136,6 +243,7 @@ function startFakeUpstream(port) {
|
||||
200,
|
||||
{
|
||||
id: "resp_test",
|
||||
thread_id: parsed.thread_id || "thread_test",
|
||||
retry_attempt: parsed.test_fail_before_response_once
|
||||
? failBeforeResponseCounts.get(`${req.url}:fail-before-response-once`) || 0
|
||||
: 0,
|
||||
@@ -269,22 +377,29 @@ async function run() {
|
||||
const logPath = path.join(tempRoot, "gateway.log");
|
||||
|
||||
const config = {
|
||||
listen_host: "127.0.0.1",
|
||||
listen_port: gatewayPort,
|
||||
upstream_base_url: `http://127.0.0.1:${upstreamPort}`,
|
||||
request_body_limit_bytes: 10 * 1024 * 1024,
|
||||
endpoints: ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"],
|
||||
reasoning_equals: [516],
|
||||
non_stream_status_code: 502,
|
||||
listen_host: "127.0.0.1",
|
||||
listen_port: gatewayPort,
|
||||
upstream_base_url: `http://127.0.0.1:${upstreamPort}`,
|
||||
request_body_limit_bytes: 10 * 1024 * 1024,
|
||||
endpoints: ["/responses", "/chat/completions", "/v1/responses", "/v1/chat/completions"],
|
||||
reasoning_equals: [516],
|
||||
retryable_status_codes: [429, 503],
|
||||
retryable_error_messages: [
|
||||
"Selected model is at capacity. Please try a different model.",
|
||||
"stream disconnected before completion: Concurrency limit exceeded for account, please retry later",
|
||||
],
|
||||
upstream_fetch_retry_attempts: 5,
|
||||
upstream_fetch_retry_backoff_ms: 25,
|
||||
non_stream_status_code: 502,
|
||||
stream_action: "strict_502",
|
||||
log_match: true,
|
||||
health_path: "/__codex_retry_gateway/health",
|
||||
};
|
||||
log_match: true,
|
||||
health_path: "/__codex_retry_gateway/health",
|
||||
};
|
||||
|
||||
await writeFile(configPath, JSON.stringify(config, null, 2), "utf8");
|
||||
|
||||
const upstream = await startFakeUpstream(upstreamPort);
|
||||
const gateway = startGateway(configPath, logPath);
|
||||
const upstream = await startFakeUpstream(upstreamPort);
|
||||
let gateway = startGateway(configPath, logPath);
|
||||
|
||||
try {
|
||||
try {
|
||||
@@ -348,6 +463,256 @@ async function run() {
|
||||
recoveredEntry?.request_body_bytes === Buffer.byteLength(recoveredPayload),
|
||||
`请求体大小记录异常: ${recoveredEntry?.request_body_bytes}`,
|
||||
);
|
||||
assert(recoveredEntry?.request_id, "请求记录未生成 request_id");
|
||||
|
||||
const threadTrackedResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ test_reasoning_tokens: 128, thread_id: "thread_nonstream" }),
|
||||
});
|
||||
const threadTrackedBody = await threadTrackedResponse.json();
|
||||
assert(threadTrackedResponse.status === 200, `thread non-stream 请求失败: ${threadTrackedResponse.status}`);
|
||||
assert(threadTrackedBody?.id === "resp_test", "thread non-stream 返回体缺少 response id");
|
||||
assert(threadTrackedBody?.thread_id === "thread_nonstream", "thread non-stream 返回体缺少 thread_id");
|
||||
|
||||
const threadRequestsResponse = await fetch(`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/requests?query=${encodeURIComponent("thread_nonstream")}`);
|
||||
const threadRequestsPayload = await threadRequestsResponse.json();
|
||||
const threadEntry = (threadRequestsPayload?.entries || []).find((entry) => entry.thread_id === "thread_nonstream");
|
||||
assert(threadEntry?.response_id === "resp_test", "non-stream 请求记录未保留 response_id");
|
||||
assert(threadEntry?.thread_id === "thread_nonstream", "non-stream 请求记录未保留 thread_id");
|
||||
|
||||
const sameRequestPayload = JSON.stringify({ test_reasoning_tokens: 128, test_request_id_marker: "same" });
|
||||
const sameRequestFirstResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: sameRequestPayload,
|
||||
});
|
||||
assert(sameRequestFirstResponse.status === 200, `相同请求首次发送失败: ${sameRequestFirstResponse.status}`);
|
||||
await sameRequestFirstResponse.json();
|
||||
|
||||
const sameRequestSecondResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: sameRequestPayload,
|
||||
});
|
||||
assert(sameRequestSecondResponse.status === 200, `相同请求二次发送失败: ${sameRequestSecondResponse.status}`);
|
||||
await sameRequestSecondResponse.json();
|
||||
|
||||
const differentBodyPayload = JSON.stringify({ test_reasoning_tokens: 128, test_request_id_marker: "different" });
|
||||
const differentBodyResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: differentBodyPayload,
|
||||
});
|
||||
assert(differentBodyResponse.status === 200, `不同请求体发送失败: ${differentBodyResponse.status}`);
|
||||
await differentBodyResponse.json();
|
||||
|
||||
const differentPathResponse = await fetch(`http://127.0.0.1:${gatewayPort}/v1/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: sameRequestPayload,
|
||||
});
|
||||
assert(differentPathResponse.status === 200, `不同路径发送失败: ${differentPathResponse.status}`);
|
||||
await differentPathResponse.json();
|
||||
|
||||
const requestIdRequestsResponse = await fetch(`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/requests?limit=40`);
|
||||
const requestIdRequestsPayload = await requestIdRequestsResponse.json();
|
||||
const sameRequestEntries = (requestIdRequestsPayload?.entries || []).filter(
|
||||
(entry) => entry.path === "/responses" && entry.request_body_bytes === Buffer.byteLength(sameRequestPayload),
|
||||
);
|
||||
assert(sameRequestEntries.length >= 2, "未找到两条相同请求记录");
|
||||
const [sameRequestEntryA, sameRequestEntryB] = sameRequestEntries;
|
||||
assert(sameRequestEntryA.request_id, "相同请求记录缺少 request_id");
|
||||
assert(sameRequestEntryA.request_id === sameRequestEntryB.request_id, "相同请求未复用 request_id");
|
||||
assert(sameRequestEntryA.seq !== sameRequestEntryB.seq, "相同请求不应复用 seq");
|
||||
|
||||
const differentBodyEntry = (requestIdRequestsPayload?.entries || []).find(
|
||||
(entry) =>
|
||||
entry.path === "/responses" &&
|
||||
entry.request_body_bytes !== Buffer.byteLength(sameRequestPayload) &&
|
||||
entry.request_body_bytes === Buffer.byteLength(differentBodyPayload),
|
||||
);
|
||||
assert(differentBodyEntry?.request_id, "不同请求体记录缺少 request_id");
|
||||
assert(differentBodyEntry.request_id !== sameRequestEntryA.request_id, "不同请求体错误复用 request_id");
|
||||
|
||||
const differentPathEntry = (requestIdRequestsPayload?.entries || []).find(
|
||||
(entry) => entry.path === "/v1/responses" && entry.request_body_bytes === Buffer.byteLength(sameRequestPayload),
|
||||
);
|
||||
assert(differentPathEntry?.request_id, "不同路径记录缺少 request_id");
|
||||
assert(differentPathEntry.request_id !== sameRequestEntryA.request_id, "不同路径错误复用 request_id");
|
||||
|
||||
const requestIdQueryResponse = await fetch(
|
||||
`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/requests?query=${encodeURIComponent(sameRequestEntryA.request_id)}`,
|
||||
);
|
||||
const requestIdQueryPayload = await requestIdQueryResponse.json();
|
||||
assert(requestIdQueryResponse.status === 200, `request_id 搜索失败: ${requestIdQueryResponse.status}`);
|
||||
assert(
|
||||
(requestIdQueryPayload?.entries || []).some((entry) => entry.request_id === sameRequestEntryA.request_id),
|
||||
"request_id 搜索未命中对应记录",
|
||||
);
|
||||
|
||||
const capacityResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ test_capacity_error: true }),
|
||||
});
|
||||
const capacityBody = await capacityResponse.json();
|
||||
assert(capacityResponse.status === 502, `capacity error 未返回 502: ${capacityResponse.status}`);
|
||||
assert(
|
||||
capacityBody?.error?.code === "upstream_error_retry_triggered",
|
||||
"capacity error 返回体未标记 retry trigger",
|
||||
);
|
||||
assert(
|
||||
capacityBody?.error?.upstream_status_code === 503,
|
||||
"capacity error 返回体未保留 upstream status",
|
||||
);
|
||||
|
||||
const capacityStatus200Response = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ test_capacity_error: true, test_capacity_status: 200 }),
|
||||
});
|
||||
const capacityStatus200Body = await capacityStatus200Response.json();
|
||||
assert(capacityStatus200Response.status === 502, `200+capacity error 未返回 502: ${capacityStatus200Response.status}`);
|
||||
assert(
|
||||
capacityStatus200Body?.error?.code === "upstream_error_retry_triggered",
|
||||
"200+capacity error 返回体未标记 retry trigger",
|
||||
);
|
||||
assert(
|
||||
capacityStatus200Body?.error?.upstream_status_code === 200,
|
||||
"200+capacity error 返回体未保留 upstream status",
|
||||
);
|
||||
|
||||
const capacityRecoveredResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ test_capacity_before_success_times: 2, test_reasoning_tokens: 128 }),
|
||||
});
|
||||
const capacityRecoveredBody = await capacityRecoveredResponse.json();
|
||||
assert(capacityRecoveredResponse.status === 200, `capacity 抖动后未自动恢复: ${capacityRecoveredResponse.status}`);
|
||||
assert(
|
||||
capacityRecoveredBody?.usage?.output_tokens_details?.reasoning_tokens === 128,
|
||||
"capacity 抖动恢复后的返回体异常",
|
||||
);
|
||||
|
||||
const requestsAfterCapacityRecoveryResponse = await fetch(`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/requests?limit=20`);
|
||||
const requestsAfterCapacityRecovery = await requestsAfterCapacityRecoveryResponse.json();
|
||||
const capacityRecoveredEntry = requestsAfterCapacityRecovery?.entries?.find(
|
||||
(entry) => entry.path === "/responses" && entry.status_code === 200 && entry.upstream_attempt_count >= 3,
|
||||
);
|
||||
assert(capacityRecoveredEntry, "capacity 抖动恢复后的请求记录未保留重试次数");
|
||||
assert(capacityRecoveredEntry.request_id, "capacity 恢复后的请求记录缺少 request_id");
|
||||
|
||||
const capacityStatus200RecoveredResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ test_capacity_before_success_times: 2, test_capacity_status: 200, test_reasoning_tokens: 128 }),
|
||||
});
|
||||
const capacityStatus200RecoveredBody = await capacityStatus200RecoveredResponse.json();
|
||||
assert(capacityStatus200RecoveredResponse.status === 200, `200+capacity 抖动后未自动恢复: ${capacityStatus200RecoveredResponse.status}`);
|
||||
assert(
|
||||
capacityStatus200RecoveredBody?.usage?.output_tokens_details?.reasoning_tokens === 128,
|
||||
"200+capacity 抖动恢复后的返回体异常",
|
||||
);
|
||||
|
||||
const requestsAfterCapacityStatus200RecoveryResponse = await fetch(`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/requests?limit=30`);
|
||||
const requestsAfterCapacityStatus200Recovery = await requestsAfterCapacityStatus200RecoveryResponse.json();
|
||||
const capacityStatus200RecoveredEntry = requestsAfterCapacityStatus200Recovery?.entries?.find(
|
||||
(entry) => entry.path === "/responses" && entry.status_code === 200 && entry.upstream_attempt_count >= 3,
|
||||
);
|
||||
assert(capacityStatus200RecoveredEntry, "200+capacity 抖动恢复后的请求记录未保留重试次数");
|
||||
|
||||
const streamCapacityResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ stream: true, test_capacity_error: true }),
|
||||
});
|
||||
const streamCapacityBody = await streamCapacityResponse.json();
|
||||
assert(streamCapacityResponse.status === 502, `stream+capacity error 未返回 502: ${streamCapacityResponse.status}`);
|
||||
assert(
|
||||
streamCapacityBody?.error?.code === "upstream_error_retry_triggered",
|
||||
"stream+capacity error 返回体未标记 retry trigger",
|
||||
);
|
||||
|
||||
const streamCapacityResponseFailed = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
stream: true,
|
||||
test_capacity_error: true,
|
||||
test_capacity_stream_event_name: "response.failed",
|
||||
test_capacity_stream_payload_shape: "response_failed",
|
||||
}),
|
||||
});
|
||||
const streamCapacityResponseFailedBody = await streamCapacityResponseFailed.json();
|
||||
assert(streamCapacityResponseFailed.status === 502, `stream+response.failed capacity error 未返回 502: ${streamCapacityResponseFailed.status}`);
|
||||
assert(
|
||||
streamCapacityResponseFailedBody?.error?.code === "upstream_error_retry_triggered",
|
||||
"stream+response.failed capacity error 返回体未标记 retry trigger",
|
||||
);
|
||||
|
||||
const streamCapacityRecoveredResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ stream: true, test_capacity_before_success_times: 2, test_reasoning_tokens: 128 }),
|
||||
});
|
||||
const streamCapacityRecoveredText = await streamCapacityRecoveredResponse.text();
|
||||
assert(streamCapacityRecoveredResponse.status === 200, `stream capacity 抖动后未自动恢复: ${streamCapacityRecoveredResponse.status}`);
|
||||
assert(streamCapacityRecoveredText.includes("hello"), "stream capacity 恢复后未拿到正常 SSE 内容");
|
||||
|
||||
const requestsAfterStreamCapacityRecoveryResponse = await fetch(`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/requests?limit=20`);
|
||||
const requestsAfterStreamCapacityRecovery = await requestsAfterStreamCapacityRecoveryResponse.json();
|
||||
const streamCapacityRecoveredEntry = requestsAfterStreamCapacityRecovery?.entries?.find(
|
||||
(entry) => entry.path === "/responses" && entry.status_code === 200 && entry.response_stream && entry.upstream_attempt_count >= 3,
|
||||
);
|
||||
assert(streamCapacityRecoveredEntry, "stream capacity 抖动恢复后的请求记录未保留重试次数");
|
||||
|
||||
const streamCapacityResponseFailedRecoveredResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
stream: true,
|
||||
test_capacity_before_success_times: 2,
|
||||
test_reasoning_tokens: 128,
|
||||
test_capacity_stream_event_name: "response.failed",
|
||||
test_capacity_stream_payload_shape: "response_failed",
|
||||
}),
|
||||
});
|
||||
const streamCapacityResponseFailedRecoveredText = await streamCapacityResponseFailedRecoveredResponse.text();
|
||||
assert(streamCapacityResponseFailedRecoveredResponse.status === 200, `stream response.failed capacity 抖动后未自动恢复: ${streamCapacityResponseFailedRecoveredResponse.status}`);
|
||||
assert(streamCapacityResponseFailedRecoveredText.includes("hello"), "stream response.failed capacity 恢复后未拿到正常 SSE 内容");
|
||||
|
||||
const requestsAfterStreamCapacityResponseFailedRecoveryResponse = await fetch(`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/requests?limit=30`);
|
||||
const requestsAfterStreamCapacityResponseFailedRecovery = await requestsAfterStreamCapacityResponseFailedRecoveryResponse.json();
|
||||
const streamCapacityResponseFailedRecoveredEntry = requestsAfterStreamCapacityResponseFailedRecovery?.entries?.find(
|
||||
(entry) => entry.path === "/responses" && entry.status_code === 200 && entry.response_stream && entry.upstream_attempt_count >= 3,
|
||||
);
|
||||
assert(streamCapacityResponseFailedRecoveredEntry, "stream response.failed capacity 恢复后的请求记录未保留重试次数");
|
||||
|
||||
const streamThreadResponse = await readSseUntilClose(
|
||||
`http://127.0.0.1:${gatewayPort}/responses`,
|
||||
{ stream: true, test_reasoning_tokens: 128, thread_id: "thread_stream_ok" },
|
||||
);
|
||||
assert(streamThreadResponse.status === 200, `stream thread 请求失败: ${streamThreadResponse.status}`);
|
||||
const streamThreadRequestsResponse = await fetch(`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/requests?query=${encodeURIComponent("thread_stream_ok")}`);
|
||||
const streamThreadRequestsPayload = await streamThreadRequestsResponse.json();
|
||||
const streamThreadEntry = (streamThreadRequestsPayload?.entries || []).find((entry) => entry.thread_id === "thread_stream_ok");
|
||||
assert(streamThreadEntry?.response_id === "resp_stream", "stream 请求记录未保留 response_id");
|
||||
assert(streamThreadEntry?.thread_id === "thread_stream_ok", "stream 请求记录未保留 thread_id");
|
||||
|
||||
const streamDisconnectedRetryResponse = await fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
stream: true,
|
||||
test_capacity_before_success_times: 2,
|
||||
test_capacity_message: "stream disconnected before completion: Concurrency limit exceeded for account, please retry later",
|
||||
test_reasoning_tokens: 128,
|
||||
}),
|
||||
});
|
||||
const streamDisconnectedRetryText = await streamDisconnectedRetryResponse.text();
|
||||
assert(streamDisconnectedRetryResponse.status === 200, `stream disconnected capacity 抖动后未自动恢复: ${streamDisconnectedRetryResponse.status}`);
|
||||
assert(streamDisconnectedRetryText.includes("hello"), "stream disconnected capacity 恢复后未拿到正常 SSE 内容");
|
||||
|
||||
for (const streamPath of [
|
||||
"/responses",
|
||||
@@ -377,12 +742,62 @@ async function run() {
|
||||
assert(!okStream.closedByError, `${streamPath} 流式 128 不应异常断开`);
|
||||
}
|
||||
|
||||
const streamProgressPromise = fetch(`http://127.0.0.1:${gatewayPort}/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ stream: true, test_reasoning_tokens: 128, test_stream_chunk_delay_ms: 180 }),
|
||||
});
|
||||
await new Promise((resolve) => setTimeout(resolve, 260));
|
||||
const midRequestsResponse = await fetch(`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/requests?limit=20`);
|
||||
const midRequestsPayload = await midRequestsResponse.json();
|
||||
const inFlightStreamEntry = midRequestsPayload?.entries?.find(
|
||||
(entry) =>
|
||||
entry.path === "/responses" &&
|
||||
entry.response_stream === true &&
|
||||
(entry.lifecycle_state === "streaming" || entry.lifecycle_state === "receive_first") &&
|
||||
(entry.stream_chunk_count || 0) >= 1,
|
||||
);
|
||||
assert(inFlightStreamEntry, "流式请求过程中未暴露进行中状态");
|
||||
assert(
|
||||
(inFlightStreamEntry?.response_bytes_received || 0) > 0,
|
||||
"流式请求过程中未累计接收字节数",
|
||||
);
|
||||
const streamProgressResponse = await streamProgressPromise;
|
||||
assert(streamProgressResponse.status === 200, `stream progress 响应状态异常: ${streamProgressResponse.status}`);
|
||||
const streamReader = streamProgressResponse.body.getReader();
|
||||
while (true) {
|
||||
const { done } = await streamReader.read();
|
||||
if (done) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
const terminatedStream = await readSseUntilClose(
|
||||
`http://127.0.0.1:${gatewayPort}/responses`,
|
||||
{ stream: true, test_force_terminate: true },
|
||||
);
|
||||
assert(terminatedStream.status === 502, `/responses 上游半路断流未返回 502: ${terminatedStream.status}`);
|
||||
|
||||
const metricsBeforeRestartResponse = await fetch(`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/status`);
|
||||
const metricsBeforeRestart = await metricsBeforeRestartResponse.json();
|
||||
assert(metricsBeforeRestartResponse.status === 200, `status API 状态异常: ${metricsBeforeRestartResponse.status}`);
|
||||
assert(metricsBeforeRestart?.metrics?.reasoning_516_count >= 1, "重启前 reasoning_516_count 未累计");
|
||||
assert(metricsBeforeRestart?.metrics?.observed_reasoning_counts?.["128"] >= 1, "重启前 reasoning 128 未累计");
|
||||
assert(metricsBeforeRestart?.metrics?.total_proxy_request_count >= 1, "重启前 total_proxy_request_count 未累计");
|
||||
|
||||
gateway.child.kill();
|
||||
await once(gateway.child, "exit");
|
||||
gateway = startGateway(configPath, logPath);
|
||||
await waitForHealth(`http://127.0.0.1:${gatewayPort}${config.health_path}`);
|
||||
|
||||
const metricsAfterRestartResponse = await fetch(`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/status`);
|
||||
const metricsAfterRestart = await metricsAfterRestartResponse.json();
|
||||
assert(metricsAfterRestartResponse.status === 200, `重启后 status API 状态异常: ${metricsAfterRestartResponse.status}`);
|
||||
assert(metricsAfterRestart?.metrics?.reasoning_516_count >= metricsBeforeRestart?.metrics?.reasoning_516_count, "重启后 reasoning_516_count 未保留");
|
||||
assert(metricsAfterRestart?.metrics?.observed_reasoning_counts?.["128"] >= metricsBeforeRestart?.metrics?.observed_reasoning_counts?.["128"], "重启后 reasoning 128 计数未保留");
|
||||
assert(metricsAfterRestart?.metrics?.total_proxy_request_count >= metricsBeforeRestart?.metrics?.total_proxy_request_count, "重启后 total_proxy_request_count 未保留");
|
||||
assert(metricsAfterRestart?.metrics?.persistent_since, "重启后未返回 persistent_since");
|
||||
|
||||
await new Promise((resolve) => setTimeout(resolve, 120));
|
||||
const logText = await readFile(logPath, "utf8");
|
||||
assert(
|
||||
@@ -391,9 +806,9 @@ async function run() {
|
||||
);
|
||||
|
||||
process.stdout.write("PASS codex-retry-gateway e2e\n");
|
||||
} finally {
|
||||
gateway.child.kill();
|
||||
upstream.close();
|
||||
} finally {
|
||||
gateway.child.kill();
|
||||
upstream.close();
|
||||
await once(upstream, "close");
|
||||
await rm(tempRoot, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
@@ -221,6 +221,8 @@ async function run() {
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
reasoning_equals: [1024],
|
||||
retryable_status_codes: [429, 503, 529],
|
||||
retryable_error_messages: ["Selected model is at capacity. Please try a different model."],
|
||||
endpoints: ["/responses", "/v1/responses"],
|
||||
non_stream_status_code: 503,
|
||||
log_match: false,
|
||||
@@ -237,6 +239,10 @@ async function run() {
|
||||
JSON.stringify(updatedGatewayConfig.reasoning_equals) === JSON.stringify([1024]),
|
||||
"Saved config file did not persist reasoning_equals",
|
||||
);
|
||||
assert(
|
||||
JSON.stringify(updatedGatewayConfig.retryable_status_codes) === JSON.stringify([429, 503, 529]),
|
||||
"Saved config file did not persist retryable_status_codes",
|
||||
);
|
||||
|
||||
const incrementalLogsResponse = await fetch(
|
||||
`http://127.0.0.1:${gatewayPort}/__codex_retry_gateway/api/logs?since_seq=${logsPayload.latest_seq}`,
|
||||
|
||||
+328
-71
@@ -1,4 +1,4 @@
|
||||
import { FormEvent, useEffect, useState } from "react";
|
||||
import { FormEvent, startTransition, useEffect, useState } from "react";
|
||||
|
||||
type PageKey = "overview" | "requests" | "profiles" | "rules" | "logs";
|
||||
type Tone = "" | "success" | "error";
|
||||
@@ -15,12 +15,17 @@ type GatewayConfig = {
|
||||
model_remap?: string;
|
||||
endpoints?: string[];
|
||||
reasoning_equals?: number[];
|
||||
retryable_status_codes?: number[];
|
||||
retryable_error_messages?: string[];
|
||||
upstream_fetch_retry_attempts?: number;
|
||||
upstream_fetch_retry_backoff_ms?: number;
|
||||
non_stream_status_code?: number;
|
||||
log_match?: boolean;
|
||||
};
|
||||
|
||||
type Metrics = {
|
||||
started_at?: string;
|
||||
persistent_since?: string | null;
|
||||
total_proxy_request_count?: number;
|
||||
inspected_response_count?: number;
|
||||
matched_response_count?: number;
|
||||
@@ -34,6 +39,8 @@ type Metrics = {
|
||||
cached_tokens?: number;
|
||||
};
|
||||
observed_reasoning_counts?: Record<string, number>;
|
||||
observed_reasoning_counts_total_keys?: number;
|
||||
observed_reasoning_counts_omitted?: number;
|
||||
};
|
||||
|
||||
type StatusPayload = {
|
||||
@@ -63,20 +70,28 @@ type Usage = {
|
||||
|
||||
type RequestEntry = {
|
||||
seq: number;
|
||||
request_id?: string | null;
|
||||
response_id?: string | null;
|
||||
thread_id?: string | null;
|
||||
lifecycle_state?: string | null;
|
||||
started_at?: string;
|
||||
first_response_at?: string | null;
|
||||
first_response_delay_ms?: number | null;
|
||||
finished_at?: string | null;
|
||||
last_activity_at?: string | null;
|
||||
duration_ms?: number | null;
|
||||
profile_name?: string;
|
||||
method?: string;
|
||||
path?: string;
|
||||
request_body_bytes?: number | null;
|
||||
response_bytes_received?: number | null;
|
||||
model?: string | null;
|
||||
requested_model?: string | null;
|
||||
forwarded_model?: string | null;
|
||||
response_stream?: boolean;
|
||||
stream_chunk_count?: number | null;
|
||||
usage_last_updated_at?: string | null;
|
||||
upstream_attempt_count?: number | null;
|
||||
matched?: boolean;
|
||||
status_code?: number | null;
|
||||
upstream_status_code?: number | null;
|
||||
@@ -114,6 +129,10 @@ type ProfileFormModel = {
|
||||
request_history_limit?: string;
|
||||
model_remap?: string;
|
||||
reasoning_equals?: string;
|
||||
retryable_status_codes?: string;
|
||||
retryable_error_messages?: string[];
|
||||
upstream_fetch_retry_attempts?: string;
|
||||
upstream_fetch_retry_backoff_ms?: string;
|
||||
endpoints?: string[];
|
||||
};
|
||||
|
||||
@@ -179,11 +198,19 @@ type ProfileFormState = {
|
||||
request_history_limit: string;
|
||||
model_remap: string;
|
||||
reasoning_equals: string;
|
||||
retryable_status_codes: string;
|
||||
retryable_error_messages: string;
|
||||
upstream_fetch_retry_attempts: string;
|
||||
upstream_fetch_retry_backoff_ms: string;
|
||||
endpoints: string;
|
||||
};
|
||||
|
||||
type RuleFormState = {
|
||||
reasoning_equals: string;
|
||||
retryable_status_codes: string;
|
||||
retryable_error_messages: string;
|
||||
upstream_fetch_retry_attempts: string;
|
||||
upstream_fetch_retry_backoff_ms: string;
|
||||
endpoints: string;
|
||||
non_stream_status_code: string;
|
||||
log_match: boolean;
|
||||
@@ -200,6 +227,20 @@ const api = {
|
||||
restore: "/__codex_retry_gateway/api/restore",
|
||||
};
|
||||
|
||||
const REQUEST_PAGE_SIZE = 20;
|
||||
const LOG_PAGE_SIZE = 200;
|
||||
|
||||
const zhNumberFormatter = new Intl.NumberFormat("zh-CN");
|
||||
const zhTimestampFormatter = new Intl.DateTimeFormat("zh-CN", {
|
||||
year: "numeric",
|
||||
month: "2-digit",
|
||||
day: "2-digit",
|
||||
hour: "2-digit",
|
||||
minute: "2-digit",
|
||||
second: "2-digit",
|
||||
hourCycle: "h23",
|
||||
});
|
||||
|
||||
type ProfileProbePayload = {
|
||||
ok: boolean;
|
||||
profile?: string;
|
||||
@@ -222,7 +263,7 @@ type ProfileProbePayload = {
|
||||
const pageCopy: Record<PageKey, { title: string; subtitle: string; index: string }> = {
|
||||
overview: {
|
||||
title: "概览",
|
||||
subtitle: "当前 gateway 运行态和本次启动以来的累计统计。",
|
||||
subtitle: "当前 gateway 运行态,以及基于持久请求记录回灌的累计统计。",
|
||||
index: "01",
|
||||
},
|
||||
requests: {
|
||||
@@ -263,12 +304,16 @@ const defaultProfileForm: ProfileFormState = {
|
||||
request_history_limit: "0",
|
||||
model_remap: "",
|
||||
reasoning_equals: "516,1034,1552",
|
||||
retryable_status_codes: "429,503",
|
||||
retryable_error_messages: "Selected model is at capacity. Please try a different model.\nstream disconnected before completion: Concurrency limit exceeded for account, please retry later",
|
||||
upstream_fetch_retry_attempts: "5",
|
||||
upstream_fetch_retry_backoff_ms: "350",
|
||||
endpoints: "/responses\n/chat/completions\n/v1/responses\n/v1/chat/completions",
|
||||
};
|
||||
|
||||
function numberFormat(value: unknown) {
|
||||
return typeof value === "number" && Number.isFinite(value)
|
||||
? new Intl.NumberFormat("zh-CN").format(value)
|
||||
? zhNumberFormatter.format(value)
|
||||
: "-";
|
||||
}
|
||||
|
||||
@@ -277,13 +322,25 @@ function timestamp(value?: string | null) {
|
||||
return "-";
|
||||
}
|
||||
const date = new Date(value);
|
||||
return Number.isNaN(date.getTime()) ? value : date.toLocaleString("zh-CN", { hour12: false });
|
||||
return Number.isNaN(date.getTime()) ? value : zhTimestampFormatter.format(date);
|
||||
}
|
||||
|
||||
function durationSeconds(value?: number | null) {
|
||||
return typeof value === "number" && Number.isFinite(value) ? `${(value / 1000).toFixed(2)} s` : "-";
|
||||
}
|
||||
|
||||
function secondsSince(startedAt: string | null | undefined, updatedAt: string | null | undefined) {
|
||||
if (!startedAt || !updatedAt) {
|
||||
return "-";
|
||||
}
|
||||
const startedMs = new Date(startedAt).getTime();
|
||||
const updatedMs = new Date(updatedAt).getTime();
|
||||
if (Number.isNaN(startedMs) || Number.isNaN(updatedMs)) {
|
||||
return "-";
|
||||
}
|
||||
return `${Math.max(0, (updatedMs - startedMs) / 1000).toFixed(1)} s`;
|
||||
}
|
||||
|
||||
function bytesFormat(value?: number | null) {
|
||||
if (typeof value !== "number" || !Number.isFinite(value)) {
|
||||
return "-";
|
||||
@@ -327,6 +384,10 @@ function cachedRatio(inputTokens?: number | null, cachedTokens?: number | null)
|
||||
return Math.max(0, Math.min(1, cached / inputTokens));
|
||||
}
|
||||
|
||||
function requestPrimaryId(entry: RequestEntry) {
|
||||
return entry.response_id || entry.request_id || "-";
|
||||
}
|
||||
|
||||
function splitList(value: string) {
|
||||
return value
|
||||
.split(/[\s,]+/)
|
||||
@@ -368,14 +429,69 @@ function profileFormFromStatus(status: StatusPayload | null): ProfileFormState {
|
||||
reasoning_equals: Array.isArray(config.reasoning_equals)
|
||||
? config.reasoning_equals.join(",")
|
||||
: defaultProfileForm.reasoning_equals,
|
||||
retryable_status_codes: Array.isArray(config.retryable_status_codes)
|
||||
? config.retryable_status_codes.join(",")
|
||||
: defaultProfileForm.retryable_status_codes,
|
||||
retryable_error_messages: Array.isArray(config.retryable_error_messages)
|
||||
? config.retryable_error_messages.join("\n")
|
||||
: defaultProfileForm.retryable_error_messages,
|
||||
upstream_fetch_retry_attempts: String(
|
||||
config.upstream_fetch_retry_attempts ?? defaultProfileForm.upstream_fetch_retry_attempts,
|
||||
),
|
||||
upstream_fetch_retry_backoff_ms: String(
|
||||
config.upstream_fetch_retry_backoff_ms ?? defaultProfileForm.upstream_fetch_retry_backoff_ms,
|
||||
),
|
||||
endpoints: Array.isArray(config.endpoints) ? config.endpoints.join("\n") : defaultProfileForm.endpoints,
|
||||
};
|
||||
}
|
||||
|
||||
function profileFormFromProfile(profile: Profile): ProfileFormState {
|
||||
const form = profile.form || {};
|
||||
return {
|
||||
...defaultProfileForm,
|
||||
name: profile.name,
|
||||
listen_host: form.listen_host || "",
|
||||
listen_port: form.listen_port || "",
|
||||
upstream_base_url: form.upstream_base_url || "",
|
||||
auth_mode: (form.auth_mode as ProfileFormState["auth_mode"]) || "passthrough",
|
||||
auth_env: form.auth_env || "",
|
||||
auth_file: form.auth_file || "",
|
||||
manual_secret: "",
|
||||
manual_secret_file: form.manual_secret_file || "",
|
||||
manual_secret_configured: Boolean(form.manual_secret_configured),
|
||||
auth_json_path: form.auth_json_path || "",
|
||||
auth_json_key: form.auth_json_key || "OPENAI_API_KEY",
|
||||
request_history_limit: form.request_history_limit || defaultProfileForm.request_history_limit,
|
||||
model_remap: form.model_remap || "",
|
||||
reasoning_equals: form.reasoning_equals || "",
|
||||
retryable_status_codes: form.retryable_status_codes || defaultProfileForm.retryable_status_codes,
|
||||
retryable_error_messages: Array.isArray(form.retryable_error_messages)
|
||||
? form.retryable_error_messages.join("\n")
|
||||
: defaultProfileForm.retryable_error_messages,
|
||||
upstream_fetch_retry_attempts:
|
||||
form.upstream_fetch_retry_attempts || defaultProfileForm.upstream_fetch_retry_attempts,
|
||||
upstream_fetch_retry_backoff_ms:
|
||||
form.upstream_fetch_retry_backoff_ms || defaultProfileForm.upstream_fetch_retry_backoff_ms,
|
||||
endpoints: Array.isArray(form.endpoints) ? form.endpoints.join("\n") : "",
|
||||
};
|
||||
}
|
||||
|
||||
function ruleFormFromStatus(status: StatusPayload | null): RuleFormState {
|
||||
const config = status?.config || {};
|
||||
return {
|
||||
reasoning_equals: Array.isArray(config.reasoning_equals) ? config.reasoning_equals.join(", ") : "",
|
||||
retryable_status_codes: Array.isArray(config.retryable_status_codes)
|
||||
? config.retryable_status_codes.join(", ")
|
||||
: defaultProfileForm.retryable_status_codes,
|
||||
retryable_error_messages: Array.isArray(config.retryable_error_messages)
|
||||
? config.retryable_error_messages.join("\n")
|
||||
: defaultProfileForm.retryable_error_messages,
|
||||
upstream_fetch_retry_attempts: String(
|
||||
config.upstream_fetch_retry_attempts ?? defaultProfileForm.upstream_fetch_retry_attempts,
|
||||
),
|
||||
upstream_fetch_retry_backoff_ms: String(
|
||||
config.upstream_fetch_retry_backoff_ms ?? defaultProfileForm.upstream_fetch_retry_backoff_ms,
|
||||
),
|
||||
endpoints: Array.isArray(config.endpoints) ? config.endpoints.join("\n") : "",
|
||||
non_stream_status_code: String(config.non_stream_status_code || 502),
|
||||
log_match: Boolean(config.log_match),
|
||||
@@ -389,6 +505,7 @@ export default function App() {
|
||||
});
|
||||
const [status, setStatus] = useState<StatusPayload | null>(null);
|
||||
const [requests, setRequests] = useState<RequestEntry[]>([]);
|
||||
const [requestsTotal, setRequestsTotal] = useState(0);
|
||||
const [requestsMeta, setRequestsMeta] = useState("正在读取请求记录...");
|
||||
const [profiles, setProfiles] = useState<Profile[]>([]);
|
||||
const [profilesMeta, setProfilesMeta] = useState("正在读取 profiles...");
|
||||
@@ -397,6 +514,7 @@ export default function App() {
|
||||
const [latestLogSeq, setLatestLogSeq] = useState(0);
|
||||
const [requestQuery, setRequestQuery] = useState("");
|
||||
const [requestFilter, setRequestFilter] = useState("all");
|
||||
const [requestLimit, setRequestLimit] = useState(REQUEST_PAGE_SIZE);
|
||||
const [ruleForm, setRuleForm] = useState<RuleFormState>(ruleFormFromStatus(null));
|
||||
const [profileForm, setProfileForm] = useState<ProfileFormState>(defaultProfileForm);
|
||||
const [ruleMessage, setRuleMessage] = useState<MessageState>({ text: "", tone: "" });
|
||||
@@ -406,6 +524,7 @@ export default function App() {
|
||||
const [deletingProfile, setDeletingProfile] = useState("");
|
||||
const [switchingTo, setSwitchingTo] = useState("");
|
||||
const [restoreRequested, setRestoreRequested] = useState(false);
|
||||
const [shouldSyncProfileFormToActive, setShouldSyncProfileFormToActive] = useState(true);
|
||||
|
||||
const metrics = status?.metrics || {};
|
||||
const tokens = metrics.token_totals || {};
|
||||
@@ -419,16 +538,18 @@ export default function App() {
|
||||
|
||||
async function loadStatus(refreshRuleForm = false) {
|
||||
const payload = await fetchJson<StatusPayload>(api.status);
|
||||
setStatus(payload);
|
||||
if (refreshRuleForm) {
|
||||
setRuleForm(ruleFormFromStatus(payload));
|
||||
setProfileForm(profileFormFromStatus(payload));
|
||||
}
|
||||
startTransition(() => {
|
||||
setStatus(payload);
|
||||
if (refreshRuleForm) {
|
||||
setRuleForm(ruleFormFromStatus(payload));
|
||||
setProfileForm(profileFormFromStatus(payload));
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async function loadRequests() {
|
||||
async function loadRequests(limitOverride = requestLimit) {
|
||||
const url = new URL(api.requests, window.location.origin);
|
||||
url.searchParams.set("limit", "100");
|
||||
url.searchParams.set("limit", String(limitOverride));
|
||||
if (requestQuery.trim()) {
|
||||
url.searchParams.set("query", requestQuery.trim());
|
||||
}
|
||||
@@ -437,49 +558,84 @@ export default function App() {
|
||||
}
|
||||
const payload = await fetchJson<RequestsPayload>(url.toString());
|
||||
const entries = payload.entries || [];
|
||||
setRequests(entries);
|
||||
setRequestsMeta(`查询命中 ${payload.total_entries ?? entries.length} 条,最新序号 ${payload.latest_seq ?? 0}。`);
|
||||
startTransition(() => {
|
||||
setRequests(entries);
|
||||
setRequestsTotal(payload.total_entries ?? entries.length);
|
||||
setRequestsMeta(
|
||||
`查询命中 ${payload.total_entries ?? entries.length} 条,当前展示最近 ${entries.length} 条,最新序号 ${payload.latest_seq ?? 0}。`,
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
async function loadProfiles() {
|
||||
const payload = await fetchJson<ProfilesPayload>(api.profiles);
|
||||
const items = payload.profiles || [];
|
||||
setProfiles(items);
|
||||
setProfilesMeta(`目录:${payload.profiles_dir || "-"};当前运行:${payload.active_profile || "-"}。`);
|
||||
startTransition(() => {
|
||||
setProfiles(items);
|
||||
setProfilesMeta(`目录:${payload.profiles_dir || "-"};当前运行:${payload.active_profile || "-"}。`);
|
||||
});
|
||||
}
|
||||
|
||||
async function loadLogs(incremental = false) {
|
||||
const url = new URL(api.logs, window.location.origin);
|
||||
if (incremental && latestLogSeq > 0) {
|
||||
url.searchParams.set("since_seq", String(latestLogSeq));
|
||||
} else {
|
||||
url.searchParams.set("limit", String(LOG_PAGE_SIZE));
|
||||
}
|
||||
const payload = await fetchJson<LogsPayload>(url.toString());
|
||||
const rendered = (payload.entries || [])
|
||||
.map((entry) => `${entry.at || "-"} ${entry.message || ""}`)
|
||||
.join("\n");
|
||||
setLogs((current) => {
|
||||
if (!incremental || latestLogSeq === 0) {
|
||||
return rendered || "当前还没有日志。";
|
||||
startTransition(() => {
|
||||
setLogs((current) => {
|
||||
if (!incremental || latestLogSeq === 0) {
|
||||
return rendered || "当前还没有日志。";
|
||||
}
|
||||
return rendered ? `${current.trim()}\n${rendered}` : current;
|
||||
});
|
||||
setLogsMeta(`已载入 ${payload.total_entries ?? payload.entries?.length ?? 0} 条日志,最新序号 ${payload.latest_seq ?? latestLogSeq}。`);
|
||||
if (typeof payload.latest_seq === "number") {
|
||||
setLatestLogSeq(payload.latest_seq);
|
||||
}
|
||||
return rendered ? `${current.trim()}\n${rendered}` : current;
|
||||
});
|
||||
setLogsMeta(`已载入 ${payload.total_entries ?? payload.entries?.length ?? 0} 条日志,最新序号 ${payload.latest_seq ?? latestLogSeq}。`);
|
||||
if (typeof payload.latest_seq === "number") {
|
||||
setLatestLogSeq(payload.latest_seq);
|
||||
}
|
||||
|
||||
async function loadPageData(targetPage: PageKey, { incrementalLogs = true } = {}) {
|
||||
if (targetPage === "requests") {
|
||||
await loadRequests();
|
||||
return;
|
||||
}
|
||||
if (targetPage === "profiles") {
|
||||
await loadProfiles();
|
||||
return;
|
||||
}
|
||||
if (targetPage === "logs") {
|
||||
await loadLogs(incrementalLogs && latestLogSeq > 0);
|
||||
}
|
||||
}
|
||||
|
||||
async function refreshLiveData() {
|
||||
async function refreshLiveData(targetPage: PageKey = page) {
|
||||
if (restoreRequested || switchingTo) {
|
||||
return;
|
||||
}
|
||||
await Promise.all([loadStatus(false), loadRequests(), loadProfiles(), loadLogs(true)]);
|
||||
const tasks: Array<Promise<unknown>> = [loadStatus(false)];
|
||||
if (targetPage !== "overview" && targetPage !== "rules") {
|
||||
tasks.push(loadPageData(targetPage, { incrementalLogs: true }));
|
||||
}
|
||||
await Promise.all(tasks);
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
window.location.hash = page;
|
||||
}, [page]);
|
||||
|
||||
useEffect(() => {
|
||||
if (page === "profiles") {
|
||||
setShouldSyncProfileFormToActive(true);
|
||||
}
|
||||
}, [page]);
|
||||
|
||||
useEffect(() => {
|
||||
const onHashChange = () => {
|
||||
const next = window.location.hash.replace("#", "") as PageKey;
|
||||
@@ -492,19 +648,53 @@ export default function App() {
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
Promise.all([loadStatus(true), loadRequests(), loadProfiles(), loadLogs(false)]).catch((error) => {
|
||||
loadStatus(true).catch((error) => {
|
||||
setRuleMessage({ text: error?.message || String(error), tone: "error" });
|
||||
});
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
const timer = window.setInterval(() => {
|
||||
refreshLiveData().catch((error) => {
|
||||
setRuleMessage({ text: error?.message || String(error), tone: "error" });
|
||||
});
|
||||
}, 2500);
|
||||
const timer = window.setInterval(() => {
|
||||
loadStatus(false).catch((error) => {
|
||||
setRuleMessage({ text: error?.message || String(error), tone: "error" });
|
||||
});
|
||||
}, 10000);
|
||||
return () => window.clearInterval(timer);
|
||||
});
|
||||
}, [restoreRequested, switchingTo]);
|
||||
|
||||
useEffect(() => {
|
||||
if (page !== "requests" && page !== "logs") {
|
||||
return;
|
||||
}
|
||||
const timer = window.setInterval(() => {
|
||||
loadPageData(page, { incrementalLogs: true }).catch((error) => {
|
||||
setRuleMessage({ text: error?.message || String(error), tone: "error" });
|
||||
});
|
||||
}, 2500);
|
||||
return () => window.clearInterval(timer);
|
||||
}, [page, requestQuery, requestFilter, latestLogSeq, requestLimit]);
|
||||
|
||||
useEffect(() => {
|
||||
if (page === "overview" || page === "rules" || page === "requests") {
|
||||
return;
|
||||
}
|
||||
loadPageData(page, { incrementalLogs: false }).catch((error) => {
|
||||
setRuleMessage({ text: error?.message || String(error), tone: "error" });
|
||||
});
|
||||
}, [page]);
|
||||
|
||||
useEffect(() => {
|
||||
if (page !== "profiles" || !shouldSyncProfileFormToActive) {
|
||||
return;
|
||||
}
|
||||
const activeProfile = profiles.find((profile) => profile.active);
|
||||
if (!activeProfile) {
|
||||
return;
|
||||
}
|
||||
setProfileForm(profileFormFromProfile(activeProfile));
|
||||
setProfileMessage({ text: "", tone: "" });
|
||||
setShouldSyncProfileFormToActive(false);
|
||||
}, [page, profiles, shouldSyncProfileFormToActive]);
|
||||
|
||||
async function saveRules(event: FormEvent) {
|
||||
event.preventDefault();
|
||||
@@ -517,6 +707,12 @@ export default function App() {
|
||||
reasoning_equals: splitList(ruleForm.reasoning_equals)
|
||||
.map((value) => Number.parseInt(value, 10))
|
||||
.filter((value) => Number.isInteger(value)),
|
||||
retryable_status_codes: splitList(ruleForm.retryable_status_codes)
|
||||
.map((value) => Number.parseInt(value, 10))
|
||||
.filter((value) => Number.isInteger(value)),
|
||||
retryable_error_messages: splitLines(ruleForm.retryable_error_messages),
|
||||
upstream_fetch_retry_attempts: Number.parseInt(ruleForm.upstream_fetch_retry_attempts, 10),
|
||||
upstream_fetch_retry_backoff_ms: Number.parseInt(ruleForm.upstream_fetch_retry_backoff_ms, 10),
|
||||
endpoints: splitLines(ruleForm.endpoints),
|
||||
non_stream_status_code: Number.parseInt(ruleForm.non_stream_status_code, 10),
|
||||
log_match: ruleForm.log_match,
|
||||
@@ -524,7 +720,6 @@ export default function App() {
|
||||
});
|
||||
setStatus(payload);
|
||||
setRuleForm(ruleFormFromStatus(payload));
|
||||
await Promise.all([loadRequests(), loadLogs(false)]);
|
||||
setRuleMessage({ text: "配置已保存,并已对当前 gateway 立即生效。", tone: "success" });
|
||||
} catch (error) {
|
||||
setRuleMessage({ text: error instanceof Error ? error.message : String(error), tone: "error" });
|
||||
@@ -532,13 +727,20 @@ export default function App() {
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
setRequestLimit(REQUEST_PAGE_SIZE);
|
||||
}, [requestQuery, requestFilter]);
|
||||
|
||||
useEffect(() => {
|
||||
if (page !== "requests") {
|
||||
return;
|
||||
}
|
||||
const timer = window.setTimeout(() => {
|
||||
loadRequests().catch((error) => {
|
||||
loadRequests(requestLimit).catch((error) => {
|
||||
setRuleMessage({ text: error?.message || String(error), tone: "error" });
|
||||
});
|
||||
}, 180);
|
||||
return () => window.clearTimeout(timer);
|
||||
}, [requestQuery, requestFilter]);
|
||||
}, [page, requestQuery, requestFilter, requestLimit]);
|
||||
|
||||
async function saveProfile(event: FormEvent) {
|
||||
event.preventDefault();
|
||||
@@ -563,6 +765,10 @@ export default function App() {
|
||||
request_history_limit: Number.parseInt(profileForm.request_history_limit, 10),
|
||||
model_remap: profileForm.model_remap,
|
||||
reasoning_equals: splitList(profileForm.reasoning_equals),
|
||||
retryable_status_codes: splitList(profileForm.retryable_status_codes),
|
||||
retryable_error_messages: splitLines(profileForm.retryable_error_messages),
|
||||
upstream_fetch_retry_attempts: Number.parseInt(profileForm.upstream_fetch_retry_attempts, 10),
|
||||
upstream_fetch_retry_backoff_ms: Number.parseInt(profileForm.upstream_fetch_retry_backoff_ms, 10),
|
||||
endpoints: splitLines(profileForm.endpoints),
|
||||
}),
|
||||
});
|
||||
@@ -579,26 +785,8 @@ export default function App() {
|
||||
}
|
||||
|
||||
function editProfile(profile: Profile) {
|
||||
const form = profile.form || {};
|
||||
setProfileForm({
|
||||
...defaultProfileForm,
|
||||
name: profile.name,
|
||||
listen_host: form.listen_host || "",
|
||||
listen_port: form.listen_port || "",
|
||||
upstream_base_url: form.upstream_base_url || "",
|
||||
auth_mode: (form.auth_mode as ProfileFormState["auth_mode"]) || "passthrough",
|
||||
auth_env: form.auth_env || "",
|
||||
auth_file: form.auth_file || "",
|
||||
manual_secret: "",
|
||||
manual_secret_file: form.manual_secret_file || "",
|
||||
manual_secret_configured: Boolean(form.manual_secret_configured),
|
||||
auth_json_path: form.auth_json_path || "",
|
||||
auth_json_key: form.auth_json_key || "OPENAI_API_KEY",
|
||||
request_history_limit: form.request_history_limit || defaultProfileForm.request_history_limit,
|
||||
model_remap: form.model_remap || "",
|
||||
reasoning_equals: form.reasoning_equals || "",
|
||||
endpoints: Array.isArray(form.endpoints) ? form.endpoints.join("\n") : "",
|
||||
});
|
||||
setShouldSyncProfileFormToActive(false);
|
||||
setProfileForm(profileFormFromProfile(profile));
|
||||
setProfileMessage({ text: "编辑后保存 profile env;如果保存的是当前运行 profile,后端会直接后台热应用。", tone: "" });
|
||||
}
|
||||
|
||||
@@ -686,7 +874,8 @@ export default function App() {
|
||||
const payload = await fetchJson<StatusPayload>(api.status);
|
||||
if (payload.config?.profile_name === profileName) {
|
||||
setSwitchingTo("");
|
||||
Promise.all([loadStatus(true), loadRequests(), loadProfiles(), loadLogs(false)]).catch(() => {});
|
||||
setShouldSyncProfileFormToActive(true);
|
||||
Promise.all([loadStatus(true), loadProfiles()]).catch(() => {});
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
@@ -776,7 +965,7 @@ export default function App() {
|
||||
</div>
|
||||
</Card>
|
||||
|
||||
<Card title="Token Totals" hint="按本次 gateway 启动以来累计;后续可以接持久化统计。">
|
||||
<Card title="Token Totals" hint="按持久请求记录累计;重启后会从 requests history 回灌。">
|
||||
<div className="token-strip">
|
||||
<TokenCard label="Input" value={numberFormat(effectiveInputTotal ?? 0)} />
|
||||
<TokenCard label="Output" value={numberFormat(tokens.output_tokens || 0)} />
|
||||
@@ -807,7 +996,8 @@ export default function App() {
|
||||
rows={[
|
||||
["516 命中", numberFormat(metrics.reasoning_516_count || 0)],
|
||||
["516 占比", percent(metrics.reasoning_516_ratio || 0)],
|
||||
["启动时间", timestamp(metrics.started_at)],
|
||||
["累计起点", timestamp(metrics.persistent_since || metrics.started_at)],
|
||||
["本次启动", timestamp(metrics.started_at)],
|
||||
["Config", status?.paths?.config_path || "-"],
|
||||
["备份", status?.state?.latest_backup_path || "-"],
|
||||
]}
|
||||
@@ -816,11 +1006,18 @@ export default function App() {
|
||||
{reasoningChips.length === 0 ? (
|
||||
<span className="chip">还没有 reasoning 观测</span>
|
||||
) : (
|
||||
reasoningChips.map(([reasoning, count]) => (
|
||||
<span className="chip" key={reasoning}>
|
||||
reasoning {reasoning}: {count}
|
||||
</span>
|
||||
))
|
||||
<>
|
||||
{reasoningChips.map(([reasoning, count]) => (
|
||||
<span className="chip" key={reasoning}>
|
||||
reasoning {reasoning}: {count}
|
||||
</span>
|
||||
))}
|
||||
{(metrics.observed_reasoning_counts_omitted || 0) > 0 ? (
|
||||
<span className="chip">
|
||||
其余 {metrics.observed_reasoning_counts_omitted} 项未展开
|
||||
</span>
|
||||
) : null}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</Card>
|
||||
@@ -851,6 +1048,14 @@ export default function App() {
|
||||
<button className="secondary" type="button" onClick={() => loadRequests()}>
|
||||
刷新请求
|
||||
</button>
|
||||
<button
|
||||
className="ghost"
|
||||
type="button"
|
||||
disabled={requests.length >= requestsTotal && requestsTotal > 0}
|
||||
onClick={() => setRequestLimit((current) => current + REQUEST_PAGE_SIZE)}
|
||||
>
|
||||
更多
|
||||
</button>
|
||||
</div>
|
||||
}
|
||||
>
|
||||
@@ -884,10 +1089,20 @@ export default function App() {
|
||||
<span className="meta-key">总</span>
|
||||
{durationSeconds(entry.duration_ms)}
|
||||
</span>
|
||||
<span className="meta-pill meta-progress">
|
||||
<span className="meta-key">更</span>
|
||||
{timestamp(entry.last_activity_at || entry.usage_last_updated_at)}
|
||||
</span>
|
||||
<span className="meta-pill meta-body">
|
||||
<span className="meta-key">体</span>
|
||||
{bytesFormat(entry.request_body_bytes)}
|
||||
</span>
|
||||
<span className="meta-pill meta-stream">
|
||||
<span className="meta-key">流</span>
|
||||
{entry.response_stream
|
||||
? `${numberFormat(entry.stream_chunk_count)} chunk / ${bytesFormat(entry.response_bytes_received)} / ${secondsSince(entry.started_at, entry.last_activity_at || entry.usage_last_updated_at || entry.finished_at)}`
|
||||
: "-"}
|
||||
</span>
|
||||
<span className="meta-pill meta-received">
|
||||
<span className="meta-key">收</span>
|
||||
{timestamp(entry.finished_at)}
|
||||
@@ -896,6 +1111,8 @@ export default function App() {
|
||||
<div className="request-subtitle">
|
||||
<code>{`${entry.method || "-"} ${entry.path || "-"}`}</code>
|
||||
<span className="hint">{entry.response_stream ? "stream" : "non-stream"}</span>
|
||||
<span className="hint">id {requestPrimaryId(entry)}</span>
|
||||
<span className="hint">thread {entry.thread_id || "-"}</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="request-badges">
|
||||
@@ -922,18 +1139,29 @@ export default function App() {
|
||||
{upstream.auth_mode || "-"} / {upstream.auth_source || "-"}
|
||||
{upstream.authorization_configured === false ? " / no auth" : ""}
|
||||
</span>
|
||||
<span className="hint">
|
||||
重试 {numberFormat(entry.upstream_attempt_count)} 次
|
||||
{entry.upstream_status_code ? ` / upstream ${entry.upstream_status_code}` : ""}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="request-block">
|
||||
<label>Token</label>
|
||||
<div className="request-token-row">
|
||||
<span className="token-pill token-in">in {numberFormat(effectiveInput)}</span>
|
||||
<span className="token-pill token-out">out {numberFormat(usage.output_tokens)}</span>
|
||||
<span className="token-pill token-cached">cached {numberFormat(usage.cached_tokens)}{cachedHitRatio !== null ? ` (${percent(cachedHitRatio)})` : ""}</span>
|
||||
<span className="token-pill token-reasoning">reasoning {numberFormat(entry.reasoning_tokens ?? usage.reasoning_tokens)}</span>
|
||||
<span className="token-pill token-total">total {numberFormat(usage.total_tokens)}</span>
|
||||
<div className="request-block">
|
||||
<label>Token</label>
|
||||
<div className="request-token-row">
|
||||
<span className="token-pill token-in">in {numberFormat(effectiveInput)}</span>
|
||||
<span className="token-pill token-out">out {numberFormat(usage.output_tokens)}</span>
|
||||
<span className="token-pill token-cached">cached {numberFormat(usage.cached_tokens)}{cachedHitRatio !== null ? ` (${percent(cachedHitRatio)})` : ""}</span>
|
||||
<span className="token-pill token-reasoning">reasoning {numberFormat(entry.reasoning_tokens ?? usage.reasoning_tokens)}</span>
|
||||
<span className="token-pill token-total">total {numberFormat(usage.total_tokens)}</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="request-block">
|
||||
<label>IDs</label>
|
||||
<code>{`response ${entry.response_id || "-"}`}</code>
|
||||
<span className="hint">{`request ${entry.request_id || "-"}`}</span>
|
||||
<span className="hint">{`thread ${entry.thread_id || "-"}`}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</article>
|
||||
);
|
||||
@@ -955,6 +1183,7 @@ export default function App() {
|
||||
className="secondary"
|
||||
type="button"
|
||||
onClick={() => {
|
||||
setShouldSyncProfileFormToActive(false);
|
||||
setProfileForm(profileFormFromStatus(status));
|
||||
setProfileMessage({ text: "", tone: "" });
|
||||
}}
|
||||
@@ -1077,6 +1306,20 @@ export default function App() {
|
||||
<Field label="reasoning_equals">
|
||||
<input value={profileForm.reasoning_equals} placeholder="516,1034,1552" onChange={(event) => setProfileForm({ ...profileForm, reasoning_equals: event.target.value })} />
|
||||
</Field>
|
||||
<Field label="retryable_status_codes">
|
||||
<input value={profileForm.retryable_status_codes} placeholder="429,503" onChange={(event) => setProfileForm({ ...profileForm, retryable_status_codes: event.target.value })} />
|
||||
</Field>
|
||||
<Field label="retryable_error_messages" hint="每行一条错误文案;上游 JSON 错误包含任一行时会转成本地重试状态码。">
|
||||
<textarea value={profileForm.retryable_error_messages} onChange={(event) => setProfileForm({ ...profileForm, retryable_error_messages: event.target.value })} />
|
||||
</Field>
|
||||
<div className="field-row">
|
||||
<Field label="upstream_fetch_retry_attempts">
|
||||
<input type="number" min={1} value={profileForm.upstream_fetch_retry_attempts} onChange={(event) => setProfileForm({ ...profileForm, upstream_fetch_retry_attempts: event.target.value })} />
|
||||
</Field>
|
||||
<Field label="upstream_fetch_retry_backoff_ms">
|
||||
<input type="number" min={0} step={50} value={profileForm.upstream_fetch_retry_backoff_ms} onChange={(event) => setProfileForm({ ...profileForm, upstream_fetch_retry_backoff_ms: event.target.value })} />
|
||||
</Field>
|
||||
</div>
|
||||
<Field label="endpoints">
|
||||
<textarea value={profileForm.endpoints} onChange={(event) => setProfileForm({ ...profileForm, endpoints: event.target.value })} />
|
||||
</Field>
|
||||
@@ -1136,6 +1379,20 @@ export default function App() {
|
||||
<Field label="reasoning_equals">
|
||||
<input value={ruleForm.reasoning_equals} placeholder="例如:516, 1034, 1552" onChange={(event) => setRuleForm({ ...ruleForm, reasoning_equals: event.target.value })} />
|
||||
</Field>
|
||||
<Field label="retryable_status_codes">
|
||||
<input value={ruleForm.retryable_status_codes} placeholder="例如:429, 503" onChange={(event) => setRuleForm({ ...ruleForm, retryable_status_codes: event.target.value })} />
|
||||
</Field>
|
||||
<Field label="retryable_error_messages" hint="每行一条错误文案;命中后会转成本地 non_stream_status_code。">
|
||||
<textarea value={ruleForm.retryable_error_messages} onChange={(event) => setRuleForm({ ...ruleForm, retryable_error_messages: event.target.value })} />
|
||||
</Field>
|
||||
<div className="field-row">
|
||||
<Field label="upstream_fetch_retry_attempts">
|
||||
<input type="number" min={1} value={ruleForm.upstream_fetch_retry_attempts} onChange={(event) => setRuleForm({ ...ruleForm, upstream_fetch_retry_attempts: event.target.value })} />
|
||||
</Field>
|
||||
<Field label="upstream_fetch_retry_backoff_ms">
|
||||
<input type="number" min={0} step={50} value={ruleForm.upstream_fetch_retry_backoff_ms} onChange={(event) => setRuleForm({ ...ruleForm, upstream_fetch_retry_backoff_ms: event.target.value })} />
|
||||
</Field>
|
||||
</div>
|
||||
<Field label="endpoints">
|
||||
<textarea value={ruleForm.endpoints} onChange={(event) => setRuleForm({ ...ruleForm, endpoints: event.target.value })} />
|
||||
</Field>
|
||||
|
||||
@@ -602,11 +602,21 @@ a {
|
||||
background: #f6e3bd;
|
||||
}
|
||||
|
||||
.meta-progress {
|
||||
color: #4a4a86;
|
||||
background: #e4e1fb;
|
||||
}
|
||||
|
||||
.meta-body {
|
||||
color: #773f33;
|
||||
background: #f8ddd5;
|
||||
}
|
||||
|
||||
.meta-stream {
|
||||
color: #5d4f1f;
|
||||
background: #f2ecbf;
|
||||
}
|
||||
|
||||
.meta-received {
|
||||
color: #425268;
|
||||
background: #e5eaef;
|
||||
|
||||
Reference in New Issue
Block a user