ADR-006 v2.1: final revision, NVIDIA provider keys, reply to 徐聪

v2.1 changes from 2nd-round review: + Emergency channel RPM: max(1, max_rpm * 0.1) + Queue 503: add Retry-After: 30 header + sidecar_backup_success Prometheus metric + Startup crypto.py key validation on boot + SQLite size limits: 100MB practical, 500MB WAL + RPM flow: per-request counting, not token-based + SSE streaming: TTFT for avg_latency_ms + Merge proxy/retry.py into core/cooldown.py Added sidecar-v2-nvidia-providers.yaml (11 keys) Co-authored-by: multica-agent <github@multica.ai>
ADR-006 v2.0: Sidecar V2 architecture revision based on review feedback
2026-06-25 15:19:21 +08:00 · 2026-06-25 14:52:39 +08:00
27 changed files with 1579 additions and 4118 deletions
@@ -1,644 +0,0 @@
 # BIZ-46 Phase3: NVIDIA Sidecar Follow-up 架构设计
 > **架构师**: 梁思筑 (architect)  
 > **日期**: 2026-06-24  
 > **状态**: 已批准，推进实施  
 > **来源**: BIZ-42 Phase2 二轮评审 follow-up  
 ---
 ## 1. 架构解耦 / 依赖注入 — SidecarContext
 ### 1.1 现状分析
 当前 `server.py` 使用 **模块级全局变量** 管理所有核心组件：
 ```python
 # server.py 全局状态（当前）
 _config: SidecarConfig
 _http_client: httpx.AsyncClient
 _priority_queue: PriorityRequestQueue
 _token_bucket: AdaptiveTokenBucket
 _prometheus: PrometheusMetrics
 _health_service: HealthService
 _pending_requests: dict[str, tuple[asyncio.Future, float]]
 _stats: dict[str, int]
 _stats_lock: asyncio.Lock
 ```
 **问题**：
 - `webui.py` 通过 `from nvidia_sidecar import server` 反向导入全局变量（循环依赖风险）
 - 单元测试需要 mock 模块级变量，无法并行运行测试
 - 未来多实例/多租户扩展需重写全部模块访问逻辑
 ### 1.2 设计方案 — SidecarContext + FastAPI Dependency Injection
 #### 1.2.1 核心数据结构
 ```python
 # context.py
 from dataclasses import dataclass, field
 import asyncio
 import httpx
 from typing import Any
@dataclass
 class SidecarContext:
    """Sidecar 全局运行时上下文 — 所有核心组件的唯一容器。
    通过 app.state.sidecar 注入 FastAPI，路由通过 Depends 获取。
    """
    config: 'SidecarConfig'
    http_client: httpx.AsyncClient
    token_bucket: 'AdaptiveTokenBucket'
    priority_queue: 'PriorityRequestQueue'
    prometheus: 'PrometheusMetrics'
    health: 'HealthService'
    pending_requests: dict[str, tuple['asyncio.Future', float]] = field(default_factory=dict)
    stats: dict[str, int] = field(default_factory=lambda: {
        "total_requests": 0,
        "nvidia_requests": 0,
        "passthrough_requests": 0,
        "ratelimited_requests": 0,
        "queue_full_rejects": 0,
        "upstream_errors": 0,
        "start_time": 0,
    })
    stats_lock: asyncio.Lock = field(default_factory=asyncio.Lock)
    async def increment_stat(self, key: str, delta: int = 1) -> None:
        """线程安全的统计计数器自增。"""
        async with self.stats_lock:
            self.stats[key] = self.stats.get(key, 0) + delta
 ```
 #### 1.2.2 注入方式
 ```python
 # server.py — lifespan 中创建 context
 from nvidia_sidecar.context import SidecarContext
@asynccontextmanager
 async def lifespan(app: FastAPI):
    ctx = SidecarContext(
        config=load_config(),
        http_client=httpx.AsyncClient(...),
        token_bucket=AdaptiveTokenBucket(...),
        priority_queue=PriorityRequestQueue(...),
        prometheus=PrometheusMetrics(),
        health=HealthService(),
    )
    app.state.sidecar = ctx  # 注入 FastAPI
    # ... worker 启动 ...
    yield
    # ... 清理 ...
 # 依赖注入函数
 def get_context(request: Request) -> SidecarContext:
    return request.app.state.sidecar
 # 路由使用
@app.post("/v1/chat/completions")
 async def chat_completions(request: Request, ctx: SidecarContext = Depends(get_context)):
    return await _handle_proxy_request(request, "/v1/chat/completions", ctx)
 ```
 #### 1.2.3 webui.py 解耦
 ```python
 # webui.py — 不再反向导入 server
 from nvidia_sidecar.context import SidecarContext
 from fastapi import Depends
 def get_webui_router():
    router = APIRouter(prefix="/api", tags=["webui"])
    def _get_ctx(request: Request) -> SidecarContext:
        return request.app.state.sidecar
    @router.get("/dashboard/stream")
    async def dashboard_stream(request: Request, ctx: SidecarContext = Depends(_get_ctx)):
        return await _dashboard_stream(request, ctx)
    @router.get("/admin/config")
    async def admin_get_config(ctx: SidecarContext = Depends(_get_ctx)):
        return await get_config(ctx)
    return router
 ```
 #### 1.2.4 Trade-off 分析
 | 维度 | 当前（全局变量） | 方案A（SidecarContext） | 方案B（FastAPI Dependency 全函数式） |
 |------|------------------|------------------------|-------------------------------------|
 | 可测试性 | 差（需 mock 模块） | 好（注入 mock context） | 优（每个依赖独立注入） |
 | 改动量 | 无 | 中等（~8 文件） | 大（每个函数签名变更） |
 | 可读性 | 一般 | 好（ctx 一目了然） | 差（参数列表膨胀） |
 | 多实例支持 | 不支持 | 支持（多 app 多 ctx） | 支持 |
 | 循环依赖 | 有（webui→server） | 消除 | 消除 |
 **决策**: 采用方案A（SidecarContext），平衡改动量与收益。
 ### 1.3 迁移计划
 分 3 步渐进迁移，每步可独立合入：
 1. **Step 1**: 创建 `context.py`，定义 `SidecarContext`，在 `lifespan` 中实例化并挂到 `app.state`
 2. **Step 2**: 路由函数改为 `Depends(get_context)`，删除模块级 `_config`、`_http_client` 等
 3. **Step 3**: `webui.py` 移除 `from nvidia_sidecar import server`，改用依赖注入
 ---
 ## 2. Prometheus 标签基数治理
 ### 2.1 现状
 当前使用 `model_id` 作为 label 的指标：
 | 指标 | Label | 风险 |
 |------|-------|------|
 | `sidecar_upstream_latency_seconds` | `model_id` | **高** — NVIDIA 模型名含版本号，可能无界增长 |
 | `sidecar_upstream_errors_total` | `status_code`, `model_id` | **中** — 组合基数 = 模型数 × 状态码数 |
 ### 2.2 基数评估
 NVIDIA API 当前已知模型约 20-30 个，但：
 - 新模型持续发布（每月 2-5 个）
 - 模型名含版本后缀（`nvidia/deepseek-ai/deepseek-v4-pro`、`nvidia/llama-3.1-70b-instruct` 等）
 - 长期运行（6 个月+）可能累积 100+ 标签组合
 **结论**: 当前基数可控（<200 组合），但长期存在膨胀风险，应提前治理。
 ### 2.3 治理方案
 | 指标 | 当前 Label | 调整后 Label | 理由 |
 |------|-----------|-------------|------|
 | `upstream_latency_seconds` | `model_id` | `provider` | provider 固定为 `nvidia`，基数=1 |
 | `upstream_errors_total` | `status_code`, `model_id` | `status_code`, `provider` | 同上 |
 **模型级信息迁移路径**：
 - 模型 ID → 结构化 JSON 日志（structlog 已支持）
 - 需要模型级延迟分析时 → 临时 `/status` API 查询或日志聚合
 ```python
 # metrics.py 调整
 self.upstream_latency_seconds: Histogram = Histogram(
    "sidecar_upstream_latency_seconds",
    "Upstream response latency in seconds",
    labelnames=["provider"],  # 原: ["model_id"]
    buckets=(...),
 )
 self.upstream_errors_total: Counter = Counter(
    "sidecar_upstream_errors_total",
    "Upstream error count by status code",
    labelnames=["status_code", "provider"],  # 原: ["status_code", "model_id"]
 )
 ```
 ```python
 # server.py 调整 — 模型信息改记日志
 model_id = _extract_model(payload) or "unknown"
 provider = "nvidia"  # 固定值，因为只有 NVIDIA 请求走 worker
 _prometheus.record_upstream_latency(provider, upstream_latency)
 if not resp.is_success:
    _prometheus.record_upstream_error(resp.status_code, provider)
 logger.info("request_completed", model_id=model_id, ...)  # JSON 日志保留模型信息
 ```
 ### 2.4 Trade-off
 | 维度 | 保留 model_id | 收敛为 provider |
 |------|--------------|----------------|
 | 基数风险 | 高（无界） | 无（固定=1） |
 | 模型级分析 | Prometheus 原生查询 | 需日志聚合 |
 | 迁移成本 | 无 | 低（改 2 个指标定义 + 调用点） |
 **决策**: 收敛为 `provider`，模型级分析通过 JSON 日志 + 日志聚合系统（ELK/Loki）完成。
 ---
 ## 3. SSE 快照共享缓存
 ### 3.1 现状
 每个 SSE 客户端每秒独立调用 `_build_snapshot()`，该方法：
 - 获取 `_stats` 字典（需锁）
 - 调用 `_token_bucket.get_status()`（需锁）
 - 调用 `_priority_queue.get_stats()`（需 asyncio.Lock）
 当 N 个仪表盘同时打开时，每秒 N 次锁竞争 + N 次重复计算。
 ### 3.2 设计方案 — 1s TTL 共享缓存
 ```python
 # webui.py
 _snapshot_cache: tuple[dict[str, Any], float] | None = None  # (data, timestamp)
 _snapshot_lock: asyncio.Lock = asyncio.Lock()
 _SNAPSHOT_TTL: float = 1.0  # 1 秒 TTL
 async def _build_snapshot_cached(ctx: SidecarContext) -> dict[str, Any]:
    """带 1s TTL 的共享快照缓存。
    多个 SSE 客户端共享同一份快照，避免重复计算和锁竞争。
    """
    global _snapshot_cache
    now = time.monotonic()
    if _snapshot_cache is not None:
        data, ts = _snapshot_cache
        if now - ts < _SNAPSHOT_TTL:
            return data
    async with _snapshot_lock:
        # Double-check（避免多个协程同时 miss 后重复构建）
        if _snapshot_cache is not None:
            data, ts = _snapshot_cache
            if now - ts < _SNAPSHOT_TTL:
                return data
        snapshot = await _build_snapshot(ctx)
        _snapshot_cache = (snapshot, now)
        return snapshot
 ```
 ### 3.3 性能收益
 | 场景 | 当前 | 优化后 |
 |------|------|--------|
 | 1 客户端 | 1 次/s 计算 | 1 次/s 计算（无变化） |
 | 5 客户端 | 5 次/s 计算，5 次锁竞争 | 1 次/s 计算，1 次锁竞争 |
 | 20 客户端 | 20 次/s 计算，20 次锁竞争 | 1 次/s 计算，1 次锁竞争 |
 ---
 ## 4. 部署支撑
 ### 4.1 Dockerfile
 ```dockerfile
 # services/nvidia_sidecar/Dockerfile
 FROM python:3.12-slim AS base
 WORKDIR /app
 # 安装依赖（利用 Docker 层缓存）
 COPY pyproject.toml .
 RUN pip install --no-cache-dir -e .
 # 复制源码
 COPY . .
 # 非 root 用户运行
 RUN useradd -r -s /bin/false sidecar
 USER sidecar
 # 健康检查
 HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
    CMD python -c "import httpx; r=httpx.get('http://127.0.0.1:9190/health'); exit(0 if r.status_code==200 else 1)"
 EXPOSE 9190 9191
 CMD ["uvicorn", "nvidia_sidecar.server:app", "--host", "0.0.0.0", "--port", "9190"]
 ```
 ### 4.2 systemd Service
 ```ini
 # services/nvidia_sidecar/deploy/nvidia-sidecar.service
 [Unit]
 Description=NVIDIA Sidecar Rate-Limiting Proxy
 After=network-online.target
 Wants=network-online.target
 [Service]
 Type=simple
 User=sidecar
 Group=sidecar
 WorkingDirectory=/opt/nvidia-sidecar
 ExecStart=/opt/nvidia-sidecar/.venv/bin/uvicorn nvidia_sidecar.server:app \
    --host 127.0.0.1 \
    --port 9190 \
    --log-level info
 Restart=always
 RestartSec=5
 # 环境变量
 EnvironmentFile=/opt/nvidia-sidecar/.env
 # 安全加固
 NoNewPrivileges=true
 ProtectSystem=strict
 ProtectHome=true
 PrivateTmp=true
 ReadWritePaths=/opt/nvidia-sidecar/logs
 # 资源限制
 LimitNOFILE=65536
 MemoryMax=512M
 [Install]
 WantedBy=multi-user.target
 ```
 ### 4.3 环境变量清单
 | 变量 | 默认值 | 说明 |
 |------|--------|------|
 | `SIDECAR_HOST` | `127.0.0.1` | 监听地址 |
 | `SIDECAR_PORT` | `9190` | 代理端口 |
 | `SIDECAR_METRICS_PORT` | `9191` | Prometheus 指标端口 |
 | `SIDECAR_UPSTREAM` | `https://integrate.api.nvidia.com/v1` | 上游 API |
 | `SIDECAR_API_KEY` | (必填) | NVIDIA API Key |
 | `SIDECAR_RATE_RPM` | `40` | 限流速率 (RPM) |
 | `SIDECAR_BUCKET_CAPACITY` | `40` | 令牌桶容量 |
 | `SIDECAR_TIMEOUT` | `60` | 请求超时 (秒) |
 | `SIDECAR_QUEUE_MAX` | `500` | 队列最大容量 |
 | `SIDECAR_LOW_TIMEOUT` | `2` | 低优先级超时 (秒) |
 | `SIDECAR_FALLBACK_PASSTHROUGH` | `true` | 队列满时是否直通 |
 | `SIDECAR_LOG_LEVEL` | `INFO` | 日志级别 |
 | `SIDECAR_ADMIN_TOKEN` | (可选) | Admin API 认证 Token |
 ### 4.4 防火墙建议
 ```
 # 仅允许内网访问代理端口
 sudo ufw allow from 192.168.1.0/24 to any port 9190
 sudo ufw allow from 192.168.1.0/24 to any port 9191
 # 禁止外网访问
 sudo ufw deny 9190
 sudo ufw deny 9191
 ```
 ---
 ## 5. Readiness HTTP Client 复用
 ### 5.1 现状
 `HealthService.check_upstream()` 每次调用创建新的 `httpx.AsyncClient`：
 ```python
 # health.py — 当前
 async def check_upstream(self, upstream_url: str, timeout: float = 5.0, api_key: str = "") -> bool:
    async with httpx.AsyncClient(timeout=timeout) as client:  # 每次新建！
        resp = await client.get(...)
 ```
 K8s/systemd 每 10-30s 探测一次，每次创建+销毁 HTTP client 带来不必要的 TCP 连接开销。
 ### 5.2 方案 — 复用主 http_client
 ```python
 # health.py — 优化后
 async def check_upstream(
    self,
    upstream_url: str,
    http_client: httpx.AsyncClient,  # 注入主 client
    api_key: str = "",
    timeout: float = 5.0,
 ) -> bool:
    try:
        headers = {}
        if api_key:
            headers["authorization"] = f"Bearer {api_key}"
        resp = await http_client.get(
            f"{upstream_url.rstrip('/')}/v1/models",
            headers=headers,
            timeout=timeout,
        )
        return resp.status_code < 500
    except Exception:
        return False
 ```
 ```python
 # server.py — 路由调用处
@app.get("/health/ready")
 async def health_ready(ctx: SidecarContext = Depends(get_context)):
    queue_size = await ctx.priority_queue.get_queue_size()
    bucket_status = ctx.token_bucket.get_status()
    return await ctx.health.readiness(
        upstream_url=ctx.config.upstream_url,
        http_client=ctx.http_client,  # 复用主 client
        upstream_api_key=ctx.config.upstream_api_key or "",
        queue_current_size=queue_size,
        queue_max_size=ctx.config.queue_max_size,
        available_tokens=bucket_status["tokens"],
        bucket_capacity=bucket_status["capacity"],
    )
 ```
 **注意**: readiness 检查使用较短 timeout (5s)，不影响主代理请求的 timeout 配置。httpx 支持per-request timeout 覆盖。
 ---
 ## 6. Retreat 并发/死锁回归测试
 ### 6.1 风险点
 `AdaptiveTokenBucket` 有两把锁：
 - `_lock` (Lock): 保护令牌消费/补充
 - `_retreat_lock` (RLock): 保护避退状态机
 潜在死锁路径：
 1. `evaluate_retreat()` 持有 `_retreat_lock` → 调用 `get_429_rate()` (也获取 `_retreat_lock`，RLock 可重入 ✅)
 2. `evaluate_retreat()` → `_apply_retreat()` → `set_rate()` → 获取 `_lock` (另一把锁)
 3. Worker 线程: `consume()` 持有 `_lock` → 不调用 `_retreat_lock` (无交叉 ✅)
 当前设计使用 RLock 已规避了重入死锁，但需要回归测试确保未来修改不引入死锁。
 ### 6.2 测试用例
 ```python
 # tests/test_retreat_concurrency.py
 import pytest
 import asyncio
 import threading
 from nvidia_sidecar.rate_limiter import AdaptiveTokenBucket, RetreatState
 class TestRetreatConcurrency:
    """避退模式并发安全回归测试。"""
    @pytest.mark.asyncio
    async def test_concurrent_record_and_evaluate(self):
        """多线程同时 record_response + evaluate_retreat 不死锁。"""
        bucket = AdaptiveTokenBucket(rate=40/60, capacity=40)
        errors: list[Exception] = []
        def worker_record():
            for i in range(1000):
                try:
                    bucket.record_response(is_429=(i % 10 == 0))
                except Exception as e:
                    errors.append(e)
        def worker_evaluate():
            for _ in range(1000):
                try:
                    bucket.evaluate_retreat()
                except Exception as e:
                    errors.append(e)
        threads = [
            threading.Thread(target=worker_record),
            threading.Thread(target=worker_record),
            threading.Thread(target=worker_evaluate),
            threading.Thread(target=worker_evaluate),
        ]
        for t in threads:
            t.start()
        for t in threads:
            t.join(timeout=10)
        # 所有线程必须在 10s 内完成（无死锁）
        assert all(not t.is_alive() for t in threads), "线程未完成，疑似死锁"
        assert not errors, f"并发错误: {errors}"
    @pytest.mark.asyncio
    async def test_concurrent_consume_and_retreat(self):
        """多线程同时 consume + evaluate_retreat 不死锁。"""
        bucket = AdaptiveTokenBucket(rate=40/60, capacity=40)
        errors: list[Exception] = []
        def worker_consume():
            for _ in range(500):
                try:
                    bucket.consume(tokens=1)
                except Exception as e:
                    errors.append(e)
        def worker_retreat():
            for _ in range(500):
                try:
                    bucket.record_response(is_429=False)
                    bucket.evaluate_retreat()
                except Exception as e:
                    errors.append(e)
        threads = [
            threading.Thread(target=worker_consume),
            threading.Thread(target=worker_consume),
            threading.Thread(target=worker_retreat),
            threading.Thread(target=worker_retreat),
        ]
        for t in threads:
            t.start()
        for t in threads:
            t.join(timeout=10)
        assert all(not t.is_alive() for t in threads), "线程未完成，疑似死锁"
        assert not errors, f"并发错误: {errors}"
    def test_retreat_state_transitions_under_load(self):
        """高负载下避退状态转换正确。"""
        bucket = AdaptiveTokenBucket(
            rate=40/60, capacity=40,
            retreat_429_threshold=0.05,
            retreat_factor=0.75,
        )
        # 模拟高 429 率
        for _ in range(100):
            bucket.record_response(is_429=True)
        state = bucket.evaluate_retreat()
        assert state == RetreatState.RETREAT
        assert bucket.get_effective_rate_rpm() < bucket.get_base_rate_rpm()
        # 模拟恢复
        for _ in range(200):
            bucket.record_response(is_429=False)
        # 需要等待 RECOVER_WINDOW
        import time
        time.sleep(0.1)  # 确保时间窗口过去
        bucket._last_state_change = 0  # 强制触发时间条件
        state = bucket.evaluate_retreat()
        assert state in (RetreatState.RECOVER, RetreatState.NORMAL)
 ```
 ---
 ## 7. Dashboard UX 优化
 ### 7.1 优化项清单
 | # | 优化项 | 实现方式 | 优先级 |
 |---|--------|---------|--------|
 | 1 | 队列柱状图 300ms 平滑动画 | CSS `transition: height 300ms ease` | P1 |
 | 2 | SSE 断连 5s 遮罩 | JS 定时器 + DOM 遮罩层 | P1 |
 | 3 | 队列图标题显示总排队数 | SSE 数据已有 `current_size`，更新标题 | P2 |
 | 4 | 页面加载同步配置 | `fetch('/api/admin/config')` 初始化表单 | P2 |
 ### 7.2 关键实现
 ```javascript
 // dashboard.html — SSE 断连检测
 let lastSSETime = Date.now();
 let reconnectMask = document.getElementById('reconnect-mask');
 eventSource.onmessage = (event) => {
    lastSSETime = Date.now();
    reconnectMask.style.display = 'none';
    // ... 更新 UI ...
 };
 // 5s 无数据 → 显示遮罩
 setInterval(() => {
    if (Date.now() - lastSSETime > 5000) {
        reconnectMask.style.display = 'flex';
    }
 }, 1000);
 // 队列柱状图动画
 // CSS: .queue-bar { transition: height 0.3s ease; }
 ```
 ```javascript
 // 页面加载时同步配置
 async function loadConfig() {
    try {
        const resp = await fetch('/api/admin/config');
        if (resp.ok) {
            const config = await resp.json();
            document.getElementById('rate-rpm').value = config.rate_rpm;
            document.getElementById('queue-max').value = config.queue_max_size;
            // ...
        }
    } catch (e) {
        console.warn('配置加载失败（可能需要 Admin Token）', e);
    }
 }
 loadConfig();
 ```
 ---
 ## 8. 实施排期
 | 阶段 | 内容 | 预估工时 | 依赖 |
 |------|------|---------|------|
 | **D1** | SidecarContext Step 1-3（解耦迁移） | 8h | 无 |
 | **D2** | Prometheus 标签收敛 + 日志增强 | 2h | D1 |
 | **D2** | SSE 共享缓存 | 2h | D1 |
 | **D2** | Readiness HTTP client 复用 | 1h | D1 |
 | **D3** | Dockerfile + systemd service | 2h | 无 |
 | **D3** | Dashboard UX 优化 | 3h | 无 |
 | **D3** | Retreat 并发回归测试 | 3h | 无 |
 | **D4** | 集成测试 + mypy strict | 4h | D1-D3 |
 | **合计** | | **25h** | |
 ---
 ## 9. 验收标准映射
 | Issue 要求 | 本文档章节 | 状态 |
 |-----------|-----------|------|
 | SidecarContext / DI 方案落地或 ADR | §1 | ✅ 详细设计 + 迁移计划 |
 | Prometheus 高基数 label 收敛 | §2 | ✅ 收敛为 provider |
 | SSE snapshot 共享缓存 | §3 | ✅ 1s TTL 设计 |
 | Dockerfile + systemd + 部署 SOP | §4 | ✅ 完整文件 |
 | readiness 复用 HTTP client | §5 | ✅ 注入主 client |
 | retreat 并发/死锁回归测试 | §6 | ✅ 测试用例 |
 | Dashboard UX 细节 | §7 | ✅ 4 项优化 |
@@ -0,0 +1,58 @@
 sequenceDiagram
    participant OC as OpenClaw
    participant GW as API Gateway
    participant LB as 负载均衡器
    participant QM as 队列管理器
    participant RL as Rate Limiter
    participant P as Provider
    participant CD as Cooldown Detector
    participant ST as 统计引擎
    OC->>GW: POST /v1/chat/completions
    GW->>LB: 路由到目标池
    Note over LB: Weighted RR 5-10s刷新<br/>weight=(max_rpm-current_rpm)/max_rpm
    LB->>RL: BEGIN IMMEDIATE 事务 检查 RPM + 预占
    alt RPM 不足
        RL->>QM: 入队等待 超时30s
        QM-->>RL: 令牌可用
    end
    RL-->>LB: 允许转发
    LB->>P: 转发请求
    P-->>LB: 响应
    alt 200 OK
        LB->>ST: INSERT ON CONFLICT 记录 usage_logs
        LB-->>GW: 正常响应
    else 429 Too Many Requests
        LB->>CD: 上报429
        CD->>P: 移入冷却池 cooldown_until=now+30s×2^n
        LB->>LB: 重新选择 Provider B
        alt Provider B 正常
            LB->>P: 转发到 Provider B
            P-->>LB: 200 OK
        end
        alt 主池全部冷却
            Note over LB: 降级 Fallback 池<br/>检查即将恢复的Provider<br/>剩余<10s 等待
            alt Fallback 可用
                LB->>P: 转发 Fallback Provider
                P-->>LB: 200 OK +降级标记
            else Fallback 也全冷却
                LB->>P: 紧急通道 1 Provider 10% RPM
                alt 紧急通道成功
                    P-->>LB: 200 OK
                else
                    LB-->>OC: 503 Service Unavailable
                    OC->>OC: OpenClaw 自身 fallback
                end
            end
        end
    end
@@ -0,0 +1,71 @@
 erDiagram
    providers ||--o{ provider_usage_logs : has
    providers ||--o{ cooldown_events : triggers
    providers ||--o| provider_health : monitors
    providers {
        string id PK
        string name
        string api_key
        string endpoint_url
        string model_prefix
        string pool
        string status
        string source
        int rpm_limit
        int tpm_limit
        float weight
        float cost_per_1k
        string cooldown_until
        string metadata
    }
    provider_usage_logs {
        string id PK
        string provider_id FK
        string model
        int prompt_tokens
        int completion_tokens
        int total_tokens
        float cost
        int request_count
        int error_count
        int avg_latency_ms
        string hour_bucket
    }
    cooldown_events {
        string id PK
        string provider_id FK
        int consecutive_count
        int cooldown_seconds
        string response_summary
        string started_at
        string ended_at
    }
    provider_health {
        string provider_id PK
        string state
        int last_latency_ms
        int last_status_code
        float success_rate_5m
        int consecutive_failures
    }
    daily_stats {
        string id PK
        string date
        string pool
        int total_requests
        int total_errors
        int total_tokens
        float total_cost
        int unique_providers
    }
    system_config {
        string key PK
        string value
        string description
    }
@@ -0,0 +1,121 @@
 # NVIDIA Provider Keys Reference for Sidecar V2
 # =============================================
 # ⚠️ SECURITY: This file contains sensitive API key material.
 # In Sidecar V2 production deployment, API keys are stored as
 # AES-256-GCM ciphertext in SQLite (providers.api_key column).
 # The plaintext keys below are for V2 initial provisioning only.
 #
 # Usage: Import into Sidecar V2 via WebUI Admin or POST /api/v2/providers
 # After import, this file should be stored in a secure location
 # (Bitwarden / password manager) and NOT kept in plaintext on disk.
 #
 # Created: 2026-06-25 | By: 梁思筑 (architect)
 # Total providers: 11 | Pool: main | RPM each: 40 | Total RPM capacity: 440
 providers:
  - account: bizwings
    email: vincent@bizwingsinc.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: "主账号"
  - account: "98053"
    email: 98053@qq.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: ""
  - account: liuweicheng84
    email: liuweicheng84@gmail.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: ""
  - account: vx18088980513
    email: vx18088980513@qq.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: ""
  - account: "64391942"
    email: 64391942@qq.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: ""
  - account: cgtest1
    email: cgtest1@bizwingsinc.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: "测试账号1"
  - account: cgtest2
    email: cgtest2@bizwingsinc.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: "测试账号2"
  - account: "15876517651"
    email: 1248106918@qq.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: ""
  - account: "19584586741"
    email: 414133763@qq.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: ""
  - account: "18874954146"
    email: 350894172@qq.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: ""
  - account: "2405483110"
    email: 2405483110@qq.com
    api_key: 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
    endpoint_url: https://integrate.api.nvidia.com/v1
    model_prefix: "nvidia/"
    pool: main
    rpm_limit: 40
    notes: ""
 # Aggregated stats
 summary:
  total_providers: 11
  total_rpm_capacity: 440
  pools:
    main: 11
    fallback: 0
@@ -0,0 +1,58 @@
 flowchart TB
    subgraph OC["OpenClaw Gateway"]
        OC_SCHED["OpenClaw 调度器"]
        OC_FB["OpenClaw Fallback<br/>传统配置链路"]
    end
    subgraph SIDECAR["Sidecar V2 systemd/Docker"]
        direction TB
        subgraph ENTRY["入口层"]
            GW["API Gateway :9190<br/>FastAPI + 路由匹配"]
        end
        subgraph CORE["核心调度层"]
            LB["负载均衡器<br/>Weighted RR 5-10s刷新"]
            QM["队列管理器<br/>FIFO + 优先级<br/>容量500 + 溢出策略"]
        end
        subgraph POOLS["Provider 池层"]
            MP["主池 Main Pool"]
            FP["Fallback 池"]
            CP["冷却池<br/>Cooldown Pool"]
        end
        subgraph FLOW["流控层"]
            RL["Rate Limiter<br/>Per-Provider Token Bucket"]
            CD["Cooldown Detector<br/>429检测+指数退避<br/>+紧急通道10%RPM"]
        end
        subgraph STATS["存储与统计层"]
            MT["Metrics :9191<br/>Prometheus"]
            ST["统计引擎<br/>Token/费用/调用量"]
            DB[("SQLite WAL<br/>sidecar_v2.db<br/>+ cron备份")]
        end
        subgraph WEBUI["WebUI 层 :9190"]
            UI["Dashboard<br/>SSE 实时推送"]
            AP["Admin API<br/>Provider CRUD<br/>Bearer Token 鉴权"]
        end
    end
    OC_SCHED --> GW
    GW --> LB
    LB --> QM
    QM --> RL
    RL --> MP
    RL --> FP
    MP -.->|"429 触发冷却"| CP
    MP -->|"全部冷却"| FP
    FP -->|"全部冷却"| OC_FB
    CP -.->|"冷却结束恢复"| MP
    RL --> CD
    CD -.->|"紧急通道 10% RPM"| MP
    LB --> MT
    MT --> ST
    ST --> DB
    DB --> UI
    AP --> DB
@@ -1,3 +0,0 @@
 __pycache__/
 *.egg-info/
 .mypy_cache/
@@ -1,40 +0,0 @@
 # NVIDIA Sidecar 限流代理 — 生产 Docker 镜像 (BIZ-46 Phase3 §4)
 #
 # 构建：
 #   docker build -t nvidia-sidecar:latest .
 #
 # 运行：
 #   docker run -d --name nvidia-sidecar \
 #     -p 127.0.0.1:9190:9190 \
 #     -p 127.0.0.1:9191:9191 \
 #     -e SIDECAR_API_KEY="nvapi-xxx" \
 #     -e SIDECAR_RATE_RPM=40 \
 #     -v $(pwd)/logs:/opt/nvidia-sidecar/logs \
 #     nvidia-sidecar:latest
 FROM python:3.12-slim AS base
 WORKDIR /app
 # 安装依赖（利用 Docker 层缓存）
 COPY pyproject.toml .
 RUN pip install --no-cache-dir fastapi>=0.115 \
    "uvicorn[standard]>=0.34" httpx>=0.28 PyYAML>=6.0 \
    structlog>=24.4 "prometheus-client>=0.21" pydantic>=2.0
 # 复制源码
 COPY . .
 # 非 root 用户运行
 RUN useradd -r -m -s /bin/false sidecar \
    && mkdir -p /opt/nvidia-sidecar/logs \
    && chown -R sidecar:sidecar /app /opt/nvidia-sidecar/logs
 USER sidecar
 # 健康检查
 HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
    CMD python -c "import httpx; r=httpx.get('http://127.0.0.1:9190/health'); exit(0 if r.status_code==200 else 1)"
 EXPOSE 9190 9191
 CMD ["uvicorn", "nvidia_sidecar.server:app", "--host", "0.0.0.0", "--port", "9190"]
@@ -1,118 +0,0 @@
 # NVIDIA Sidecar 限流代理
 为 NVIDIA API 提供**优先级排队 + 令牌桶限流**的透明代理层。
 > BIZ-46 Phase3: 架构解耦、Prometheus 标签治理、SSE 共享缓存、部署支撑、测试完善、Dashboard UX 优化。
 ## 快速启动
 ```bash
 pip install .
 nvidia-sidecar
 ```
 监听 `127.0.0.1:9190`，代理到 NVIDIA API。
 ## 环境变量
 | 变量 | 默认值 | 说明 |
 |------|--------|------|
 | `SIDECAR_HOST` | `127.0.0.1` | 监听地址 |
 | `SIDECAR_PORT` | `9190` | 监听端口 |
 | `SIDECAR_METRICS_PORT` | `9191` | Metrics 端口 |
 | `SIDECAR_UPSTREAM` | `https://integrate.api.nvidia.com/v1` | 上游 API 地址 |
 | `SIDECAR_API_KEY` | — | NVIDIA API Key（必填） |
 | `SIDECAR_RATE_RPM` | `40` | 每分钟请求数限制 |
 | `SIDECAR_BUCKET_CAPACITY` | `40` | 令牌桶容量 |
 | `SIDECAR_TIMEOUT` | `60` | 上游请求超时（秒） |
 | `SIDECAR_QUEUE_MAX` | `500` | 队列最大长度 |
 | `SIDECAR_LOW_TIMEOUT` | `2.0` | 低优先级令牌等待超时（秒） |
 | `SIDECAR_FALLBACK_PASSTHROUGH` | `true` | 队列满时是否直通上游 |
 | `SIDECAR_LOG_LEVEL` | `INFO` | 日志级别 |
 ## YAML 配置
 ```yaml
 listen_port: 9292
 rate_rpm: 60
 upstream_api_key: "nvapi-xxx"
 ```
 ```bash
 nvidia-sidecar --config /etc/nvidia-sidecar.yaml
 ```
 ## API 端点
 | 路径 | 方法 | 说明 |
 |------|------|------|
 | `/v1/chat/completions` | POST | OpenAI Chat Completions 代理 |
 | `/v1/completions` | POST | OpenAI Completions 代理（legacy） |
 | `/v1/embeddings` | POST | OpenAI Embeddings 代理 |
 | `/v1/models` | GET | 模型列表代理 |
 | `/health` | GET | 存活检查 (liveness) |
 | `/health/ready` | GET | 就绪检查 (readiness，含上游连通性) |
 | `/status` | GET | 调试用完整状态（限流器 + 队列 + 避退） |
 | `/api/dashboard/stream` | GET | SSE 仪表盘实时推送 |
 | `/api/dashboard` | GET | 仪表盘 HTML 页面 |
 | `/api/admin/config` | GET/POST | 配置查询/热重载（需 Admin Token） |
 | `/metrics` | :9191 | Prometheus 指标端点（独立端口） |
 ## 部署方式
 ### Docker（推荐）
 ```bash
 # 构建
 docker build -t nvidia-sidecar:latest .
 # 运行
 docker run -d --name nvidia-sidecar \
  -p 127.0.0.1:9190:9190 \
  -p 127.0.0.1:9191:9191 \
  -e SIDECAR_API_KEY="nvapi-xxx" \
  nvidia-sidecar:latest
 ```
 ### systemd
 ```bash
 # 安装
 sudo cp deploy/nvidia-sidecar.service /etc/systemd/system/
 sudo systemctl daemon-reload
 sudo systemctl enable nvidia-sidecar
 # 配置环境变量
 sudo cp deploy/.env.example /opt/nvidia-sidecar/.env
 sudo vim /opt/nvidia-sidecar/.env  # 填入实际值
 # 启动
 sudo systemctl start nvidia-sidecar
 sudo journalctl -u nvidia-sidecar -f  # 查看日志
 ```
 ### 环境变量清单
 详见 `deploy/.env.example`。
 ### 防火墙建议
 ```bash
 # 仅允许内网访问代理端口
 sudo ufw allow from 192.168.1.0/24 to any port 9190
 sudo ufw allow from 192.168.1.0/24 to any port 9191
 # 禁止外网访问
 sudo ufw deny 9190
 sudo ufw deny 9191
 ```
 ## 架构
 ```
 请求 → 网关识别 → [NVIDIA: 优先级排队 → 令牌桶限流] → httpx → NVIDIA API
                → [非 NVIDIA: 直通] → httpx → 上游
 ```
 - **四级优先级**: URGENT > HIGH > NORMAL > LOW（通过 `X-Priority` header 指定）
 - **队列满策略**: PASSTHROUGH（直通）/ REJECT（503）/ DROP_LOWEST（丢弃最低优先级）
 - **令牌桶**: 40 RPM，线程安全，支持阻塞/非阻塞消费
@@ -1,41 +0,0 @@
 """
 NVIDIA Sidecar 限流代理 — 核心代理模块。
 为 OpenAI Chat Completions 兼容 API 提供四层防护：
    1. 请求接收（FastAPI）
    2. 网关识别 → 非 NVIDIA 直通
    3. 优先级排队 → 令牌桶限流
    4. httpx 异步转发到 NVIDIA 上游
 """
 from __future__ import annotations
 from nvidia_sidecar.config import SidecarConfig, load_config
 from nvidia_sidecar.rate_limiter import (
    Priority,
    TokenBucket,
    is_nvidia_gateway,
    normalize_gateway_name,
 )
 from nvidia_sidecar.priority_queue import (
    PriorityQueueItem,
    PriorityRequestQueue,
    QueueFullError,
    QueueFullPassthrough,
    QueueFullPolicy,
 )
 __version__ = "0.1.0"
 __all__ = [
    "SidecarConfig",
    "load_config",
    "Priority",
    "TokenBucket",
    "is_nvidia_gateway",
    "normalize_gateway_name",
    "PriorityQueueItem",
    "PriorityRequestQueue",
    "QueueFullError",
    "QueueFullPassthrough",
    "QueueFullPolicy",
 ]
@@ -1,221 +0,0 @@
 """
 NVIDIA Sidecar 限流代理 — 配置管理模块 (§3.1)
 集中管理 Sidecar 运行参数，支持环境变量覆盖和 YAML 配置文件。
 """
 from __future__ import annotations
 import os
 import warnings
 from dataclasses import dataclass, field
 from pathlib import Path
 from typing import Any
@dataclass
 class SidecarConfig:
    """Sidecar 运行配置数据类。
    所有字段可通过环境变量覆盖，优先级：环境变量 > YAML 配置文件 > 默认值。
    """
    # ---- 网络 ----
    listen_host: str = field(
        default="127.0.0.1",
        metadata={"env": "SIDECAR_HOST"},
    )
    listen_port: int = field(
        default=9190,
        metadata={"env": "SIDECAR_PORT"},
    )
    metrics_port: int = field(
        default=9191,
        metadata={"env": "SIDECAR_METRICS_PORT"},
    )
    # ---- 上游 ----
    upstream_url: str = field(
        default="https://integrate.api.nvidia.com/v1",
        metadata={"env": "SIDECAR_UPSTREAM"},
    )
    upstream_api_key: str = field(
        default="",
        metadata={"env": "SIDECAR_API_KEY"},
    )
    # ---- 限流 ----
    rate_rpm: int = field(
        default=40,
        metadata={"env": "SIDECAR_RATE_RPM"},
    )
    bucket_capacity: int = field(
        default=40,
        metadata={"env": "SIDECAR_BUCKET_CAPACITY"},
    )
    # ---- 超时 ----
    request_timeout: float = field(
        default=60.0,
        metadata={"env": "SIDECAR_TIMEOUT"},
    )
    # ---- 队列 ----
    queue_max_size: int = field(
        default=500,
        metadata={"env": "SIDECAR_QUEUE_MAX"},
    )
    low_priority_timeout: float = field(
        default=2.0,
        metadata={"env": "SIDECAR_LOW_TIMEOUT"},
    )
    # ---- 降级 ----
    fallback_enabled_passthrough: bool = field(
        default=True,
        metadata={"env": "SIDECAR_FALLBACK_PASSTHROUGH"},
    )
    # ---- 日志 ----
    log_level: str = field(
        default="INFO",
        metadata={"env": "SIDECAR_LOG_LEVEL"},
    )
 def _apply_env_overrides(config: SidecarConfig) -> SidecarConfig:
    """用环境变量覆盖配置字段。
    遍历 SidecarConfig 的 dataclass fields，对每个声明了 ``metadata={"env": ...}``
    的字段检查环境变量是否存在，存在则用对应类型转换后覆盖。
    """
    import dataclasses as _dc
    # 使用 typing.get_type_hints 解析 from __future__ import annotations
    # 引入的字符串化类型注解 (PEP 563)
    try:
        resolved_types = __import__("typing").get_type_hints(type(config))
    except Exception:
        resolved_types = {}
    for fld in _dc.fields(config):
        env_key: str | None = fld.metadata.get("env")
        if env_key is None:
            continue
        env_val = os.environ.get(env_key)
        if env_val is None:
            continue
        target_type = resolved_types.get(fld.name, fld.type)
        target_type_name: str = getattr(target_type, "__name__", str(target_type))
        try:
            if target_type is bool or target_type == "bool":
                parsed: bool = env_val.strip().lower() in ("true", "1", "yes", "on")
                setattr(config, fld.name, parsed)
            elif target_type is int or target_type == "int":
                setattr(config, fld.name, int(env_val))
            elif target_type is float or target_type == "float":
                setattr(config, fld.name, float(env_val))
            else:
                setattr(config, fld.name, env_val)
        except (ValueError, TypeError) as exc:
            warnings.warn(
                f"无法将环境变量 {env_key}={env_val!r} 转换为 {target_type_name}: {exc}"
            )
    return config
 def _validate_config(config: SidecarConfig) -> list[str]:
    """验证配置合理性，返回警告/问题列表。"""
    issues: list[str] = []
    # 端口冲突检查
    if config.listen_port == config.metrics_port:
        issues.append(
            f"listen_port ({config.listen_port}) 与 metrics_port ({config.metrics_port}) 相同"
        )
    # rate_rpm 边界检查
    if config.rate_rpm <= 0:
        issues.append(
            f"rate_rpm ({config.rate_rpm}) 无效，回退到默认值 40"
        )
        config.rate_rpm = 40
    # queue_max_size 合理性
    if config.queue_max_size <= 0:
        issues.append(
            f"queue_max_size ({config.queue_max_size}) 无效，回退到默认值 500"
        )
        config.queue_max_size = 500
    # request_timeout 合理性
    if config.request_timeout <= 0:
        issues.append(
            f"request_timeout ({config.request_timeout}) 无效，回退到默认值 60"
        )
        config.request_timeout = 60.0
    elif config.request_timeout > 300.0:
        issues.append(
            f"request_timeout ({config.request_timeout}) 异常偏高，已截断为 300"
        )
        config.request_timeout = 300.0
    return issues
 def load_config(path: str | None = None) -> SidecarConfig:
    """加载 Sidecar 配置。
    加载顺序（后者覆盖前者）：
    1. 默认值（SidecarConfig dataclass defaults）
    2. YAML 配置文件（如果 path 提供）
    3. 环境变量覆盖
    Args:
        path: 可选 YAML 配置文件路径。为 None 时只使用默认值 + 环境变量。
    Returns:
        经过验证的 SidecarConfig 实例。
    Raises:
        FileNotFoundError: path 指定的文件不存在。
        yaml.YAMLError: YAML 解析失败。
    """
    config = SidecarConfig()
    if path is not None:
        import yaml
        cfg_path = Path(path)
        if not cfg_path.is_file():
            raise FileNotFoundError(f"配置文件不存在: {cfg_path}")
        try:
            raw: dict[str, Any] = yaml.safe_load(cfg_path.read_text(encoding="utf-8")) or {}
        except yaml.YAMLError as exc:
            raise yaml.YAMLError(f"YAML 解析失败 ({cfg_path}): {exc}") from exc
        # 覆盖已声明的字段
        for fld_name in (
            "listen_host", "listen_port", "metrics_port",
            "upstream_url", "upstream_api_key",
            "rate_rpm", "bucket_capacity",
            "request_timeout",
            "queue_max_size", "low_priority_timeout",
            "fallback_enabled_passthrough",
            "log_level",
        ):
            if fld_name in raw:
                setattr(config, fld_name, raw[fld_name])
    # 环境变量覆盖（最高优先级）
    config = _apply_env_overrides(config)
    # 验证
    issues = _validate_config(config)
    for issue in issues:
        warnings.warn(issue)
    return config
@@ -1,75 +0,0 @@
 """
 NVIDIA Sidecar — SidecarContext 依赖注入容器 (§BIZ-46 Phase3)
 将所有模块级全局状态收敛为单一 dataclass，通过 FastAPI app.state 注入，
 消除 webui.py → server 的反向导入，支持可测试性和多实例扩展。
 设计文档: docs/architecture/BIZ-46_Phase3_Architecture_Design.md §1
 """
 from __future__ import annotations
 import asyncio
 import time
 from dataclasses import dataclass, field
 from typing import TYPE_CHECKING, Any
 import httpx
 if TYPE_CHECKING:
    from nvidia_sidecar.config import SidecarConfig
    from nvidia_sidecar.rate_limiter import AdaptiveTokenBucket
    from nvidia_sidecar.priority_queue import PriorityRequestQueue
    from nvidia_sidecar.metrics import PrometheusMetrics
    from nvidia_sidecar.health import HealthService
@dataclass
 class SidecarContext:
    """Sidecar 全局运行时上下文 — 所有核心组件的唯一容器。
    通过 ``app.state.sidecar`` 注入 FastAPI，路由通过 ``Depends(get_context)`` 获取。
    """
    # ---- 核心组件 ----
    config: SidecarConfig
    http_client: httpx.AsyncClient
    token_bucket: AdaptiveTokenBucket
    priority_queue: PriorityRequestQueue
    prometheus: PrometheusMetrics
    health: HealthService
    # ---- 运行时状态 ----
    pending_requests: dict[str, tuple["asyncio.Future[Any]", float]] = field(default_factory=dict)
    """request_id → (response future, enqueued_at) 的映射。"""
    stats: dict[str, int] = field(default_factory=lambda: {
        "total_requests": 0,
        "nvidia_requests": 0,
        "passthrough_requests": 0,
        "ratelimited_requests": 0,
        "queue_full_rejects": 0,
        "upstream_errors": 0,
        "start_time": 0,
    })
    stats_lock: asyncio.Lock = field(default_factory=asyncio.Lock)
    # ---- 缓存 ----
    snapshot_cache: tuple["dict[str, Any]", float] | None = None
    """SSE 快照共享缓存: (data, timestamp)。"""
    snapshot_cache_lock: asyncio.Lock = field(default_factory=asyncio.Lock)
    SNAPSHOT_CACHE_TTL: float = 1.0
    # ---- 便捷方法 ----
    async def increment_stat(self, key: str, delta: int = 1) -> None:
        """线程安全的统计计数器自增。"""
        async with self.stats_lock:
            self.stats[key] = self.stats.get(key, 0) + delta
    @property
    def uptime_seconds(self) -> int:
        """服务运行时长（秒）。"""
        st = self.stats.get("start_time", 0)
        return int(time.time() - st) if st else 0
@@ -1,31 +0,0 @@
 # NVIDIA Sidecar 环境变量清单 (BIZ-46 Phase3 §4)
 # 复制为 .env 后按需修改，供 Docker / systemd 使用。
 # 网络
 SIDECAR_HOST=127.0.0.1
 SIDECAR_PORT=9190
 SIDECAR_METRICS_PORT=9191
 # 上游 API（必填）
 SIDECAR_UPSTREAM=https://integrate.api.nvidia.com/v1
 SIDECAR_API_KEY=nvapi-your-key-here
 # 限流
 SIDECAR_RATE_RPM=40
 SIDECAR_BUCKET_CAPACITY=40
 # 超时
 SIDECAR_TIMEOUT=60
 # 队列
 SIDECAR_QUEUE_MAX=500
 SIDECAR_LOW_TIMEOUT=2
 # 降级
 SIDECAR_FALLBACK_PASSTHROUGH=true
 # 日志
 SIDECAR_LOG_LEVEL=INFO
 # Admin API 认证（可选，不设置则跳过认证）
 # SIDECAR_ADMIN_TOKEN=your-admin-token-here
@@ -1,49 +0,0 @@
 # NVIDIA Sidecar 限流代理 — systemd service (BIZ-46 Phase3 §4)
 #
 # 安装：
 #   sudo cp deploy/nvidia-sidecar.service /etc/systemd/system/
 #   sudo systemctl daemon-reload
 #   sudo systemctl enable nvidia-sidecar
 #   sudo systemctl start nvidia-sidecar
 #
 # 运维：
 #   sudo systemctl status nvidia-sidecar
 #   sudo journalctl -u nvidia-sidecar -f
 [Unit]
 Description=NVIDIA Sidecar Rate-Limiting Proxy
 Documentation=https://github.com/bizwings/nvidia-sidecar
 After=network-online.target
 Wants=network-online.target
 [Service]
 Type=simple
 User=sidecar
 Group=sidecar
 WorkingDirectory=/opt/nvidia-sidecar
 ExecStart=/opt/nvidia-sidecar/.venv/bin/uvicorn nvidia_sidecar.server:app \
    --host 127.0.0.1 \
    --port 9190 \
    --log-level info
 Restart=always
 RestartSec=5
 # 环境变量
 EnvironmentFile=/opt/nvidia-sidecar/.env
 # 安全加固
 NoNewPrivileges=true
 ProtectSystem=strict
 ProtectHome=true
 PrivateTmp=true
 ReadWritePaths=/opt/nvidia-sidecar/logs
 # 资源限制
 LimitNOFILE=65536
 MemoryMax=512M
 # 启动延迟（等待网络就绪）
 ExecStartPre=/bin/sleep 1
 [Install]
 WantedBy=multi-user.target
@@ -1,198 +0,0 @@
 """
 NVIDIA Sidecar 限流代理 — 健康检查端点 (§3.6)
 提供 Kubernetes / systemd 兼容的健康检查：
    GET /health       — 存活检查
    GET /health/ready — 就绪检查（含上游连通性）
 BIZ-46 Phase3: Readiness HTTP Client 复用 — 注入主 http_client，
 不再每次检查创建新 client，降低 K8s/systemd 高频探测的连接开销。
 """
 from __future__ import annotations
 import time
 from dataclasses import dataclass
 from typing import Any
 import httpx
@dataclass
 class HealthService:
    """健康检查服务。
    封装存活检查和就绪检查的逻辑，供 server.py 路由调用。
    """
    start_time: float = 0.0
    version: str = "0.1.0"
    def __post_init__(self) -> None:
        if self.start_time == 0.0:
            self.start_time = time.time()
    @property
    def uptime_seconds(self) -> float:
        """服务运行时长（秒）。"""
        return time.time() - self.start_time
    async def check_upstream(
        self,
        upstream_url: str,
        http_client: httpx.AsyncClient,
        timeout: float = 5.0,
        api_key: str = "",
    ) -> bool:
        """检查上游连通性（复用注入的 http_client，BIZ-46 Phase3）。
        Args:
            upstream_url: NVIDIA API base URL。
            http_client: 复用的 httpx.AsyncClient（来自 ctx）。
            timeout: 超时秒数（per-request override）。
            api_key: 可选的 API Key 用于认证。
        Returns:
            True 上游可达。
        """
        try:
            headers: dict[str, str] = {}
            if api_key:
                headers["authorization"] = f"Bearer {api_key}"
            resp = await http_client.get(
                f"{upstream_url.rstrip('/')}/v1/models",
                headers=headers,
                timeout=timeout,
            )
            return resp.status_code < 500
        except Exception:
            return False
    def check_queue_healthy(
        self,
        current_size: int,
        max_size: int,
        threshold_ratio: float = 0.9,
    ) -> bool:
        """检查队列是否健康（未接近满载）。
        Args:
            current_size: 当前队列长度。
            max_size: 队列最大容量。
            threshold_ratio: 告警阈值比例，默认 0.9。
        Returns:
            True 队列健康。
        """
        if max_size <= 0:
            return True
        return current_size < max_size * threshold_ratio
    def check_token_bucket_healthy(
        self,
        available_tokens: float,
        capacity: int,
        threshold: float = 0.05,
    ) -> bool:
        """检查令牌桶是否健康（token 未耗尽）。
        Args:
            available_tokens: 当前可用令牌数。
            capacity: 桶容量。
            threshold: 令牌数低于此比例视为不健康。
        Returns:
            True 令牌桶健康。
        """
        if capacity <= 0:
            return False
        return available_tokens > capacity * threshold
    def liveness(self) -> dict[str, Any]:
        """存活检查响应。
        Returns:
            liveness JSON payload。
        """
        return {
            "status": "ok",
            "uptime": round(self.uptime_seconds, 1),
            "version": self.version,
        }
    async def readiness(
        self,
        upstream_url: str,
        upstream_api_key: str = "",
        queue_current_size: int = 0,
        queue_max_size: int = 500,
        available_tokens: float = 0.0,
        bucket_capacity: int = 40,
        http_client: httpx.AsyncClient | None = None,
    ) -> dict[str, Any]:
        """就绪检查响应。
        Args:
            upstream_url: 上游 API 地址。
            upstream_api_key: API Key。
            queue_current_size: 当前队列长度。
            queue_max_size: 队列最大容量。
            available_tokens: 当前令牌数。
            bucket_capacity: 桶容量。
            http_client: 复用的 httpx.AsyncClient（BIZ-46 Phase3）。
                为 None 时回退到每次创建新 client（兼容旧调用）。
        Returns:
            readiness JSON payload。
        """
        if http_client is not None:
            upstream_ok = await self.check_upstream(
                upstream_url, http_client=http_client, api_key=upstream_api_key,
            )
        else:
            # 向后兼容：无 http_client 时沿用旧行为
            upstream_ok = await self.check_upstream_standalone(
                upstream_url, api_key=upstream_api_key,
            )
        queue_ok = self.check_queue_healthy(queue_current_size, queue_max_size)
        token_ok = self.check_token_bucket_healthy(available_tokens, bucket_capacity)
        all_ready = upstream_ok and queue_ok and token_ok
        return {
            "ready": all_ready,
            "upstream_reachable": upstream_ok,
            "queue_healthy": queue_ok,
            "token_bucket_healthy": token_ok,
        }
    async def check_upstream_standalone(
        self,
        upstream_url: str,
        timeout: float = 5.0,
        api_key: str = "",
    ) -> bool:
        """独立检查上游连通性（向后兼容，每次创建新 client）。
        Args:
            upstream_url: NVIDIA API base URL。
            timeout: 超时秒数。
            api_key: 可选的 API Key。
        Returns:
            True 上游可达。
        """
        try:
            headers: dict[str, str] = {}
            if api_key:
                headers["authorization"] = f"Bearer {api_key}"
            async with httpx.AsyncClient(timeout=timeout) as client:
                resp = await client.get(
                    f"{upstream_url.rstrip('/')}/v1/models",
                    headers=headers,
                )
                return resp.status_code < 500
        except Exception:
            return False
@@ -1,277 +0,0 @@
 """
 NVIDIA Sidecar 限流代理 — Prometheus 指标端点 (§3.5)
 10 个指标，独立端口 :9191，与代理端口 :9190 分离。
 BIZ-46 Phase3: Prometheus 标签基数治理 — model_id label 收敛为 provider。
 - upstream_latency_seconds: model_id → provider (固定值 "nvidia", 基数=1)
 - upstream_errors_total: model_id → provider
 - 模型级信息迁移到 structlog JSON 日志
 """
 from __future__ import annotations
 import time
 import threading
 from typing import Any
 from prometheus_client import (
    CollectorRegistry,
    Counter,
    Gauge,
    Histogram,
    generate_latest,
    make_asgi_app,
 )
 class PrometheusMetrics:
    """Sidecar Prometheus 指标收集器。
    线程安全，所有公开方法通过 ``threading.Lock`` 保护。
    """
    def __init__(self, registry: CollectorRegistry | None = None) -> None:
        """初始化所有 10 个 Prometheus 指标。
        Args:
            registry: 可选自定义 Registry；None 则使用默认全局 registry。
        """
        self._registry: CollectorRegistry = registry or CollectorRegistry()
        self._lock: threading.Lock = threading.Lock()
        self._start_time: float = time.time()
        # ---- 1. 总请求数（按优先级 + 状态分组） ----
        self.requests_total: Counter = Counter(
            "sidecar_requests_total",
            "Total requests processed by priority and status",
            labelnames=["priority", "status"],
            registry=self._registry,
        )
        # ---- 2. 可用令牌数 ----
        self.tokens_available: Gauge = Gauge(
            "sidecar_tokens_available",
            "Current number of available tokens",
            registry=self._registry,
        )
        # ---- 3. 令牌生成速率 ----
        self.tokens_rate: Gauge = Gauge(
            "sidecar_tokens_rate",
            "Current token generation rate (tokens per minute)",
            registry=self._registry,
        )
        # ---- 4. 各优先级队列深度 ----
        self.queue_depth: Gauge = Gauge(
            "sidecar_queue_depth",
            "Queue depth by priority",
            labelnames=["priority"],
            registry=self._registry,
        )
        # ---- 5. 队列等待时间 Histogram ----
        self.queue_latency_seconds: Histogram = Histogram(
            "sidecar_queue_latency_seconds",
            "Request wait time in queue in seconds",
            labelnames=["priority"],
            buckets=(0.01, 0.05, 0.1, 0.5, 1.0, 2.0, 5.0, 10.0, 30.0, 60.0, 120.0),
            registry=self._registry,
        )
        # ---- 6. 上游响应延迟 Histogram（label 收敛: model_id → provider） ----
        self.upstream_latency_seconds: Histogram = Histogram(
            "sidecar_upstream_latency_seconds",
            "Upstream response latency in seconds",
            labelnames=["provider"],  # BIZ-46: was ["model_id"], converged to fixed-cardinality provider
            buckets=(0.1, 0.5, 1.0, 2.0, 5.0, 10.0, 30.0, 60.0, 120.0, 300.0, 600.0),
            registry=self._registry,
        )
        # ---- 7. 上游错误计数（label 收敛: model_id → provider） ----
        self.upstream_errors_total: Counter = Counter(
            "sidecar_upstream_errors_total",
            "Upstream error count by status code and provider",
            labelnames=["status_code", "provider"],  # BIZ-46: was ["model_id"], converged
            registry=self._registry,
        )
        # ---- 8. 降级直通次数 ----
        self.fallback_passthrough_total: Counter = Counter(
            "sidecar_fallback_passthrough_total",
            "Total fallback / passthrough events (queue full or sidecar unavailable)",
            registry=self._registry,
        )
        # ---- 9. 健康状态 ----
        self.health_status: Gauge = Gauge(
            "sidecar_health_status",
            "Sidecar health: 0=unhealthy, 1=healthy",
            registry=self._registry,
        )
        # ---- 10. 运行时长 ----
        self.uptime_seconds: Gauge = Gauge(
            "sidecar_uptime_seconds",
            "Process uptime in seconds",
            registry=self._registry,
        )
        # 避退模式指标（附加，不计入基础 10 个）
        self.retreat_state: Gauge = Gauge(
            "sidecar_retreat_state",
            "Adaptive retreat state: 0=NORMAL, 1=RETREAT, 2=RECOVER",
            registry=self._registry,
        )
        self.effective_rate_rpm: Gauge = Gauge(
            "sidecar_effective_rate_rpm",
            "Current effective rate in RPM (after retreat adjustments)",
            registry=self._registry,
        )
        self.upstream_429_rate: Gauge = Gauge(
            "sidecar_upstream_429_rate",
            "Upstream 429 rate over the retreat observation window (0.0-1.0)",
            registry=self._registry,
        )
        # 初始化
        self.health_status.set(1)
    # ---- ASGI app 生成 ----
    def build_asgi_app(self) -> Any:
        """生成 Prometheus ASGI 应用，挂载到独立端口。
        Returns:
            可传给 uvicorn 的 ASGI app。
        """
        return make_asgi_app(registry=self._registry)
    # ---- 指标记录方法 ----
    def record_request(self, priority: str, status: str) -> None:
        """记录一次请求。
        Args:
            priority: 优先级名（URGENT / HIGH / NORMAL / LOW）。
            status: 状态（success / ratelimited / error）。
        """
        with self._lock:
            self.requests_total.labels(priority=priority, status=status).inc()
    def record_queue_latency(self, priority: str, seconds: float) -> None:
        """记录排队延迟。
        Args:
            priority: 优先级名。
            seconds: 排队等待秒数。
        """
        with self._lock:
            self.queue_latency_seconds.labels(priority=priority).observe(seconds)
    def record_upstream(self, status_code: int, provider: str) -> None:
        """记录上游响应（label 收敛: provider 替代 model_id，BIZ-46 Phase3）。
        Args:
            status_code: HTTP 状态码。
            provider: 上游提供商标识（固定 "nvidia"）。
        """
        with self._lock:
            self.upstream_latency_seconds.labels(provider=provider).observe(0.0)
    def record_upstream_error(self, status_code: int, provider: str) -> None:
        """记录上游错误（label 收敛: provider 替代 model_id，BIZ-46 Phase3）。
        Args:
            status_code: 错误 HTTP 状态码。
            provider: 上游提供商标识（固定 "nvidia"）。
        """
        with self._lock:
            self.upstream_errors_total.labels(
                status_code=str(status_code), provider=provider
            ).inc()
    def record_upstream_latency(self, provider: str, seconds: float) -> None:
        """记录上游响应延迟（label 收敛: provider 替代 model_id，BIZ-46 Phase3）。
        Args:
            provider: 上游提供商标识（固定 "nvidia"）。
            seconds: 响应延迟秒数。
        """
        with self._lock:
            self.upstream_latency_seconds.labels(provider=provider).observe(seconds)
    def update_token_status(self, tokens: float, rate_per_minute: float) -> None:
        """更新令牌桶状态。
        Args:
            tokens: 当前可用令牌数。
            rate_per_minute: 每分钟速率。
        """
        with self._lock:
            self.tokens_available.set(tokens)
            self.tokens_rate.set(rate_per_minute)
    def update_queue_depth(self, depths: dict[str, int]) -> None:
        """更新各优先级队列深度。
        Args:
            depths: {priority_name: count} 映射。
        """
        with self._lock:
            # 先清零所有已知标签再设置，避免残留旧值
            for pri in ("URGENT", "HIGH", "NORMAL", "LOW"):
                self.queue_depth.labels(priority=pri).set(depths.get(pri, 0))
    def increment_fallback(self) -> None:
        """降级直通计数 +1。"""
        with self._lock:
            self.fallback_passthrough_total.inc()
    def set_health(self, healthy: bool) -> None:
        """设置健康状态。
        Args:
            healthy: True=健康, False=不健康。
        """
        with self._lock:
            self.health_status.set(1 if healthy else 0)
    def update_uptime(self) -> None:
        """更新运行时长。"""
        with self._lock:
            self.uptime_seconds.set(time.time() - self._start_time)
    # ---- 避退模式指标 ----
    def update_retreat_metrics(
        self,
        retreat_state: str,
        effective_rate_rpm: float,
        upstream_429_rate: float,
    ) -> None:
        """更新避退模式指标。
        Args:
            retreat_state: "normal" / "retreat" / "recover".
            effective_rate_rpm: 当前实际速率 (RPM)。
            upstream_429_rate: 上游 429 率 (0.0-1.0)。
        """
        state_map: dict[str, int] = {"normal": 0, "retreat": 1, "recover": 2}
        with self._lock:
            self.retreat_state.set(state_map.get(retreat_state, 0))
            self.effective_rate_rpm.set(effective_rate_rpm)
            self.upstream_429_rate.set(upstream_429_rate)
    # ---- 导出 ----
    def generate_latest(self) -> bytes:
        """生成 Prometheus 文本格式的指标数据。
        Returns:
            Prometheus 文本格式 bytes。
        """
        with self._lock:
            self.update_uptime()
        return generate_latest(self._registry)
@@ -1,253 +0,0 @@
 """
 NVIDIA Sidecar 限流代理 — 四级优先级请求队列模块 (§3.3)
 管理待处理的 NVIDIA API 请求，按优先级 + FIFO 出队。
 支持三种队列满策略：PASSTHROUGH / REJECT / DROP_LOWEST。
 """
 from __future__ import annotations
 import asyncio
 import heapq
 import time
 import uuid
 from dataclasses import dataclass, field
 from enum import Enum
 from typing import Any
 from nvidia_sidecar.rate_limiter import Priority
 # ---------------------------------------------------------------------------
 # 队列满策略
 # ---------------------------------------------------------------------------
 class QueueFullPolicy(str, Enum):
    """队列满时的处理策略。"""
    PASSTHROUGH = "passthrough"   # 直通上游，绕过排队（fail-open 子策略）
    REJECT = "reject"             # 返回 503 Service Unavailable
    DROP_LOWEST = "drop_lowest"   # 丢弃队列中最低优先级元素，插入新请求
 # ---------------------------------------------------------------------------
 # 队列元素
 # ---------------------------------------------------------------------------
@dataclass(order=True)
 class PriorityQueueItem:
    """优先级队列元素。
    ``sort_index`` 由 ``(priority, timestamp)`` 组成，
    Python 的 ``__lt__`` 按字段顺序比较：先比 priority，再比 timestamp。
    数值越小越优先（URGENT=1 优于 HIGH=2）。
    """
    sort_index: tuple[int, float] = field(compare=True)
    priority: Priority = field(compare=False)
    request_id: str = field(compare=False)
    payload: dict[str, Any] = field(compare=False)
    enqueued_at: float = field(compare=False)
    headers: dict[str, str] = field(default_factory=dict, compare=False)
 # ---------------------------------------------------------------------------
 # 优先级请求队列
 # ---------------------------------------------------------------------------
 class QueueFullError(Exception):
    """队列已满且策略为 REJECT 时抛出。"""
    pass
 class QueueFullPassthrough(Exception):
    """队列已满且策略为 PASSTHROUGH 时抛出，由调用方绕过队列直通上游。"""
    pass
 class PriorityRequestQueue:
    """异步线程安全的四级优先级请求队列。
    内部使用 ``asyncio.Lock`` 保护并发操作，
    基于 ``heapq`` + ``asyncio.Event`` 实现阻塞出队。
    """
    def __init__(self, max_size: int = 500) -> None:
        """初始化优先级队列。
        Args:
            max_size: 队列最大容量。
        Raises:
            ValueError: max_size <= 0。
        """
        if max_size <= 0:
            raise ValueError(f"max_size 必须为正整数，当前值: {max_size}")
        self.max_size: int = max_size
        self._heap: list[PriorityQueueItem] = []
        self._lock: asyncio.Lock = asyncio.Lock()
        self._not_empty: asyncio.Event = asyncio.Event()
        self._full_policy: QueueFullPolicy = QueueFullPolicy.PASSTHROUGH
        # 统计
        self._total_enqueued: int = 0
        self._total_dequeued: int = 0
        self._total_dropped: int = 0
    # ---- 队列满策略 ---- 
    def set_full_policy(self, policy: QueueFullPolicy) -> None:
        """设置队列满时的处理策略。
        Args:
            policy: QueueFullPolicy 枚举值。
        """
        self._full_policy = policy
    @property
    def full_policy(self) -> QueueFullPolicy:
        """当前队列满策略。"""
        return self._full_policy
    # ---- 动态容量调整 ----
    def set_max_size(self, new_size: int) -> tuple[bool, str]:
        """动态调整队列最大容量（热重载）。
        缩小操作受保护：如果 new_size 小于当前排队数，拒绝变更并
        提示当前队列深度。
        Args:
            new_size: 新的最大容量。
        Returns:
            (成功标志, 消息)。成功时标志为 True，消息含新旧容量对比；
            失败时标志为 False，消息含拒绝原因和当前深度。
        Raises:
            ValueError: new_size <= 0。
        """
        if new_size <= 0:
            raise ValueError(f"max_size 必须为正整数，当前值: {new_size}")
        current = len(self._heap)
        if new_size < current:
            return (False, f"拒绝缩小：新上限 {new_size} < 当前排队数 {current}，需要先排空或提升上限")
        old = self.max_size
        self.max_size = new_size
        return (True, f"队列上限已调整：{old} → {new_size}{'（当前排队 ' + str(current) + '）' if current > 0 else ''}")
    # ---- 入队 ----
    async def put(
        self,
        item: dict[str, Any],
        priority: Priority = Priority.NORMAL,
        headers: dict[str, str] | None = None,
    ) -> str:
        """将请求放入队列。
        Args:
            item: 请求体（JSON 序列化的 dict）。
            priority: 请求优先级，默认 NORMAL。
            headers: 原始请求 headers。
        Returns:
            分配的唯一 request_id。
        Raises:
            QueueFullError: 队列满且策略为 REJECT。
        """
        request_id = str(uuid.uuid4())
        headers = headers or {}
        queue_item = PriorityQueueItem(
            sort_index=(int(priority), time.monotonic()),
            priority=priority,
            request_id=request_id,
            payload=item,
            enqueued_at=time.monotonic(),
            headers=headers,
        )
        async with self._lock:
            queue_size = len(self._heap)
            if queue_size >= self.max_size:
                if self._full_policy == QueueFullPolicy.REJECT:
                    raise QueueFullError(
                        f"队列已满 ({queue_size}/{self.max_size})，策略: reject"
                    )
                elif self._full_policy == QueueFullPolicy.DROP_LOWEST:
                    # 丢弃 heap 中优先级最低（值最大）的元素
                    # heap 是最小堆，找最大值需要遍历
                    max_val_item = max(self._heap, key=lambda x: x.sort_index)
                    self._heap.remove(max_val_item)
                    heapq.heapify(self._heap)
                    self._total_dropped += 1
                # PASSTHROUGH 策略：不插入队列，抛异常让调用方绕过排队
                else:
                    raise QueueFullPassthrough(
                        f"队列已满 ({queue_size}/{self.max_size})，策略: passthrough"
                    )
            heapq.heappush(self._heap, queue_item)
            self._total_enqueued += 1
        self._not_empty.set()
        return request_id
    # ---- 出队 ----
    async def get(self, timeout: float = 1.0) -> PriorityQueueItem | None:
        """从队列取出下一个元素（阻塞、优先级排序）。
        Args:
            timeout: 阻塞等待的最大秒数，默认 1.0。
        Returns:
            优先级最高的队列元素；超时无元素时返回 None。
        """
        deadline = time.monotonic() + timeout
        while True:
            async with self._lock:
                if self._heap:
                    item = heapq.heappop(self._heap)
                    self._total_dequeued += 1
                    if not self._heap:
                        self._not_empty.clear()
                    return item
            # 队列为空，等待新元素入队
            remaining = deadline - time.monotonic()
            if remaining <= 0:
                return None
            try:
                await asyncio.wait_for(
                    self._not_empty.wait(),
                    timeout=remaining,
                )
            except asyncio.TimeoutError:
                return None
    # ---- 状态查询 ----
    async def get_queue_size(self) -> int:
        """返回当前队列长度。"""
        async with self._lock:
            return len(self._heap)
    async def get_stats(self) -> dict[str, Any]:
        """返回队列统计信息。"""
        async with self._lock:
            depth_by_priority: dict[str, int] = {}
            for item in self._heap:
                key = item.priority.name
                depth_by_priority[key] = depth_by_priority.get(key, 0) + 1
            return {
                "max_size": self.max_size,
                "current_size": len(self._heap),
                "total_enqueued": self._total_enqueued,
                "total_dequeued": self._total_dequeued,
                "total_dropped": self._total_dropped,
                "depth_by_priority": depth_by_priority,
                "full_policy": self._full_policy.value,
                "utilization": len(self._heap) / self.max_size if self.max_size > 0 else 0.0,
            }
@@ -1,48 +0,0 @@
 [project]
 name = "nvidia_sidecar"
 version = "0.1.0"
 description = "NVIDIA Sidecar 限流代理 — 为 NVIDIA API 提供优先级排队 + 令牌桶限流"
 readme = "README.md"
 license = { text = "MIT" }
 requires-python = ">=3.12"
 dependencies = [
    "fastapi>=0.115",
    "uvicorn[standard]>=0.34",
    "httpx>=0.28",
    "PyYAML>=6.0",
    "structlog>=24.4",
    "prometheus-client>=0.21",
    "pydantic>=2.0",
 ]
 [project.optional-dependencies]
 dev = [
    "pytest>=8.3",
    "pytest-asyncio>=0.24",
    "httpx>=0.28",
    "mypy>=1.14",
    "types-PyYAML",
 ]
 [project.scripts]
 nvidia-sidecar = "nvidia_sidecar.server:main"
 [build-system]
 requires = ["setuptools>=75", "wheel"]
 build-backend = "setuptools.build_meta"
 [tool.setuptools]
 packages = ["nvidia_sidecar"]
 [tool.setuptools.package-dir]
 # Flat layout: __init__.py + all .py files at project root
 "nvidia_sidecar" = "."
 [tool.mypy]
 python_version = "3.12"
 strict = true
 warn_return_any = true
 warn_unused_configs = true
 [[tool.mypy.overrides]]
 module = "structlog.*"
 ignore_missing_imports = true
@@ -1,438 +0,0 @@
 """
 NVIDIA Sidecar 限流代理 — 令牌桶 + 网关识别模块 (§3.2)
 从 BIZ-26 rate_limiter.py 提取核心限流逻辑，去除多线程调度器、缓存管理等。
 保留：Priority, TokenBucket, is_nvidia_gateway, normalize_gateway_name。
 """
 from __future__ import annotations
 import time
 import threading
 from enum import IntEnum
 from typing import Any
 # ---------------------------------------------------------------------------
 # 优先级枚举
 # ---------------------------------------------------------------------------
 class Priority(IntEnum):
    """请求优先级（数值越小优先级越高）。"""
    URGENT = 1
    HIGH = 2
    NORMAL = 3
    LOW = 4
 # ---------------------------------------------------------------------------
 # NVIDIA 网关别名集
 # ---------------------------------------------------------------------------
 NVIDIA_GATEWAY_ALIASES: set[str] = {
    "nvidia",
    "nvidia-gateway",
    "nvidiavx",
    "nvidiavx18088980513",
 }
 def is_nvidia_gateway(value: str | None) -> bool:
    """判断给定网关名/模型全路径是否属于 NVIDIA 网关。
    Args:
        value: 网关名（如 ``"nvidia"``）或模型全路径前缀
               （如 ``"nvidia/deepseek-ai/deepseek-v4-pro"``）。
               None 时直接返回 False。
    Returns:
        True 当 value 的 provider 部分匹配已知 NVIDIA 别名。
    """
    if value is None:
        return False
    # 提取 provider 前缀：取 "/" 前第一个部分
    provider = value.split("/", 1)[0].lower().strip()
    return provider in NVIDIA_GATEWAY_ALIASES
 def normalize_gateway_name(value: str | None) -> str | None:
    """规范化网关名：提取 provider 前缀并转为小写。
    Args:
        value: 网关名或模型全路径。None 时返回 None。
    Returns:
        provider 前缀的小写形式，或 None。
    """
    if value is None:
        return None
    return value.split("/", 1)[0].lower().strip()
 # ---------------------------------------------------------------------------
 # 令牌桶（线程安全）
 # ---------------------------------------------------------------------------
 class TokenBucket:
    """线程安全的令牌桶实现。
    支持固定速率令牌补充和消费，带有溢出保护和可选的阻塞等待。
    """
    def __init__(self, rate: float = 40 / 60, capacity: int = 40) -> None:
        """初始化令牌桶。
        Args:
            rate: 令牌补充速率（令牌/秒）。默认 40/60 ≈ 0.667 token/s（40 RPM）。
            capacity: 桶最大容量（令牌数）。默认 40。
        """
        self._rate: float = float(rate)
        self._capacity: int = int(capacity)
        self._tokens: float = float(capacity)  # 启动时桶满
        self._last_refill: float = time.monotonic()
        self._lock: threading.Lock = threading.Lock()
    # ---- 内部方法 ----
    def _refill(self) -> None:
        """补充令牌（调用方需持有 _lock）。
        根据距上次补充的时间差计算新增令牌数，不超过 capacity。
        """
        now = time.monotonic()
        elapsed = now - self._last_refill
        if elapsed > 0 and self._rate > 0:
            new_tokens = elapsed * self._rate
            self._tokens = min(self._tokens + new_tokens, float(self._capacity))
        self._last_refill = now
    # ---- 公开方法 ----
    def consume(self, tokens: int = 1) -> bool:
        """尝试立即消费令牌（非阻塞）。
        Args:
            tokens: 要消费的令牌数，默认 1。
        Returns:
            True 消费成功；False 令牌不足。
        """
        if tokens <= 0:
            return True
        with self._lock:
            self._refill()
            if self._tokens >= tokens:
                self._tokens -= tokens
                return True
            return False
    def try_consume(self, tokens: int = 1, timeout: float = 2.0) -> bool:
        """尝试在指定时间内消费令牌（阻塞）。
        Args:
            tokens: 要消费的令牌数，默认 1。
            timeout: 最大等待秒数，默认 2.0。
        Returns:
            True 在超时前成功消费；False 超时。
        """
        if tokens <= 0:
            return True
        deadline = time.monotonic() + timeout
        while True:
            with self._lock:
                self._refill()
                if self._tokens >= tokens:
                    self._tokens -= tokens
                    return True
            # 释放锁后计算剩余等待时间
            remaining = deadline - time.monotonic()
            if remaining <= 0:
                return False
            # 等待到下一个令牌应该补充的时间点
            sleep_time = min(remaining, max(0.05, 1.0 / self._rate) if self._rate > 0 else remaining)
            time.sleep(sleep_time)
    def wait_for_token(self, timeout: float | None = None) -> bool:
        """等待并尝试消费 1 个令牌。
        Args:
            timeout: 最大等待秒数；None 表示无限等待（不推荐）。
        Returns:
            True 成功消费；False 超时。
        """
        return self.try_consume(tokens=1, timeout=timeout if timeout is not None else float("inf"))
    def get_status(self) -> dict[str, Any]:
        """获取令牌桶当前状态。
        Returns:
            包含 tokens, capacity, rate_per_minute, utilization 的字典。
        """
        with self._lock:
            self._refill()
            rate_per_minute = self._rate * 60.0
            utilization = 0.0 if self._capacity == 0 else (
                (self._capacity - self._tokens) / self._capacity
            )
            return {
                "tokens": round(self._tokens, 2),
                "capacity": self._capacity,
                "rate_per_minute": round(rate_per_minute, 1),
                "utilization": round(utilization, 4),
            }
    # ---- 属性 ----
    @property
    def rate(self) -> float:
        """当前令牌补充速率（令牌/秒）。"""
        return self._rate
    @property
    def capacity(self) -> int:
        """桶容量。"""
        return self._capacity
    # ---- 动态速率调整（供 AdaptiveTokenBucket 使用） ----
    def set_rate(self, rate: float) -> None:
        """动态调整令牌补充速率（令牌/秒）。
        Args:
            rate: 新速率（令牌/秒）。
        """
        with self._lock:
            self._refill()  # 先补充现有令牌再切换速率
            self._rate = float(rate)
 # ---------------------------------------------------------------------------
 # 避退模式：AdaptiveTokenBucket (§ADR-009)
 # ---------------------------------------------------------------------------
 class RetreatState:
    """避退状态机常量。"""
    NORMAL: str = "normal"
    RETREAT: str = "retreat"
    RECOVER: str = "recover"
 class AdaptiveTokenBucket(TokenBucket):
    """自适应避退令牌桶（ADR-009）。
    监控上游 429 率（60s 滑动窗口），自动调整发射速率：
    - 429 率 < 5%   → NORMAL，保持基准速率
    - 429 率 5-10%  → RETREAT，速率 × 0.75
    - 429 率 10-20% → RETREAT，再次降速
    - 429 率 > 20%  → RETREAT，最低 5 RPM + 告警
    - 连续 120s 429 率 < 2% → RECOVER，逐步 +2 RPM 恢复
    线程安全，继承 TokenBucket 的所有公共接口。
    """
    # ADR-009 参数（可通过构造函数覆盖）
    RETREAT_WINDOW_SECONDS: float = 60.0
    RETREAT_429_THRESHOLD: float = 0.05
    RETREAT_FACTOR: float = 0.75
    RETREAT_MIN_RPM: float = 5.0
    RECOVER_WINDOW_SECONDS: float = 120.0
    RECOVER_429_THRESHOLD: float = 0.02
    RECOVER_INCREMENT_RPM: float = 2.0
    def __init__(
        self,
        rate: float = 40 / 60,
        capacity: int = 40,
        *,
        retreat_window_seconds: float = 60.0,
        retreat_429_threshold: float = 0.05,
        retreat_factor: float = 0.75,
        retreat_min_rpm: float = 5.0,
        recover_window_seconds: float = 120.0,
        recover_429_threshold: float = 0.02,
        recover_increment_rpm: float = 2.0,
    ) -> None:
        """初始化自适应避退令牌桶。
        Args:
            rate: 基准令牌补充速率（令牌/秒）。默认 40/60 ≈ 0.667 token/s。
            capacity: 桶最大容量。默认 40。
            retreat_window_seconds: 429 率滑动窗口大小（秒）。
            retreat_429_threshold: 触发避退的 429 率阈值。
            retreat_factor: 每次避退速率乘数。
            retreat_min_rpm: 避退最低 RPM。
            recover_window_seconds: 恢复观察窗口大小（秒）。
            recover_429_threshold: 触发恢复的 429 率阈值。
            recover_increment_rpm: 每次恢复增加的 RPM。
        """
        super().__init__(rate=rate, capacity=capacity)
        # 基准速率（不变）
        self._base_rate: float = float(rate)
        # 避退参数
        self.RETREAT_WINDOW_SECONDS = retreat_window_seconds
        self.RETREAT_429_THRESHOLD = retreat_429_threshold
        self.RETREAT_FACTOR = retreat_factor
        self.RETREAT_MIN_RPM = retreat_min_rpm
        self.RECOVER_WINDOW_SECONDS = recover_window_seconds
        self.RECOVER_429_THRESHOLD = recover_429_threshold
        self.RECOVER_INCREMENT_RPM = recover_increment_rpm
        # 避退状态机
        self._retreat_state: str = RetreatState.NORMAL
        # 429 滑动窗口：[(timestamp, is_429), ...]
        self._429_window: list[tuple[float, bool]] = []
        # 上次状态变更时间
        self._last_state_change: float = time.monotonic()
        # 避退状态锁（RLock 防止 evaluate_retreat() → get_429_rate() 重入死锁）
        self._retreat_lock: threading.RLock = threading.RLock()
    # ---- 429 反馈 ----
    def record_response(self, is_429: bool) -> None:
        """记录一次上游响应是否为 429。
        Args:
            is_429: True 表示上游返回了 429。
        """
        now = time.monotonic()
        with self._retreat_lock:
            self._429_window.append((now, is_429))
            # 清理超出观察窗口的旧记录
            cutoff = now - max(
                self.RETREAT_WINDOW_SECONDS,
                self.RECOVER_WINDOW_SECONDS,
            )
            self._429_window = [
                (ts, flag) for ts, flag in self._429_window
                if ts >= cutoff
            ]
    def get_429_rate(self, window_seconds: float | None = None) -> float:
        """获取指定窗口内的 429 率。
        Args:
            window_seconds: 滑动窗口大小；None 使用 RETREAT_WINDOW_SECONDS。
        Returns:
            0.0-1.0 之间的 429 率。
        """
        ws = window_seconds or self.RETREAT_WINDOW_SECONDS
        now = time.monotonic()
        with self._retreat_lock:
            in_window = [flag for ts, flag in self._429_window if now - ts <= ws]
            if not in_window:
                return 0.0
            return sum(1 for f in in_window if f) / len(in_window)
    # ---- 避退状态评估 ----
    def evaluate_retreat(self) -> str:
        """评估并更新避退状态，返回新状态名。
        每次调用根据当前 429 率 + 持续时间决定是否进入 RETREAT / RECOVER。
        Returns:
            "normal" / "retreat" / "recover"。
        """
        now = time.monotonic()
        with self._retreat_lock:
            retreat_rate = self.get_429_rate(self.RETREAT_WINDOW_SECONDS)
            recover_rate = self.get_429_rate(self.RECOVER_WINDOW_SECONDS)
            if self._retreat_state == RetreatState.NORMAL:
                if retreat_rate >= self.RETREAT_429_THRESHOLD:
                    self._retreat_state = RetreatState.RETREAT
                    self._last_state_change = now
                    self._apply_retreat()
            elif self._retreat_state == RetreatState.RETREAT:
                # 持续高 429 率 → 再次降速
                if retreat_rate >= self.RETREAT_429_THRESHOLD * 2:
                    # 429 > 10%，再次降速
                    if self._rate > self.RETREAT_MIN_RPM / 60.0:
                        self._apply_retreat()
                elif recover_rate < self.RECOVER_429_THRESHOLD:
                    time_in_low = now - self._last_state_change
                    if time_in_low >= self.RECOVER_WINDOW_SECONDS:
                        self._retreat_state = RetreatState.RECOVER
                        self._last_state_change = now
                        self._apply_recover()
            elif self._retreat_state == RetreatState.RECOVER:
                if retreat_rate >= self.RETREAT_429_THRESHOLD:
                    # 恢复期间 429 回升，重新进入避退
                    self._retreat_state = RetreatState.RETREAT
                    self._last_state_change = now
                    self._apply_retreat()
                elif self._rate >= self._base_rate:
                    # 已恢复到基准速率
                    self._rate = self._base_rate
                    self._retreat_state = RetreatState.NORMAL
                    self._last_state_change = now
                else:
                    # 继续逐步恢复
                    self._apply_recover()
            return self._retreat_state
    def _apply_retreat(self) -> None:
        """执行一次避退降速。"""
        new_rate: float = max(
            self.RETREAT_MIN_RPM / 60.0,
            self._rate * self.RETREAT_FACTOR,
        )
        self._rate = new_rate
    def _apply_recover(self) -> None:
        """执行一次恢复提速。"""
        increment: float = self.RECOVER_INCREMENT_RPM / 60.0
        new_rate: float = min(self._base_rate, self._rate + increment)
        self._rate = new_rate
    # ---- 状态查询 ----
    def get_retreat_state(self) -> str:
        """获取当前避退状态。
        Returns:
            "normal" / "retreat" / "recover"。
        """
        with self._retreat_lock:
            return self._retreat_state
    def get_effective_rate_rpm(self) -> float:
        """获取当前实际速率（RPM），考虑避退乘数。
        Returns:
            当前每分钟速率。
        """
        with self._lock:
            return self._rate * 60.0
    def get_base_rate_rpm(self) -> float:
        """获取基准速率（RPM），即未避退时的速率。
        Returns:
            基准每分钟速率。
        """
        return self._base_rate * 60.0
    def reset_to_base(self) -> None:
        """手动重置到基准速率（用于运维干预）。"""
        with self._retreat_lock:
            self._rate = self._base_rate
            self._retreat_state = RetreatState.NORMAL
            self._last_state_change = time.monotonic()
            self._429_window.clear()
@@ -1,822 +0,0 @@
 """
 NVIDIA Sidecar 限流代理 — FastAPI 代理主入口 (§3.4)
 完整的 API 代理链路：
    接收 → 网关识别 → [NVIDIA: 排队 → 令牌限流] → httpx 转发 → 返回
 非 NVIDIA 请求直通上游，NVIDIA 请求经过四级优先级队列 + 令牌桶限流。
 BIZ-46 Phase3: 架构解耦 — 所有全局状态收敛为 SidecarContext (§1)
 """
 from __future__ import annotations
 import asyncio
 import logging
 import time
 from collections.abc import AsyncGenerator
 from contextlib import asynccontextmanager
 from typing import Any
 import httpx
 import structlog
 import uvicorn
 from fastapi import Depends, FastAPI, Request, Response
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import JSONResponse, StreamingResponse
 from nvidia_sidecar.config import load_config, SidecarConfig
 from nvidia_sidecar.context import SidecarContext
 from nvidia_sidecar.rate_limiter import (
    Priority,
    AdaptiveTokenBucket,
    is_nvidia_gateway,
 )
 from nvidia_sidecar.priority_queue import (
    PriorityRequestQueue,
    QueueFullError,
    QueueFullPassthrough,
    QueueFullPolicy,
 )
 from nvidia_sidecar.metrics import PrometheusMetrics
 from nvidia_sidecar.health import HealthService
 from nvidia_sidecar.webui import webui_router
 # ---------------------------------------------------------------------------
 # 结构化日志
 # ---------------------------------------------------------------------------
 structlog.configure(
    processors=[
        structlog.stdlib.filter_by_level,
        structlog.stdlib.add_logger_name,
        structlog.stdlib.add_log_level,
        structlog.stdlib.PositionalArgumentsFormatter(),
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.StackInfoRenderer(),
        structlog.processors.format_exc_info,
        structlog.processors.UnicodeDecoder(),
        structlog.processors.JSONRenderer(),
    ],
    context_class=dict,
    logger_factory=structlog.PrintLoggerFactory(),
    wrapper_class=structlog.stdlib.BoundLogger,
    cache_logger_on_first_use=True,
 )
 logger: structlog.stdlib.BoundLogger = structlog.get_logger("nvidia_sidecar")
 # ---------------------------------------------------------------------------
 # FastAPI 依赖注入
 # ---------------------------------------------------------------------------
 def get_context(request: Request) -> SidecarContext:
    """从 app.state 获取 SidecarContext（FastAPI 依赖注入）。"""
    return request.app.state.sidecar  # type: ignore[no-any-return]
 # ---------------------------------------------------------------------------
 # 工具函数
 # ---------------------------------------------------------------------------
 def _extract_model(body: Any) -> str | None:
    """从请求体中提取模型标识符（兼容 OpenAI Chat/Completions 格式）。
    Args:
        body: 已解析的 JSON 请求体。
    Returns:
        模型标识符字符串，或 None。
    """
    if isinstance(body, dict):
        return str(body.get("model", "")) or None
    return None
 def _resolve_priority(headers: dict[str, str]) -> Priority:
    """从请求 headers 解析优先级。
    检查 ``X-Priority`` header，值为 ``urgent``/``high``/``normal``/``low``，
    不区分大小写。默认 NORMAL。
    """
    raw = headers.get("x-priority", "").strip().lower()
    mapping: dict[str, Priority] = {
        "urgent": Priority.URGENT,
        "high": Priority.HIGH,
        "normal": Priority.NORMAL,
        "low": Priority.LOW,
    }
    return mapping.get(raw, Priority.NORMAL)
 # ---------------------------------------------------------------------------
 # 上游转发
 # ---------------------------------------------------------------------------
 async def _forward_to_upstream(
    ctx: SidecarContext,
    method: str,
    path: str,
    body: bytes | None,
    headers: dict[str, str],
    stream: bool = False,
 ) -> httpx.Response:
    """将请求转发到 NVIDIA 上游 API。
    Args:
        ctx: SidecarContext 运行时上下文。
        method: HTTP 方法。
        path: 请求路径（如 ``/v1/chat/completions``）。
        body: 原始请求体 bytes。
        headers: 要转发的请求 headers（会追加 Authorization）。
        stream: 是否请求流式响应。
    Returns:
        httpx.Response 对象。
    Raises:
        httpx.HTTPError: HTTP 请求失败。
    """
    upstream_url = ctx.config.upstream_url.rstrip("/") + path
    forward_headers: dict[str, str] = {
        k: v for k, v in headers.items()
        if k.lower() not in ("host", "content-length", "transfer-encoding")
    }
    if ctx.config.upstream_api_key:
        forward_headers["authorization"] = f"Bearer {ctx.config.upstream_api_key}"
    elif "authorization" not in {k.lower() for k in forward_headers}:
        forward_headers["authorization"] = "Bearer nvidia"
    try:
        req = ctx.http_client.build_request(
            method=method,
            url=upstream_url,
            headers=forward_headers,
            content=body,
            timeout=ctx.config.request_timeout,
        )
        response = await ctx.http_client.send(req, stream=stream)
        return response
    except httpx.TimeoutException:
        logger.warning("upstream_timeout", path=path, timeout=ctx.config.request_timeout)
        raise
    except httpx.HTTPError as exc:
        logger.error("upstream_error", path=path, error=str(exc))
        raise
 # ---------------------------------------------------------------------------
 # worker 协程：消费优先级队列 + 令牌桶 + 转发
 # ---------------------------------------------------------------------------
 async def _worker_loop(ctx: SidecarContext) -> None:
    """后台 worker：持续从优先级队列取请求 → 令牌限流 → 转发 → 设置 future 结果。
    Args:
        ctx: SidecarContext 运行时上下文。
    """
    log = logger.bind(worker="main")
    log.info("worker_started")
    while True:
        try:
            queue_item = await ctx.priority_queue.get(timeout=1.0)
            if queue_item is None:
                continue
            request_id = queue_item.request_id
            payload = queue_item.payload
            headers = queue_item.headers
            enqueued_at = queue_item.enqueued_at
            # 查找对应的 pending future
            pending_entry = ctx.pending_requests.get(request_id)
            if pending_entry is None:
                log.warning("orphan_request", request_id=request_id)
                continue
            future, _ = pending_entry
            # 低优先级令牌等待超时处理
            if queue_item.priority == Priority.LOW:
                # 放线程池执行阻塞的令牌桶调用
                got_token = await asyncio.to_thread(
                    ctx.token_bucket.try_consume,
                    tokens=1,
                    timeout=ctx.config.low_priority_timeout,
                )
                if not got_token:
                    log.info("low_priority_timeout", request_id=request_id)
                    await ctx.increment_stat("ratelimited_requests")
                    ctx.prometheus.record_request(queue_item.priority.name, "ratelimited")
                    if not future.done():
                        future.set_exception(
                            _RateLimitedError(
                                f"低优先级请求令牌等待超时 ({ctx.config.low_priority_timeout}s)"
                            )
                        )
                    ctx.pending_requests.pop(request_id, None)
                    continue
            else:
                # 非低优先级：在 worker 内轮询等待令牌，避免重入队导致 future 悬挂
                got_token = await asyncio.to_thread(ctx.token_bucket.consume, tokens=1)
                if not got_token:
                    token_deadline = time.monotonic() + ctx.config.request_timeout
                    while not got_token:
                        await asyncio.sleep(0.1)
                        got_token = await asyncio.to_thread(ctx.token_bucket.consume, tokens=1)
                        if time.monotonic() > token_deadline:
                            break
                    if not got_token:
                        log.warning(
                            "token_wait_timeout",
                            request_id=request_id,
                            priority=queue_item.priority.name,
                            timeout=ctx.config.request_timeout,
                        )
                        await ctx.increment_stat("ratelimited_requests")
                        ctx.prometheus.record_request(queue_item.priority.name, "ratelimited")
                        if not future.done():
                            future.set_exception(
                                _RateLimitedError(
                                    f"令牌等待超时 ({ctx.config.request_timeout:.0f}s)"
                                )
                            )
                        ctx.pending_requests.pop(request_id, None)
                        continue
            # 转发到上游
            upstream_start = time.monotonic()
            try:
                path = headers.get("x-original-path", "/v1/chat/completions")
                method = headers.get("x-original-method", "POST")
                # 过滤内部 headers
                clean_headers = {
                    k: v for k, v in headers.items()
                    if not k.startswith("x-original-") and not k.startswith("x-request-id")
                }
                resp = await _forward_to_upstream(
                    ctx=ctx,
                    method=method,
                    path=path,
                    body=payload.get("_raw_body"),
                    headers=clean_headers,
                    stream=payload.get("stream", False),
                )
                upstream_latency = time.monotonic() - upstream_start
                queue_latency = time.monotonic() - enqueued_at
                total_latency = upstream_latency + queue_latency
                is_429: bool = resp.status_code == 429
                ctx.token_bucket.record_response(is_429)
                # 避退状态评估 + 指标更新
                ctx.token_bucket.evaluate_retreat()
                retreat_state = ctx.token_bucket.get_retreat_state()
                effective_rpm = ctx.token_bucket.get_effective_rate_rpm()
                upstream_429_rate = ctx.token_bucket.get_429_rate()
                ctx.prometheus.update_retreat_metrics(retreat_state, effective_rpm, upstream_429_rate)
                # 模型级信息写入 JSON 日志 (BIZ-46 Phase3: provider label 收敛后保留)
                model_id = _extract_model(payload) or "unknown"
                log.info(
                    "request_completed",
                    request_id=request_id,
                    status=resp.status_code,
                    model_id=model_id,
                    upstream_latency=round(upstream_latency, 3),
                    queue_latency=round(queue_latency, 3),
                    total_latency=round(total_latency, 3),
                    retreat_state=retreat_state,
                    effective_rpm=round(effective_rpm, 1),
                )
                # 记录 Prometheus 指标 — provider 收敛（BIZ-46 Phase3）
                provider = "nvidia"
                ctx.prometheus.record_upstream_latency(provider, upstream_latency)
                if not resp.is_success:
                    ctx.prometheus.record_upstream_error(resp.status_code, provider)
                ctx.prometheus.record_request(queue_item.priority.name, "success" if resp.is_success else "error")
                ctx.prometheus.record_queue_latency(queue_item.priority.name, queue_latency)
                if not future.done():
                    future.set_result(resp)
            except (httpx.HTTPError, OSError) as exc:
                log.error("upstream_request_failed", request_id=request_id, error=str(exc))
                await ctx.increment_stat("upstream_errors")
                ctx.prometheus.record_request(queue_item.priority.name, "error")
                ctx.prometheus.set_health(False)
                if not future.done():
                    future.set_exception(exc)
            ctx.pending_requests.pop(request_id, None)
        except asyncio.CancelledError:
            log.info("worker_cancelled")
            break
        except Exception:
            log.exception("worker_unexpected_error")
 # ---------------------------------------------------------------------------
 # PASSTHROUGH 直通路径（队列满 + PASSTHROUGH 策略）
 # ---------------------------------------------------------------------------
 async def _passthrough_with_rate_limit(
    ctx: SidecarContext,
    request: Request,
    path: str,
    body_bytes: bytes,
    raw_headers: dict[str, str],
    priority: Priority,
 ) -> Response:
    """队列满时的 PASSSTHROUGH 直通路径：仍受令牌桶限流，但不排队。
    Args:
        ctx: SidecarContext 运行时上下文。
        request: FastAPI Request。
        path: 请求路径。
        body_bytes: 原始请求体。
        raw_headers: 请求 headers。
        priority: 请求优先级。
    Returns:
        FastAPI Response。
    """
    await ctx.increment_stat("passthrough_requests")
    ctx.prometheus.increment_fallback()
    # 低优先级走令牌桶等待
    if priority == Priority.LOW:
        got_token = await asyncio.to_thread(
            ctx.token_bucket.try_consume,
            tokens=1,
            timeout=ctx.config.low_priority_timeout,
        )
        if not got_token:
            await ctx.increment_stat("ratelimited_requests")
            ctx.prometheus.record_request(priority.name, "ratelimited")
            return JSONResponse(
                status_code=429,
                content={
                    "error": {
                        "message": f"令牌不足（队列满 + passthrough），超时 {ctx.config.low_priority_timeout}s",
                        "type": "RateLimitedError",
                    }
                },
            )
    else:
        got_token = await asyncio.to_thread(ctx.token_bucket.consume, tokens=1)
        if not got_token:
            deadline = time.monotonic() + ctx.config.request_timeout
            while not got_token:
                await asyncio.sleep(0.1)
                got_token = await asyncio.to_thread(ctx.token_bucket.consume, tokens=1)
                if time.monotonic() > deadline:
                    await ctx.increment_stat("ratelimited_requests")
                    ctx.prometheus.record_request(priority.name, "ratelimited")
                    return JSONResponse(
                        status_code=429,
                        content={
                            "error": {
                                "message": f"令牌不足（队列满 + passthrough），等待超时 {ctx.config.request_timeout:.0f}s",
                                "type": "RateLimitedError",
                            }
                        },
                    )
    # 拿到令牌，直接转发
    try:
        clean_headers = {k: v for k, v in raw_headers.items()}
        resp = await _forward_to_upstream(
            ctx=ctx,
            method=request.method,
            path=path,
            body=body_bytes if body_bytes else None,
            headers=clean_headers,
            stream=False,
        )
        retreat_state = ctx.token_bucket.get_retreat_state()
        ctx.token_bucket.evaluate_retreat()
        ctx.prometheus.update_retreat_metrics(
            retreat_state,
            ctx.token_bucket.get_effective_rate_rpm(),
            ctx.token_bucket.get_429_rate(),
        )
        return _build_response(resp)
    except Exception as exc:
        status, msg = _map_exception(exc)
        logger.error("passthrough_error", path=path, error=str(exc))
        ctx.prometheus.set_health(False)
        return JSONResponse(
            status_code=status,
            content={"error": {"message": msg, "type": type(exc).__name__}},
        )
 # ---------------------------------------------------------------------------
 # 自定义异常
 # ---------------------------------------------------------------------------
 class _RateLimitedError(Exception):
    """429 限流错误。"""
    pass
 # ---------------------------------------------------------------------------
 # 异常处理矩阵 (§3.4)
 # ---------------------------------------------------------------------------
 _EXCEPTION_MATRIX: dict[type[Exception], tuple[int, str]] = {
    _RateLimitedError: (429, "Too Many Requests — 令牌不足"),
    QueueFullError: (503, "Service Unavailable — 队列已满"),
    httpx.TimeoutException: (504, "Gateway Timeout — 上游超时"),
    httpx.ConnectError: (502, "Bad Gateway — 上游连接失败"),
    httpx.HTTPStatusError: (502, "Bad Gateway — 上游返回错误状态"),
 }
 def _map_exception(exc: Exception) -> tuple[int, str]:
    """将异常映射为 HTTP 状态码 + 错误信息。"""
    for exc_type, (status, msg) in _EXCEPTION_MATRIX.items():
        if isinstance(exc, exc_type):
            return status, msg
    return 500, f"Internal Server Error — {type(exc).__name__}"
 # ---------------------------------------------------------------------------
 # FastAPI 应用 + lifespan
 # ---------------------------------------------------------------------------
@asynccontextmanager
 async def lifespan(app: FastAPI) -> AsyncGenerator[None, Any]:
    """应用生命周期管理：初始化/清理全局资源。
    BIZ-46 Phase3: 所有资源收敛到 SidecarContext，挂载于 app.state.sidecar。
    """
    # 启动
    config: SidecarConfig = load_config()
    logging.getLogger().setLevel(config.log_level.upper())
    http_client: httpx.AsyncClient = httpx.AsyncClient(
        timeout=httpx.Timeout(config.request_timeout),
        limits=httpx.Limits(
            max_connections=100,
            max_keepalive_connections=20,
        ),
    )
    priority_queue: PriorityRequestQueue = PriorityRequestQueue(max_size=config.queue_max_size)
    token_bucket: AdaptiveTokenBucket = AdaptiveTokenBucket(
        rate=config.rate_rpm / 60.0,
        capacity=config.bucket_capacity,
    )
    prometheus: PrometheusMetrics = PrometheusMetrics()
    health: HealthService = HealthService()
    ctx: SidecarContext = SidecarContext(
        config=config,
        http_client=http_client,
        token_bucket=token_bucket,
        priority_queue=priority_queue,
        prometheus=prometheus,
        health=health,
    )
    ctx.stats["start_time"] = int(time.time())
    app.state.sidecar = ctx  # 注入 FastAPI
    # 启动 worker 协程
    worker_task = asyncio.create_task(_worker_loop(ctx))
    # 在独立端口 :9191 启动 Prometheus metrics 服务器
    metrics_app = prometheus.build_asgi_app()
    metrics_config = uvicorn.Config(
        metrics_app,
        host=config.listen_host,
        port=config.metrics_port,
        log_level="error",
    )
    metrics_server = uvicorn.Server(metrics_config)
    _metrics_task = asyncio.create_task(metrics_server.serve())
    # CORS 中间件（严维序评审 #8）
    app.add_middleware(
        CORSMiddleware,
        allow_origins=["*"],
        allow_credentials=False,
        allow_methods=["*"],
        allow_headers=["*"],
    )
    # 挂载 webui 子路由
    app.include_router(webui_router)
    # upstream_api_key 启动检查（严维序评审 #5）
    if not config.upstream_api_key:
        logger.warning(
            "upstream_api_key_empty",
            message="SIDECAR_API_KEY 未设置，NVIDIA 请求将因 401 认证失败",
        )
    logger.info(
        "sidecar_started",
        host=config.listen_host,
        port=config.listen_port,
        metrics_port=config.metrics_port,
        rate_rpm=config.rate_rpm,
        queue_max=config.queue_max_size,
        retreat_enabled=True,
    )
    yield  # app 运行中
    # 关闭
    worker_task.cancel()
    try:
        await worker_task
    except asyncio.CancelledError:
        pass
    _metrics_task.cancel()
    try:
        await _metrics_task
    except asyncio.CancelledError:
        pass
    await http_client.aclose()
    logger.info("sidecar_stopped")
 def _mask_api_key(key: str) -> str:
    """对 API Key 进行脱敏处理，仅保留前 4 位以供识别。"""
    if not key:
        return ""
    if len(key) <= 4:
        return key[:2] + "****"
    return key[:4] + "****"
 app: FastAPI = FastAPI(
    title="NVIDIA Sidecar Rate-Limiting Proxy",
    version="0.1.0",
    lifespan=lifespan,
 )
 # ---------------------------------------------------------------------------
 # 核心代理处理器
 # ---------------------------------------------------------------------------
 async def _handle_proxy_request(ctx: SidecarContext, request: Request, path: str) -> Response:
    """统一的代理请求处理入口。
    执行完整链路：
    1. 解析请求体 → 提取 model
    2. 网关识别 → 非 NVIDIA 直通
    3. NVIDIA → 排队 + 令牌限流 + 转发
    """
    await ctx.increment_stat("total_requests")
    # 解析请求
    body_bytes: bytes = await request.body()
    raw_headers: dict[str, str] = dict(request.headers)
    # 尝试解析 JSON body
    body_json: dict[str, Any] = {}
    try:
        if body_bytes:
            body_json = __import__("json").loads(body_bytes)
    except (ValueError, TypeError):
        body_json = {}
    # 提取 model 进行网关识别
    model: str | None = _extract_model(body_json)
    is_nvidia: bool = is_nvidia_gateway(model)
    # 非 NVIDIA → 直接转发
    if not is_nvidia:
        await ctx.increment_stat("passthrough_requests")
        try:
            resp = await _forward_to_upstream(
                ctx=ctx,
                method=request.method,
                path=path,
                body=body_bytes if body_bytes else None,
                headers=raw_headers,
                stream=body_json.get("stream", False),
            )
            return _build_response(resp)
        except Exception as exc:
            status, msg = _map_exception(exc)
            logger.error("passthrough_error", path=path, error=str(exc))
            return JSONResponse(
                status_code=status,
                content={"error": {"message": msg, "type": type(exc).__name__}},
            )
    # NVIDIA → 排队 + 限流 + 转发
    await ctx.increment_stat("nvidia_requests")
    priority: Priority = _resolve_priority(raw_headers)
    # 注入内部元数据到 payload
    payload_for_queue: dict[str, Any] = dict(body_json)
    payload_for_queue["_raw_body"] = body_bytes
    # 尝试入队；PASSTHROUGH 策略下队列满时走直通路径
    try:
        request_id = await ctx.priority_queue.put(
            item=payload_for_queue,
            priority=priority,
            headers={
                **raw_headers,
                "x-original-path": path,
                "x-original-method": request.method,
            },
        )
    except QueueFullError:
        await ctx.increment_stat("queue_full_rejects")
        return JSONResponse(
            status_code=503,
            content={
                "error": {
                    "message": "队列已满，当前策略: reject",
                    "type": "QueueFullError",
                }
            },
        )
    except QueueFullPassthrough:
        await ctx.increment_stat("passthrough_requests")
        logger.info("queue_full_passthrough", path=path)
        return await _passthrough_with_rate_limit(ctx, request, path, body_bytes, raw_headers, priority)
    # 创建 future 并注册到 pending
    loop = asyncio.get_running_loop()
    future: asyncio.Future[httpx.Response] = loop.create_future()
    ctx.pending_requests[request_id] = (future, time.monotonic())
    try:
        resp = await future
        return _build_response(resp)
    except _RateLimitedError as exc:
        return JSONResponse(
            status_code=429,
            content={
                "error": {
                    "message": str(exc),
                    "type": "RateLimitedError",
                }
            },
        )
    except Exception as exc:
        status, msg = _map_exception(exc)
        logger.error("proxy_error", path=path, request_id=request_id, error=str(exc))
        return JSONResponse(
            status_code=status,
            content={"error": {"message": msg, "type": type(exc).__name__}},
        )
 def _build_response(resp: httpx.Response) -> Response:
    """将 httpx.Response 转换为 FastAPI Response。
    支持 JSON 和流式 (SSE) 两种响应类型。
    """
    content_type = resp.headers.get("content-type", "")
    # 流式响应 (SSE)
    if "text/event-stream" in content_type or "stream" in content_type:
        return StreamingResponse(
            content=resp.aiter_bytes(),
            status_code=resp.status_code,
            headers={
                k: v for k, v in resp.headers.items()
                if k.lower() not in ("content-encoding", "transfer-encoding")
            },
            media_type="text/event-stream",
        )
    # 普通 JSON 响应
    return Response(
        content=resp.content,
        status_code=resp.status_code,
        headers={
            k: v for k, v in resp.headers.items()
            if k.lower() not in ("content-encoding", "transfer-encoding")
        },
        media_type=content_type or "application/json",
    )
 # ---------------------------------------------------------------------------
 # 路由
 # ---------------------------------------------------------------------------
@app.get("/health")
 async def health(ctx: SidecarContext = Depends(get_context)) -> dict[str, Any]:
    """存活检查 (liveness)。"""
    return ctx.health.liveness()
@app.get("/health/ready")
 async def health_ready(ctx: SidecarContext = Depends(get_context)) -> dict[str, Any]:
    """就绪检查 (readiness)，含上游连通性。
    BIZ-46 Phase3: 复用 ctx.http_client，不再每次创建新 client。
    """
    queue_size = await ctx.priority_queue.get_queue_size()
    bucket_status = ctx.token_bucket.get_status()
    return await ctx.health.readiness(
        upstream_url=ctx.config.upstream_url,
        upstream_api_key=ctx.config.upstream_api_key or "",
        queue_current_size=queue_size,
        queue_max_size=ctx.config.queue_max_size,
        available_tokens=bucket_status["tokens"],
        bucket_capacity=bucket_status["capacity"],
        http_client=ctx.http_client,  # 复用主 client
    )
@app.get("/status")
 async def status(ctx: SidecarContext = Depends(get_context)) -> dict[str, Any]:
    """调试用：限流器 + 队列 + 避退完整状态。"""
    queue_stats = await ctx.priority_queue.get_stats()
    bucket_status = ctx.token_bucket.get_status()
    return {
        "requests": {
            "total": ctx.stats["total_requests"],
            "nvidia": ctx.stats["nvidia_requests"],
            "passthrough": ctx.stats["passthrough_requests"],
            "ratelimited": ctx.stats["ratelimited_requests"],
        },
        "errors": {
            "queue_full_rejects": ctx.stats["queue_full_rejects"],
            "upstream_errors": ctx.stats["upstream_errors"],
        },
        "queue": queue_stats,
        "token_bucket": bucket_status,
        "retreat": {
            "state": ctx.token_bucket.get_retreat_state(),
            "effective_rpm": round(ctx.token_bucket.get_effective_rate_rpm(), 1),
            "base_rpm": round(ctx.token_bucket.get_base_rate_rpm(), 1),
            "upstream_429_rate": round(ctx.token_bucket.get_429_rate(), 4),
        },
        "uptime_seconds": ctx.uptime_seconds,
    }
 # ---- OpenAI 兼容端点 ----
@app.post("/v1/chat/completions")
 async def chat_completions(request: Request, ctx: SidecarContext = Depends(get_context)) -> Response:
    """OpenAI Chat Completions API 代理（含流式支持）。"""
    return await _handle_proxy_request(ctx, request, "/v1/chat/completions")
@app.post("/v1/completions")
 async def completions(request: Request, ctx: SidecarContext = Depends(get_context)) -> Response:
    """OpenAI Completions API 代理（legacy）。"""
    return await _handle_proxy_request(ctx, request, "/v1/completions")
@app.post("/v1/embeddings")
 async def embeddings(request: Request, ctx: SidecarContext = Depends(get_context)) -> Response:
    """OpenAI Embeddings API 代理。"""
    return await _handle_proxy_request(ctx, request, "/v1/embeddings")
@app.get("/v1/models")
@app.get("/v1/models/{model_id:path}")
 async def list_models(request: Request, model_id: str | None = None, ctx: SidecarContext = Depends(get_context)) -> Response:
    """OpenAI Models API 代理。"""
    path = f"/v1/models/{model_id}" if model_id else "/v1/models"
    return await _handle_proxy_request(ctx, request, path)
 # ---- 通用代理（catch-all 用于非标准 NVIDIA 端点） ----
@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"])
 async def catch_all(request: Request, path: str, ctx: SidecarContext = Depends(get_context)) -> Response:
    """通用代理端点：转发任何未匹配的路径到上游。"""
    target_path = f"/{path}" if not path.startswith("/") else path
    return await _handle_proxy_request(ctx, request, target_path)
 # ---------------------------------------------------------------------------
 # 入口
 # ---------------------------------------------------------------------------
 def main() -> None:
    """开发/调试入口。"""
    import uvicorn
    cfg: SidecarConfig = load_config()
    uvicorn.run(
        "nvidia_sidecar.server:app",
        host=cfg.listen_host,
        port=cfg.listen_port,
        log_level=cfg.log_level.lower(),
    )
 if __name__ == "__main__":
    main()
@@ -1,327 +0,0 @@
 <!DOCTYPE html>
 <html lang="zh-CN">
 <head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>NVIDIA Sidecar — 实时仪表盘</title>
  <script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.7/dist/chart.umd.min.js"></script>
  <style>
    * { margin: 0; padding: 0; box-sizing: border-box; }
    body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; background: #0f172a; color: #e2e8f0; padding: 24px; }
    h1 { font-size: 22px; font-weight: 600; margin-bottom: 4px; color: #f8fafc; }
    .subtitle { color: #94a3b8; font-size: 13px; margin-bottom: 24px; }
    .grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(380px, 1fr)); gap: 20px; margin-bottom: 24px; }
    .card { background: #1e293b; border-radius: 12px; padding: 20px; border: 1px solid #334155; }
    .card h2 { font-size: 15px; font-weight: 600; color: #94a3b8; margin-bottom: 14px; text-transform: uppercase; letter-spacing: 0.05em; }
    .card canvas { max-height: 220px; }
    .stat-row { display: flex; gap: 16px; flex-wrap: wrap; }
    .stat { flex: 1; min-width: 100px; background: #0f172a; border-radius: 8px; padding: 12px; text-align: center; border: 1px solid #334155; }
    .stat .value { font-size: 28px; font-weight: 700; color: #38bdf8; }
    .stat .label { font-size: 11px; color: #64748b; margin-top: 4px; text-transform: uppercase; }
    .stat.warn .value { color: #f59e0b; }
    .stat.danger .value { color: #ef4444; }
    .retreat-badge { display: inline-block; padding: 2px 10px; border-radius: 999px; font-size: 12px; font-weight: 600; }
    .retreat-badge.normal { background: #065f46; color: #6ee7b7; }
    .retreat-badge.retreat { background: #78350f; color: #fbbf24; }
    .retreat-badge.recover { background: #1e3a5f; color: #60a5fa; }
    .config-panel { background: #1e293b; border-radius: 12px; padding: 20px; border: 1px solid #334155; }
    .config-panel h2 { font-size: 15px; font-weight: 600; color: #94a3b8; margin-bottom: 14px; text-transform: uppercase; letter-spacing: 0.05em; }
    .config-row { display: flex; align-items: center; gap: 12px; margin-bottom: 12px; flex-wrap: wrap; }
    .config-row label { min-width: 100px; font-size: 13px; color: #cbd5e1; }
    .config-row input, .config-row select { background: #0f172a; border: 1px solid #334155; border-radius: 6px; color: #e2e8f0; padding: 6px 10px; font-size: 13px; }
    .config-row input[type="range"] { width: 140px; }
    .config-row button { background: #38bdf8; color: #0f172a; border: none; border-radius: 6px; padding: 6px 16px; font-size: 13px; font-weight: 600; cursor: pointer; }
    .config-row button:hover { background: #7dd3fc; }
    .config-row button:disabled { background: #475569; cursor: not-allowed; }
    .toast { position: fixed; top: 16px; right: 16px; padding: 10px 20px; border-radius: 8px; font-size: 13px; z-index: 999; animation: fadeInOut 3s; }
    .toast.success { background: #065f46; color: #6ee7b7; }
    .toast.error { background: #7f1d1d; color: #fca5a5; }
    @keyframes fadeInOut { 0% { opacity: 0; transform: translateY(-8px); } 10% { opacity: 1; transform: translateY(0); } 80% { opacity: 1; } 100% { opacity: 0; } }
    .disconnected { background: #7f1d1d; color: #fca5a5; padding: 4px 10px; border-radius: 4px; font-size: 12px; display: inline-block; margin-left: 8px; }
    .connected { background: #065f46; color: #6ee7b7; padding: 4px 10px; border-radius: 4px; font-size: 12px; display: inline-block; margin-left: 8px; }
    /* BIZ-46 Phase3: 队列柱状图 300ms 平滑动画 */
    .queue-bar { transition: height 0.3s ease; }
    /* BIZ-46 Phase3: SSE 断连 5s 半透明遮罩 */
    #reconnect-mask {
      display: none;
      position: fixed;
      top: 0; left: 0; right: 0; bottom: 0;
      background: rgba(15, 23, 42, 0.85);
      z-index: 1000;
      justify-content: center;
      align-items: center;
      flex-direction: column;
    }
    #reconnect-mask.visible { display: flex; }
    #reconnect-mask .mask-icon { font-size: 48px; margin-bottom: 16px; }
    #reconnect-mask .mask-text { color: #94a3b8; font-size: 16px; font-weight: 500; }
    #reconnect-mask .mask-sub { color: #64748b; font-size: 13px; margin-top: 8px; }
  </style>
 </head>
 <body>
  <!-- BIZ-46 Phase3: SSE 断连遮罩 -->
  <div id="reconnect-mask">
    <div class="mask-icon">⚠️</div>
    <div class="mask-text">数据暂不可用</div>
    <div class="mask-sub">SSE 连接中断，正在重连…</div>
  </div>
  <h1>🚀 NVIDIA Sidecar 实时仪表盘
    <span id="conn-status" class="connected">已连接</span>
  </h1>
  <p class="subtitle">令牌桶限流 · 优先级队列 · 避退模式 · 实时监控</p>
  <!-- 状态卡片 -->
  <div class="stat-row" style="margin-bottom: 24px;">
    <div class="stat"><div class="value" id="val-total">0</div><div class="label">总请求</div></div>
    <div class="stat"><div class="value" id="val-nvidia">0</div><div class="label">NVIDIA 请求</div></div>
    <div class="stat"><div class="value" id="val-rate">0</div><div class="label">当前 RPM</div></div>
    <div class="stat"><div class="value" id="val-429">0%</div><div class="label">上游 429 率</div></div>
    <div class="stat"><div class="value" id="val-retreat">正常</div><div class="label">避退状态</div></div>
    <div class="stat"><div class="value" id="val-uptime">0s</div><div class="label">运行时间</div></div>
  </div>
  <!-- 图表 -->
  <div class="grid">
    <div class="card">
      <h2>📊 令牌桶使用率</h2>
      <canvas id="chart-tokens"></canvas>
    </div>
    <div class="card">
      <!-- BIZ-46 Phase3: 队列图标题显示总排队数 -->
      <h2>📈 队列深度 <span id="queue-total" style="font-size:13px;color:#38bdf8;">(共 0)</span></h2>
      <canvas id="chart-queue"></canvas>
    </div>
    <div class="card">
      <h2>📉 请求吞吐量 (最近 20 点)</h2>
      <canvas id="chart-throughput"></canvas>
    </div>
    <div class="card">
      <h2>⚙️ 速率历史</h2>
      <canvas id="chart-rate"></canvas>
    </div>
  </div>
  <!-- 配置面板 -->
  <div class="config-panel">
    <h2>🔧 实时配置</h2>
    <div class="config-row">
      <label>速率 (RPM)</label>
      <input type="range" id="cfg-rate-rpm" min="1" max="100" value="40" oninput="document.getElementById('cfg-rate-val').textContent=this.value">
      <span id="cfg-rate-val" style="min-width:30px;">40</span>
    </div>
    <div class="config-row">
      <label>队列上限</label>
      <input type="number" id="cfg-queue-max" value="500" min="1" max="2000" style="width:80px;">
    </div>
    <div class="config-row">
      <button onclick="applyConfig()">应用配置</button>
    </div>
  </div>
 <script>
 // SSE 连接
 let evtSource = null;
 let dataHistory = { throughput: [], rates: [] };
 const MAX_HISTORY = 20;
 let lastSSETime = Date.now();
 // BIZ-46 Phase3: SSE 断连 5s 遮罩
 function checkReconnect() {
  const mask = document.getElementById('reconnect-mask');
  if (Date.now() - lastSSETime > 5000) {
    mask.classList.add('visible');
  }
 }
 setInterval(checkReconnect, 1000);
 function connectSSE() {
  if (evtSource) evtSource.close();
  evtSource = new EventSource('/api/dashboard/stream');
  evtSource.onmessage = (e) => {
    try {
      const snap = JSON.parse(e.data);
      lastSSETime = Date.now();
      // 隐藏断连遮罩
      document.getElementById('reconnect-mask').classList.remove('visible');
      updateDashboard(snap);
      document.getElementById('conn-status').className = 'connected';
      document.getElementById('conn-status').textContent = '已连接';
    } catch (err) {
      document.getElementById('conn-status').className = 'disconnected';
      document.getElementById('conn-status').textContent = '解析错误';
    }
  };
  evtSource.onerror = () => {
    document.getElementById('conn-status').className = 'disconnected';
    document.getElementById('conn-status').textContent = '断开 - 重连中';
  };
 }
 // 初始化 Chart.js
 const ctxTokens = document.getElementById('chart-tokens').getContext('2d');
 const chartTokens = new Chart(ctxTokens, {
  type: 'doughnut',
  data: {
    labels: ['已用令牌', '可用令牌'],
    datasets: [{ data: [0, 40], backgroundColor: ['#ef4444', '#22c55e'], borderWidth: 0 }]
  },
  options: { responsive: true, maintainAspectRatio: true, cutout: '65%', plugins: { legend: { position: 'bottom', labels: { color: '#94a3b8' } } },
    // BIZ-46 Phase3: 300ms 平滑动画
    animation: { duration: 300 } }
 });
 const ctxQueue = document.getElementById('chart-queue').getContext('2d');
 const chartQueue = new Chart(ctxQueue, {
  type: 'bar',
  data: {
    labels: ['URGENT', 'HIGH', 'NORMAL', 'LOW'],
    datasets: [{ label: '排队数', data: [0, 0, 0, 0], backgroundColor: ['#ef4444', '#f59e0b', '#38bdf8', '#a78bfa'] }]
  },
  options: { responsive: true, maintainAspectRatio: true,
    scales: { y: { beginAtZero: true, ticks: { color: '#94a3b8' } }, x: { ticks: { color: '#94a3b8' } } },
    plugins: { legend: { display: false } },
    // BIZ-46 Phase3: 300ms 平滑动画
    animation: { duration: 300 } }
 });
 const ctxThroughput = document.getElementById('chart-throughput').getContext('2d');
 const chartThroughput = new Chart(ctxThroughput, {
  type: 'line',
  data: { labels: [], datasets: [
    { label: '成功', data: [], borderColor: '#22c55e', backgroundColor: '#22c55e20', fill: false, tension: 0.3, pointRadius: 2 },
    { label: '429', data: [], borderColor: '#f59e0b', backgroundColor: '#f59e0b20', fill: false, tension: 0.3, pointRadius: 2 },
    { label: '直通', data: [], borderColor: '#a78bfa', backgroundColor: '#a78bfa20', fill: false, tension: 0.3, pointRadius: 2 },
  ]},
  options: { responsive: true, maintainAspectRatio: true,
    scales: { y: { beginAtZero: true, ticks: { color: '#94a3b8' } }, x: { ticks: { color: '#94a3b8' } } },
    plugins: { legend: { position: 'bottom', labels: { color: '#94a3b8' } } },
    animation: { duration: 300 } }
 });
 const ctxRate = document.getElementById('chart-rate').getContext('2d');
 const chartRate = new Chart(ctxRate, {
  type: 'line',
  data: { labels: [], datasets: [
    { label: '有效 RPM', data: [], borderColor: '#38bdf8', fill: false, tension: 0.3, pointRadius: 2 },
    { label: '基准 RPM', data: [], borderColor: '#64748b', fill: false, tension: 0.3, pointRadius: 2, borderDash: [4, 4] },
  ]},
  options: { responsive: true, maintainAspectRatio: true,
    scales: { y: { beginAtZero: true, ticks: { color: '#94a3b8' } }, x: { ticks: { color: '#94a3b8' } } },
    plugins: { legend: { position: 'bottom', labels: { color: '#94a3b8' } } },
    animation: { duration: 300 } }
 });
 function updateDashboard(snap) {
  const r = snap.requests || {};
  const tb = snap.token_bucket || {};
  const rt = snap.retreat || {};
  document.getElementById('val-total').textContent = (r.total || 0).toLocaleString();
  document.getElementById('val-nvidia').textContent = (r.nvidia || 0).toLocaleString();
  document.getElementById('val-rate').textContent = Math.round(rt.effective_rpm || 40);
  document.getElementById('val-429').textContent = ((rt.upstream_429_rate || 0) * 100).toFixed(1) + '%';
  document.getElementById('val-uptime').textContent = fmtDuration(snap.uptime_seconds || 0);
  const retreatEl = document.getElementById('val-retreat');
  const state = rt.state || 'normal';
  retreatEl.textContent = state === 'retreat' ? '⚠️ 避退' : state === 'recover' ? '↗ 恢复中' : '✅ 正常';
  retreatEl.style.color = state === 'retreat' ? '#f59e0b' : state === 'recover' ? '#60a5fa' : '#22c55e';
  chartTokens.data.datasets[0].data = [
    Math.round((tb.capacity || 40) - (tb.tokens || 40)),
    Math.round(tb.tokens || 0)
  ];
  chartTokens.update();
  const qs = snap.queue || {};
  const perPriority = qs.per_priority || {};
  const totalQueued = perPriority.URGENT + perPriority.HIGH + perPriority.NORMAL + perPriority.LOW || qs.current_size || 0;
  chartQueue.data.datasets[0].data = [
    perPriority.URGENT || 0,
    perPriority.HIGH || 0,
    perPriority.NORMAL || 0,
    perPriority.LOW || 0
  ];
  chartQueue.update();
  // BIZ-46 Phase3: 队列图标题显示总排队数
  document.getElementById('queue-total').textContent = '(共 ' + totalQueued + ')';
  const now = new Date().toLocaleTimeString();
  const prev = dataHistory.throughput.length > 0 ? dataHistory.throughput[dataHistory.throughput.length - 1].nvidia : 0;
  const throughput = Math.max(0, (r.nvidia || 0) - prev);
  dataHistory.throughput.push({ time: now, nvidia: throughput, ratelimited: r.ratelimited || 0, passthrough: r.passthrough || 0 });
  dataHistory.rates.push({ time: now, effective: rt.effective_rpm || 40, base: rt.base_rpm || 40 });
  if (dataHistory.throughput.length > MAX_HISTORY) dataHistory.throughput.shift();
  if (dataHistory.rates.length > MAX_HISTORY) dataHistory.rates.shift();
  chartThroughput.data.labels = dataHistory.throughput.map(d => d.time);
  chartThroughput.data.datasets[0].data = dataHistory.throughput.map(d => d.nvidia);
  chartThroughput.data.datasets[1].data = dataHistory.throughput.map(d => d.ratelimited);
  chartThroughput.data.datasets[2].data = dataHistory.throughput.map(d => d.passthrough);
  chartThroughput.update();
  chartRate.data.labels = dataHistory.rates.map(d => d.time);
  chartRate.data.datasets[0].data = dataHistory.rates.map(d => d.effective);
  chartRate.data.datasets[1].data = dataHistory.rates.map(d => d.base);
  chartRate.update();
 }
 function fmtDuration(s) {
  if (s < 60) return s + 's';
  if (s < 3600) return Math.floor(s/60) + 'm ' + (s%60) + 's';
  return Math.floor(s/3600) + 'h ' + Math.floor((s%3600)/60) + 'm';
 }
 async function applyConfig() {
  const btn = document.querySelector('.config-row button');
  btn.disabled = true;
  try {
    const resp = await fetch('/api/admin/config', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        rate_rpm: parseInt(document.getElementById('cfg-rate-rpm').value),
        queue_max_size: parseInt(document.getElementById('cfg-queue-max').value),
      })
    });
    const result = await resp.json();
    showToast(resp.ok ? 'success' : 'error', resp.ok ? '配置已更新' : (result.detail || '配置更新失败'));
  } catch (err) {
    showToast('error', '请求失败: ' + err.message);
  }
  btn.disabled = false;
 }
 function showToast(type, msg) {
  const t = document.createElement('div');
  t.className = 'toast ' + type;
  t.textContent = msg;
  document.body.appendChild(t);
  setTimeout(() => t.remove(), 3000);
 }
 // BIZ-46 Phase3: 页面加载时同步当前配置值
 async function loadConfig() {
  try {
    const resp = await fetch('/api/admin/config');
    if (resp.ok) {
      const config = await resp.json();
      document.getElementById('cfg-rate-rpm').value = config.rate_rpm || 40;
      document.getElementById('cfg-rate-val').textContent = config.rate_rpm || 40;
      document.getElementById('cfg-queue-max').value = config.queue_max_size || 500;
    }
  } catch (e) {
    console.warn('配置加载失败（可能需要 Admin Token）', e);
  }
 }
 loadConfig();
 connectSSE();
 </script>
 </body>
 </html>
@@ -1 +0,0 @@
 # nvidia_sidecar tests
@@ -1,207 +0,0 @@
 """
 避退模式并发/死锁回归测试 (BIZ-46 Phase3 6)
 覆盖多线程场景下的 AdaptiveTokenBucket 线程安全性：
 - 并发 record_response + evaluate_retreat
 - 并发 consume + record_response + evaluate_retreat
 - 高负载下避退状态转换正确性
 设计文档: docs/architecture/BIZ-46_Phase3_Architecture_Design.md 6
 """
 from __future__ import annotations
 import threading
 import time
 import pytest
 from nvidia_sidecar.rate_limiter import AdaptiveTokenBucket, RetreatState
 class TestRetreatConcurrency:
    """避退模式并发安全回归测试。"""
    @pytest.mark.asyncio
    async def test_concurrent_record_and_evaluate(self) -> None:
        """多线程同时 record_response + evaluate_retreat 不死锁。
        4 个线程同时操作：
        - 2 个线程执行 record_response (1000 次)
        - 2 个线程执行 evaluate_retreat (1000 次)
        所有线程必须在 10s 内完成，否则判定为死锁。
        """
        bucket = AdaptiveTokenBucket(rate=40 / 60, capacity=40)
        errors: list[Exception] = []
        def worker_record() -> None:
            for i in range(1000):
                try:
                    bucket.record_response(is_429=(i % 10 == 0))
                except Exception as e:
                    errors.append(e)
        def worker_evaluate() -> None:
            for _ in range(1000):
                try:
                    bucket.evaluate_retreat()
                except Exception as e:
                    errors.append(e)
        threads = [
            threading.Thread(target=worker_record),
            threading.Thread(target=worker_record),
            threading.Thread(target=worker_evaluate),
            threading.Thread(target=worker_evaluate),
        ]
        for t in threads:
            t.start()
        for t in threads:
            t.join(timeout=10)
        alive_threads = [t for t in threads if t.is_alive()]
        assert not alive_threads, (
            f"{len(alive_threads)} 个线程未完成，疑似死锁"
        )
        assert not errors, f"并发错误: {errors}"
    @pytest.mark.asyncio
    async def test_concurrent_consume_and_retreat(self) -> None:
        """多线程同时 consume + record_response + evaluate_retreat 不死锁。
        覆盖 _lock (TokenBucket) 和 _retreat_lock (AdaptiveTokenBucket)
        同时被不同线程持有时的交叉锁场景。
        """
        bucket = AdaptiveTokenBucket(rate=40 / 60, capacity=40)
        errors: list[Exception] = []
        def worker_consume() -> None:
            for _ in range(500):
                try:
                    bucket.consume(tokens=1)
                except Exception as e:
                    errors.append(e)
        def worker_retreat() -> None:
            for _ in range(500):
                try:
                    bucket.record_response(is_429=False)
                    bucket.evaluate_retreat()
                except Exception as e:
                    errors.append(e)
        threads = [
            threading.Thread(target=worker_consume),
            threading.Thread(target=worker_consume),
            threading.Thread(target=worker_retreat),
            threading.Thread(target=worker_retreat),
        ]
        for t in threads:
            t.start()
        for t in threads:
            t.join(timeout=10)
        alive_threads = [t for t in threads if t.is_alive()]
        assert not alive_threads, (
            f"{len(alive_threads)} 个线程未完成，疑似死锁"
        )
        assert not errors, f"并发错误: {errors}"
    @pytest.mark.asyncio
    async def test_retreat_state_transitions_under_load(self) -> None:
        """高负载下避退状态转换正确。
        1. 注入 100 个 429 → 验证进入 RETREAT
        2. 注入 200 个成功 → 手动推进时间 → 验证恢复
        """
        bucket = AdaptiveTokenBucket(
            rate=40 / 60,
            capacity=40,
            retreat_window_seconds=0.1,
            retreat_429_threshold=0.05,
            retreat_factor=0.75,
            retreat_min_rpm=5.0,
            recover_window_seconds=0.01,
        )
        # 阶段 1：模拟高 429 率
        for _ in range(100):
            bucket.record_response(is_429=True)
        state = bucket.evaluate_retreat()
        assert state == RetreatState.RETREAT, (
            f"高 429 率应触发避退，实际: {state}"
        )
        assert bucket.get_effective_rate_rpm() < bucket.get_base_rate_rpm(), (
            f"避退后速率应低于基准，实际: "
            f"{bucket.get_effective_rate_rpm()} vs {bucket.get_base_rate_rpm()}"
        )
        # 阶段 2：模拟恢复
        time.sleep(0.15)  # 等待 429 从短窗口中过期
        for _ in range(200):
            bucket.record_response(is_429=False)
        for _ in range(10):
            state = bucket.evaluate_retreat()
        assert state in (RetreatState.RECOVER, RetreatState.NORMAL), (
            f"恢复后应为 RECOVER 或 NORMAL，实际: {state}"
        )
    @pytest.mark.asyncio
    async def test_try_consume_concurrency_safety(self) -> None:
        """并发 try_consume 不死锁。"""
        bucket = AdaptiveTokenBucket(rate=40 / 60, capacity=40)
        errors: list[Exception] = []
        results: list[bool] = []
        def worker() -> None:
            for _ in range(200):
                try:
                    got = bucket.try_consume(tokens=1, timeout=0.1)
                    results.append(got)
                except Exception as e:
                    errors.append(e)
        threads = [threading.Thread(target=worker) for _ in range(8)]
        for t in threads:
            t.start()
        for t in threads:
            t.join(timeout=10)
        alive = [t for t in threads if t.is_alive()]
        assert not alive, f"{len(alive)} 个线程未完成，疑似死锁"
        assert not errors, f"并发错误: {errors}"
        successful = sum(1 for r in results if r)
        assert successful > 0, (
            f"令牌桶应至少成功消费一些令牌，成功: {successful}/{len(results)}"
        )
    @pytest.mark.asyncio
    async def test_high_load_state_coherence(self) -> None:
        """高负载下令牌桶状态一致性：消费总量 ≤ 初始 token + 补充量。"""
        bucket = AdaptiveTokenBucket(rate=10.0, capacity=100)
        consumed_count: list[int] = [0]
        lock = threading.Lock()
        def worker() -> None:
            local_consumed = 0
            for _ in range(50):
                if bucket.consume(tokens=1):
                    local_consumed += 1
                time.sleep(0.001)
            with lock:
                consumed_count[0] += local_consumed
        threads = [threading.Thread(target=worker) for _ in range(10)]
        for t in threads:
            t.start()
        for t in threads:
            t.join(timeout=15)
        max_expected = 100 + int(10.0 * 5)
        assert consumed_count[0] <= max_expected, (
            f"消费量异常: {consumed_count[0]}，应 ≤ {max_expected}"
        )
@@ -1,325 +0,0 @@
 """
 NVIDIA Sidecar — WebUI 后端 API
 提供仪表盘 SSE 实时推送 + 配置热重载 API。
 BIZ-46 Phase3:
 - 架构解耦：移除反向导入 server，改用 Depends(get_context) (§1)
 - SSE 共享缓存：1s TTL snapshot cache，多客户端不重复构建 (§3)
 - Dashboard UX：页面加载同步配置 + 队列深度标题 (§7)
 """
 from __future__ import annotations
 import asyncio
 import json
 import os
 import time
 from pathlib import Path
 from typing import Any, AsyncGenerator
 import structlog
 from fastapi import APIRouter, Depends, HTTPException, Request
 from fastapi.responses import HTMLResponse, JSONResponse, StreamingResponse
 from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
 from pydantic import BaseModel
 from nvidia_sidecar.context import SidecarContext
 webui_router: APIRouter = APIRouter(prefix="/api", tags=["webui"])
 logger: structlog.stdlib.BoundLogger = structlog.get_logger("nvidia_sidecar.webui")
 STATIC_DIR: Path = Path(__file__).parent / "static"
 # dashboard.html 缓存（严维序评审 #6 / 梁思筑评审 #8：避免每次请求读磁盘）
 _dashboard_html_cache: tuple[str, float] | None = None
 _DASHBOARD_CACHE_TTL: float = 300.0  # 5 分钟
 # Admin API 认证（严维序评审 #1）
 _ADMIN_TOKEN: str | None = os.environ.get("SIDECAR_ADMIN_TOKEN")
 _admin_auth_scheme: HTTPBearer = HTTPBearer(auto_error=False)
 def _get_ctx(request: Request) -> SidecarContext:
    """获取 SidecarContext（webui 路由级注入，避免循环导入 server）。"""
    return request.app.state.sidecar  # type: ignore[no-any-return]
 # ---------------------------------------------------------------------------
 # 配置热重载模型
 # ---------------------------------------------------------------------------
 class ConfigPatch(BaseModel):
    """可在线修改的配置字段。"""
    rate_rpm: int | None = None
    queue_max_size: int | None = None
    fallback_enabled_passthrough: bool | None = None
 # ---------------------------------------------------------------------------
 # SSE 快照构建（BIZ-46 Phase3: 1s TTL 共享缓存）
 # ---------------------------------------------------------------------------
 async def _build_snapshot(ctx: SidecarContext) -> dict[str, Any]:
    """构建当前状态快照（从 SidecarContext 读取，含队列深度）。
    BIZ-46 Phase3: 不再通过反向导入 server 访问全局变量。
    """
    try:
        bucket_status = ctx.token_bucket.get_status()
        now = time.time()
        queue_data: dict[str, Any] = {"current_size": 0, "per_priority": {}}
        try:
            queue_stats = await ctx.priority_queue.get_stats()
            queue_data = {
                "max_size": queue_stats.get("max_size", 0),
                "current_size": queue_stats.get("current_size", 0),
                "per_priority": queue_stats.get("depth_by_priority", {}),
                "total_enqueued": queue_stats.get("total_enqueued", 0),
                "total_dequeued": queue_stats.get("total_dequeued", 0),
                "total_dropped": queue_stats.get("total_dropped", 0),
            }
        except Exception:
            logger.warning(
                "queue_stats_unavailable",
                message="队列统计获取失败，仪表盘队列深度可能不准确",
            )
        return {
            "timestamp": now,
            "uptime_seconds": ctx.uptime_seconds,
            "token_bucket": bucket_status,
            "queue": queue_data,
            "retreat": {
                "state": ctx.token_bucket.get_retreat_state(),
                "effective_rpm": round(ctx.token_bucket.get_effective_rate_rpm(), 1),
                "base_rpm": round(ctx.token_bucket.get_base_rate_rpm(), 1),
                "upstream_429_rate": round(ctx.token_bucket.get_429_rate(), 4),
            },
            "requests": {
                "total": ctx.stats.get("total_requests", 0),
                "nvidia": ctx.stats.get("nvidia_requests", 0),
                "passthrough": ctx.stats.get("passthrough_requests", 0),
                "ratelimited": ctx.stats.get("ratelimited_requests", 0),
            },
            "errors": {
                "queue_full_rejects": ctx.stats.get("queue_full_rejects", 0),
                "upstream_errors": ctx.stats.get("upstream_errors", 0),
            },
        }
    except Exception:
        logger.exception("snapshot_build_error")
        return {"error": "snapshot_unavailable", "timestamp": time.time()}
 async def _build_snapshot_cached(ctx: SidecarContext) -> dict[str, Any]:
    """带 1s TTL 的共享快照缓存（BIZ-46 Phase3 §3）。
    多个 SSE 客户端共享同一份快照，避免重复计算和锁竞争。
    性能收益：
    - 1 客户端: 1 次/s 计算（无变化）
    - 5 客户端: ~5 次/s → 1 次/s
    - 20 客户端: ~20 次/s → 1 次/s
    """
    now_cache = time.monotonic()
    if ctx.snapshot_cache is not None:
        data, ts = ctx.snapshot_cache
        if now_cache - ts < ctx.SNAPSHOT_CACHE_TTL:
            return data
    async with ctx.snapshot_cache_lock:
        # Double-check（避免多个协程同时 miss 后重复构建）
        if ctx.snapshot_cache is not None:
            data, ts = ctx.snapshot_cache
            if now_cache - ts < ctx.SNAPSHOT_CACHE_TTL:
                return data
        snapshot = await _build_snapshot(ctx)
        ctx.snapshot_cache = (snapshot, now_cache)
        return snapshot
 # ---------------------------------------------------------------------------
 # 仪表盘 SSE 推送
 # ---------------------------------------------------------------------------
 async def _dashboard_stream(request: Request, ctx: SidecarContext) -> StreamingResponse:
    """SSE 实时推送 Sidecar 完整状态快照（每秒一次）。
    供 dashboard.html 的 EventSource 消费。
    BIZ-46 Phase3: 使用共享缓存 _build_snapshot_cached，多客户端不重复计算。
    """
    async def event_generator() -> AsyncGenerator[str, None]:
        first_frame = True
        while True:
            if await request.is_disconnected():
                break
            try:
                snapshot: dict[str, Any] = await _build_snapshot_cached(ctx)
                payload_sse = f"data: {json.dumps(snapshot, ensure_ascii=False)}\n\n"
                if first_frame:
                    payload_sse = f"retry: 3000\n{payload_sse}"
                    first_frame = False
                yield payload_sse
            except Exception:
                logger.exception("dashboard_sse_error")
                yield f"data: {json.dumps({'error': 'internal'})}\n\n"
            await asyncio.sleep(1.0)
    return StreamingResponse(
        event_generator(),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "X-Accel-Buffering": "no",
        },
    )
 # ---------------------------------------------------------------------------
 # 配置热重载
 # ---------------------------------------------------------------------------
 async def get_config(ctx: SidecarContext) -> dict[str, Any]:
    """获取当前完整配置（从 SidecarContext 读取）。"""
    config = ctx.config
    effective_rpm = float(ctx.token_bucket.get_effective_rate_rpm())
    return {
        "listen_host": config.listen_host,
        "listen_port": config.listen_port,
        "metrics_port": config.metrics_port,
        "upstream_url": config.upstream_url,
        "upstream_api_key": _mask_api_key(config.upstream_api_key),
        "rate_rpm": round(effective_rpm, 1),
        "bucket_capacity": config.bucket_capacity,
        "request_timeout": config.request_timeout,
        "queue_max_size": config.queue_max_size,
        "low_priority_timeout": config.low_priority_timeout,
        "fallback_enabled_passthrough": config.fallback_enabled_passthrough,
        "log_level": config.log_level,
    }
 async def update_config(body: ConfigPatch, ctx: SidecarContext) -> JSONResponse:
    """在线修改配置项并即时生效。"""
    config = ctx.config
    changed: list[str] = []
    if body.rate_rpm is not None:
        if body.rate_rpm <= 0:
            raise HTTPException(status_code=400, detail="rate_rpm must be > 0")
        config.rate_rpm = body.rate_rpm
        ctx.token_bucket.set_rate(body.rate_rpm / 60.0)
        changed.append("rate_rpm")
    if body.queue_max_size is not None:
        if body.queue_max_size <= 0:
            raise HTTPException(status_code=400, detail="queue_max_size must be > 0")
        ok, msg = ctx.priority_queue.set_max_size(body.queue_max_size)
        if not ok:
            raise HTTPException(status_code=400, detail=msg)
        config.queue_max_size = body.queue_max_size
        changed.append("queue_max_size")
        logger.info("queue_max_size_updated", detail=msg)
    if body.fallback_enabled_passthrough is not None:
        config.fallback_enabled_passthrough = body.fallback_enabled_passthrough
        changed.append("fallback_enabled_passthrough")
    logger.info("config_updated", changed=changed)
    return JSONResponse(
        content={"status": "ok", "changed": changed},
    )
 def _mask_api_key(key: str) -> str:
    """对 API Key 进行脱敏处理，仅保留前 4 位以供识别。
    严维序评审 #2 / 沈路明评审 #3：防止 API Key 明文泄露。
    """
    if not key:
        return ""
    if len(key) <= 4:
        return key[:2] + "****"
    return key[:4] + "****"
 # ---------------------------------------------------------------------------
 # 路由注册
 # ---------------------------------------------------------------------------
@webui_router.get("/dashboard/stream")
 async def dashboard_stream(
    request: Request,
    ctx: SidecarContext = Depends(_get_ctx),
 ) -> StreamingResponse:
    """SSE 仪表盘实时推送端点（BIZ-46 Phase3: 使用共享缓存）。"""
    return await _dashboard_stream(request, ctx)
 async def _verify_admin_auth(
    credentials: HTTPAuthorizationCredentials | None = Depends(_admin_auth_scheme),
 ) -> None:
    """Admin API Bearer Token 认证（严维序评审 #1）。
    若设置了 SIDECAR_ADMIN_TOKEN 环境变量，则要求请求携带匹配的 Bearer Token。
    未设置时跳过认证（开发/测试环境）。
    """
    if _ADMIN_TOKEN is None:
        return  # 未配置认证 token，允许无认证访问
    if credentials is None:
        raise HTTPException(status_code=401, detail="需要 Bearer Token 认证（Admin API）")
    if credentials.credentials != _ADMIN_TOKEN:
        raise HTTPException(status_code=403, detail="Admin Token 无效")
@webui_router.get("/admin/config")
 async def admin_get_config(
    _auth: None = Depends(_verify_admin_auth),
    ctx: SidecarContext = Depends(_get_ctx),
 ) -> JSONResponse:
    """获取当前配置（需要 Admin 认证）。"""
    return JSONResponse(content=await get_config(ctx))
@webui_router.post("/admin/config")
 async def admin_update_config(
    body: ConfigPatch,
    _auth: None = Depends(_verify_admin_auth),
    ctx: SidecarContext = Depends(_get_ctx),
 ) -> JSONResponse:
    """在线修改配置（热重载，需要 Admin 认证）。"""
    return await update_config(body, ctx)
 # ---------------------------------------------------------------------------
 # 仪表盘静态页面
 # ---------------------------------------------------------------------------
 def _get_dashboard_html() -> str:
    """获取仪表盘 HTML（带缓存，严维序评审 #6 / 梁思筑评审 #8）。
    首次加载后缓存 5 分钟，避免每次请求读磁盘。
    """
    global _dashboard_html_cache
    now = time.monotonic()
    if _dashboard_html_cache is not None:
        cached_content, cached_at = _dashboard_html_cache
        if now - cached_at < _DASHBOARD_CACHE_TTL:
            return cached_content
    dashboard_path = STATIC_DIR / "dashboard.html"
    if dashboard_path.is_file():
        content = dashboard_path.read_text(encoding="utf-8")
        _dashboard_html_cache = (content, now)
        return content
    return "<h1>dashboard.html not found</h1>"
@webui_router.get("/dashboard", include_in_schema=False)
 async def dashboard_page() -> HTMLResponse:
    """仪表盘 HTML 页面（含缓存策略）。"""
    return HTMLResponse(content=_get_dashboard_html())