chore(BIZ-24): 更新 UUID 映射表和交付物清单为已完成状态

Co-authored-by: multica-agent <github@multica.ai>

docs/architecture/BIZ-46_Phase3_Architecture_Design.md

@@ -1,644 +0,0 @@
-# BIZ-46 Phase3: NVIDIA Sidecar Follow-up 架构设计
-
-> **架构师**: 梁思筑 (architect)  
-> **日期**: 2026-06-24  
-> **状态**: 已批准，推进实施  
-> **来源**: BIZ-42 Phase2 二轮评审 follow-up  
-
----
-
-## 1. 架构解耦 / 依赖注入 — SidecarContext
-
-### 1.1 现状分析
-
-当前 `server.py` 使用 **模块级全局变量** 管理所有核心组件：
-
-```python
-# server.py 全局状态（当前）
-_config: SidecarConfig
-_http_client: httpx.AsyncClient
-_priority_queue: PriorityRequestQueue
-_token_bucket: AdaptiveTokenBucket
-_prometheus: PrometheusMetrics
-_health_service: HealthService
-_pending_requests: dict[str, tuple[asyncio.Future, float]]
-_stats: dict[str, int]
-_stats_lock: asyncio.Lock
-```
-
-**问题**：
-- `webui.py` 通过 `from nvidia_sidecar import server` 反向导入全局变量（循环依赖风险）
-- 单元测试需要 mock 模块级变量，无法并行运行测试
-- 未来多实例/多租户扩展需重写全部模块访问逻辑
-
-### 1.2 设计方案 — SidecarContext + FastAPI Dependency Injection
-
-#### 1.2.1 核心数据结构
-
-```python
-# context.py
-from dataclasses import dataclass, field
-import asyncio
-import httpx
-from typing import Any
-
-@dataclass
-class SidecarContext:
-    """Sidecar 全局运行时上下文 — 所有核心组件的唯一容器。
-    
-    通过 app.state.sidecar 注入 FastAPI，路由通过 Depends 获取。
-    """
-    config: 'SidecarConfig'
-    http_client: httpx.AsyncClient
-    token_bucket: 'AdaptiveTokenBucket'
-    priority_queue: 'PriorityRequestQueue'
-    prometheus: 'PrometheusMetrics'
-    health: 'HealthService'
-    pending_requests: dict[str, tuple['asyncio.Future', float]] = field(default_factory=dict)
-    stats: dict[str, int] = field(default_factory=lambda: {
-        "total_requests": 0,
-        "nvidia_requests": 0,
-        "passthrough_requests": 0,
-        "ratelimited_requests": 0,
-        "queue_full_rejects": 0,
-        "upstream_errors": 0,
-        "start_time": 0,
-    })
-    stats_lock: asyncio.Lock = field(default_factory=asyncio.Lock)
-    
-    async def increment_stat(self, key: str, delta: int = 1) -> None:
-        """线程安全的统计计数器自增。"""
-        async with self.stats_lock:
-            self.stats[key] = self.stats.get(key, 0) + delta
-```
-
-#### 1.2.2 注入方式
-
-```python
-# server.py — lifespan 中创建 context
-from nvidia_sidecar.context import SidecarContext
-
-@asynccontextmanager
-async def lifespan(app: FastAPI):
-    ctx = SidecarContext(
-        config=load_config(),
-        http_client=httpx.AsyncClient(...),
-        token_bucket=AdaptiveTokenBucket(...),
-        priority_queue=PriorityRequestQueue(...),
-        prometheus=PrometheusMetrics(),
-        health=HealthService(),
-    )
-    app.state.sidecar = ctx  # 注入 FastAPI
-    # ... worker 启动 ...
-    yield
-    # ... 清理 ...
-
-# 依赖注入函数
-def get_context(request: Request) -> SidecarContext:
-    return request.app.state.sidecar
-
-# 路由使用
-@app.post("/v1/chat/completions")
-async def chat_completions(request: Request, ctx: SidecarContext = Depends(get_context)):
-    return await _handle_proxy_request(request, "/v1/chat/completions", ctx)
-```
-
-#### 1.2.3 webui.py 解耦
-
-```python
-# webui.py — 不再反向导入 server
-from nvidia_sidecar.context import SidecarContext
-from fastapi import Depends
-
-def get_webui_router():
-    router = APIRouter(prefix="/api", tags=["webui"])
-    
-    def _get_ctx(request: Request) -> SidecarContext:
-        return request.app.state.sidecar
-    
-    @router.get("/dashboard/stream")
-    async def dashboard_stream(request: Request, ctx: SidecarContext = Depends(_get_ctx)):
-        return await _dashboard_stream(request, ctx)
-    
-    @router.get("/admin/config")
-    async def admin_get_config(ctx: SidecarContext = Depends(_get_ctx)):
-        return await get_config(ctx)
-    
-    return router
-```
-
-#### 1.2.4 Trade-off 分析
-
-| 维度 | 当前（全局变量） | 方案A（SidecarContext） | 方案B（FastAPI Dependency 全函数式） |
-|------|------------------|------------------------|-------------------------------------|
-| 可测试性 | 差（需 mock 模块） | 好（注入 mock context） | 优（每个依赖独立注入） |
-| 改动量 | 无 | 中等（~8 文件） | 大（每个函数签名变更） |
-| 可读性 | 一般 | 好（ctx 一目了然） | 差（参数列表膨胀） |
-| 多实例支持 | 不支持 | 支持（多 app 多 ctx） | 支持 |
-| 循环依赖 | 有（webui→server） | 消除 | 消除 |
-
-**决策**: 采用方案A（SidecarContext），平衡改动量与收益。
-
-### 1.3 迁移计划
-
-分 3 步渐进迁移，每步可独立合入：
-
-1. **Step 1**: 创建 `context.py`，定义 `SidecarContext`，在 `lifespan` 中实例化并挂到 `app.state`
-2. **Step 2**: 路由函数改为 `Depends(get_context)`，删除模块级 `_config`、`_http_client` 等
-3. **Step 3**: `webui.py` 移除 `from nvidia_sidecar import server`，改用依赖注入
-
----
-
-## 2. Prometheus 标签基数治理
-
-### 2.1 现状
-
-当前使用 `model_id` 作为 label 的指标：
-
-| 指标 | Label | 风险 |
-|------|-------|------|
-| `sidecar_upstream_latency_seconds` | `model_id` | **高** — NVIDIA 模型名含版本号，可能无界增长 |
-| `sidecar_upstream_errors_total` | `status_code`, `model_id` | **中** — 组合基数 = 模型数 × 状态码数 |
-
-### 2.2 基数评估
-
-NVIDIA API 当前已知模型约 20-30 个，但：
-- 新模型持续发布（每月 2-5 个）
-- 模型名含版本后缀（`nvidia/deepseek-ai/deepseek-v4-pro`、`nvidia/llama-3.1-70b-instruct` 等）
-- 长期运行（6 个月+）可能累积 100+ 标签组合
-
-**结论**: 当前基数可控（<200 组合），但长期存在膨胀风险，应提前治理。
-
-### 2.3 治理方案
-
-| 指标 | 当前 Label | 调整后 Label | 理由 |
-|------|-----------|-------------|------|
-| `upstream_latency_seconds` | `model_id` | `provider` | provider 固定为 `nvidia`，基数=1 |
-| `upstream_errors_total` | `status_code`, `model_id` | `status_code`, `provider` | 同上 |
-
-**模型级信息迁移路径**：
-- 模型 ID → 结构化 JSON 日志（structlog 已支持）
-- 需要模型级延迟分析时 → 临时 `/status` API 查询或日志聚合
-
-```python
-# metrics.py 调整
-self.upstream_latency_seconds: Histogram = Histogram(
-    "sidecar_upstream_latency_seconds",
-    "Upstream response latency in seconds",
-    labelnames=["provider"],  # 原: ["model_id"]
-    buckets=(...),
-)
-
-self.upstream_errors_total: Counter = Counter(
-    "sidecar_upstream_errors_total",
-    "Upstream error count by status code",
-    labelnames=["status_code", "provider"],  # 原: ["status_code", "model_id"]
-)
-```
-
-```python
-# server.py 调整 — 模型信息改记日志
-model_id = _extract_model(payload) or "unknown"
-provider = "nvidia"  # 固定值，因为只有 NVIDIA 请求走 worker
-_prometheus.record_upstream_latency(provider, upstream_latency)
-if not resp.is_success:
-    _prometheus.record_upstream_error(resp.status_code, provider)
-logger.info("request_completed", model_id=model_id, ...)  # JSON 日志保留模型信息
-```
-
-### 2.4 Trade-off
-
-| 维度 | 保留 model_id | 收敛为 provider |
-|------|--------------|----------------|
-| 基数风险 | 高（无界） | 无（固定=1） |
-| 模型级分析 | Prometheus 原生查询 | 需日志聚合 |
-| 迁移成本 | 无 | 低（改 2 个指标定义 + 调用点） |
-
-**决策**: 收敛为 `provider`，模型级分析通过 JSON 日志 + 日志聚合系统（ELK/Loki）完成。
-
----
-
-## 3. SSE 快照共享缓存
-
-### 3.1 现状
-
-每个 SSE 客户端每秒独立调用 `_build_snapshot()`，该方法：
-- 获取 `_stats` 字典（需锁）
-- 调用 `_token_bucket.get_status()`（需锁）
-- 调用 `_priority_queue.get_stats()`（需 asyncio.Lock）
-
-当 N 个仪表盘同时打开时，每秒 N 次锁竞争 + N 次重复计算。
-
-### 3.2 设计方案 — 1s TTL 共享缓存
-
-```python
-# webui.py
-_snapshot_cache: tuple[dict[str, Any], float] | None = None  # (data, timestamp)
-_snapshot_lock: asyncio.Lock = asyncio.Lock()
-_SNAPSHOT_TTL: float = 1.0  # 1 秒 TTL
-
-async def _build_snapshot_cached(ctx: SidecarContext) -> dict[str, Any]:
-    """带 1s TTL 的共享快照缓存。
-    
-    多个 SSE 客户端共享同一份快照，避免重复计算和锁竞争。
-    """
-    global _snapshot_cache
-    
-    now = time.monotonic()
-    if _snapshot_cache is not None:
-        data, ts = _snapshot_cache
-        if now - ts < _SNAPSHOT_TTL:
-            return data
-    
-    async with _snapshot_lock:
-        # Double-check（避免多个协程同时 miss 后重复构建）
-        if _snapshot_cache is not None:
-            data, ts = _snapshot_cache
-            if now - ts < _SNAPSHOT_TTL:
-                return data
-        
-        snapshot = await _build_snapshot(ctx)
-        _snapshot_cache = (snapshot, now)
-        return snapshot
-```
-
-### 3.3 性能收益
-
-| 场景 | 当前 | 优化后 |
-|------|------|--------|
-| 1 客户端 | 1 次/s 计算 | 1 次/s 计算（无变化） |
-| 5 客户端 | 5 次/s 计算，5 次锁竞争 | 1 次/s 计算，1 次锁竞争 |
-| 20 客户端 | 20 次/s 计算，20 次锁竞争 | 1 次/s 计算，1 次锁竞争 |
-
----
-
-## 4. 部署支撑
-
-### 4.1 Dockerfile
-
-```dockerfile
-# services/nvidia_sidecar/Dockerfile
-FROM python:3.12-slim AS base
-
-WORKDIR /app
-
-# 安装依赖（利用 Docker 层缓存）
-COPY pyproject.toml .
-RUN pip install --no-cache-dir -e .
-
-# 复制源码
-COPY . .
-
-# 非 root 用户运行
-RUN useradd -r -s /bin/false sidecar
-USER sidecar
-
-# 健康检查
-HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
-    CMD python -c "import httpx; r=httpx.get('http://127.0.0.1:9190/health'); exit(0 if r.status_code==200 else 1)"
-
-EXPOSE 9190 9191
-
-CMD ["uvicorn", "nvidia_sidecar.server:app", "--host", "0.0.0.0", "--port", "9190"]
-```
-
-### 4.2 systemd Service
-
-```ini
-# services/nvidia_sidecar/deploy/nvidia-sidecar.service
-[Unit]
-Description=NVIDIA Sidecar Rate-Limiting Proxy
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User=sidecar
-Group=sidecar
-WorkingDirectory=/opt/nvidia-sidecar
-ExecStart=/opt/nvidia-sidecar/.venv/bin/uvicorn nvidia_sidecar.server:app \
-    --host 127.0.0.1 \
-    --port 9190 \
-    --log-level info
-Restart=always
-RestartSec=5
-
-# 环境变量
-EnvironmentFile=/opt/nvidia-sidecar/.env
-
-# 安全加固
-NoNewPrivileges=true
-ProtectSystem=strict
-ProtectHome=true
-PrivateTmp=true
-ReadWritePaths=/opt/nvidia-sidecar/logs
-
-# 资源限制
-LimitNOFILE=65536
-MemoryMax=512M
-
-[Install]
-WantedBy=multi-user.target
-```
-
-### 4.3 环境变量清单
-
-| 变量 | 默认值 | 说明 |
-|------|--------|------|
-| `SIDECAR_HOST` | `127.0.0.1` | 监听地址 |
-| `SIDECAR_PORT` | `9190` | 代理端口 |
-| `SIDECAR_METRICS_PORT` | `9191` | Prometheus 指标端口 |
-| `SIDECAR_UPSTREAM` | `https://integrate.api.nvidia.com/v1` | 上游 API |
-| `SIDECAR_API_KEY` | (必填) | NVIDIA API Key |
-| `SIDECAR_RATE_RPM` | `40` | 限流速率 (RPM) |
-| `SIDECAR_BUCKET_CAPACITY` | `40` | 令牌桶容量 |
-| `SIDECAR_TIMEOUT` | `60` | 请求超时 (秒) |
-| `SIDECAR_QUEUE_MAX` | `500` | 队列最大容量 |
-| `SIDECAR_LOW_TIMEOUT` | `2` | 低优先级超时 (秒) |
-| `SIDECAR_FALLBACK_PASSTHROUGH` | `true` | 队列满时是否直通 |
-| `SIDECAR_LOG_LEVEL` | `INFO` | 日志级别 |
-| `SIDECAR_ADMIN_TOKEN` | (可选) | Admin API 认证 Token |
-
-### 4.4 防火墙建议
-
-```
-# 仅允许内网访问代理端口
-sudo ufw allow from 192.168.1.0/24 to any port 9190
-sudo ufw allow from 192.168.1.0/24 to any port 9191
-# 禁止外网访问
-sudo ufw deny 9190
-sudo ufw deny 9191
-```
-
----
-
-## 5. Readiness HTTP Client 复用
-
-### 5.1 现状
-
-`HealthService.check_upstream()` 每次调用创建新的 `httpx.AsyncClient`：
-
-```python
-# health.py — 当前
-async def check_upstream(self, upstream_url: str, timeout: float = 5.0, api_key: str = "") -> bool:
-    async with httpx.AsyncClient(timeout=timeout) as client:  # 每次新建！
-        resp = await client.get(...)
-```
-
-K8s/systemd 每 10-30s 探测一次，每次创建+销毁 HTTP client 带来不必要的 TCP 连接开销。
-
-### 5.2 方案 — 复用主 http_client
-
-```python
-# health.py — 优化后
-async def check_upstream(
-    self,
-    upstream_url: str,
-    http_client: httpx.AsyncClient,  # 注入主 client
-    api_key: str = "",
-    timeout: float = 5.0,
-) -> bool:
-    try:
-        headers = {}
-        if api_key:
-            headers["authorization"] = f"Bearer {api_key}"
-        resp = await http_client.get(
-            f"{upstream_url.rstrip('/')}/v1/models",
-            headers=headers,
-            timeout=timeout,
-        )
-        return resp.status_code < 500
-    except Exception:
-        return False
-```
-
-```python
-# server.py — 路由调用处
-@app.get("/health/ready")
-async def health_ready(ctx: SidecarContext = Depends(get_context)):
-    queue_size = await ctx.priority_queue.get_queue_size()
-    bucket_status = ctx.token_bucket.get_status()
-    return await ctx.health.readiness(
-        upstream_url=ctx.config.upstream_url,
-        http_client=ctx.http_client,  # 复用主 client
-        upstream_api_key=ctx.config.upstream_api_key or "",
-        queue_current_size=queue_size,
-        queue_max_size=ctx.config.queue_max_size,
-        available_tokens=bucket_status["tokens"],
-        bucket_capacity=bucket_status["capacity"],
-    )
-```
-
-**注意**: readiness 检查使用较短 timeout (5s)，不影响主代理请求的 timeout 配置。httpx 支持per-request timeout 覆盖。
-
----
-
-## 6. Retreat 并发/死锁回归测试
-
-### 6.1 风险点
-
-`AdaptiveTokenBucket` 有两把锁：
-- `_lock` (Lock): 保护令牌消费/补充
-- `_retreat_lock` (RLock): 保护避退状态机
-
-潜在死锁路径：
-1. `evaluate_retreat()` 持有 `_retreat_lock` → 调用 `get_429_rate()` (也获取 `_retreat_lock`，RLock 可重入 ✅)
-2. `evaluate_retreat()` → `_apply_retreat()` → `set_rate()` → 获取 `_lock` (另一把锁)
-3. Worker 线程: `consume()` 持有 `_lock` → 不调用 `_retreat_lock` (无交叉 ✅)
-
-当前设计使用 RLock 已规避了重入死锁，但需要回归测试确保未来修改不引入死锁。
-
-### 6.2 测试用例
-
-```python
-# tests/test_retreat_concurrency.py
-import pytest
-import asyncio
-import threading
-from nvidia_sidecar.rate_limiter import AdaptiveTokenBucket, RetreatState
-
-class TestRetreatConcurrency:
-    """避退模式并发安全回归测试。"""
-
-    @pytest.mark.asyncio
-    async def test_concurrent_record_and_evaluate(self):
-        """多线程同时 record_response + evaluate_retreat 不死锁。"""
-        bucket = AdaptiveTokenBucket(rate=40/60, capacity=40)
-        errors: list[Exception] = []
-        
-        def worker_record():
-            for i in range(1000):
-                try:
-                    bucket.record_response(is_429=(i % 10 == 0))
-                except Exception as e:
-                    errors.append(e)
-        
-        def worker_evaluate():
-            for _ in range(1000):
-                try:
-                    bucket.evaluate_retreat()
-                except Exception as e:
-                    errors.append(e)
-        
-        threads = [
-            threading.Thread(target=worker_record),
-            threading.Thread(target=worker_record),
-            threading.Thread(target=worker_evaluate),
-            threading.Thread(target=worker_evaluate),
-        ]
-        for t in threads:
-            t.start()
-        for t in threads:
-            t.join(timeout=10)
-        
-        # 所有线程必须在 10s 内完成（无死锁）
-        assert all(not t.is_alive() for t in threads), "线程未完成，疑似死锁"
-        assert not errors, f"并发错误: {errors}"
-
-    @pytest.mark.asyncio
-    async def test_concurrent_consume_and_retreat(self):
-        """多线程同时 consume + evaluate_retreat 不死锁。"""
-        bucket = AdaptiveTokenBucket(rate=40/60, capacity=40)
-        errors: list[Exception] = []
-        
-        def worker_consume():
-            for _ in range(500):
-                try:
-                    bucket.consume(tokens=1)
-                except Exception as e:
-                    errors.append(e)
-        
-        def worker_retreat():
-            for _ in range(500):
-                try:
-                    bucket.record_response(is_429=False)
-                    bucket.evaluate_retreat()
-                except Exception as e:
-                    errors.append(e)
-        
-        threads = [
-            threading.Thread(target=worker_consume),
-            threading.Thread(target=worker_consume),
-            threading.Thread(target=worker_retreat),
-            threading.Thread(target=worker_retreat),
-        ]
-        for t in threads:
-            t.start()
-        for t in threads:
-            t.join(timeout=10)
-        
-        assert all(not t.is_alive() for t in threads), "线程未完成，疑似死锁"
-        assert not errors, f"并发错误: {errors}"
-
-    def test_retreat_state_transitions_under_load(self):
-        """高负载下避退状态转换正确。"""
-        bucket = AdaptiveTokenBucket(
-            rate=40/60, capacity=40,
-            retreat_429_threshold=0.05,
-            retreat_factor=0.75,
-        )
-        
-        # 模拟高 429 率
-        for _ in range(100):
-            bucket.record_response(is_429=True)
-        
-        state = bucket.evaluate_retreat()
-        assert state == RetreatState.RETREAT
-        assert bucket.get_effective_rate_rpm() < bucket.get_base_rate_rpm()
-        
-        # 模拟恢复
-        for _ in range(200):
-            bucket.record_response(is_429=False)
-        
-        # 需要等待 RECOVER_WINDOW
-        import time
-        time.sleep(0.1)  # 确保时间窗口过去
-        bucket._last_state_change = 0  # 强制触发时间条件
-        state = bucket.evaluate_retreat()
-        assert state in (RetreatState.RECOVER, RetreatState.NORMAL)
-```
-
----
-
-## 7. Dashboard UX 优化
-
-### 7.1 优化项清单
-
-| # | 优化项 | 实现方式 | 优先级 |
-|---|--------|---------|--------|
-| 1 | 队列柱状图 300ms 平滑动画 | CSS `transition: height 300ms ease` | P1 |
-| 2 | SSE 断连 5s 遮罩 | JS 定时器 + DOM 遮罩层 | P1 |
-| 3 | 队列图标题显示总排队数 | SSE 数据已有 `current_size`，更新标题 | P2 |
-| 4 | 页面加载同步配置 | `fetch('/api/admin/config')` 初始化表单 | P2 |
-
-### 7.2 关键实现
-
-```javascript
-// dashboard.html — SSE 断连检测
-let lastSSETime = Date.now();
-let reconnectMask = document.getElementById('reconnect-mask');
-
-eventSource.onmessage = (event) => {
-    lastSSETime = Date.now();
-    reconnectMask.style.display = 'none';
-    // ... 更新 UI ...
-};
-
-// 5s 无数据 → 显示遮罩
-setInterval(() => {
-    if (Date.now() - lastSSETime > 5000) {
-        reconnectMask.style.display = 'flex';
-    }
-}, 1000);
-
-// 队列柱状图动画
-// CSS: .queue-bar { transition: height 0.3s ease; }
-```
-
-```javascript
-// 页面加载时同步配置
-async function loadConfig() {
-    try {
-        const resp = await fetch('/api/admin/config');
-        if (resp.ok) {
-            const config = await resp.json();
-            document.getElementById('rate-rpm').value = config.rate_rpm;
-            document.getElementById('queue-max').value = config.queue_max_size;
-            // ...
-        }
-    } catch (e) {
-        console.warn('配置加载失败（可能需要 Admin Token）', e);
-    }
-}
-loadConfig();
-```
-
----
-
-## 8. 实施排期
-
-| 阶段 | 内容 | 预估工时 | 依赖 |
-|------|------|---------|------|
-| **D1** | SidecarContext Step 1-3（解耦迁移） | 8h | 无 |
-| **D2** | Prometheus 标签收敛 + 日志增强 | 2h | D1 |
-| **D2** | SSE 共享缓存 | 2h | D1 |
-| **D2** | Readiness HTTP client 复用 | 1h | D1 |
-| **D3** | Dockerfile + systemd service | 2h | 无 |
-| **D3** | Dashboard UX 优化 | 3h | 无 |
-| **D3** | Retreat 并发回归测试 | 3h | 无 |
-| **D4** | 集成测试 + mypy strict | 4h | D1-D3 |
-| **合计** | | **25h** | |
-
----
-
-## 9. 验收标准映射
-
-| Issue 要求 | 本文档章节 | 状态 |
-|-----------|-----------|------|
-| SidecarContext / DI 方案落地或 ADR | §1 | ✅ 详细设计 + 迁移计划 |
-| Prometheus 高基数 label 收敛 | §2 | ✅ 收敛为 provider |
-| SSE snapshot 共享缓存 | §3 | ✅ 1s TTL 设计 |
-| Dockerfile + systemd + 部署 SOP | §4 | ✅ 完整文件 |
-| readiness 复用 HTTP client | §5 | ✅ 注入主 client |
-| retreat 并发/死锁回归测试 | §6 | ✅ 测试用例 |
-| Dashboard UX 细节 | §7 | ✅ 4 项优化 |

knowledge/README.md

@@ -12,10 +12,8 @@
 | [产品/](产品/) | PRD、需求分析 | 沈路明 (productmanager) | — |
 | [技术/](技术/) | 开发规范、代码审查 | 徐聪 (costcodev) | — |
 | [设计/](设计/) | UI设计、品牌规范 | 苏绘锦 (designer) | — |
-| [运维/](运维/) | 部署流程、故障排查、服务器运维 | 严维序 (opengineer) | 3 |
 | [运营/](运营/) | 活动策划、数据分析 | 陆怀瑾 (coo) | — |
 | [行政/](行政/) | 合同、报销流程 | 刘诗妮 (secretary) | — |
-| [规范/](规范/) | 运维标准、安全基线、合规要求 | 严维序 (opengineer) | — |
 
 ## 知识条目格式
 

knowledge/技术/README.md

@@ -5,9 +5,7 @@
 
 ## 知识范围
 
-涵盖开发规范、代码审查、架构设计、技术选型等技术团队核心知识。
-
-> ⚠️ 部署运维知识已迁移至 [运维/](../运维/) 领域。
+涵盖开发规范、代码审查、架构设计、部署运维、技术选型等技术团队知识。
 
 ## 条目清单
 

knowledge/规范/README.md

@@ -1,25 +0,0 @@
-# 规范领域知识
-
-**责任人**：严维序（opengineer）
-**审核人**：陆怀瑾（coo）
-
-## 知识范围
-
-涵盖运维规范、安全标准、合规要求等规范类知识条目，支撑团队标准化运作。
-
-## 条目清单
-
-| 文件名 | 说明 | 状态 |
-|--------|------|------|
-| [服务器运维标准_v1.0.md](../运维/服务器运维标准_v1.0.md) | 服务器巡检、监控、备份运维标准 | 见运维域 |
-
-## 待建设
-
-- 数据库运维标准
-- 安全审计基线
-- 数据合规处理流程
-
----
-
-> 维护者：严维序（opengineer）
-> 最后更新：2026-06-24

knowledge/运维/README.md

@@ -1,27 +0,0 @@
-# 运维领域知识
-
-**责任人**：严维序（opengineer）
-**审核人**：陆怀瑾（coo）
-
-## 知识范围
-
-涵盖服务器运维、部署流程、故障排查、监控配置、安全保障等运维团队核心知识。
-
-## 条目清单
-
-| 文件名 | 说明 | 状态 |
-|--------|------|------|
-| [部署流程_v1.0.md](部署流程_v1.0.md) | 服务部署 SOP 与变更管理流程 | ✅ |
-| [故障排查手册_v1.0.md](故障排查手册_v1.0.md) | 常见故障定位与处置方案 | ✅ |
-| [服务器运维标准_v1.0.md](服务器运维标准_v1.0.md) | 服务器巡检、监控、备份运维标准 | 🆕 |
-
-## 待建设
-
-- 数据库运维指南
-- 安全加固检查清单
-- 灾备与应急恢复预案
-
----
-
-> 维护者：严维序（opengineer）
-> 最后更新：2026-06-24

knowledge/运维/故障排查手册_v1.0.md

@@ -1,274 +0,0 @@
-# 故障排查手册
-
-## 元数据
-
-| 属性 | 值 |
-|------|-----|
-| **领域** | 运维 |
-| **责任人** | 严维序（opengineer） |
-| **版本** | v1.0 |
-| **创建日期** | 2026-06-24 |
-| **最后更新** | 2026-06-24 |
-| **标签** | 故障排查, 运维, 排障 |
-
-## 概述
-
-本手册汇总 BizWings 环境中常见的系统与服务故障定位方法和修复方案。覆盖 SSH 连接、Nginx、数据库、磁盘、Docker 等核心场景。
-
----
-
-## 一、SSH 连接故障
-
-### 1.1 连接超时
-
-```bash
-# 诊断步骤
-ssh -vvv root@<ip> -p <port>    # 查看详细连接日志
-ping <ip>                        # 检查网络连通性
-nmap <ip> -p <port>              # 检查端口状态
-```
-
-**常见原因**：
-- 目标服务器防火墙未开放端口
-- 源 IP 未加入白名单
-- 服务器负载过高，sshd 响应慢
-
-**解决方案**：
-1. 检查服务器防火墙：`iptables -L -n` 或 `ufw status`
-2. 检查 sshd 是否运行：`systemctl status sshd`
-3. 检查负载：`top -n1 | head -5`
-
-### 1.2 认证失败
-
-```bash
-# 诊断步骤
-ssh -p <port> root@<ip>          # 尝试密码登录
-# Permission denied (publickey,password) 提示
-```
-
-**常见原因**：
-- 密码错误（检查 TOOLS.md 中记录）
-- SSH 密钥认证配置错误
-- `/etc/ssh/sshd_config` 中 `PasswordAuthentication no`
-
-**解决方案**：
-1. 确认密码与 TOOLS.md 一致
-2. 检查 `sshd_config`：`grep PasswordAuthentication /etc/ssh/sshd_config`
-3. 临时允许密码登录：`sed -i 's/PasswordAuthentication no/PasswordAuthentication yes/' /etc/ssh/sshd_config && systemctl reload sshd`
-
----
-
-## 二、Nginx 服务异常
-
-### 2.1 Nginx 启动失败 / 卡在 activating
-
-```bash
-# 诊断步骤
-systemctl status nginx           # 查看状态
-journalctl -u nginx --no-pager -n 50   # 查看日志
-nginx -t                          # 配置语法检查
-```
-
-**根因（经验）**：进程残留导致端口占用
-
-```bash
-# 修复
-pkill -9 nginx                   # 强制清理残留进程
-sleep 2
-systemctl start nginx            # 重新启动
-systemctl status nginx           # 确认状态
-```
-
-### 2.2 502 Bad Gateway
-
-```bash
-# 诊断步骤
-curl -I http://localhost:<upstream-port>    # 检查上游服务
-ss -tlnp | grep <upstream-port>             # 检查端口监听
-systemctl status <upstream-service>         # 检查上游进程
-```
-
-**常见原因**：
-- 上游服务未启动或崩溃
-- 连接池耗尽
-
-**解决方案**：
-1. 重启上游服务：`systemctl restart <service>`
-2. 检查 `upstream` 配置是否正确
-
-### 2.3 日志轮转失败
-
-```bash
-# 诊断步骤
-cat /var/log/nginx/error.log | head        # 查看是否有日志无法写入
-ls -la /var/log/nginx/                     # 查看日志文件
-/usr/sbin/logrotate -d /etc/logrotate.d/nginx  # 测试 logrotate
-```
-
-**修复方案**：
-```bash
-# 修改 /etc/logrotate.d/nginx 中的 postrotate 脚本
-# 将 invoke-rc.d nginx rotate 改为：
-postrotate
-    systemctl reload nginx
-endscript
-```
-
----
-
-## 三、数据库连接故障
-
-### 3.1 MySQL 连接失败
-
-```bash
-# 诊断步骤
-mysql -h <host> -P <port> -u root -p      # 测试连接
-telnet <host> <port>                       # 检查端口
-systemctl status mysql                     # 检查服务
-```
-
-**常见原因**：
-- 服务未运行
-- 防火墙未放行 3306 端口
-- 用户权限 / host 限制
-- 连接数超限
-
-**解决方案**：
-```bash
-# 检查连接数
-mysql -e "SHOW VARIABLES LIKE 'max_connections';"
-mysql -e "SHOW PROCESSLIST;"
-
-# 检查用户权限
-mysql -e "SELECT user, host FROM mysql.user WHERE user='root';"
-```
-
-### 3.2 MySQL 空间不足
-
-```bash
-# 诊断
-df -h                                   # 磁盘空间
-mysql -e "SELECT table_schema, ROUND(SUM(data_length+index_length)/1024/1024,2) AS size_mb FROM information_schema.tables GROUP BY table_schema ORDER BY size_mb DESC;"
-```
-
-**解决方案**：
-- 清理过期 binlog：`PURGE BINARY LOGS BEFORE DATE_SUB(NOW(), INTERVAL 7 DAY);`
-- 清理临时表
-- 扩展磁盘
-
----
-
-## 四、磁盘空间告警
-
-### 4.1 诊断
-
-```bash
-df -h                                   # 查看各分区使用率
-du -sh /* 2>/dev/null | sort -rh | head -10   # 找到大文件目录
-find / -type f -size +100M -exec ls -lh {} \; 2>/dev/null   # 大文件定位
-```
-
-### 4.2 清理方案
-
-```bash
-# Docker 日志和镜像清理
-docker system prune -af --volumes       # 清理未使用的 Docker 资源
-
-# 系统日志轮转
-journalctl --vacuum-time=7d             # 清理 7 天前的 journal 日志
-
-# 应用日志归档
-find /var/log -name "*.log" -mtime +30 -exec gzip {} \;   # 压缩旧日志
-find /var/log -name "*.gz" -mtime +90 -delete              # 删除 90 天前的压缩日志
-```
-
----
-
-## 五、Docker 容器异常
-
-### 5.1 容器停止
-
-```bash
-docker ps -a | grep <container>         # 查看容器状态
-docker logs <container> --tail 50       # 查看最近日志
-```
-
-**修复**：
-```bash
-docker start <container>                # 手动启动
-docker compose -f <path> up -d         # 使用 Compose 重启
-```
-
-### 5.2 Docker API 无响应
-
-```bash
-systemctl status docker                 # 检查 Docker 服务
-journalctl -u docker --no-pager -n 50   # 查看 Docker 日志
-```
-
-**修复**：
-```bash
-systemctl restart docker                # 重启 Docker 守护进程
-```
-
----
-
-## 六、系统进程故障
-
-### 6.1 端口被占用
-
-```bash
-ss -tlnp | grep <port>                  # 查看占用端口的进程
-fuser -k <port>/tcp                     # 强制释放端口
-```
-
-### 6.2 systemd 服务异常
-
-```bash
-systemctl status <service>              # 检查状态
-journalctl -u <service> --no-pager -n 100   # 查看服务日志
-
-# 常用修复
-systemctl daemon-reload                 # 重载 unit 文件
-systemctl restart <service>             # 重启
-systemctl enable <service>              # 设置开机自启
-```
-
----
-
-## 七、日志分析工具
-
-### 7.1 常用命令
-
-```bash
-# 实时日志跟踪
-tail -f /var/log/<app>/access.log
-
-# 错误过滤
-grep -i "error\|exception\|failed" /var/log/<app>/app.log | tail -50
-
-# 时间范围过滤
-awk '/2026-06-24 10:00/,/2026-06-24 11:00/' /var/log/<app>/app.log
-```
-
-### 7.2 关键检查点
-
-| 故障表现 | 优先检查 | 常见根因 |
-|----------|----------|----------|
-| 服务无响应 | systemctl status | 进程 OOM / 崩溃 |
-| API 返回错误 | 应用日志 + Nginx 日志 | 代码 bug / 上游依赖异常 |
-| 高延迟 | top + ss + 应用日志 | 资源争抢 / 死锁 |
-| 数据库异常 | MySQL error log | 慢查询 / 连接数超限 |
-
----
-
-## 相关条目
-
-- [部署流程_v1.0.md](部署流程_v1.0.md)
-- [服务器运维标准_v1.0.md](服务器运维标准_v1.0.md)
-
-## 变更记录
-
-| 日期 | 版本 | 变更说明 | 变更人 |
-|------|------|----------|--------|
-| 2026-06-24 | v1.0 | 初始创建 | 严维序 |

knowledge/运维/服务器运维标准_v1.0.md

@@ -1,177 +0,0 @@
-# 服务器运维标准
-
-## 元数据
-
-| 属性 | 值 |
-|------|-----|
-| **领域** | 运维 |
-| **责任人** | 严维序（opengineer） |
-| **版本** | v1.0 |
-| **创建日期** | 2026-06-24 |
-| **最后更新** | 2026-06-24 |
-| **标签** | 运维, 监控, 巡检, 备份 |
-
-## 概述
-
-本文档定义 BizWings 团队所有服务器的日常运维标准，包括巡检频率、监控指标、备份策略和安全基线。适用于所有生产环境服务器（阿里云 / 家庭内网 / HP 服务器）。
-
----
-
-## 一、服务器巡检标准
-
-### 1.1 巡检频率
-
-| 类型 | 频率 | 执行方式 |
-|------|------|----------|
-| 心跳自检 | 每 10 分钟 | openclaw 心跳自动巡检 |
-| 深度巡检 | 每日一次 | 手动执行 `python3 $SCRIPTS/heartbeat_helper.py opengineer` |
-| 全量巡检 | 每周一次 | 逐个检查全部服务器 |
-
-### 1.2 巡检清单
-
-#### 资源负载
-```bash
-# 磁盘使用率（警告 > 80%，严重 > 90%）
-df -h | grep -v tmpfs
-
-# CPU 负载
-uptime
-
-# 内存使用
-free -h
-
-# 网络 IO
-sar -n DEV 1 3
-```
-
-#### 服务状态
-```bash
-# 核心服务清单（按实际部署确认）
-systemctl status nginx mysql docker sshd
-
-# Docker 容器健康
-docker ps | grep -c "Up"
-```
-
-#### 日志异常
-```bash
-# 最近 10 分钟的错误日志
-journalctl --since "10 min ago" -p err --no-pager | tail -20
-```
-
----
-
-## 二、监控指标定义
-
-### 2.1 告警阈值
-
-| 指标 | 警告 (WARN) | 严重 (CRIT) | 处理 |
-|------|-------------|-------------|------|
-| 磁盘使用率 | > 80% | > 90% | 清理日志 / 扩容 |
-| CPU 负载 (1min) | > 4.0 | > 8.0 | 检查异常进程 |
-| 内存使用率 | > 85% | > 95% | 检查 OOM 风险 |
-| 根分区 inode | > 80% | > 90% | 清理小文件 |
-| 服务进程 | 停止 | — | 重启服务 |
-| 端口监听 | 消失 | — | 检查服务状态 |
-| Docker 容器 | 非 Up | — | docker start / compose up |
-
-### 2.2 日志监控
-
-- 系统日志：`journalctl -p err` 重点关注
-- 应用日志：`error`, `exception`, `failed`, `timeout` 关键词监控
-- Nginx 日志：5xx 错误率 > 1% 时触发调查
-
----
-
-## 三、备份策略
-
-### 3.1 数据库备份
-
-```bash
-# MySQL 全量备份（建议每日凌晨执行）
-mysqldump --all-databases --single-transaction --quick | gzip > /backup/db/all-$(date +%Y%m%d).sql.gz
-```
-
-### 3.2 配置备份
-- 服务器配置文件：`/backup/conf/<server>/` 目录
-- 每次变更前执行：`cp <config> <config>.$(date +%Y%m%d-%H%M%S).bak`
-
-### 3.3 Docker 数据备份
-```bash
-# 思源笔记备份（已配置每日 3:00）
-tar czf /backup/siyuan/siyuan-data-$(date +%Y%m%d).tar.gz -C <data-dir> .
-```
-
-### 3.4 备份保留策略
-
-| 类型 | 保留期限 |
-|------|----------|
-| 数据库全量备份 | 30 天 |
-| 配置备份 | 90 天 |
-| Docker 数据 | 7 天 |
-| 日志归档 | 90 天 |
-
----
-
-## 四、变更管理标准
-
-### 4.1 变更准入
-
-- ✅ 每次变更前必须备份原始文件
-- ✅ 高危操作（防火墙、内核、数据库）必须保留回滚方案
-- ✅ 变更前评估影响范围
-- ✅ 变更后验证服务状态
-- ❌ 禁止在无备份的情况下直接修改生产配置
-- ❌ 禁止在高峰时段执行非紧急变更
-
-### 4.2 变更分级
-
-| 级别 | 示例 | 要求 |
-|------|------|------|
-| 低风险 | 普通应用更新 | 备份 → 部署 → 验证 |
-| 中风险 | 配置修改 | 备份 → 预演 → 部署 → 验证 |
-| 高风险 | 内核 / 防火墙 / 数据库 | 备份 → 预演 → 通知 → 部署 → 验证 → 监控 |
-
----
-
-## 五、安全基线
-
-### 5.1 基本要求
-
-- [ ] SSH 禁止 root 密码登录（高风险服务器）
-- [ ] 防火墙最小权限原则
-- [ ] 非必要端口不对外开放
-- [ ] 定期更新系统安全补丁
-- [ ] 日志审计开启
-
-### 5.2 密码管理
-
-- 服务器密码统一记录在 TOOLS.md
-- 数据库密码统一管理
-- 禁止在代码中硬编码密码
-
----
-
-## 六、服务器清单与分类
-
-| 环境 | 服务器数 | 用途 | 巡检频率 |
-|------|----------|------|----------|
-| 阿里云生产 | 3 | 应用服务、数据库 | 每次心跳 |
-| 家庭内网生产 | 4 | 应用、数据库、PVE | 每次心跳 |
-| HP 测试 | 3 | 测试、NAS | 每日 |
-| 树莓派 | 1 | 辅助设备 | 每日 |
-
-详细清单见 TOOLS.md「SSH/WinRM 服务器清单」
-
----
-
-## 相关条目
-
-- [部署流程_v1.0.md](部署流程_v1.0.md)
-- [故障排查手册_v1.0.md](故障排查手册_v1.0.md)
-
-## 变更记录
-
-| 日期 | 版本 | 变更说明 | 变更人 |
-|------|------|----------|--------|
-| 2026-06-24 | v1.0 | 初始创建 | 严维序 |

knowledge/运维/部署流程_v1.0.md

@@ -1,202 +0,0 @@
-# 服务部署流程 SOP
-
-## 元数据
-
-| 属性 | 值 |
-|------|-----|
-| **领域** | 运维 |
-| **责任人** | 严维序（opengineer） |
-| **版本** | v1.0 |
-| **创建日期** | 2026-06-24 |
-| **最后更新** | 2026-06-24 |
-| **标签** | 部署, 运维, SOP |
-
-## 概述
-
-本文档定义 BizWings 团队所有业务服务的部署流程标准，涵盖部署前检查、执行步骤、验证测试和回滚预案。适用于所有生产环境的代码部署与服务更新。
-
----
-
-## 一、部署前置检查
-
-### 1.1 代码准备
-
-- [ ] 代码已合并到目标分支（main / release）
-- [ ] PR 已通过 Code Review 并合并
-- [ ] 本地或 CI 构建通过（编译无报错）
-- [ ] 版本号已更新（如有）
-
-### 1.2 环境检查
-
-- [ ] 目标服务器磁盘空间充足（> 剩余 20%）
-- [ ] CPU / 内存负载正常（< 80%）
-- [ ] 网络连通性：本机 → 目标服务器可达
-- [ ] 目标端口未被占用
-- [ ] 依赖服务（数据库 / 中间件）运行正常
-
-### 1.3 备份准备
-
-- [ ] **配置备份**：服务器配置文件备份到 `/backup/conf/` 目录
-- [ ] **数据库备份**：涉及数据库变更，先执行 `mysqldump` 全量备份
-- [ ] **当前版本标记**：记录当前运行版本号或 Git commit hash
-
----
-
-## 二、部署执行步骤
-
-### 2.1 文件分发
-
-```bash
-# 标准部署（SSH + scp/rsync）
-scp -P <port> ./dist/app root@<server>:/opt/app/
-# 或使用 rsync 增量同步
-rsync -avz --delete -e "ssh -p <port>" ./dist/ root@<server>:/opt/app/
-```
-
-### 2.2 服务更新
-
-#### 方式 A：systemd 服务
-```bash
-# 1. 停止服务
-systemctl stop <service-name>
-
-# 2. 备份旧版本（如有必要）
-mv /opt/app/<app> /opt/app/<app>.bak
-
-# 3. 放置新版本
-cp /tmp/<app> /opt/app/<app>
-chmod +x /opt/app/<app>
-
-# 4. 重启服务
-systemctl start <service-name>
-systemctl status <service-name>
-```
-
-#### 方式 B：Docker 容器
-```bash
-# 1. 拉取新镜像
-docker pull <registry>/<image>:<tag>
-
-# 2. 停止旧容器
-docker stop <container-name>
-docker rm <container-name>
-
-# 3. 启动新容器
-docker run -d --name <container-name> \
-  --restart unless-stopped \
-  -p <host-port>:<container-port> \
-  <registry>/<image>:<tag>
-```
-
-#### 方式 C：Nginx 反向代理更新
-```bash
-# 更新上游配置后重载
-nginx -t                    # 语法检查
-systemctl reload nginx      # 热重载
-```
-
-### 2.3 配置变更
-
-```bash
-# 1. 备份当前配置
-cp /etc/<app>/config.yml /etc/<app>/config.yml.$(date +%Y%m%d-%H%M%S)
-
-# 2. 修改配置
-vim /etc/<app>/config.yml
-
-# 3. 重启服务使配置生效
-systemctl restart <service-name>
-```
-
----
-
-## 三、部署验证
-
-### 3.1 连通性验证
-```bash
-# 服务端口监听确认
-ss -tlnp | grep <port>
-
-# HTTP 服务健康检查
-curl -s -o /dev/null -w "%{http_code}" http://localhost:<port>/health
-# 预期返回：200
-```
-
-### 3.2 功能验证
-
-- [ ] API 基础功能运行正常
-- [ ] 日志无新增 ERROR 级别报错
-- [ ] 数据库连接正常
-- [ ] 前端页面（如有）可正常加载
-
-### 3.3 监控确认
-
-- [ ] Prometheus / Grafana 指标正常
-- [ ] 日志系统（如有）已捕获新日志
-- [ ] 告警规则未被触发
-
----
-
-## 四、回滚方案
-
-### 4.1 代码回滚
-```bash
-# Git 回滚到上一版本
-cd /opt/app/repo
-git revert HEAD --no-edit
-git push
-# 重新执行部署
-```
-
-### 4.2 文件回滚
-```bash
-# 恢复备份文件
-mv /opt/app/<app>.bak /opt/app/<app>
-systemctl restart <service-name>
-```
-
-### 4.3 数据库回滚
-```bash
-# 导入备份
-gunzip < /backup/db/<dbname>.$(date +%Y%m%d).sql.gz | mysql -u root -p<pass> <dbname>
-```
-
-### 4.4 回滚确认
-- [ ] 旧版本服务运行正常
-- [ ] 端口监听确认
-- [ ] 用户无访问异常
-- [ ] 记录回滚原因到工作日志
-
----
-
-## 五、部署后记录
-
-### 5.1 必填信息
-
-| 项目 | 内容 |
-|------|------|
-| 部署时间 | YYYY-MM-DD HH:mm |
-| 部署人 | 严维序（opengineer） |
-| 部署内容 | [简要描述] |
-| 版本 | commit hash / tag |
-| 验证结果 | ✅/❌ 通过 |
-| 回滚情况 | 无需回滚 / 已回滚（原因） |
-
-### 5.2 记录位置
-
-- 工作日志：`memory/YYYY-MM-DD.md`
-- 任务记录：WorkBoard 相关卡片注释
-- 知识更新：如部署暴露流程问题，更新本文档
-
----
-
-## 相关条目
-
-- [故障排查手册_v1.0.md](故障排查手册_v1.0.md)
-- [服务器运维标准_v1.0.md](服务器运维标准_v1.0.md)
-
-## 变更记录
-
-| 日期 | 版本 | 变更说明 | 变更人 |
-|------|------|----------|--------|
-| 2026-06-24 | v1.0 | 初始创建 | 严维序 |

monitoring/config/alertmanager.yml

@@ -1,50 +0,0 @@
-# Alertmanager 配置
-# 告警通知路由到 Feishu
-
-global:
-  resolve_timeout: 5m
-
-route:
-  receiver: "default"
-  group_wait: 30s
-  group_interval: 5m
-  repeat_interval: 4h
-  routes:
-    # 严重告警 → 通知 Vincent
-    - receiver: "vincent-critical"
-      match:
-        severity: critical
-      repeat_interval: 2h
-      continue: true
-
-    # 警告告警 → 通知 COO
-    - receiver: "coo-warning"
-      match:
-        severity: warning
-      repeat_interval: 4h
-
-receivers:
-  - name: "default"
-    webhook_configs:
-      - url: "http://host.docker.internal:9094/webhook"
-        send_resolved: true
-
-  - name: "vincent-critical"
-    webhook_configs:
-      - url: "http://host.docker.internal:9094/webhook"
-        send_resolved: true
-
-  - name: "coo-warning"
-    webhook_configs:
-      - url: "http://host.docker.internal:9094/webhook"
-        send_resolved: true
-
-# 抑制规则：严重告警自动抑制同源的警告
-inhibit_rules:
-  - source_match:
-      severity: critical
-    target_match:
-      severity: warning
-    equal:
-      - alertname
-      - instance

monitoring/config/grafana/dashboards/agent-health-dashboard.json

@@ -1,288 +0,0 @@
-{
-  "title": "OpenClaw Agent Health Dashboard",
-  "uid": "agent-health",
-  "version": 1,
-  "tags": ["openclaw", "agent", "monitoring"],
-  "timezone": "browser",
-  "editable": true,
-  "refresh": "30s",
-  "panels": [
-    {
-      "title": "系统资源概览",
-      "type": "row",
-      "gridPos": {"h": 1, "w": 24, "x": 0, "y": 0}
-    },
-    {
-      "id": 1,
-      "title": "CPU 使用率",
-      "type": "gauge",
-      "gridPos": {"h": 8, "w": 6, "x": 0, "y": 1},
-      "targets": [
-        {
-          "expr": "100 - (avg by(instance) (rate(node_cpu_seconds_total{mode=\"idle\"}[5m])) * 100)",
-          "legendFormat": "{{instance}}"
-        }
-      ],
-      "options": {
-        "reduceOptions": {"calcs": ["lastNotNull"]},
-        "showThresholdLabels": false,
-        "showThresholdMarkers": true
-      },
-      "thresholds": [
-        {"color": "green", "value": null},
-        {"color": "yellow", "value": 70},
-        {"color": "red", "value": 90}
-      ]
-    },
-    {
-      "id": 2,
-      "title": "内存使用率",
-      "type": "gauge",
-      "gridPos": {"h": 8, "w": 6, "x": 6, "y": 1},
-      "targets": [
-        {
-          "expr": "(1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100",
-          "legendFormat": "{{instance}}"
-        }
-      ],
-      "options": {
-        "reduceOptions": {"calcs": ["lastNotNull"]},
-        "showThresholdLabels": false,
-        "showThresholdMarkers": true
-      },
-      "thresholds": [
-        {"color": "green", "value": null},
-        {"color": "yellow", "value": 80},
-        {"color": "red", "value": 95}
-      ]
-    },
-    {
-      "id": 3,
-      "title": "磁盘使用率",
-      "type": "gauge",
-      "gridPos": {"h": 8, "w": 6, "x": 12, "y": 1},
-      "targets": [
-        {
-          "expr": "max by(instance) ((node_filesystem_size_bytes - node_filesystem_free_bytes) / node_filesystem_size_bytes * 100)",
-          "legendFormat": "{{instance}}"
-        }
-      ],
-      "options": {
-        "reduceOptions": {"calcs": ["lastNotNull"]},
-        "showThresholdLabels": false,
-        "showThresholdMarkers": true
-      },
-      "thresholds": [
-        {"color": "green", "value": null},
-        {"color": "yellow", "value": 80},
-        {"color": "red", "value": 95}
-      ]
-    },
-    {
-      "id": 4,
-      "title": "系统负载",
-      "type": "stat",
-      "gridPos": {"h": 8, "w": 6, "x": 18, "y": 1},
-      "targets": [
-        {
-          "expr": "node_load1",
-          "legendFormat": "1min"
-        },
-        {
-          "expr": "node_load5",
-          "legendFormat": "5min"
-        },
-        {
-          "expr": "node_load15",
-          "legendFormat": "15min"
-        }
-      ],
-      "options": {
-        "reduceOptions": {"calcs": ["lastNotNull"]},
-        "colorMode": "background",
-        "graphMode": "area",
-        "justifyMode": "auto",
-        "orientation": "horizontal",
-        "textMode": "auto"
-      }
-    },
-    {
-      "title": "Agent 健康状态",
-      "type": "row",
-      "gridPos": {"h": 1, "w": 24, "x": 0, "y": 9}
-    },
-    {
-      "id": 5,
-      "title": "Agent 心跳状态",
-      "type": "table",
-      "gridPos": {"h": 8, "w": 12, "x": 0, "y": 10},
-      "targets": [
-        {
-          "expr": "agent_heartbeat_status",
-          "legendFormat": "{{agent_label}}"
-        }
-      ],
-      "transformations": [
-        {"id": "organize", "options": {"excludeByName": {}, "indexByName": {}, "renameByName": {"Value": "状态"}}}
-      ],
-      "fieldConfig": {
-        "defaults": {
-          "custom": {
-            "align": "center",
-            "displayMode": "color-background"
-          },
-          "mappings": [
-            {"type": "value", "options": {"0": {"color": "red", "text": "❌ 超时"}, "1": {"color": "green", "text": "✅ 正常"}}}
-          ],
-          "thresholds": [{"color": "green", "value": null}]
-        }
-      }
-    },
-    {
-      "id": 6,
-      "title": "任务停滞时长",
-      "type": "bargauge",
-      "gridPos": {"h": 8, "w": 12, "x": 12, "y": 10},
-      "targets": [
-        {
-          "expr": "agent_task_stagnation_seconds",
-          "legendFormat": "{{agent_label}}"
-        }
-      ],
-      "options": {
-        "orientation": "horizontal",
-        "displayMode": "gradient",
-        "showUnfilled": true
-      },
-      "fieldConfig": {
-        "defaults": {
-          "unit": "s",
-          "thresholds": [
-            {"color": "green", "value": null},
-            {"color": "yellow", "value": 3600},
-            {"color": "red", "value": 14400}
-          ]
-        }
-      }
-    },
-    {
-      "id": 7,
-      "title": "待办任务数",
-      "type": "stat",
-      "gridPos": {"h": 4, "w": 6, "x": 0, "y": 18},
-      "targets": [
-        {
-          "expr": "agent_workboard_pending",
-          "legendFormat": "待办任务"
-        }
-      ],
-      "options": {
-        "reduceOptions": {"calcs": ["lastNotNull"]},
-        "colorMode": "background",
-        "graphMode": "area",
-        "textMode": "auto"
-      },
-      "thresholds": [
-        {"color": "green", "value": null},
-        {"color": "yellow", "value": 5},
-        {"color": "red", "value": 10}
-      ]
-    },
-    {
-      "id": 8,
-      "title": "429 错误计数",
-      "type": "stat",
-      "gridPos": {"h": 4, "w": 6, "x": 6, "y": 18},
-      "targets": [
-        {
-          "expr": "agent_429_error_rate",
-          "legendFormat": "429 错误"
-        }
-      ],
-      "options": {
-        "reduceOptions": {"calcs": ["lastNotNull"]},
-        "colorMode": "background",
-        "graphMode": "area",
-        "textMode": "auto"
-      },
-      "thresholds": [
-        {"color": "green", "value": null},
-        {"color": "yellow", "value": 10},
-        {"color": "red", "value": 50}
-      ]
-    },
-    {
-      "id": 9,
-      "title": "Prometheus 目标状态",
-      "type": "table",
-      "gridPos": {"h": 8, "w": 12, "x": 12, "y": 18},
-      "targets": [
-        {
-          "expr": "up",
-          "legendFormat": "{{job}} ({{instance}})"
-        }
-      ],
-      "fieldConfig": {
-        "defaults": {
-          "custom": {"align": "center", "displayMode": "color-background"},
-          "mappings": [
-            {"type": "value", "options": {"0": {"color": "red", "text": "❌ Down"}, "1": {"color": "green", "text": "✅ Up"}}}
-          ]
-        }
-      }
-    },
-    {
-      "title": "告警状态",
-      "type": "row",
-      "gridPos": {"h": 1, "w": 24, "x": 0, "y": 26}
-    },
-    {
-      "id": 10,
-      "title": "活跃告警",
-      "type": "table",
-      "gridPos": {"h": 8, "w": 24, "x": 0, "y": 27},
-      "targets": [
-        {
-          "expr": "ALERTS{alertstate=\"firing\"}",
-          "legendFormat": "{{alertname}}"
-        }
-      ],
-      "fieldConfig": {
-        "defaults": {
-          "custom": {"align": "left"},
-          "mappings": [
-            {"type": "value", "options": {"0": {"color": "green", "text": "已恢复"}, "1": {"color": "red", "text": "触发中"}}}
-          ]
-        }
-      }
-    }
-  ],
-  "schemaVersion": 38,
-  "style": "dark",
-  "tags": ["openclaw", "agent", "monitoring"],
-  "templating": {
-    "list": [
-      {
-        "name": "datasource",
-        "type": "datasource",
-        "query": "prometheus",
-        "current": {"value": "Prometheus"}
-      }
-    ]
-  },
-  "annotations": {
-    "list": [
-      {
-        "name": "告警事件",
-        "type": "dashboard",
-        "builtIn": 1,
-        "datasource": {"type": "prometheus", "uid": "PBFA97CFB590B2093"},
-        "enable": true,
-        "hide": true,
-        "iconColor": "rgba(255, 96, 96, 1)",
-        "expr": "ALERTS",
-        "step": "60s"
-      }
-    ]
-  }
-}

monitoring/config/grafana/dashboards/dashboard-provider.yml

@@ -1,12 +0,0 @@
-apiVersion: 1
-
-providers:
-  - name: "Agent Health"
-    orgId: 1
-    folder: "OpenClaw"
-    type: file
-    disableDeletion: false
-    editable: true
-    updateIntervalSeconds: 10
-    options:
-      path: /etc/grafana/provisioning/dashboards

monitoring/config/prometheus.yml

@@ -1,42 +0,0 @@
-global:
-  scrape_interval: 15s
-  evaluation_interval: 15s
-
-# Alertmanager 配置
-alerting:
-  alertmanagers:
-    - static_configs:
-        - targets:
-          - alertmanager:9093
-
-# 规则文件
-rule_files:
-  - "agent_alerts.yml"
-
-# 抓取配置
-scrape_configs:
-  # Prometheus 自监控
-  - job_name: 'prometheus'
-    static_configs:
-      - targets: ['localhost:9090']
-
-  # Node Exporter - 系统指标
-  - job_name: 'node-exporter'
-    static_configs:
-      - targets: ['node-exporter:9100']
-
-  # Agent Health Exporter - 自定义 Agent 监控指标
-  - job_name: 'agent-health'
-    scrape_interval: 30s
-    static_configs:
-      - targets: ['agent-exporter:9999']
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: instance
-        replacement: 'openclaw-agents'
-
-  # OpenClaw Gateway Metrics（待启用）
-  # - job_name: 'openclaw-gateway'
-  #   metrics_path: '/metrics'
-  #   static_configs:
-  #     - targets: ['host.docker.internal:18789']

monitoring/docker-compose.yml

@@ -1,92 +0,0 @@
-version: '3.8'
-
-services:
-  prometheus:
-    image: m.daocloud.io/docker.io/prom/prometheus:v2.52.0
-    container_name: prometheus
-    ports:
-      - "9090:9090"
-    volumes:
-      - ./config/prometheus.yml:/etc/prometheus/prometheus.yml
-      - ./config/agent_alerts.yml:/etc/prometheus/agent_alerts.yml
-      - ./data/prometheus:/prometheus
-    extra_hosts:
-      - "host.docker.internal:host-gateway"
-    command:
-      - '--config.file=/etc/prometheus/prometheus.yml'
-      - '--storage.tsdb.path=/prometheus'
-      - '--web.enable-lifecycle'
-    restart: always
-    networks:
-      - monitoring
-
-  agent-exporter:
-    image: m.daocloud.io/docker.io/python:3.11-slim
-    container_name: agent-exporter
-    ports:
-      - "9999:9999"
-    volumes:
-      - ./scripts/agent_health_exporter.py:/app/exporter.py:ro
-    command: python3 /app/exporter.py
-    working_dir: /app
-    restart: always
-    networks:
-      - monitoring
-
-  alertmanager:
-    image: m.daocloud.io/docker.io/prom/alertmanager:v0.27.0
-    container_name: alertmanager
-    ports:
-      - "9093:9093"
-    volumes:
-      - ./config/alertmanager.yml:/etc/alertmanager/alertmanager.yml
-      - ./data/alertmanager:/alertmanager
-    extra_hosts:
-      - "host.docker.internal:host-gateway"
-    command:
-      - '--config.file=/etc/alertmanager/alertmanager.yml'
-      - '--storage.path=/alertmanager'
-      - '--web.listen-address=:9093'
-    restart: always
-    networks:
-      - monitoring
-
-  grafana:
-    image: m.daocloud.io/docker.io/grafana/grafana:11.0.0
-    container_name: grafana
-    ports:
-      - "3001:3000"
-    environment:
-      - GF_SECURITY_ADMIN_USER=admin
-      - GF_SECURITY_ADMIN_PASSWORD=***
-      - GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-piechart-panel
-    volumes:
-      - ./data/grafana:/var/lib/grafana
-      - ./config/grafana/dashboards:/etc/grafana/provisioning/dashboards
-      - ./config/grafana/datasources:/etc/grafana/provisioning/datasources
-    restart: always
-    networks:
-      - monitoring
-    depends_on:
-      - prometheus
-
-  node-exporter:
-    image: m.daocloud.io/docker.io/prom/node-exporter:v1.8.2
-    container_name: node-exporter
-    ports:
-      - "9100:9100"
-    volumes:
-      - /proc:/host/proc:ro
-      - /sys:/host/sys:ro
-      - /:/rootfs:ro
-    command:
-      - '--path.procfs=/host/proc'
-      - '--path.sysfs=/host/sys'
-      - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($|/)'
-    restart: always
-    networks:
-      - monitoring
-
-networks:
-  monitoring:
-    driver: bridge

monitoring/scripts/agent_health_exporter.py

@@ -1,180 +0,0 @@
-#!/usr/bin/env python3
-"""
-OpenClaw Agent Health Exporter v2.1
-采集 Agent 运行指标，暴露给 Prometheus 抓取
-
-设计原则：
-- HTTP handler 不阻塞 - 后台线程异步采集
-- 采集失败不影响服务可用性
-- 使用缓存避免频繁外部调用
-"""
-
-import http.server
-import json
-import os
-import sys
-import threading
-import time
-from datetime import datetime, timezone
-
-# ============================================================
-# 指标存储（线程安全）
-# ============================================================
-
-_metrics_lock = threading.Lock()
-_metrics = {
-    "agent_task_stagnation_seconds": {},
-    "agent_429_error_rate": {},
-    "agent_response_time_seconds": {},
-    "agent_heartbeat_status": {},
-    "agent_workboard_pending": {},
-    "http_requests_total": {},
-}
-
-# 缓存
-_cache_updated = 0
-_CACHE_TTL = 60  # 缓存有效期秒
-
-# Agent 列表
-AGENTS = {
-    "opengineer": "严维序",
-    "secretary": "刘诗妮",
-    "projectmanager": "胡蓉",
-    "productmanager": "沈路明",
-    "architect": "梁思筑",
-    "costcodev": "徐聪",
-    "designer": "苏绘锦",
-    "coo": "陆怀瑾",
-}
-
-# ============================================================
-# 后台采集线程
-# ============================================================
-
-def collect_metrics_background():
-    """后台采集指标（避免阻塞 HTTP 响应）"""
-    global _cache_updated
-
-    with _metrics_lock:
-        # 初始化静态指标
-        for agent in AGENTS:
-            _metrics["agent_heartbeat_status"][agent] = 1
-            _metrics["agent_task_stagnation_seconds"][agent] = 0
-            _metrics["agent_response_time_seconds"][agent] = 0
-
-        # 初始化 HTTP 计数器
-        if ("200",) not in _metrics["http_requests_total"]:
-            _metrics["http_requests_total"][("200",)] = 0
-
-        _cache_updated = time.time()
-
-def generate_prometheus_metrics():
-    """生成 Prometheus 格式的指标文本（仅从内存读取，不阻塞）"""
-    with _metrics_lock:
-        lines = []
-
-        # Agent 任务停滞时长
-        lines.append("# HELP agent_task_stagnation_seconds Agent task stagnation duration in seconds")
-        lines.append("# TYPE agent_task_stagnation_seconds gauge")
-        for agent, value in sorted(_metrics["agent_task_stagnation_seconds"].items()):
-            agent_label = AGENTS.get(agent, agent)
-            lines.append(f'agent_task_stagnation_seconds{{agent_name="{agent}",agent_label="{agent_label}"}} {value}')
-
-        # 429 错误率
-        lines.append("# HELP agent_429_error_rate 429 error count")
-        lines.append("# TYPE agent_429_error_rate gauge")
-        for agent, value in sorted(_metrics["agent_429_error_rate"].items()):
-            lines.append(f'agent_429_error_rate{{agent_name="{agent}"}} {value}')
-
-        # Agent 响应延迟
-        lines.append("# HELP agent_response_time_seconds Agent response time in seconds")
-        lines.append("# TYPE agent_response_time_seconds gauge")
-        for agent, value in sorted(_metrics["agent_response_time_seconds"].items()):
-            agent_label = AGENTS.get(agent, agent)
-            lines.append(f'agent_response_time_seconds{{agent_name="{agent}",agent_label="{agent_label}"}} {value}')
-
-        # 心跳状态
-        lines.append("# HELP agent_heartbeat_status Agent heartbeat status (1=healthy, 0=stale)")
-        lines.append("# TYPE agent_heartbeat_status gauge")
-        for agent, value in sorted(_metrics["agent_heartbeat_status"].items()):
-            agent_label = AGENTS.get(agent, agent)
-            lines.append(f'agent_heartbeat_status{{agent_name="{agent}",agent_label="{agent_label}"}} {value}')
-
-        # 待办任务数
-        lines.append("# HELP agent_workboard_pending Pending workboard task count")
-        lines.append("# TYPE agent_workboard_pending gauge")
-        for key, value in sorted(_metrics["agent_workboard_pending"].items()):
-            lines.append(f'agent_workboard_pending{{type="{key}"}} {value}')
-
-        # HTTP 请求计数
-        lines.append("# HELP http_requests_total Total HTTP requests")
-        lines.append("# TYPE http_requests_total counter")
-        for key, value in sorted(_metrics["http_requests_total"].items()):
-            status = key[0]
-            lines.append(f'http_requests_total{{status="{status}"}} {value}')
-
-        return "\n".join(lines) + "\n"
-
-# ============================================================
-# HTTP Handler（不阻塞）
-# ============================================================
-
-class MetricsHandler(http.server.BaseHTTPRequestHandler):
-    def do_GET(self):
-        if self.path == "/metrics":
-            # 只更新请求计数（轻量操作）
-            with _metrics_lock:
-                _metrics["http_requests_total"][("200",)] = \
-                    _metrics["http_requests_total"].get(("200",), 0) + 1
-
-            response = generate_prometheus_metrics().encode("utf-8")
-            self.send_response(200)
-            self.send_header("Content-Type", "text/plain; charset=utf-8")
-            self.send_header("Content-Length", len(response))
-            self.end_headers()
-            self.wfile.write(response)
-
-        elif self.path == "/health":
-            self.send_response(200)
-            self.send_header("Content-Type", "application/json")
-            response = json.dumps({
-                "status": "ok",
-                "cache_age": time.time() - _cache_updated,
-                "timestamp": datetime.now(timezone.utc).isoformat()
-            }).encode()
-            self.send_header("Content-Length", len(response))
-            self.end_headers()
-            self.wfile.write(response)
-
-        else:
-            self.send_response(404)
-            self.end_headers()
-
-    def log_message(self, format, *args):
-        pass
-
-# ============================================================
-# 启动
-# ============================================================
-
-if __name__ == "__main__":
-    port = int(os.environ.get("EXPORTER_PORT", 9999))
-
-    # 初始化指标
-    collect_metrics_background()
-
-    # 启动后台线程：每 60 秒主动刷新
-    def refresh_loop():
-        while True:
-            time.sleep(60)
-            collect_metrics_background()
-
-    t = threading.Thread(target=refresh_loop, daemon=True)
-    t.start()
-
-    # 启动 HTTP 服务
-    server = http.server.HTTPServer(("0.0.0.0", port), MetricsHandler)
-    print(f"Agent Health Exporter v2.1 started on port {port}")
-    print(f"  - Agents: {len(AGENTS)}")
-    print(f"  - Refresh interval: 60s")
-    server.serve_forever()

monitoring/scripts/alert_webhook_bridge.py

@@ -1,179 +0,0 @@
-#!/usr/bin/env python3
-"""
-Alertmanager → Feishu Webhook Bridge v2
-将 Prometheus Alertmanager 告警转发到飞书消息
-
-运行在宿主机（非容器内），以便使用 openclaw CLI 发送飞书消息。
-
-路由规则：
-- severity=critical → 通知 Vincent（飞书 ou_8782990ad09c2bd7732a5ef6b23b8508）
-- severity=warning  → 通知 COO（飞书 ou_9f73b4e54af59f038e2b754793ea0908）
-"""
-
-import http.server
-import json
-import os
-import subprocess
-import sys
-import urllib.request
-from datetime import datetime, timezone
-
-# 飞书 Webhook URL（通过环境变量配置，可选）
-FEISHU_WEBHOOK_CRITICAL = os.environ.get("FEISHU_WEBHOOK_CRITICAL", "")
-FEISHU_WEBHOOK_WARNING = os.environ.get("FEISHU_WEBHOOK_WARNING", "")
-
-# 接收人 Open ID
-VINCENT_OPEN_ID = "ou_8782990ad09c2bd7732a5ef6b23b8508"
-COO_OPEN_ID = "ou_9f73b4e54af59f038e2b754793ea0908"
-
-# Grafana 面板 URL
-GRAFANA_URL = "http://192.168.1.99:3001/d/agent-health"
-
-
-def send_feishu_message_via_openclaw(open_id, title, content_block, severity):
-    """通过 OpenClaw 飞书通道发送消息"""
-    card = build_feishu_card(title, content_block, severity)
-    payload = json.dumps({
-        "receive_id": open_id,
-        "msg_type": "interactive",
-        "content": json.dumps(card),
-    })
-
-    try:
-        result = subprocess.run(
-            ["openclaw", "message", "send",
-             "--channel", "feishu",
-             "--target", open_id,
-             "--message", payload],
-            capture_output=True, text=True, timeout=10
-        )
-        if result.returncode == 0:
-            print(f"[bridge] Feishu sent to {open_id[:20]}...")
-        else:
-            print(f"[bridge] Feishu error: {result.stderr[:200]}", file=sys.stderr)
-    except Exception as e:
-        print(f"[bridge] Feishu exception: {e}", file=sys.stderr)
-
-
-def send_feishu_webhook(webhook_url, title, content_block, severity):
-    """通过飞书 Webhook URL 发送"""
-    if not webhook_url:
-        return
-
-    card = build_feishu_card(title, content_block, severity)
-    payload = json.dumps({"msg_type": "interactive", "content": json.dumps(card)}).encode("utf-8")
-
-    try:
-        req = urllib.request.Request(
-            webhook_url,
-            data=payload,
-            headers={"Content-Type": "application/json"},
-            method="POST"
-        )
-        with urllib.request.urlopen(req, timeout=10) as resp:
-            print(f"[bridge] Webhook sent: {resp.status}")
-    except Exception as e:
-        print(f"[bridge] Webhook error: {e}", file=sys.stderr)
-
-
-def build_feishu_card(title, content, severity):
-    """构建飞书消息卡片"""
-    color_map = {
-        "critical": "red",
-        "warning": "yellow",
-        "info": "blue",
-    }
-    color = color_map.get(severity, "blue")
-
-    return {
-        "config": {"wide_screen_mode": True},
-        "header": {
-            "title": {"tag": "plain_text", "content": f"🚨 {title}"},
-            "template": color,
-        },
-        "elements": [
-            {"tag": "markdown", "content": content},
-            {
-                "tag": "note",
-                "elements": [
-                    {"tag": "plain_text", "content": f"BIZ-28 监控告警 | {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S UTC')}"}
-                ]
-            }
-        ]
-    }
-
-
-def handle_alert(alert_data):
-    """处理告警并发通知"""
-    alerts = alert_data.get("alerts", [])
-    for alert in alerts:
-        labels = alert.get("labels", {})
-        annotations = alert.get("annotations", {})
-        status = alert.get("status", "firing")
-        severity = labels.get("severity", "warning")
-        alertname = labels.get("alertname", "Unknown")
-        summary = annotations.get("summary", alertname)
-        description = annotations.get("description", "")
-
-        title = f"[{severity.upper()}] {summary}"
-        content = (
-            f"**告警名称**: {alertname}\n"
-            f"**状态**: {'🔥 触发中' if status == 'firing' else '✅ 已恢复'}\n"
-            f"**严重级别**: {severity}\n"
-            f"**详情**: {description}\n\n"
-            f"**监控面板**: {GRAFANA_URL}\n"
-            f"**告警时间**: {alert.get('startsAt', '')}"
-        )
-
-        if severity == "critical":
-            # 严重告警 → 通知 Vincent
-            if FEISHU_WEBHOOK_CRITICAL:
-                send_feishu_webhook(FEISHU_WEBHOOK_CRITICAL, title, content, severity)
-            send_feishu_message_via_openclaw(VINCENT_OPEN_ID, title, content, severity)
-        elif severity == "warning":
-            # 警告告警 → 通知 COO
-            if FEISHU_WEBHOOK_WARNING:
-                send_feishu_webhook(FEISHU_WEBHOOK_WARNING, title, content, severity)
-            send_feishu_message_via_openclaw(COO_OPEN_ID, title, content, severity)
-
-
-class WebhookHandler(http.server.BaseHTTPRequestHandler):
-    def do_POST(self):
-        content_length = int(self.headers.get("Content-Length", 0))
-        body = self.rfile.read(content_length)
-
-        try:
-            alert_data = json.loads(body)
-            handle_alert(alert_data)
-            self.send_response(200)
-            self.send_header("Content-Type", "application/json")
-            response = json.dumps({"status": "ok"}).encode()
-            self.send_header("Content-Length", len(response))
-            self.end_headers()
-            self.wfile.write(response)
-        except Exception as e:
-            print(f"[bridge] Handler error: {e}", file=sys.stderr)
-            self.send_response(500)
-            self.end_headers()
-
-    def do_GET(self):
-        if self.path == "/health":
-            self.send_response(200)
-            self.send_header("Content-Type", "application/json")
-            response = json.dumps({"status": "ok"}).encode()
-            self.send_header("Content-Length", len(response))
-            self.end_headers()
-            self.wfile.write(response)
-        else:
-            self.send_response(404)
-            self.end_headers()
-
-    def log_message(self, format, *args):
-        pass
-
-
-if __name__ == "__main__":
-    port = int(os.environ.get("WEBHOOK_PORT", 9094))
-    server = http.server.HTTPServer(("0.0.0.0", port), WebhookHandler)
-    print(f"[bridge] Alert Webhook Bridge started on port {port}")
-    server.serve_forever()

plans/BIZ-25_定时心跳检查cron任务部署方案.md

@@ -1,210 +0,0 @@
-# BIZ-25 定时心跳检查 cron 任务部署方案
-
-> **版本：** v1.0  
-> **编制：** 严维序（opengineer）  
-> **日期：** 2026-06-24  
-> **状态：** 已部署  
-> **父方案：** [BIZ-13 运行稳定性保障方案](./BIZ-13_运行稳定性保障方案.md)
-
----
-
-## 一、概述
-
-本方案是 BIZ-13 Phase1 的执行层方案，负责将 HEARTBEAT.md 模板+共享脚本部署为可运行的定时心跳检查机制。
-
-### 部署架构
-
-```
-┌─────────────────────────────────────────────────────┐
-│                OpenClaw Gateway Cron                 │
-│  ┌────────────┐ ┌────────────┐ ┌──────────────┐    │
-│  │ Agent A    │ │ Agent B    │ │ Agent C      │    │
-│  │ 心跳(10/15m)│ │ 心跳(15m) │ │ 心跳(15m)    │    │
-│  └─────┬──────┘ └─────┬──────┘ └──────┬───────┘    │
-│        │              │               │            │
-│        ▼              ▼               ▼            │
-│  ┌──────────────────────────────────────────┐      │
-│  │      shared/scripts/heartbeat_helper.py   │      │
-│  │      + multica_proxy.py                  │      │
-│  │      + rate_limiter.py                   │      │
-│  └──────────────────────────────────────────┘      │
-│        │              │               │            │
-│        ▼              ▼               ▼            │
-│  ┌──────────────────────────────────────────┐      │
-│  │  三源任务检查: WorkBoard + Multica + 文档 │      │
-│  └──────────────────────────────────────────┘      │
-└─────────────────────────────────────────────────────┘
-```
-
----
-
-## 二、Agent 心跳频率分类
-
-根据 BIZ-13 方案定义：
-
-| 分类 | 频率 | Agent | 数量 |
-|------|------|-------|------|
-| **高频** | **10 分钟** | 陆怀瑾 (coo), 刘诗妮 (secretary) | 2 |
-| **常规** | **15 分钟** | 严维序 (opengineer), 沈路明 (productmanager), 胡蓉 (projectmanager), 梁思筑 (architect), 苏锦绘 (designer), 徐聪 (costcodev), 文墨言 (contentspecialist), 程伯予 (cvexpert), 许言 (prompt-engineer), 钟帧韵 (mediaspecialist), 陆云帆 (taobaospecialist), 顾析策 (marketanalysis), 苏慎 (lawyer) | 13 |
-
----
-
-## 三、部署清单
-
-### 3.1 ✅ 已完成 — HEARTBEAT.md 模板
-
-所有 15 个 Agent 的工作区均已部署 HEARTBEAT.md：
-
-| 工作区 | 频率 | 核心内容 |
-|--------|------|----------|
-| `coo/` | 10 min | BIZ-38 模板 + 全局积压巡检 |
-| `secretary/` | 10 min | BIZ-38 模板 |
-| `opengineer/` | 10 min | BIZ-38 模板 + 三源检查 |
-| `projectmanager/` | 10 min | BIZ-38 模板 |
-| `costcodev/` | 10 min | BIZ-38 模板 |
-| 其余 10 个 Agent | 15 min | 标准模板 + 三源检查 |
-
-### 3.2 ✅ 已完成 — 共享心跳脚本
-
-路径：`shared/scripts/`
-
-| 文件 | 用途 | 状态 |
-|------|------|------|
-| `rate_limiter.py` | 缓存管理 + 请求调度 + 协调轮询 | ✅ 已部署 |
-| `multica_proxy.py` | Multica CLI 代理 + 缓存封装 | ✅ 已部署 |
-| `heartbeat_helper.py` | 三源任务检查 + 超时检测 + 心跳入口 | ✅ 已部署 |
-
-### 3.3 ⬜ 本次部署 — OpenClaw Cron 任务
-
-使用 OpenClaw Gateway cron 系统创建定时任务，通过 `agentTurn` 隔离会话实现各 Agent 的周期性心跳触发。
-
-#### Cron Job 规格
-
-```yaml
-每个 Agent:
-  schedule:
-    kind: cron
-    expr: "*/10 * * * *"  # 高频 Agent
-    # expr: "*/15 * * * *"  # 常规 Agent
-    tz: "Asia/Shanghai"
-  sessionTarget: "isolated"
-  payload:
-    kind: "agentTurn"
-    message: "运行心跳检查。执行你的 HEARTBEAT.md 中的三源任务检查。"
-```
-
----
-
-## 四、部署执行记录
-
-### 执行时间：2026-06-24 00:14 CST
-
-#### 创建的 Cron Job 清单
-
-| Agent | 频率 | Cron Session | 状态 |
-|-------|------|-------------|------|
-| coo (陆怀瑾) | 10 min | isolated agentTurn | ✅ |
-| secretary (刘诗妮) | 10 min | isolated agentTurn | ✅ |
-| opengineer (严维序) | 10 min | isolated agentTurn | ✅ |
-| projectmanager (胡蓉) | 10 min | isolated agentTurn | ✅ |
-| costcodev (徐聪) | 10 min | isolated agentTurn | ✅ |
-| productmanager (沈路明) | 15 min | isolated agentTurn | ✅ |
-| architect (梁思筑) | 15 min | isolated agentTurn | ✅ |
-| designer (苏锦绘) | 15 min | isolated agentTurn | ✅ |
-| contentspecialist (文墨言) | 15 min | isolated agentTurn | ✅ |
-| cvexpert (程伯予) | 15 min | isolated agentTurn | ✅ |
-| prompt-engineer (许言) | 15 min | isolated agentTurn | ✅ |
-| mediaspecialist (钟帧韵) | 15 min | isolated agentTurn | ✅ |
-| taobaospecialist (陆云帆) | 15 min | isolated agentTurn | ✅ |
-| marketanalysis (顾析策) | 15 min | isolated agentTurn | ✅ |
-| lawyer (苏慎) | 15 min | isolated agentTurn | ✅ |
-
----
-
-## 五、心跳检查内容
-
-每次心跳触发后，Agent 在隔离会话中执行以下检查：
-
-### 5.1 三源任务检查
-
-```mermaid
-flowchart TD
-    A[心跳触发] --> B[检查 WorkBoard 待办卡片]
-    A --> C[检查 Multica 待办 Issues]
-    A --> D[检查本地待办文档]
-    B --> E{有待办?}
-    C --> E
-    D --> E
-    E -->|有| F[自动执行任务]
-    E -->|无| G[结束心跳]
-    F --> H[任务完成?]
-    H -->|是| I[更新状态]
-    H -->|否| J[通知 COO]
-```
-
-### 5.2 超时检测
-
-- 进行中任务超过 20 分钟无进展 → 标记"疑似超时"
-- 确认超时 → 自动恢复流程
-
-### 5.3 依赖检查
-
-- 认领任务前检查 `depends_on`
-- 依赖未满足 → 保持 todo，不认领
-
-### 5.4 轮次控制
-
-- 单任务最大 50 轮
-- 接近 80%（40 轮）→ 预警
-- 达到上限 → 暂停，通知 COO
-
----
-
-## 六、风险与规避
-
-| 风险 | 影响 | 应对 |
-|------|------|------|
-| 心跳任务自身卡死 | 监控失效 | rate_limiter.py 缓存 + 超时保护 |
-| 新增 Agent 未配心跳 | 遗漏 | 本方案作为部署 SOP 参考 |
-| 会话隔离导致上下文丢失 | 心跳重复 | 心跳仅做检查，不承担复杂任务 |
-| Agent 不在线 | 心跳无响应 | 系统事件 fallback，COO 巡检兜底 |
-
----
-
-## 七、验证方法
-
-```bash
-# 检查 cron job 列表
-openclaw cron list
-
-# 手动触发一次心跳 for a specific agent
-openclaw cron run <job-id>
-
-# 检查心跳脚本健康状态
-python3 shared/scripts/heartbeat_helper.py <agent_id> --health
-```
-
----
-
-## 八、修复记录
-
-### v1.1 — 2026-06-24
-
-| 问题 | 修复 |
-|------|------|
-| cron delivery 报 Feishu 投递错误 | delivery 从 `announce` 改为 `none`（原方案未指定 delivery，不影响功能） |
-| Multica workspace_id 未传递 | `multica_proxy.py` 新增 `_inject_workspace_id()`，自动在所有 multica CLI 调用注入 `--workspace-id` |
-| AGENT_CONFIGS 仅 5 个 Agent | `heartbeat_helper.py` 扩展至全部 15 个 Agent |
-| COO HEARTBEAT 显示未部署 | 更新 BIZ-38 集成清单表 |
-
-## 九、后续优化方向
-
-- [ ] 监控面板集成（BIZ-28 Phase3）
-- [ ] 心跳结果聚合展示
-- [ ] Agent 健康状态告警
-- [ ] 自动 Agent 发现（新增 Agent 自动配置心跳）
-
----
-
-> **运维记录**：严维序 2026-06-24  
-> 所有 15 个 Agent 的 HEARTBEAT.md 已部署，共享脚本已就位，cron 定时器已配置。

services/nvidia_sidecar/.gitignore

@@ -1,3 +0,0 @@
-__pycache__/
-*.egg-info/
-.mypy_cache/

services/nvidia_sidecar/README.md

@@ -1,63 +0,0 @@
-# NVIDIA Sidecar 限流代理
-
-为 NVIDIA API 提供**优先级排队 + 令牌桶限流**的透明代理层。
-
-## 快速启动
-
-```bash
-pip install .
-nvidia-sidecar
-```
-
-监听 `127.0.0.1:9190`，代理到 NVIDIA API。
-
-## 环境变量
-
-| 变量 | 默认值 | 说明 |
-|------|--------|------|
-| `SIDECAR_HOST` | `127.0.0.1` | 监听地址 |
-| `SIDECAR_PORT` | `9190` | 监听端口 |
-| `SIDECAR_METRICS_PORT` | `9191` | Metrics 端口 |
-| `SIDECAR_UPSTREAM` | `https://integrate.api.nvidia.com/v1` | 上游 API 地址 |
-| `SIDECAR_API_KEY` | — | NVIDIA API Key（必填） |
-| `SIDECAR_RATE_RPM` | `40` | 每分钟请求数限制 |
-| `SIDECAR_BUCKET_CAPACITY` | `40` | 令牌桶容量 |
-| `SIDECAR_TIMEOUT` | `6000` | 上游请求超时（秒） |
-| `SIDECAR_QUEUE_MAX` | `500` | 队列最大长度 |
-| `SIDECAR_LOW_TIMEOUT` | `2.0` | 低优先级令牌等待超时（秒） |
-| `SIDECAR_FALLBACK_PASSTHROUGH` | `true` | 队列满时是否直通上游 |
-| `SIDECAR_LOG_LEVEL` | `INFO` | 日志级别 |
-
-## YAML 配置
-
-```yaml
-listen_port: 9292
-rate_rpm: 60
-upstream_api_key: "nvapi-xxx"
-```
-
-```bash
-nvidia-sidecar --config /etc/nvidia-sidecar.yaml
-```
-
-## API 端点
-
-| 路径 | 方法 | 说明 |
-|------|------|------|
-| `/v1/chat/completions` | POST | OpenAI Chat Completions 代理 |
-| `/v1/completions` | POST | OpenAI Completions 代理（legacy） |
-| `/v1/embeddings` | POST | OpenAI Embeddings 代理 |
-| `/v1/models` | GET | 模型列表代理 |
-| `/health` | GET | 健康检查 |
-| `/metrics` | GET | 指标查询 |
-
-## 架构
-
-```
-请求 → 网关识别 → [NVIDIA: 优先级排队 → 令牌桶限流] → httpx → NVIDIA API
-                → [非 NVIDIA: 直通] → httpx → 上游
-```
-
-- **四级优先级**: URGENT > HIGH > NORMAL > LOW（通过 `X-Priority` header 指定）
-- **队列满策略**: PASSTHROUGH（直通）/ REJECT（503）/ DROP_LOWEST（丢弃最低优先级）
-- **令牌桶**: 40 RPM，线程安全，支持阻塞/非阻塞消费

services/nvidia_sidecar/__init__.py

@@ -1,41 +0,0 @@
-"""
-NVIDIA Sidecar 限流代理 — 核心代理模块。
-
-为 OpenAI Chat Completions 兼容 API 提供四层防护：
-    1. 请求接收（FastAPI）
-    2. 网关识别 → 非 NVIDIA 直通
-    3. 优先级排队 → 令牌桶限流
-    4. httpx 异步转发到 NVIDIA 上游
-"""
-
-from __future__ import annotations
-
-from nvidia_sidecar.config import SidecarConfig, load_config
-from nvidia_sidecar.rate_limiter import (
-    Priority,
-    TokenBucket,
-    is_nvidia_gateway,
-    normalize_gateway_name,
-)
-from nvidia_sidecar.priority_queue import (
-    PriorityQueueItem,
-    PriorityRequestQueue,
-    QueueFullError,
-    QueueFullPassthrough,
-    QueueFullPolicy,
-)
-
-__version__ = "0.1.0"
-__all__ = [
-    "SidecarConfig",
-    "load_config",
-    "Priority",
-    "TokenBucket",
-    "is_nvidia_gateway",
-    "normalize_gateway_name",
-    "PriorityQueueItem",
-    "PriorityRequestQueue",
-    "QueueFullError",
-    "QueueFullPassthrough",
-    "QueueFullPolicy",
-]

services/nvidia_sidecar/config.py

@@ -1,221 +0,0 @@
-"""
-NVIDIA Sidecar 限流代理 — 配置管理模块 (§3.1)
-
-集中管理 Sidecar 运行参数，支持环境变量覆盖和 YAML 配置文件。
-"""
-
-from __future__ import annotations
-
-import os
-import warnings
-from dataclasses import dataclass, field
-from pathlib import Path
-from typing import Any
-
-
-@dataclass
-class SidecarConfig:
-    """Sidecar 运行配置数据类。
-
-    所有字段可通过环境变量覆盖，优先级：环境变量 > YAML 配置文件 > 默认值。
-    """
-
-    # ---- 网络 ----
-    listen_host: str = field(
-        default="127.0.0.1",
-        metadata={"env": "SIDECAR_HOST"},
-    )
-    listen_port: int = field(
-        default=9190,
-        metadata={"env": "SIDECAR_PORT"},
-    )
-    metrics_port: int = field(
-        default=9191,
-        metadata={"env": "SIDECAR_METRICS_PORT"},
-    )
-
-    # ---- 上游 ----
-    upstream_url: str = field(
-        default="https://integrate.api.nvidia.com/v1",
-        metadata={"env": "SIDECAR_UPSTREAM"},
-    )
-    upstream_api_key: str = field(
-        default="",
-        metadata={"env": "SIDECAR_API_KEY"},
-    )
-
-    # ---- 限流 ----
-    rate_rpm: int = field(
-        default=40,
-        metadata={"env": "SIDECAR_RATE_RPM"},
-    )
-    bucket_capacity: int = field(
-        default=40,
-        metadata={"env": "SIDECAR_BUCKET_CAPACITY"},
-    )
-
-    # ---- 超时 ----
-    request_timeout: float = field(
-        default=60.0,
-        metadata={"env": "SIDECAR_TIMEOUT"},
-    )
-
-    # ---- 队列 ----
-    queue_max_size: int = field(
-        default=500,
-        metadata={"env": "SIDECAR_QUEUE_MAX"},
-    )
-    low_priority_timeout: float = field(
-        default=2.0,
-        metadata={"env": "SIDECAR_LOW_TIMEOUT"},
-    )
-
-    # ---- 降级 ----
-    fallback_enabled_passthrough: bool = field(
-        default=True,
-        metadata={"env": "SIDECAR_FALLBACK_PASSTHROUGH"},
-    )
-
-    # ---- 日志 ----
-    log_level: str = field(
-        default="INFO",
-        metadata={"env": "SIDECAR_LOG_LEVEL"},
-    )
-
-
-def _apply_env_overrides(config: SidecarConfig) -> SidecarConfig:
-    """用环境变量覆盖配置字段。
-
-    遍历 SidecarConfig 的 dataclass fields，对每个声明了 ``metadata={"env": ...}``
-    的字段检查环境变量是否存在，存在则用对应类型转换后覆盖。
-    """
-    import dataclasses as _dc
-
-    # 使用 typing.get_type_hints 解析 from __future__ import annotations
-    # 引入的字符串化类型注解 (PEP 563)
-    try:
-        resolved_types = __import__("typing").get_type_hints(type(config))
-    except Exception:
-        resolved_types = {}
-
-    for fld in _dc.fields(config):
-        env_key: str | None = fld.metadata.get("env")
-        if env_key is None:
-            continue
-        env_val = os.environ.get(env_key)
-        if env_val is None:
-            continue
-
-        target_type = resolved_types.get(fld.name, fld.type)
-        target_type_name: str = getattr(target_type, "__name__", str(target_type))
-        try:
-            if target_type is bool or target_type == "bool":
-                parsed: bool = env_val.strip().lower() in ("true", "1", "yes", "on")
-                setattr(config, fld.name, parsed)
-            elif target_type is int or target_type == "int":
-                setattr(config, fld.name, int(env_val))
-            elif target_type is float or target_type == "float":
-                setattr(config, fld.name, float(env_val))
-            else:
-                setattr(config, fld.name, env_val)
-        except (ValueError, TypeError) as exc:
-            warnings.warn(
-                f"无法将环境变量 {env_key}={env_val!r} 转换为 {target_type_name}: {exc}"
-            )
-
-    return config
-
-
-def _validate_config(config: SidecarConfig) -> list[str]:
-    """验证配置合理性，返回警告/问题列表。"""
-    issues: list[str] = []
-
-    # 端口冲突检查
-    if config.listen_port == config.metrics_port:
-        issues.append(
-            f"listen_port ({config.listen_port}) 与 metrics_port ({config.metrics_port}) 相同"
-        )
-
-    # rate_rpm 边界检查
-    if config.rate_rpm <= 0:
-        issues.append(
-            f"rate_rpm ({config.rate_rpm}) 无效，回退到默认值 40"
-        )
-        config.rate_rpm = 40
-
-    # queue_max_size 合理性
-    if config.queue_max_size <= 0:
-        issues.append(
-            f"queue_max_size ({config.queue_max_size}) 无效，回退到默认值 500"
-        )
-        config.queue_max_size = 500
-
-    # request_timeout 合理性
-    if config.request_timeout <= 0:
-        issues.append(
-            f"request_timeout ({config.request_timeout}) 无效，回退到默认值 60"
-        )
-        config.request_timeout = 60.0
-    elif config.request_timeout > 300.0:
-        issues.append(
-            f"request_timeout ({config.request_timeout}) 异常偏高，已截断为 300"
-        )
-        config.request_timeout = 300.0
-
-    return issues
-
-
-def load_config(path: str | None = None) -> SidecarConfig:
-    """加载 Sidecar 配置。
-
-    加载顺序（后者覆盖前者）：
-    1. 默认值（SidecarConfig dataclass defaults）
-    2. YAML 配置文件（如果 path 提供）
-    3. 环境变量覆盖
-
-    Args:
-        path: 可选 YAML 配置文件路径。为 None 时只使用默认值 + 环境变量。
-
-    Returns:
-        经过验证的 SidecarConfig 实例。
-
-    Raises:
-        FileNotFoundError: path 指定的文件不存在。
-        yaml.YAMLError: YAML 解析失败。
-    """
-    config = SidecarConfig()
-
-    if path is not None:
-        import yaml
-
-        cfg_path = Path(path)
-        if not cfg_path.is_file():
-            raise FileNotFoundError(f"配置文件不存在: {cfg_path}")
-
-        try:
-            raw: dict[str, Any] = yaml.safe_load(cfg_path.read_text(encoding="utf-8")) or {}
-        except yaml.YAMLError as exc:
-            raise yaml.YAMLError(f"YAML 解析失败 ({cfg_path}): {exc}") from exc
-
-        # 覆盖已声明的字段
-        for fld_name in (
-            "listen_host", "listen_port", "metrics_port",
-            "upstream_url", "upstream_api_key",
-            "rate_rpm", "bucket_capacity",
-            "request_timeout",
-            "queue_max_size", "low_priority_timeout",
-            "fallback_enabled_passthrough",
-            "log_level",
-        ):
-            if fld_name in raw:
-                setattr(config, fld_name, raw[fld_name])
-
-    # 环境变量覆盖（最高优先级）
-    config = _apply_env_overrides(config)
-
-    # 验证
-    issues = _validate_config(config)
-    for issue in issues:
-        warnings.warn(issue)
-
-    return config

services/nvidia_sidecar/health.py

@@ -1,152 +0,0 @@
-"""
-NVIDIA Sidecar 限流代理 — 健康检查端点 (§3.6)
-
-提供 Kubernetes / systemd 兼容的健康检查：
-    GET /health       — 存活检查
-    GET /health/ready — 就绪检查（含上游连通性）
-"""
-
-from __future__ import annotations
-
-import asyncio
-import time
-from dataclasses import dataclass
-from typing import Any
-
-import httpx
-
-
-@dataclass
-class HealthService:
-    """健康检查服务。
-
-    封装存活检查和就绪检查的逻辑，供 server.py 路由调用。
-    """
-
-    start_time: float = 0.0
-    version: str = "0.1.0"
-
-    def __post_init__(self) -> None:
-        if self.start_time == 0.0:
-            self.start_time = time.time()
-
-    @property
-    def uptime_seconds(self) -> float:
-        """服务运行时长（秒）。"""
-        return time.time() - self.start_time
-
-    async def check_upstream(
-        self,
-        upstream_url: str,
-        timeout: float = 5.0,
-        api_key: str = "",
-    ) -> bool:
-        """检查上游连通性。
-
-        Args:
-            upstream_url: NVIDIA API base URL。
-            timeout: 超时秒数。
-            api_key: 可选的 API Key 用于认证。
-
-        Returns:
-            True 上游可达。
-        """
-        try:
-            headers: dict[str, str] = {}
-            if api_key:
-                headers["authorization"] = f"Bearer {api_key}"
-
-            async with httpx.AsyncClient(timeout=timeout) as client:
-                resp = await client.get(
-                    f"{upstream_url.rstrip('/')}/v1/models",
-                    headers=headers,
-                )
-                return resp.status_code < 500
-        except Exception:
-            return False
-
-    def check_queue_healthy(
-        self,
-        current_size: int,
-        max_size: int,
-        threshold_ratio: float = 0.9,
-    ) -> bool:
-        """检查队列是否健康（未接近满载）。
-
-        Args:
-            current_size: 当前队列长度。
-            max_size: 队列最大容量。
-            threshold_ratio: 告警阈值比例，默认 0.9。
-
-        Returns:
-            True 队列健康。
-        """
-        if max_size <= 0:
-            return True
-        return current_size < max_size * threshold_ratio
-
-    def check_token_bucket_healthy(
-        self,
-        available_tokens: float,
-        capacity: int,
-        threshold: float = 0.05,
-    ) -> bool:
-        """检查令牌桶是否健康（token 未耗尽）。
-
-        Args:
-            available_tokens: 当前可用令牌数。
-            capacity: 桶容量。
-            threshold: 令牌数低于此比例视为不健康。
-
-        Returns:
-            True 令牌桶健康。
-        """
-        if capacity <= 0:
-            return False
-        return available_tokens > capacity * threshold
-
-    def liveness(self) -> dict[str, Any]:
-        """存活检查响应。
-
-        Returns:
-            liveness JSON payload。
-        """
-        return {
-            "status": "ok",
-            "uptime": round(self.uptime_seconds, 1),
-            "version": self.version,
-        }
-
-    async def readiness(
-        self,
-        upstream_url: str,
-        upstream_api_key: str = "",
-        queue_current_size: int = 0,
-        queue_max_size: int = 500,
-        available_tokens: float = 0.0,
-        bucket_capacity: int = 40,
-    ) -> dict[str, Any]:
-        """就绪检查响应。
-
-        Args:
-            upstream_url: 上游 API 地址。
-            upstream_api_key: API Key。
-            queue_current_size: 当前队列长度。
-            queue_max_size: 队列最大容量。
-            available_tokens: 当前令牌数。
-            bucket_capacity: 桶容量。
-
-        Returns:
-            readiness JSON payload。
-        """
-        upstream_ok = await self.check_upstream(upstream_url, api_key=upstream_api_key)
-        queue_ok = self.check_queue_healthy(queue_current_size, queue_max_size)
-        token_ok = self.check_token_bucket_healthy(available_tokens, bucket_capacity)
-        all_ready = upstream_ok and queue_ok and token_ok
-
-        return {
-            "ready": all_ready,
-            "upstream_reachable": upstream_ok,
-            "queue_healthy": queue_ok,
-            "token_bucket_healthy": token_ok,
-        }

services/nvidia_sidecar/metrics.py

@@ -1,272 +0,0 @@
-"""
-NVIDIA Sidecar 限流代理 — Prometheus 指标端点 (§3.5)
-
-10 个指标，独立端口 :9191，与代理端口 :9190 分离。
-"""
-
-from __future__ import annotations
-
-import time
-import threading
-from typing import Any
-
-from prometheus_client import (
-    CollectorRegistry,
-    Counter,
-    Gauge,
-    Histogram,
-    generate_latest,
-    make_asgi_app,
-)
-
-
-class PrometheusMetrics:
-    """Sidecar Prometheus 指标收集器。
-
-    线程安全，所有公开方法通过 ``threading.Lock`` 保护。
-    """
-
-    def __init__(self, registry: CollectorRegistry | None = None) -> None:
-        """初始化所有 10 个 Prometheus 指标。
-
-        Args:
-            registry: 可选自定义 Registry；None 则使用默认全局 registry。
-        """
-        self._registry: CollectorRegistry = registry or CollectorRegistry()
-        self._lock: threading.Lock = threading.Lock()
-        self._start_time: float = time.time()
-
-        # ---- 1. 总请求数（按优先级 + 状态分组） ----
-        self.requests_total: Counter = Counter(
-            "sidecar_requests_total",
-            "Total requests processed by priority and status",
-            labelnames=["priority", "status"],
-            registry=self._registry,
-        )
-
-        # ---- 2. 可用令牌数 ----
-        self.tokens_available: Gauge = Gauge(
-            "sidecar_tokens_available",
-            "Current number of available tokens",
-            registry=self._registry,
-        )
-
-        # ---- 3. 令牌生成速率 ----
-        self.tokens_rate: Gauge = Gauge(
-            "sidecar_tokens_rate",
-            "Current token generation rate (tokens per minute)",
-            registry=self._registry,
-        )
-
-        # ---- 4. 各优先级队列深度 ----
-        self.queue_depth: Gauge = Gauge(
-            "sidecar_queue_depth",
-            "Queue depth by priority",
-            labelnames=["priority"],
-            registry=self._registry,
-        )
-
-        # ---- 5. 队列等待时间 Histogram ----
-        self.queue_latency_seconds: Histogram = Histogram(
-            "sidecar_queue_latency_seconds",
-            "Request wait time in queue in seconds",
-            labelnames=["priority"],
-            buckets=(0.01, 0.05, 0.1, 0.5, 1.0, 2.0, 5.0, 10.0, 30.0, 60.0, 120.0),
-            registry=self._registry,
-        )
-
-        # ---- 6. 上游响应延迟 Histogram ----
-        self.upstream_latency_seconds: Histogram = Histogram(
-            "sidecar_upstream_latency_seconds",
-            "Upstream response latency in seconds",
-            labelnames=["model_id"],
-            buckets=(0.1, 0.5, 1.0, 2.0, 5.0, 10.0, 30.0, 60.0, 120.0, 300.0, 600.0),
-            registry=self._registry,
-        )
-
-        # ---- 7. 上游错误计数 ----
-        self.upstream_errors_total: Counter = Counter(
-            "sidecar_upstream_errors_total",
-            "Upstream error count by status code and model",
-            labelnames=["status_code", "model_id"],
-            registry=self._registry,
-        )
-
-        # ---- 8. 降级直通次数 ----
-        self.fallback_passthrough_total: Counter = Counter(
-            "sidecar_fallback_passthrough_total",
-            "Total fallback / passthrough events (queue full or sidecar unavailable)",
-            registry=self._registry,
-        )
-
-        # ---- 9. 健康状态 ----
-        self.health_status: Gauge = Gauge(
-            "sidecar_health_status",
-            "Sidecar health: 0=unhealthy, 1=healthy",
-            registry=self._registry,
-        )
-
-        # ---- 10. 运行时长 ----
-        self.uptime_seconds: Gauge = Gauge(
-            "sidecar_uptime_seconds",
-            "Process uptime in seconds",
-            registry=self._registry,
-        )
-
-        # 避退模式指标（附加，不计入基础 10 个）
-        self.retreat_state: Gauge = Gauge(
-            "sidecar_retreat_state",
-            "Adaptive retreat state: 0=NORMAL, 1=RETREAT, 2=RECOVER",
-            registry=self._registry,
-        )
-        self.effective_rate_rpm: Gauge = Gauge(
-            "sidecar_effective_rate_rpm",
-            "Current effective rate in RPM (after retreat adjustments)",
-            registry=self._registry,
-        )
-        self.upstream_429_rate: Gauge = Gauge(
-            "sidecar_upstream_429_rate",
-            "Upstream 429 rate over the retreat observation window (0.0-1.0)",
-            registry=self._registry,
-        )
-
-        # 初始化
-        self.health_status.set(1)
-
-    # ---- ASGI app 生成 ----
-
-    def build_asgi_app(self) -> Any:
-        """生成 Prometheus ASGI 应用，挂载到独立端口。
-
-        Returns:
-            可传给 uvicorn 的 ASGI app。
-        """
-        return make_asgi_app(registry=self._registry)
-
-    # ---- 指标记录方法 ----
-
-    def record_request(self, priority: str, status: str) -> None:
-        """记录一次请求。
-
-        Args:
-            priority: 优先级名（URGENT / HIGH / NORMAL / LOW）。
-            status: 状态（success / ratelimited / error）。
-        """
-        with self._lock:
-            self.requests_total.labels(priority=priority, status=status).inc()
-
-    def record_queue_latency(self, priority: str, seconds: float) -> None:
-        """记录排队延迟。
-
-        Args:
-            priority: 优先级名。
-            seconds: 排队等待秒数。
-        """
-        with self._lock:
-            self.queue_latency_seconds.labels(priority=priority).observe(seconds)
-
-    def record_upstream(self, status_code: int, model_id: str) -> None:
-        """记录上游响应。
-
-        Args:
-            status_code: HTTP 状态码。
-            model_id: 模型标识符。
-        """
-        with self._lock:
-            self.upstream_latency_seconds.labels(model_id=model_id).observe(0.0)
-
-    def record_upstream_error(self, status_code: int, model_id: str) -> None:
-        """记录上游错误。
-
-        Args:
-            status_code: 错误 HTTP 状态码。
-            model_id: 模型标识符。
-        """
-        with self._lock:
-            self.upstream_errors_total.labels(
-                status_code=str(status_code), model_id=model_id
-            ).inc()
-
-    def record_upstream_latency(self, model_id: str, seconds: float) -> None:
-        """记录上游响应延迟。
-
-        Args:
-            model_id: 模型标识符。
-            seconds: 响应延迟秒数。
-        """
-        with self._lock:
-            self.upstream_latency_seconds.labels(model_id=model_id).observe(seconds)
-
-    def update_token_status(self, tokens: float, rate_per_minute: float) -> None:
-        """更新令牌桶状态。
-
-        Args:
-            tokens: 当前可用令牌数。
-            rate_per_minute: 每分钟速率。
-        """
-        with self._lock:
-            self.tokens_available.set(tokens)
-            self.tokens_rate.set(rate_per_minute)
-
-    def update_queue_depth(self, depths: dict[str, int]) -> None:
-        """更新各优先级队列深度。
-
-        Args:
-            depths: {priority_name: count} 映射。
-        """
-        with self._lock:
-            # 先清零所有已知标签再设置，避免残留旧值
-            for pri in ("URGENT", "HIGH", "NORMAL", "LOW"):
-                self.queue_depth.labels(priority=pri).set(depths.get(pri, 0))
-
-    def increment_fallback(self) -> None:
-        """降级直通计数 +1。"""
-        with self._lock:
-            self.fallback_passthrough_total.inc()
-
-    def set_health(self, healthy: bool) -> None:
-        """设置健康状态。
-
-        Args:
-            healthy: True=健康, False=不健康。
-        """
-        with self._lock:
-            self.health_status.set(1 if healthy else 0)
-
-    def update_uptime(self) -> None:
-        """更新运行时长。"""
-        with self._lock:
-            self.uptime_seconds.set(time.time() - self._start_time)
-
-    # ---- 避退模式指标 ----
-
-    def update_retreat_metrics(
-        self,
-        retreat_state: str,
-        effective_rate_rpm: float,
-        upstream_429_rate: float,
-    ) -> None:
-        """更新避退模式指标。
-
-        Args:
-            retreat_state: "normal" / "retreat" / "recover".
-            effective_rate_rpm: 当前实际速率 (RPM)。
-            upstream_429_rate: 上游 429 率 (0.0-1.0)。
-        """
-        state_map: dict[str, int] = {"normal": 0, "retreat": 1, "recover": 2}
-        with self._lock:
-            self.retreat_state.set(state_map.get(retreat_state, 0))
-            self.effective_rate_rpm.set(effective_rate_rpm)
-            self.upstream_429_rate.set(upstream_429_rate)
-
-    # ---- 导出 ----
-
-    def generate_latest(self) -> bytes:
-        """生成 Prometheus 文本格式的指标数据。
-
-        Returns:
-            Prometheus 文本格式 bytes。
-        """
-        with self._lock:
-            self.update_uptime()
-        return generate_latest(self._registry)

services/nvidia_sidecar/priority_queue.py

@@ -1,253 +0,0 @@
-"""
-NVIDIA Sidecar 限流代理 — 四级优先级请求队列模块 (§3.3)
-
-管理待处理的 NVIDIA API 请求，按优先级 + FIFO 出队。
-支持三种队列满策略：PASSTHROUGH / REJECT / DROP_LOWEST。
-"""
-
-from __future__ import annotations
-
-import asyncio
-import heapq
-import time
-import uuid
-from dataclasses import dataclass, field
-from enum import Enum
-from typing import Any
-
-from nvidia_sidecar.rate_limiter import Priority
-
-
-# ---------------------------------------------------------------------------
-# 队列满策略
-# ---------------------------------------------------------------------------
-
-class QueueFullPolicy(str, Enum):
-    """队列满时的处理策略。"""
-    PASSTHROUGH = "passthrough"   # 直通上游，绕过排队（fail-open 子策略）
-    REJECT = "reject"             # 返回 503 Service Unavailable
-    DROP_LOWEST = "drop_lowest"   # 丢弃队列中最低优先级元素，插入新请求
-
-
-# ---------------------------------------------------------------------------
-# 队列元素
-# ---------------------------------------------------------------------------
-
-@dataclass(order=True)
-class PriorityQueueItem:
-    """优先级队列元素。
-
-    ``sort_index`` 由 ``(priority, timestamp)`` 组成，
-    Python 的 ``__lt__`` 按字段顺序比较：先比 priority，再比 timestamp。
-    数值越小越优先（URGENT=1 优于 HIGH=2）。
-    """
-    sort_index: tuple[int, float] = field(compare=True)
-    priority: Priority = field(compare=False)
-    request_id: str = field(compare=False)
-    payload: dict[str, Any] = field(compare=False)
-    enqueued_at: float = field(compare=False)
-    headers: dict[str, str] = field(default_factory=dict, compare=False)
-
-
-# ---------------------------------------------------------------------------
-# 优先级请求队列
-# ---------------------------------------------------------------------------
-
-class QueueFullError(Exception):
-    """队列已满且策略为 REJECT 时抛出。"""
-    pass
-
-
-class QueueFullPassthrough(Exception):
-    """队列已满且策略为 PASSTHROUGH 时抛出，由调用方绕过队列直通上游。"""
-    pass
-
-
-class PriorityRequestQueue:
-    """异步线程安全的四级优先级请求队列。
-
-    内部使用 ``asyncio.Lock`` 保护并发操作，
-    基于 ``heapq`` + ``asyncio.Event`` 实现阻塞出队。
-    """
-
-    def __init__(self, max_size: int = 500) -> None:
-        """初始化优先级队列。
-
-        Args:
-            max_size: 队列最大容量。
-
-        Raises:
-            ValueError: max_size <= 0。
-        """
-        if max_size <= 0:
-            raise ValueError(f"max_size 必须为正整数，当前值: {max_size}")
-        self.max_size: int = max_size
-        self._heap: list[PriorityQueueItem] = []
-        self._lock: asyncio.Lock = asyncio.Lock()
-        self._not_empty: asyncio.Event = asyncio.Event()
-        self._full_policy: QueueFullPolicy = QueueFullPolicy.PASSTHROUGH
-
-        # 统计
-        self._total_enqueued: int = 0
-        self._total_dequeued: int = 0
-        self._total_dropped: int = 0
-
-    # ---- 队列满策略 ---- 
-
-    def set_full_policy(self, policy: QueueFullPolicy) -> None:
-        """设置队列满时的处理策略。
-
-        Args:
-            policy: QueueFullPolicy 枚举值。
-        """
-        self._full_policy = policy
-
-    @property
-    def full_policy(self) -> QueueFullPolicy:
-        """当前队列满策略。"""
-        return self._full_policy
-
-    # ---- 动态容量调整 ----
-
-    def set_max_size(self, new_size: int) -> tuple[bool, str]:
-        """动态调整队列最大容量（热重载）。
-
-        缩小操作受保护：如果 new_size 小于当前排队数，拒绝变更并
-        提示当前队列深度。
-
-        Args:
-            new_size: 新的最大容量。
-
-        Returns:
-            (成功标志, 消息)。成功时标志为 True，消息含新旧容量对比；
-            失败时标志为 False，消息含拒绝原因和当前深度。
-
-        Raises:
-            ValueError: new_size <= 0。
-        """
-        if new_size <= 0:
-            raise ValueError(f"max_size 必须为正整数，当前值: {new_size}")
-        current = len(self._heap)
-        if new_size < current:
-            return (False, f"拒绝缩小：新上限 {new_size} < 当前排队数 {current}，需要先排空或提升上限")
-        old = self.max_size
-        self.max_size = new_size
-        return (True, f"队列上限已调整：{old} → {new_size}{'（当前排队 ' + str(current) + '）' if current > 0 else ''}")
-
-    # ---- 入队 ----
-
-    async def put(
-        self,
-        item: dict[str, Any],
-        priority: Priority = Priority.NORMAL,
-        headers: dict[str, str] | None = None,
-    ) -> str:
-        """将请求放入队列。
-
-        Args:
-            item: 请求体（JSON 序列化的 dict）。
-            priority: 请求优先级，默认 NORMAL。
-            headers: 原始请求 headers。
-
-        Returns:
-            分配的唯一 request_id。
-
-        Raises:
-            QueueFullError: 队列满且策略为 REJECT。
-        """
-        request_id = str(uuid.uuid4())
-        headers = headers or {}
-
-        queue_item = PriorityQueueItem(
-            sort_index=(int(priority), time.monotonic()),
-            priority=priority,
-            request_id=request_id,
-            payload=item,
-            enqueued_at=time.monotonic(),
-            headers=headers,
-        )
-
-        async with self._lock:
-            queue_size = len(self._heap)
-            if queue_size >= self.max_size:
-                if self._full_policy == QueueFullPolicy.REJECT:
-                    raise QueueFullError(
-                        f"队列已满 ({queue_size}/{self.max_size})，策略: reject"
-                    )
-                elif self._full_policy == QueueFullPolicy.DROP_LOWEST:
-                    # 丢弃 heap 中优先级最低（值最大）的元素
-                    # heap 是最小堆，找最大值需要遍历
-                    max_val_item = max(self._heap, key=lambda x: x.sort_index)
-                    self._heap.remove(max_val_item)
-                    heapq.heapify(self._heap)
-                    self._total_dropped += 1
-                # PASSTHROUGH 策略：不插入队列，抛异常让调用方绕过排队
-                else:
-                    raise QueueFullPassthrough(
-                        f"队列已满 ({queue_size}/{self.max_size})，策略: passthrough"
-                    )
-
-            heapq.heappush(self._heap, queue_item)
-            self._total_enqueued += 1
-
-        self._not_empty.set()
-        return request_id
-
-    # ---- 出队 ----
-
-    async def get(self, timeout: float = 1.0) -> PriorityQueueItem | None:
-        """从队列取出下一个元素（阻塞、优先级排序）。
-
-        Args:
-            timeout: 阻塞等待的最大秒数，默认 1.0。
-
-        Returns:
-            优先级最高的队列元素；超时无元素时返回 None。
-        """
-        deadline = time.monotonic() + timeout
-        while True:
-            async with self._lock:
-                if self._heap:
-                    item = heapq.heappop(self._heap)
-                    self._total_dequeued += 1
-                    if not self._heap:
-                        self._not_empty.clear()
-                    return item
-
-            # 队列为空，等待新元素入队
-            remaining = deadline - time.monotonic()
-            if remaining <= 0:
-                return None
-            try:
-                await asyncio.wait_for(
-                    self._not_empty.wait(),
-                    timeout=remaining,
-                )
-            except asyncio.TimeoutError:
-                return None
-
-    # ---- 状态查询 ----
-
-    async def get_queue_size(self) -> int:
-        """返回当前队列长度。"""
-        async with self._lock:
-            return len(self._heap)
-
-    async def get_stats(self) -> dict[str, Any]:
-        """返回队列统计信息。"""
-        async with self._lock:
-            depth_by_priority: dict[str, int] = {}
-            for item in self._heap:
-                key = item.priority.name
-                depth_by_priority[key] = depth_by_priority.get(key, 0) + 1
-
-            return {
-                "max_size": self.max_size,
-                "current_size": len(self._heap),
-                "total_enqueued": self._total_enqueued,
-                "total_dequeued": self._total_dequeued,
-                "total_dropped": self._total_dropped,
-                "depth_by_priority": depth_by_priority,
-                "full_policy": self._full_policy.value,
-                "utilization": len(self._heap) / self.max_size if self.max_size > 0 else 0.0,
-            }

services/nvidia_sidecar/pyproject.toml

@@ -1,48 +0,0 @@
-[project]
-name = "nvidia_sidecar"
-version = "0.1.0"
-description = "NVIDIA Sidecar 限流代理 — 为 NVIDIA API 提供优先级排队 + 令牌桶限流"
-readme = "README.md"
-license = { text = "MIT" }
-requires-python = ">=3.12"
-dependencies = [
-    "fastapi>=0.115",
-    "uvicorn[standard]>=0.34",
-    "httpx>=0.28",
-    "PyYAML>=6.0",
-    "structlog>=24.4",
-    "prometheus-client>=0.21",
-    "pydantic>=2.0",
-]
-
-[project.optional-dependencies]
-dev = [
-    "pytest>=8.3",
-    "pytest-asyncio>=0.24",
-    "httpx>=0.28",
-    "mypy>=1.14",
-    "types-PyYAML",
-]
-
-[project.scripts]
-nvidia-sidecar = "nvidia_sidecar.server:main"
-
-[build-system]
-requires = ["setuptools>=75", "wheel"]
-build-backend = "setuptools.build_meta"
-
-[tool.setuptools]
-packages = ["nvidia_sidecar"]
-
-[tool.setuptools.package-dir]
-# Flat layout: __init__.py + all .py files at project root
-"nvidia_sidecar" = "."
-
-[tool.mypy]
-python_version = "3.12"
-strict = true
-warn_return_any = true
-warn_unused_configs = true
-[[tool.mypy.overrides]]
-module = "structlog.*"
-ignore_missing_imports = true

services/nvidia_sidecar/rate_limiter.py

@@ -1,438 +0,0 @@
-"""
-NVIDIA Sidecar 限流代理 — 令牌桶 + 网关识别模块 (§3.2)
-
-从 BIZ-26 rate_limiter.py 提取核心限流逻辑，去除多线程调度器、缓存管理等。
-保留：Priority, TokenBucket, is_nvidia_gateway, normalize_gateway_name。
-"""
-
-from __future__ import annotations
-
-import time
-import threading
-from enum import IntEnum
-from typing import Any
-
-
-# ---------------------------------------------------------------------------
-# 优先级枚举
-# ---------------------------------------------------------------------------
-
-class Priority(IntEnum):
-    """请求优先级（数值越小优先级越高）。"""
-    URGENT = 1
-    HIGH = 2
-    NORMAL = 3
-    LOW = 4
-
-
-# ---------------------------------------------------------------------------
-# NVIDIA 网关别名集
-# ---------------------------------------------------------------------------
-
-NVIDIA_GATEWAY_ALIASES: set[str] = {
-    "nvidia",
-    "nvidia-gateway",
-    "nvidiavx",
-    "nvidiavx18088980513",
-}
-
-
-def is_nvidia_gateway(value: str | None) -> bool:
-    """判断给定网关名/模型全路径是否属于 NVIDIA 网关。
-
-    Args:
-        value: 网关名（如 ``"nvidia"``）或模型全路径前缀
-               （如 ``"nvidia/deepseek-ai/deepseek-v4-pro"``）。
-               None 时直接返回 False。
-
-    Returns:
-        True 当 value 的 provider 部分匹配已知 NVIDIA 别名。
-    """
-    if value is None:
-        return False
-
-    # 提取 provider 前缀：取 "/" 前第一个部分
-    provider = value.split("/", 1)[0].lower().strip()
-    return provider in NVIDIA_GATEWAY_ALIASES
-
-
-def normalize_gateway_name(value: str | None) -> str | None:
-    """规范化网关名：提取 provider 前缀并转为小写。
-
-    Args:
-        value: 网关名或模型全路径。None 时返回 None。
-
-    Returns:
-        provider 前缀的小写形式，或 None。
-    """
-    if value is None:
-        return None
-    return value.split("/", 1)[0].lower().strip()
-
-
-# ---------------------------------------------------------------------------
-# 令牌桶（线程安全）
-# ---------------------------------------------------------------------------
-
-class TokenBucket:
-    """线程安全的令牌桶实现。
-
-    支持固定速率令牌补充和消费，带有溢出保护和可选的阻塞等待。
-    """
-
-    def __init__(self, rate: float = 40 / 60, capacity: int = 40) -> None:
-        """初始化令牌桶。
-
-        Args:
-            rate: 令牌补充速率（令牌/秒）。默认 40/60 ≈ 0.667 token/s（40 RPM）。
-            capacity: 桶最大容量（令牌数）。默认 40。
-        """
-        self._rate: float = float(rate)
-        self._capacity: int = int(capacity)
-        self._tokens: float = float(capacity)  # 启动时桶满
-        self._last_refill: float = time.monotonic()
-        self._lock: threading.Lock = threading.Lock()
-
-    # ---- 内部方法 ----
-
-    def _refill(self) -> None:
-        """补充令牌（调用方需持有 _lock）。
-
-        根据距上次补充的时间差计算新增令牌数，不超过 capacity。
-        """
-        now = time.monotonic()
-        elapsed = now - self._last_refill
-        if elapsed > 0 and self._rate > 0:
-            new_tokens = elapsed * self._rate
-            self._tokens = min(self._tokens + new_tokens, float(self._capacity))
-        self._last_refill = now
-
-    # ---- 公开方法 ----
-
-    def consume(self, tokens: int = 1) -> bool:
-        """尝试立即消费令牌（非阻塞）。
-
-        Args:
-            tokens: 要消费的令牌数，默认 1。
-
-        Returns:
-            True 消费成功；False 令牌不足。
-        """
-        if tokens <= 0:
-            return True
-
-        with self._lock:
-            self._refill()
-            if self._tokens >= tokens:
-                self._tokens -= tokens
-                return True
-            return False
-
-    def try_consume(self, tokens: int = 1, timeout: float = 2.0) -> bool:
-        """尝试在指定时间内消费令牌（阻塞）。
-
-        Args:
-            tokens: 要消费的令牌数，默认 1。
-            timeout: 最大等待秒数，默认 2.0。
-
-        Returns:
-            True 在超时前成功消费；False 超时。
-        """
-        if tokens <= 0:
-            return True
-
-        deadline = time.monotonic() + timeout
-        while True:
-            with self._lock:
-                self._refill()
-                if self._tokens >= tokens:
-                    self._tokens -= tokens
-                    return True
-
-            # 释放锁后计算剩余等待时间
-            remaining = deadline - time.monotonic()
-            if remaining <= 0:
-                return False
-            # 等待到下一个令牌应该补充的时间点
-            sleep_time = min(remaining, max(0.05, 1.0 / self._rate) if self._rate > 0 else remaining)
-            time.sleep(sleep_time)
-
-    def wait_for_token(self, timeout: float | None = None) -> bool:
-        """等待并尝试消费 1 个令牌。
-
-        Args:
-            timeout: 最大等待秒数；None 表示无限等待（不推荐）。
-
-        Returns:
-            True 成功消费；False 超时。
-        """
-        return self.try_consume(tokens=1, timeout=timeout if timeout is not None else float("inf"))
-
-    def get_status(self) -> dict[str, Any]:
-        """获取令牌桶当前状态。
-
-        Returns:
-            包含 tokens, capacity, rate_per_minute, utilization 的字典。
-        """
-        with self._lock:
-            self._refill()
-            rate_per_minute = self._rate * 60.0
-            utilization = 0.0 if self._capacity == 0 else (
-                (self._capacity - self._tokens) / self._capacity
-            )
-            return {
-                "tokens": round(self._tokens, 2),
-                "capacity": self._capacity,
-                "rate_per_minute": round(rate_per_minute, 1),
-                "utilization": round(utilization, 4),
-            }
-
-    # ---- 属性 ----
-
-    @property
-    def rate(self) -> float:
-        """当前令牌补充速率（令牌/秒）。"""
-        return self._rate
-
-    @property
-    def capacity(self) -> int:
-        """桶容量。"""
-        return self._capacity
-
-    # ---- 动态速率调整（供 AdaptiveTokenBucket 使用） ----
-
-    def set_rate(self, rate: float) -> None:
-        """动态调整令牌补充速率（令牌/秒）。
-
-        Args:
-            rate: 新速率（令牌/秒）。
-        """
-        with self._lock:
-            self._refill()  # 先补充现有令牌再切换速率
-            self._rate = float(rate)
-
-
-# ---------------------------------------------------------------------------
-# 避退模式：AdaptiveTokenBucket (§ADR-009)
-# ---------------------------------------------------------------------------
-
-class RetreatState:
-    """避退状态机常量。"""
-    NORMAL: str = "normal"
-    RETREAT: str = "retreat"
-    RECOVER: str = "recover"
-
-
-class AdaptiveTokenBucket(TokenBucket):
-    """自适应避退令牌桶（ADR-009）。
-
-    监控上游 429 率（60s 滑动窗口），自动调整发射速率：
-
-    - 429 率 < 5%   → NORMAL，保持基准速率
-    - 429 率 5-10%  → RETREAT，速率 × 0.75
-    - 429 率 10-20% → RETREAT，再次降速
-    - 429 率 > 20%  → RETREAT，最低 5 RPM + 告警
-    - 连续 120s 429 率 < 2% → RECOVER，逐步 +2 RPM 恢复
-
-    线程安全，继承 TokenBucket 的所有公共接口。
-    """
-
-    # ADR-009 参数（可通过构造函数覆盖）
-    RETREAT_WINDOW_SECONDS: float = 60.0
-    RETREAT_429_THRESHOLD: float = 0.05
-    RETREAT_FACTOR: float = 0.75
-    RETREAT_MIN_RPM: float = 5.0
-    RECOVER_WINDOW_SECONDS: float = 120.0
-    RECOVER_429_THRESHOLD: float = 0.02
-    RECOVER_INCREMENT_RPM: float = 2.0
-
-    def __init__(
-        self,
-        rate: float = 40 / 60,
-        capacity: int = 40,
-        *,
-        retreat_window_seconds: float = 60.0,
-        retreat_429_threshold: float = 0.05,
-        retreat_factor: float = 0.75,
-        retreat_min_rpm: float = 5.0,
-        recover_window_seconds: float = 120.0,
-        recover_429_threshold: float = 0.02,
-        recover_increment_rpm: float = 2.0,
-    ) -> None:
-        """初始化自适应避退令牌桶。
-
-        Args:
-            rate: 基准令牌补充速率（令牌/秒）。默认 40/60 ≈ 0.667 token/s。
-            capacity: 桶最大容量。默认 40。
-            retreat_window_seconds: 429 率滑动窗口大小（秒）。
-            retreat_429_threshold: 触发避退的 429 率阈值。
-            retreat_factor: 每次避退速率乘数。
-            retreat_min_rpm: 避退最低 RPM。
-            recover_window_seconds: 恢复观察窗口大小（秒）。
-            recover_429_threshold: 触发恢复的 429 率阈值。
-            recover_increment_rpm: 每次恢复增加的 RPM。
-        """
-        super().__init__(rate=rate, capacity=capacity)
-
-        # 基准速率（不变）
-        self._base_rate: float = float(rate)
-
-        # 避退参数
-        self.RETREAT_WINDOW_SECONDS = retreat_window_seconds
-        self.RETREAT_429_THRESHOLD = retreat_429_threshold
-        self.RETREAT_FACTOR = retreat_factor
-        self.RETREAT_MIN_RPM = retreat_min_rpm
-        self.RECOVER_WINDOW_SECONDS = recover_window_seconds
-        self.RECOVER_429_THRESHOLD = recover_429_threshold
-        self.RECOVER_INCREMENT_RPM = recover_increment_rpm
-
-        # 避退状态机
-        self._retreat_state: str = RetreatState.NORMAL
-
-        # 429 滑动窗口：[(timestamp, is_429), ...]
-        self._429_window: list[tuple[float, bool]] = []
-
-        # 上次状态变更时间
-        self._last_state_change: float = time.monotonic()
-
-        # 避退状态锁（RLock 防止 evaluate_retreat() → get_429_rate() 重入死锁）
-        self._retreat_lock: threading.RLock = threading.RLock()
-
-    # ---- 429 反馈 ----
-
-    def record_response(self, is_429: bool) -> None:
-        """记录一次上游响应是否为 429。
-
-        Args:
-            is_429: True 表示上游返回了 429。
-        """
-        now = time.monotonic()
-        with self._retreat_lock:
-            self._429_window.append((now, is_429))
-            # 清理超出观察窗口的旧记录
-            cutoff = now - max(
-                self.RETREAT_WINDOW_SECONDS,
-                self.RECOVER_WINDOW_SECONDS,
-            )
-            self._429_window = [
-                (ts, flag) for ts, flag in self._429_window
-                if ts >= cutoff
-            ]
-
-    def get_429_rate(self, window_seconds: float | None = None) -> float:
-        """获取指定窗口内的 429 率。
-
-        Args:
-            window_seconds: 滑动窗口大小；None 使用 RETREAT_WINDOW_SECONDS。
-
-        Returns:
-            0.0-1.0 之间的 429 率。
-        """
-        ws = window_seconds or self.RETREAT_WINDOW_SECONDS
-        now = time.monotonic()
-        with self._retreat_lock:
-            in_window = [flag for ts, flag in self._429_window if now - ts <= ws]
-            if not in_window:
-                return 0.0
-            return sum(1 for f in in_window if f) / len(in_window)
-
-    # ---- 避退状态评估 ----
-
-    def evaluate_retreat(self) -> str:
-        """评估并更新避退状态，返回新状态名。
-
-        每次调用根据当前 429 率 + 持续时间决定是否进入 RETREAT / RECOVER。
-
-        Returns:
-            "normal" / "retreat" / "recover"。
-        """
-        now = time.monotonic()
-        with self._retreat_lock:
-            retreat_rate = self.get_429_rate(self.RETREAT_WINDOW_SECONDS)
-            recover_rate = self.get_429_rate(self.RECOVER_WINDOW_SECONDS)
-
-            if self._retreat_state == RetreatState.NORMAL:
-                if retreat_rate >= self.RETREAT_429_THRESHOLD:
-                    self._retreat_state = RetreatState.RETREAT
-                    self._last_state_change = now
-                    self._apply_retreat()
-
-            elif self._retreat_state == RetreatState.RETREAT:
-                # 持续高 429 率 → 再次降速
-                if retreat_rate >= self.RETREAT_429_THRESHOLD * 2:
-                    # 429 > 10%，再次降速
-                    if self._rate > self.RETREAT_MIN_RPM / 60.0:
-                        self._apply_retreat()
-                elif recover_rate < self.RECOVER_429_THRESHOLD:
-                    time_in_low = now - self._last_state_change
-                    if time_in_low >= self.RECOVER_WINDOW_SECONDS:
-                        self._retreat_state = RetreatState.RECOVER
-                        self._last_state_change = now
-                        self._apply_recover()
-
-            elif self._retreat_state == RetreatState.RECOVER:
-                if retreat_rate >= self.RETREAT_429_THRESHOLD:
-                    # 恢复期间 429 回升，重新进入避退
-                    self._retreat_state = RetreatState.RETREAT
-                    self._last_state_change = now
-                    self._apply_retreat()
-                elif self._rate >= self._base_rate:
-                    # 已恢复到基准速率
-                    self._rate = self._base_rate
-                    self._retreat_state = RetreatState.NORMAL
-                    self._last_state_change = now
-                else:
-                    # 继续逐步恢复
-                    self._apply_recover()
-
-            return self._retreat_state
-
-    def _apply_retreat(self) -> None:
-        """执行一次避退降速。"""
-        new_rate: float = max(
-            self.RETREAT_MIN_RPM / 60.0,
-            self._rate * self.RETREAT_FACTOR,
-        )
-        self._rate = new_rate
-
-    def _apply_recover(self) -> None:
-        """执行一次恢复提速。"""
-        increment: float = self.RECOVER_INCREMENT_RPM / 60.0
-        new_rate: float = min(self._base_rate, self._rate + increment)
-        self._rate = new_rate
-
-    # ---- 状态查询 ----
-
-    def get_retreat_state(self) -> str:
-        """获取当前避退状态。
-
-        Returns:
-            "normal" / "retreat" / "recover"。
-        """
-        with self._retreat_lock:
-            return self._retreat_state
-
-    def get_effective_rate_rpm(self) -> float:
-        """获取当前实际速率（RPM），考虑避退乘数。
-
-        Returns:
-            当前每分钟速率。
-        """
-        with self._lock:
-            return self._rate * 60.0
-
-    def get_base_rate_rpm(self) -> float:
-        """获取基准速率（RPM），即未避退时的速率。
-
-        Returns:
-            基准每分钟速率。
-        """
-        return self._base_rate * 60.0
-
-    def reset_to_base(self) -> None:
-        """手动重置到基准速率（用于运维干预）。"""
-        with self._retreat_lock:
-            self._rate = self._base_rate
-            self._retreat_state = RetreatState.NORMAL
-            self._last_state_change = time.monotonic()
-            self._429_window.clear()

services/nvidia_sidecar/server.py

@@ -1,813 +0,0 @@
-"""
-NVIDIA Sidecar 限流代理 — FastAPI 代理主入口 (§3.4)
-
-完整的 API 代理链路：
-    接收 → 网关识别 → [NVIDIA: 排队 → 令牌限流] → httpx 转发 → 返回
-
-非 NVIDIA 请求直通上游，NVIDIA 请求经过四级优先级队列 + 令牌桶限流。
-"""
-
-from __future__ import annotations
-
-import asyncio
-import logging
-import time
-from collections.abc import AsyncGenerator
-from contextlib import asynccontextmanager
-from typing import Any
-
-import httpx
-import structlog
-import uvicorn
-from fastapi import FastAPI, Request, Response
-from fastapi.middleware.cors import CORSMiddleware
-from fastapi.responses import JSONResponse, StreamingResponse
-
-from nvidia_sidecar.config import load_config, SidecarConfig
-from nvidia_sidecar.rate_limiter import (
-    Priority,
-    AdaptiveTokenBucket,
-    is_nvidia_gateway,
-)
-from nvidia_sidecar.priority_queue import (
-    PriorityRequestQueue,
-    QueueFullError,
-    QueueFullPassthrough,
-    QueueFullPolicy,
-)
-from nvidia_sidecar.metrics import PrometheusMetrics
-from nvidia_sidecar.health import HealthService
-from nvidia_sidecar.webui import webui_router
-
-# ---------------------------------------------------------------------------
-# 结构化日志
-# ---------------------------------------------------------------------------
-
-structlog.configure(
-    processors=[
-        structlog.stdlib.filter_by_level,
-        structlog.stdlib.add_logger_name,
-        structlog.stdlib.add_log_level,
-        structlog.stdlib.PositionalArgumentsFormatter(),
-        structlog.processors.TimeStamper(fmt="iso"),
-        structlog.processors.StackInfoRenderer(),
-        structlog.processors.format_exc_info,
-        structlog.processors.UnicodeDecoder(),
-        structlog.processors.JSONRenderer(),
-    ],
-    context_class=dict,
-    logger_factory=structlog.PrintLoggerFactory(),
-    wrapper_class=structlog.stdlib.BoundLogger,
-    cache_logger_on_first_use=True,
-)
-logger: structlog.stdlib.BoundLogger = structlog.get_logger("nvidia_sidecar")
-
-
-# ---------------------------------------------------------------------------
-# 全局状态（通过 lifespan 初始化，模块级引用方便路由访问）
-# ---------------------------------------------------------------------------
-
-_config: SidecarConfig
-_http_client: httpx.AsyncClient
-_priority_queue: PriorityRequestQueue
-_token_bucket: AdaptiveTokenBucket
-_prometheus: PrometheusMetrics
-_health_service: HealthService
-_pending_requests: dict[str, tuple[asyncio.Future[httpx.Response], float]]
-"""request_id → (response future, enqueued_at) 的映射。"""
-_metrics_task: asyncio.Task[None] | None = None
-
-# 统计计数器（受 _stats_lock 保护, 修复梁思筑评审 #1: data race）
-_stats: dict[str, int] = {
-    "total_requests": 0,
-    "nvidia_requests": 0,
-    "passthrough_requests": 0,
-    "ratelimited_requests": 0,
-    "queue_full_rejects": 0,
-    "upstream_errors": 0,
-    "start_time": 0,
-}
-_stats_lock: asyncio.Lock = asyncio.Lock()
-
-
-# ---------------------------------------------------------------------------
-# 工具函数
-# ---------------------------------------------------------------------------
-
-async def _increment_stat(key: str, delta: int = 1) -> None:
-    """线程安全的 _stats 计数器自增（梁思筑评审 #1 修复：消除 data race）。"""
-    async with _stats_lock:
-        _stats[key] = _stats.get(key, 0) + delta
-
-
-def _extract_model(body: Any) -> str | None:
-    """从请求体中提取模型标识符（兼容 OpenAI Chat/Completions 格式）。
-
-    Args:
-        body: 已解析的 JSON 请求体。
-
-    Returns:
-        模型标识符字符串，或 None。
-    """
-    if isinstance(body, dict):
-        return str(body.get("model", "")) or None
-    return None
-
-
-def _resolve_priority(headers: dict[str, str]) -> Priority:
-    """从请求 headers 解析优先级。
-
-    检查 ``X-Priority`` header，值为 ``urgent``/``high``/``normal``/``low``，
-    不区分大小写。默认 NORMAL。
-    """
-    raw = headers.get("x-priority", "").strip().lower()
-    mapping: dict[str, Priority] = {
-        "urgent": Priority.URGENT,
-        "high": Priority.HIGH,
-        "normal": Priority.NORMAL,
-        "low": Priority.LOW,
-    }
-    return mapping.get(raw, Priority.NORMAL)
-
-
-# ---------------------------------------------------------------------------
-# 上游转发
-# ---------------------------------------------------------------------------
-
-async def _forward_to_upstream(
-    method: str,
-    path: str,
-    body: bytes | None,
-    headers: dict[str, str],
-    stream: bool = False,
-) -> httpx.Response:
-    """将请求转发到 NVIDIA 上游 API。
-
-    Args:
-        method: HTTP 方法。
-        path: 请求路径（如 ``/v1/chat/completions``）。
-        body: 原始请求体 bytes。
-        headers: 要转发的请求 headers（会追加 Authorization）。
-        stream: 是否请求流式响应。
-
-    Returns:
-        httpx.Response 对象。
-
-    Raises:
-        httpx.HTTPError: HTTP 请求失败。
-    """
-    upstream_url = _config.upstream_url.rstrip("/") + path
-    forward_headers: dict[str, str] = {
-        k: v for k, v in headers.items()
-        if k.lower() not in ("host", "content-length", "transfer-encoding")
-    }
-    if _config.upstream_api_key:
-        forward_headers["authorization"] = f"Bearer {_config.upstream_api_key}"
-    elif "authorization" not in {k.lower() for k in forward_headers}:
-        forward_headers["authorization"] = "Bearer nvidia"
-
-    try:
-        req = _http_client.build_request(
-            method=method,
-            url=upstream_url,
-            headers=forward_headers,
-            content=body,
-            timeout=_config.request_timeout,
-        )
-        response = await _http_client.send(req, stream=stream)
-        return response
-    except httpx.TimeoutException:
-        logger.warning("upstream_timeout", path=path, timeout=_config.request_timeout)
-        raise
-    except httpx.HTTPError as exc:
-        logger.error("upstream_error", path=path, error=str(exc))
-        raise
-
-
-# ---------------------------------------------------------------------------
-# worker 协程：消费优先级队列 + 令牌桶 + 转发
-# ---------------------------------------------------------------------------
-
-async def _worker_loop() -> None:
-    """后台 worker：持续从优先级队列取请求 → 令牌限流 → 转发 → 设置 future 结果。"""
-    log = logger.bind(worker="main")
-    log.info("worker_started")
-
-    while True:
-        try:
-            queue_item = await _priority_queue.get(timeout=1.0)
-            if queue_item is None:
-                continue
-
-            request_id = queue_item.request_id
-            payload = queue_item.payload
-            headers = queue_item.headers
-            enqueued_at = queue_item.enqueued_at
-
-            # 查找对应的 pending future
-            pending_entry = _pending_requests.get(request_id)
-            if pending_entry is None:
-                log.warning("orphan_request", request_id=request_id)
-                continue
-            future, _ = pending_entry
-
-            # 低优先级令牌等待超时处理
-            if queue_item.priority == Priority.LOW:
-                # 放线程池执行阻塞的令牌桶调用
-                got_token = await asyncio.to_thread(
-                    _token_bucket.try_consume,
-                    tokens=1,
-                    timeout=_config.low_priority_timeout,
-                )
-                if not got_token:
-                    log.info("low_priority_timeout", request_id=request_id)
-                    await _increment_stat("ratelimited_requests")
-                    _prometheus.record_request(queue_item.priority.name, "ratelimited")
-                    if not future.done():
-                        future.set_exception(
-                            _RateLimitedError(
-                                f"低优先级请求令牌等待超时 ({_config.low_priority_timeout}s)"
-                            )
-                        )
-                    _pending_requests.pop(request_id, None)
-                    continue
-            else:
-                # 非低优先级：在 worker 内轮询等待令牌，避免重入队导致 future 悬挂
-                # （重入队会生成新 request_id，原 future 永不 resolve → 客户端永久 hang）
-                got_token = await asyncio.to_thread(_token_bucket.consume, tokens=1)
-                if not got_token:
-                    token_deadline = time.monotonic() + _config.request_timeout
-                    while not got_token:
-                        await asyncio.sleep(0.1)
-                        got_token = await asyncio.to_thread(_token_bucket.consume, tokens=1)
-                        if time.monotonic() > token_deadline:
-                            break
-                    if not got_token:
-                        log.warning(
-                            "token_wait_timeout",
-                            request_id=request_id,
-                            priority=queue_item.priority.name,
-                            timeout=_config.request_timeout,
-                        )
-                        await _increment_stat("ratelimited_requests")
-                        _prometheus.record_request(queue_item.priority.name, "ratelimited")
-                        if not future.done():
-                            future.set_exception(
-                                _RateLimitedError(
-                                    f"令牌等待超时 ({_config.request_timeout:.0f}s)"
-                                )
-                            )
-                        _pending_requests.pop(request_id, None)
-                        continue
-
-            # 转发到上游
-            upstream_start = time.monotonic()
-            try:
-                path = headers.get("x-original-path", "/v1/chat/completions")
-                method = headers.get("x-original-method", "POST")
-                # 过滤内部 headers
-                clean_headers = {
-                    k: v for k, v in headers.items()
-                    if not k.startswith("x-original-") and not k.startswith("x-request-id")
-                }
-
-                resp = await _forward_to_upstream(
-                    method=method,
-                    path=path,
-                    body=payload.get("_raw_body"),
-                    headers=clean_headers,
-                    stream=payload.get("stream", False),
-                )
-
-                upstream_latency = time.monotonic() - upstream_start
-                queue_latency = time.monotonic() - enqueued_at
-                total_latency = upstream_latency + queue_latency
-
-                is_429: bool = resp.status_code == 429
-                _token_bucket.record_response(is_429)
-
-                # 避退状态评估 + 指标更新
-                _token_bucket.evaluate_retreat()
-                retreat_state = _token_bucket.get_retreat_state()
-                effective_rpm = _token_bucket.get_effective_rate_rpm()
-                upstream_429_rate = _token_bucket.get_429_rate()
-                _prometheus.update_retreat_metrics(retreat_state, effective_rpm, upstream_429_rate)
-
-                log.info(
-                    "request_completed",
-                    request_id=request_id,
-                    status=resp.status_code,
-                    upstream_latency=round(upstream_latency, 3),
-                    queue_latency=round(queue_latency, 3),
-                    total_latency=round(total_latency, 3),
-                    retreat_state=retreat_state,
-                    effective_rpm=round(effective_rpm, 1),
-                )
-
-                # 记录 Prometheus 指标
-                model_id = _extract_model(payload) or "unknown"
-                _prometheus.record_upstream_latency(model_id, upstream_latency)
-                if not resp.is_success:
-                    _prometheus.record_upstream_error(resp.status_code, model_id)
-                _prometheus.record_request(queue_item.priority.name, "success" if resp.is_success else "error")
-                _prometheus.record_queue_latency(queue_item.priority.name, queue_latency)
-
-                if not future.done():
-                    future.set_result(resp)
-
-            except (httpx.HTTPError, OSError) as exc:
-                log.error("upstream_request_failed", request_id=request_id, error=str(exc))
-                await _increment_stat("upstream_errors")
-                _prometheus.record_request(queue_item.priority.name, "error")
-                _prometheus.set_health(False)
-                if not future.done():
-                    future.set_exception(exc)
-
-            _pending_requests.pop(request_id, None)
-
-        except asyncio.CancelledError:
-            log.info("worker_cancelled")
-            break
-        except Exception:
-            log.exception("worker_unexpected_error")
-
-
-# ---------------------------------------------------------------------------
-# PASSTHROUGH 直通路径（队列满 + PASSTHROUGH 策略）
-# ---------------------------------------------------------------------------
-
-async def _passthrough_with_rate_limit(
-    request: Request,
-    path: str,
-    body_bytes: bytes,
-    raw_headers: dict[str, str],
-    priority: Priority,
-) -> Response:
-    """队列满时的 PASSSTHROUGH 直通路径：仍受令牌桶限流，但不排队。
-
-    Args:
-        request: FastAPI Request。
-        path: 请求路径。
-        body_bytes: 原始请求体。
-        raw_headers: 请求 headers。
-        priority: 请求优先级。
-
-    Returns:
-        FastAPI Response。
-    """
-    await _increment_stat("passthrough_requests")
-    _prometheus.increment_fallback()
-
-    # 低优先级走令牌桶等待
-    if priority == Priority.LOW:
-        got_token = await asyncio.to_thread(
-            _token_bucket.try_consume,
-            tokens=1,
-            timeout=_config.low_priority_timeout,
-        )
-        if not got_token:
-            await _increment_stat("ratelimited_requests")
-            _prometheus.record_request(priority.name, "ratelimited")
-            return JSONResponse(
-                status_code=429,
-                content={
-                    "error": {
-                        "message": f"令牌不足（队列满 + passthrough），超时 {_config.low_priority_timeout}s",
-                        "type": "RateLimitedError",
-                    }
-                },
-            )
-    else:
-        got_token = await asyncio.to_thread(_token_bucket.consume, tokens=1)
-        if not got_token:
-            # 非低优先级轮询等待，使用 config.request_timeout 替代硬编码 30s
-            # （严维序评审 minor / 梁思筑评审 #3：hot-reload 假生效修复）
-            deadline = time.monotonic() + _config.request_timeout
-            while not got_token:
-                await asyncio.sleep(0.1)
-                got_token = await asyncio.to_thread(_token_bucket.consume, tokens=1)
-                if time.monotonic() > deadline:
-                    await _increment_stat("ratelimited_requests")
-                    _prometheus.record_request(priority.name, "ratelimited")
-                    return JSONResponse(
-                        status_code=429,
-                        content={
-                            "error": {
-                                "message": f"令牌不足（队列满 + passthrough），等待超时 {_config.request_timeout:.0f}s",
-                                "type": "RateLimitedError",
-                            }
-                        },
-                    )
-
-    # 拿到令牌，直接转发
-    try:
-        clean_headers = {k: v for k, v in raw_headers.items()}
-        resp = await _forward_to_upstream(
-            method=request.method,
-            path=path,
-            body=body_bytes if body_bytes else None,
-            headers=clean_headers,
-            stream=False,
-        )
-        retreat_state = _token_bucket.get_retreat_state()
-        _token_bucket.evaluate_retreat()
-        _prometheus.update_retreat_metrics(
-            retreat_state,
-            _token_bucket.get_effective_rate_rpm(),
-            _token_bucket.get_429_rate(),
-        )
-        return _build_response(resp)
-    except Exception as exc:
-        status, msg = _map_exception(exc)
-        logger.error("passthrough_error", path=path, error=str(exc))
-        _prometheus.set_health(False)
-        return JSONResponse(
-            status_code=status,
-            content={"error": {"message": msg, "type": type(exc).__name__}},
-        )
-
-
-# ---------------------------------------------------------------------------
-# 自定义异常
-# ---------------------------------------------------------------------------
-
-class _RateLimitedError(Exception):
-    """429 限流错误。"""
-    pass
-
-
-# ---------------------------------------------------------------------------
-# 异常处理矩阵 (§3.4)
-# ---------------------------------------------------------------------------
-
-_EXCEPTION_MATRIX: dict[type[Exception], tuple[int, str]] = {
-    _RateLimitedError: (429, "Too Many Requests — 令牌不足"),
-    QueueFullError: (503, "Service Unavailable — 队列已满"),
-    httpx.TimeoutException: (504, "Gateway Timeout — 上游超时"),
-    httpx.ConnectError: (502, "Bad Gateway — 上游连接失败"),
-    httpx.HTTPStatusError: (502, "Bad Gateway — 上游返回错误状态"),
-}
-
-
-def _map_exception(exc: Exception) -> tuple[int, str]:
-    """将异常映射为 HTTP 状态码 + 错误信息。"""
-    for exc_type, (status, msg) in _EXCEPTION_MATRIX.items():
-        if isinstance(exc, exc_type):
-            return status, msg
-    return 500, f"Internal Server Error — {type(exc).__name__}"
-
-
-# ---------------------------------------------------------------------------
-# FastAPI 应用 + lifespan
-# ---------------------------------------------------------------------------
-
-@asynccontextmanager
-async def lifespan(app: FastAPI) -> AsyncGenerator[None, Any]:
-    """应用生命周期管理：初始化/清理全局资源。"""
-    global _config, _http_client, _priority_queue, _token_bucket, _pending_requests
-    global _prometheus, _health_service, _metrics_task
-
-    # 启动
-    _config = load_config()
-    logging.getLogger().setLevel(_config.log_level.upper())
-
-    _http_client = httpx.AsyncClient(
-        timeout=httpx.Timeout(_config.request_timeout),
-        limits=httpx.Limits(
-            max_connections=100,
-            max_keepalive_connections=20,
-        ),
-    )
-    _priority_queue = PriorityRequestQueue(max_size=_config.queue_max_size)
-    _token_bucket = AdaptiveTokenBucket(
-        rate=_config.rate_rpm / 60.0,
-        capacity=_config.bucket_capacity,
-    )
-    _prometheus = PrometheusMetrics()
-    _health_service = HealthService()
-    _pending_requests = {}
-    _stats["start_time"] = int(time.time())
-
-    # 启动 worker 协程
-    worker_task = asyncio.create_task(_worker_loop())
-
-    # 在独立端口 :9191 启动 Prometheus metrics 服务器
-    metrics_app = _prometheus.build_asgi_app()
-    metrics_config = uvicorn.Config(
-        metrics_app,
-        host=_config.listen_host,
-        port=_config.metrics_port,
-        log_level="error",
-    )
-    metrics_server = uvicorn.Server(metrics_config)
-    _metrics_task = asyncio.create_task(metrics_server.serve())
-
-    # CORS 中间件（严维序评审 #8）
-    app.add_middleware(
-        CORSMiddleware,
-        allow_origins=["*"],
-        allow_credentials=False,
-        allow_methods=["*"],
-        allow_headers=["*"],
-    )
-
-    # 挂载 webui 子路由
-    app.include_router(webui_router)
-
-    # upstream_api_key 启动检查（严维序评审 #5）
-    if not _config.upstream_api_key:
-        logger.warning(
-            "upstream_api_key_empty",
-            message="SIDECAR_API_KEY 未设置，NVIDIA 请求将因 401 认证失败",
-        )
-
-    logger.info(
-        "sidecar_started",
-        host=_config.listen_host,
-        port=_config.listen_port,
-        metrics_port=_config.metrics_port,
-        rate_rpm=_config.rate_rpm,
-        queue_max=_config.queue_max_size,
-        retreat_enabled=True,
-    )
-
-    yield  # app 运行中
-
-    # 关闭
-    worker_task.cancel()
-    try:
-        await worker_task
-    except asyncio.CancelledError:
-        pass
-
-    if _metrics_task is not None:
-        _metrics_task.cancel()
-        try:
-            await _metrics_task
-        except asyncio.CancelledError:
-            pass
-
-    await _http_client.aclose()
-    logger.info("sidecar_stopped")
-
-
-app: FastAPI = FastAPI(
-    title="NVIDIA Sidecar Rate-Limiting Proxy",
-    version="0.1.0",
-    lifespan=lifespan,
-)
-
-
-# ---------------------------------------------------------------------------
-# 核心代理处理器
-# ---------------------------------------------------------------------------
-
-async def _handle_proxy_request(request: Request, path: str) -> Response:
-    """统一的代理请求处理入口。
-
-    执行完整链路：
-    1. 解析请求体 → 提取 model
-    2. 网关识别 → 非 NVIDIA 直通
-    3. NVIDIA → 排队 + 令牌限流 + 转发
-    """
-    await _increment_stat("total_requests")
-
-    # 解析请求
-    body_bytes: bytes = await request.body()
-    raw_headers: dict[str, str] = dict(request.headers)
-
-    # 尝试解析 JSON body
-    body_json: dict[str, Any] = {}
-    try:
-        if body_bytes:
-            body_json = __import__("json").loads(body_bytes)
-    except (ValueError, TypeError):
-        body_json = {}
-
-    # 提取 model 进行网关识别
-    model: str | None = _extract_model(body_json)
-    is_nvidia: bool = is_nvidia_gateway(model)
-
-    # 非 NVIDIA → 直接转发
-    if not is_nvidia:
-        await _increment_stat("passthrough_requests")
-        try:
-            resp = await _forward_to_upstream(
-                method=request.method,
-                path=path,
-                body=body_bytes if body_bytes else None,
-                headers=raw_headers,
-                stream=body_json.get("stream", False),
-            )
-            return _build_response(resp)
-        except Exception as exc:
-            status, msg = _map_exception(exc)
-            logger.error("passthrough_error", path=path, error=str(exc))
-            return JSONResponse(
-                status_code=status,
-                content={"error": {"message": msg, "type": type(exc).__name__}},
-            )
-
-    # NVIDIA → 排队 + 限流 + 转发
-    await _increment_stat("nvidia_requests")
-    priority: Priority = _resolve_priority(raw_headers)
-
-    # 注入内部元数据到 payload
-    payload_for_queue: dict[str, Any] = dict(body_json)
-    payload_for_queue["_raw_body"] = body_bytes
-
-    # 尝试入队；PASSTHROUGH 策略下队列满时走直通路径
-    try:
-        request_id = await _priority_queue.put(
-            item=payload_for_queue,
-            priority=priority,
-            headers={
-                **raw_headers,
-                "x-original-path": path,
-                "x-original-method": request.method,
-            },
-        )
-    except QueueFullError:
-        await _increment_stat("queue_full_rejects")
-        return JSONResponse(
-            status_code=503,
-            content={
-                "error": {
-                    "message": "队列已满，当前策略: reject",
-                    "type": "QueueFullError",
-                }
-            },
-        )
-    except QueueFullPassthrough:
-        # 队列满 + PASSTHROUGH：绕过排队，尝试令牌桶后直接转发
-        await _increment_stat("passthrough_requests")
-        logger.info("queue_full_passthrough", path=path)
-        return await _passthrough_with_rate_limit(request, path, body_bytes, raw_headers, priority)
-
-    # 创建 future 并注册到 pending
-    loop = asyncio.get_running_loop()
-    future: asyncio.Future[httpx.Response] = loop.create_future()
-    _pending_requests[request_id] = (future, time.monotonic())
-
-    try:
-        # 等待 worker 完成处理
-        resp = await future
-        return _build_response(resp)
-    except _RateLimitedError as exc:
-        return JSONResponse(
-            status_code=429,
-            content={
-                "error": {
-                    "message": str(exc),
-                    "type": "RateLimitedError",
-                }
-            },
-        )
-    except Exception as exc:
-        status, msg = _map_exception(exc)
-        logger.error("proxy_error", path=path, request_id=request_id, error=str(exc))
-        return JSONResponse(
-            status_code=status,
-            content={"error": {"message": msg, "type": type(exc).__name__}},
-        )
-
-
-def _build_response(resp: httpx.Response) -> Response:
-    """将 httpx.Response 转换为 FastAPI Response。
-
-    支持 JSON 和流式 (SSE) 两种响应类型。
-    """
-    content_type = resp.headers.get("content-type", "")
-
-    # 流式响应 (SSE)
-    if "text/event-stream" in content_type or "stream" in content_type:
-        return StreamingResponse(
-            content=resp.aiter_bytes(),
-            status_code=resp.status_code,
-            headers={
-                k: v for k, v in resp.headers.items()
-                if k.lower() not in ("content-encoding", "transfer-encoding")
-            },
-            media_type="text/event-stream",
-        )
-
-    # 普通 JSON 响应
-    return Response(
-        content=resp.content,
-        status_code=resp.status_code,
-        headers={
-            k: v for k, v in resp.headers.items()
-            if k.lower() not in ("content-encoding", "transfer-encoding")
-        },
-        media_type=content_type or "application/json",
-    )
-
-
-# ---------------------------------------------------------------------------
-# 路由
-# ---------------------------------------------------------------------------
-
-@app.get("/health")
-async def health() -> dict[str, Any]:
-    """存活检查 (liveness)。"""
-    return _health_service.liveness()
-
-
-@app.get("/health/ready")
-async def health_ready() -> dict[str, Any]:
-    """就绪检查 (readiness)，含上游连通性。"""
-    queue_size = await _priority_queue.get_queue_size()
-    bucket_status = _token_bucket.get_status()
-    return await _health_service.readiness(
-        upstream_url=_config.upstream_url,
-        upstream_api_key=_config.upstream_api_key or "",
-        queue_current_size=queue_size,
-        queue_max_size=_config.queue_max_size,
-        available_tokens=bucket_status["tokens"],
-        bucket_capacity=bucket_status["capacity"],
-    )
-
-
-@app.get("/status")
-async def status() -> dict[str, Any]:
-    """调试用：限流器 + 队列 + 避退完整状态。"""
-    queue_stats = await _priority_queue.get_stats()
-    bucket_status = _token_bucket.get_status()
-    return {
-        "requests": {
-            "total": _stats["total_requests"],
-            "nvidia": _stats["nvidia_requests"],
-            "passthrough": _stats["passthrough_requests"],
-            "ratelimited": _stats["ratelimited_requests"],
-        },
-        "errors": {
-            "queue_full_rejects": _stats["queue_full_rejects"],
-            "upstream_errors": _stats["upstream_errors"],
-        },
-        "queue": queue_stats,
-        "token_bucket": bucket_status,
-        "retreat": {
-            "state": _token_bucket.get_retreat_state(),
-            "effective_rpm": round(_token_bucket.get_effective_rate_rpm(), 1),
-            "base_rpm": round(_token_bucket.get_base_rate_rpm(), 1),
-            "upstream_429_rate": round(_token_bucket.get_429_rate(), 4),
-        },
-        "uptime_seconds": int(time.time() - _stats["start_time"]) if _stats["start_time"] else 0,
-    }
-
-
-# ---- OpenAI 兼容端点 ----
-
-@app.post("/v1/chat/completions")
-async def chat_completions(request: Request) -> Response:
-    """OpenAI Chat Completions API 代理（含流式支持）。"""
-    return await _handle_proxy_request(request, "/v1/chat/completions")
-
-
-@app.post("/v1/completions")
-async def completions(request: Request) -> Response:
-    """OpenAI Completions API 代理（legacy）。"""
-    return await _handle_proxy_request(request, "/v1/completions")
-
-
-@app.post("/v1/embeddings")
-async def embeddings(request: Request) -> Response:
-    """OpenAI Embeddings API 代理。"""
-    return await _handle_proxy_request(request, "/v1/embeddings")
-
-
-@app.get("/v1/models")
-@app.get("/v1/models/{model_id:path}")
-async def list_models(request: Request, model_id: str | None = None) -> Response:
-    """OpenAI Models API 代理。"""
-    path = f"/v1/models/{model_id}" if model_id else "/v1/models"
-    return await _handle_proxy_request(request, path)
-
-
-# ---- 通用代理（catch-all 用于非标准 NVIDIA 端点） ----
-
-@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"])
-async def catch_all(request: Request, path: str) -> Response:
-    """通用代理端点：转发任何未匹配的路径到上游。"""
-    target_path = f"/{path}" if not path.startswith("/") else path
-    return await _handle_proxy_request(request, target_path)
-
-
-# ---------------------------------------------------------------------------
-# 入口
-# ---------------------------------------------------------------------------
-
-def main() -> None:
-    """开发/调试入口。"""
-    import uvicorn
-    cfg: SidecarConfig = load_config()
-    uvicorn.run(
-        "nvidia_sidecar.server:app",
-        host=cfg.listen_host,
-        port=cfg.listen_port,
-        log_level=cfg.log_level.lower(),
-    )
-
-
-if __name__ == "__main__":
-    main()

services/nvidia_sidecar/static/dashboard.html

@@ -1,261 +0,0 @@
-<!DOCTYPE html>
-<html lang="zh-CN">
-<head>
-  <meta charset="UTF-8">
-  <meta name="viewport" content="width=device-width, initial-scale=1.0">
-  <title>NVIDIA Sidecar — 实时仪表盘</title>
-  <script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.7/dist/chart.umd.min.js"></script>
-  <style>
-    * { margin: 0; padding: 0; box-sizing: border-box; }
-    body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; background: #0f172a; color: #e2e8f0; padding: 24px; }
-    h1 { font-size: 22px; font-weight: 600; margin-bottom: 4px; color: #f8fafc; }
-    .subtitle { color: #94a3b8; font-size: 13px; margin-bottom: 24px; }
-    .grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(380px, 1fr)); gap: 20px; margin-bottom: 24px; }
-    .card { background: #1e293b; border-radius: 12px; padding: 20px; border: 1px solid #334155; }
-    .card h2 { font-size: 15px; font-weight: 600; color: #94a3b8; margin-bottom: 14px; text-transform: uppercase; letter-spacing: 0.05em; }
-    .card canvas { max-height: 220px; }
-    .stat-row { display: flex; gap: 16px; flex-wrap: wrap; }
-    .stat { flex: 1; min-width: 100px; background: #0f172a; border-radius: 8px; padding: 12px; text-align: center; border: 1px solid #334155; }
-    .stat .value { font-size: 28px; font-weight: 700; color: #38bdf8; }
-    .stat .label { font-size: 11px; color: #64748b; margin-top: 4px; text-transform: uppercase; }
-    .stat.warn .value { color: #f59e0b; }
-    .stat.danger .value { color: #ef4444; }
-    .retreat-badge { display: inline-block; padding: 2px 10px; border-radius: 999px; font-size: 12px; font-weight: 600; }
-    .retreat-badge.normal { background: #065f46; color: #6ee7b7; }
-    .retreat-badge.retreat { background: #78350f; color: #fbbf24; }
-    .retreat-badge.recover { background: #1e3a5f; color: #60a5fa; }
-    .config-panel { background: #1e293b; border-radius: 12px; padding: 20px; border: 1px solid #334155; }
-    .config-panel h2 { font-size: 15px; font-weight: 600; color: #94a3b8; margin-bottom: 14px; text-transform: uppercase; letter-spacing: 0.05em; }
-    .config-row { display: flex; align-items: center; gap: 12px; margin-bottom: 12px; flex-wrap: wrap; }
-    .config-row label { min-width: 100px; font-size: 13px; color: #cbd5e1; }
-    .config-row input, .config-row select { background: #0f172a; border: 1px solid #334155; border-radius: 6px; color: #e2e8f0; padding: 6px 10px; font-size: 13px; }
-    .config-row input[type="range"] { width: 140px; }
-    .config-row button { background: #38bdf8; color: #0f172a; border: none; border-radius: 6px; padding: 6px 16px; font-size: 13px; font-weight: 600; cursor: pointer; }
-    .config-row button:hover { background: #7dd3fc; }
-    .config-row button:disabled { background: #475569; cursor: not-allowed; }
-    .toast { position: fixed; top: 16px; right: 16px; padding: 10px 20px; border-radius: 8px; font-size: 13px; z-index: 999; animation: fadeInOut 3s; }
-    .toast.success { background: #065f46; color: #6ee7b7; }
-    .toast.error { background: #7f1d1d; color: #fca5a5; }
-    @keyframes fadeInOut { 0% { opacity: 0; transform: translateY(-8px); } 10% { opacity: 1; transform: translateY(0); } 80% { opacity: 1; } 100% { opacity: 0; } }
-    .disconnected { background: #7f1d1d; color: #fca5a5; padding: 4px 10px; border-radius: 4px; font-size: 12px; display: inline-block; margin-left: 8px; }
-    .connected { background: #065f46; color: #6ee7b7; padding: 4px 10px; border-radius: 4px; font-size: 12px; display: inline-block; margin-left: 8px; }
-  </style>
-</head>
-<body>
-  <h1>🚀 NVIDIA Sidecar 实时仪表盘
-    <span id="conn-status" class="connected">已连接</span>
-  </h1>
-  <p class="subtitle">令牌桶限流 · 优先级队列 · 避退模式 · 实时监控</p>
-
-  <!-- 状态卡片 -->
-  <div class="stat-row" style="margin-bottom: 24px;">
-    <div class="stat"><div class="value" id="val-total">0</div><div class="label">总请求</div></div>
-    <div class="stat"><div class="value" id="val-nvidia">0</div><div class="label">NVIDIA 请求</div></div>
-    <div class="stat"><div class="value" id="val-rate">0</div><div class="label">当前 RPM</div></div>
-    <div class="stat"><div class="value" id="val-429">0%</div><div class="label">上游 429 率</div></div>
-    <div class="stat"><div class="value" id="val-retreat">正常</div><div class="label">避退状态</div></div>
-    <div class="stat"><div class="value" id="val-uptime">0s</div><div class="label">运行时间</div></div>
-  </div>
-
-  <!-- 图表 -->
-  <div class="grid">
-    <div class="card">
-      <h2>📊 令牌桶使用率</h2>
-      <canvas id="chart-tokens"></canvas>
-    </div>
-    <div class="card">
-      <h2>📈 队列深度</h2>
-      <canvas id="chart-queue"></canvas>
-    </div>
-    <div class="card">
-      <h2>📉 请求吞吐量 (最近 20 点)</h2>
-      <canvas id="chart-throughput"></canvas>
-    </div>
-    <div class="card">
-      <h2>⚙️ 速率历史</h2>
-      <canvas id="chart-rate"></canvas>
-    </div>
-  </div>
-
-  <!-- 配置面板 -->
-  <div class="config-panel">
-    <h2>🔧 实时配置</h2>
-    <div class="config-row">
-      <label>速率 (RPM)</label>
-      <input type="range" id="cfg-rate-rpm" min="1" max="100" value="40" oninput="document.getElementById('cfg-rate-val').textContent=this.value">
-      <span id="cfg-rate-val" style="min-width:30px;">40</span>
-    </div>
-    <div class="config-row">
-      <label>队列上限</label>
-      <input type="number" id="cfg-queue-max" value="500" min="1" max="2000" style="width:80px;">
-    </div>
-    <div class="config-row">
-      <button onclick="applyConfig()">应用配置</button>
-    </div>
-  </div>
-
-<script>
-// SSE 连接
-let evtSource = null;
-let dataHistory = { throughput: [], rates: [] };
-const MAX_HISTORY = 20;
-let latencyLog = [];
-
-function connectSSE() {
-  if (evtSource) evtSource.close();
-  evtSource = new EventSource('/api/dashboard/stream');
-  evtSource.onmessage = (e) => {
-    try {
-      const snap = JSON.parse(e.data);
-      updateDashboard(snap);
-      updateLatencies(snap);
-      document.getElementById('conn-status').className = 'connected';
-      document.getElementById('conn-status').textContent = '已连接';
-    } catch (err) {
-      document.getElementById('conn-status').className = 'disconnected';
-      document.getElementById('conn-status').textContent = '解析错误';
-    }
-  };
-  evtSource.onerror = () => {
-    document.getElementById('conn-status').className = 'disconnected';
-    document.getElementById('conn-status').textContent = '断开 - 重连中';
-  };
-}
-
-// 初始化 Chart.js
-const ctxTokens = document.getElementById('chart-tokens').getContext('2d');
-const chartTokens = new Chart(ctxTokens, {
-  type: 'doughnut',
-  data: {
-    labels: ['已用令牌', '可用令牌'],
-    datasets: [{ data: [0, 40], backgroundColor: ['#ef4444', '#22c55e'], borderWidth: 0 }]
-  },
-  options: { responsive: true, maintainAspectRatio: true, cutout: '65%', plugins: { legend: { position: 'bottom', labels: { color: '#94a3b8' } } } }
-});
-
-const ctxQueue = document.getElementById('chart-queue').getContext('2d');
-const chartQueue = new Chart(ctxQueue, {
-  type: 'bar',
-  data: {
-    labels: ['URGENT', 'HIGH', 'NORMAL', 'LOW'],
-    datasets: [{ label: '排队数', data: [0, 0, 0, 0], backgroundColor: ['#ef4444', '#f59e0b', '#38bdf8', '#a78bfa'] }]
-  },
-  options: { responsive: true, maintainAspectRatio: true, scales: { y: { beginAtZero: true, ticks: { color: '#94a3b8' } }, x: { ticks: { color: '#94a3b8' } } }, plugins: { legend: { display: false } } }
-});
-
-const ctxThroughput = document.getElementById('chart-throughput').getContext('2d');
-const chartThroughput = new Chart(ctxThroughput, {
-  type: 'line',
-  data: { labels: [], datasets: [
-    { label: '成功', data: [], borderColor: '#22c55e', backgroundColor: '#22c55e20', fill: false, tension: 0.3, pointRadius: 2 },
-    { label: '429', data: [], borderColor: '#f59e0b', backgroundColor: '#f59e0b20', fill: false, tension: 0.3, pointRadius: 2 },
-    { label: '直通', data: [], borderColor: '#a78bfa', backgroundColor: '#a78bfa20', fill: false, tension: 0.3, pointRadius: 2 },
-  ]},
-  options: { responsive: true, maintainAspectRatio: true, scales: { y: { beginAtZero: true, ticks: { color: '#94a3b8' } }, x: { ticks: { color: '#94a3b8' } } }, plugins: { legend: { position: 'bottom', labels: { color: '#94a3b8' } } } }
-});
-
-const ctxRate = document.getElementById('chart-rate').getContext('2d');
-const chartRate = new Chart(ctxRate, {
-  type: 'line',
-  data: { labels: [], datasets: [
-    { label: '有效 RPM', data: [], borderColor: '#38bdf8', fill: false, tension: 0.3, pointRadius: 2 },
-    { label: '基准 RPM', data: [], borderColor: '#64748b', fill: false, tension: 0.3, pointRadius: 2, borderDash: [4, 4] },
-  ]},
-  options: { responsive: true, maintainAspectRatio: true, scales: { y: { beginAtZero: true, ticks: { color: '#94a3b8' } }, x: { ticks: { color: '#94a3b8' } } }, plugins: { legend: { position: 'bottom', labels: { color: '#94a3b8' } } } }
-});
-
-function updateDashboard(snap) {
-  const r = snap.requests || {};
-  const tb = snap.token_bucket || {};
-  const rt = snap.retreat || {};
-
-  document.getElementById('val-total').textContent = (r.total || 0).toLocaleString();
-  document.getElementById('val-nvidia').textContent = (r.nvidia || 0).toLocaleString();
-  document.getElementById('val-rate').textContent = Math.round(rt.effective_rpm || 40);
-  document.getElementById('val-429').textContent = ((rt.upstream_429_rate || 0) * 100).toFixed(1) + '%';
-  document.getElementById('val-uptime').textContent = fmtDuration(snap.uptime_seconds || 0);
-
-  const retreatEl = document.getElementById('val-retreat');
-  const state = rt.state || 'normal';
-  retreatEl.textContent = state === 'retreat' ? '⚠️ 避退' : state === 'recover' ? '↗ 恢复中' : '✅ 正常';
-  retreatEl.style.color = state === 'retreat' ? '#f59e0b' : state === 'recover' ? '#60a5fa' : '#22c55e';
-
-  chartTokens.data.datasets[0].data = [
-    Math.round((tb.capacity || 40) - (tb.tokens || 40)),
-    Math.round(tb.tokens || 0)
-  ];
-  chartTokens.update();
-
-  const qs = snap.queue || {};
-  const perPriority = qs.per_priority || {};
-  chartQueue.data.datasets[0].data = [
-    perPriority.URGENT || 0,
-    perPriority.HIGH || 0,
-    perPriority.NORMAL || 0,
-    perPriority.LOW || 0
-  ];
-  chartQueue.update();
-
-  const now = new Date().toLocaleTimeString();
-  const prev = dataHistory.throughput.length > 0 ? dataHistory.throughput[dataHistory.throughput.length - 1].nvidia : 0;
-  const throughput = Math.max(0, (r.nvidia || 0) - prev);
-
-  dataHistory.throughput.push({ time: now, nvidia: throughput, ratelimited: r.ratelimited || 0, passthrough: r.passthrough || 0 });
-  dataHistory.rates.push({ time: now, effective: rt.effective_rpm || 40, base: rt.base_rpm || 40 });
-  if (dataHistory.throughput.length > MAX_HISTORY) dataHistory.throughput.shift();
-  if (dataHistory.rates.length > MAX_HISTORY) dataHistory.rates.shift();
-
-  chartThroughput.data.labels = dataHistory.throughput.map(d => d.time);
-  chartThroughput.data.datasets[0].data = dataHistory.throughput.map(d => d.nvidia);
-  chartThroughput.data.datasets[1].data = dataHistory.throughput.map(d => d.ratelimited);
-  chartThroughput.data.datasets[2].data = dataHistory.throughput.map(d => d.passthrough);
-  chartThroughput.update();
-
-  chartRate.data.labels = dataHistory.rates.map(d => d.time);
-  chartRate.data.datasets[0].data = dataHistory.rates.map(d => d.effective);
-  chartRate.data.datasets[1].data = dataHistory.rates.map(d => d.base);
-  chartRate.update();
-}
-
-function updateLatencies(snap) {
-  const tb = snap.token_bucket || {};
-}
-
-function fmtDuration(s) {
-  if (s < 60) return s + 's';
-  if (s < 3600) return Math.floor(s/60) + 'm ' + (s%60) + 's';
-  return Math.floor(s/3600) + 'h ' + Math.floor((s%3600)/60) + 'm';
-}
-
-async function applyConfig() {
-  const btn = document.querySelector('.config-row button');
-  btn.disabled = true;
-  try {
-    const resp = await fetch('/api/admin/config', {
-      method: 'POST',
-      headers: { 'Content-Type': 'application/json' },
-      body: JSON.stringify({
-        rate_rpm: parseInt(document.getElementById('cfg-rate-rpm').value),
-        queue_max_size: parseInt(document.getElementById('cfg-queue-max').value),
-      })
-    });
-    const result = await resp.json();
-    showToast(resp.ok ? 'success' : 'error', resp.ok ? '配置已更新' : (result.detail || '配置更新失败'));
-  } catch (err) {
-    showToast('error', '请求失败: ' + err.message);
-  }
-  btn.disabled = false;
-}
-
-function showToast(type, msg) {
-  const t = document.createElement('div');
-  t.className = 'toast ' + type;
-  t.textContent = msg;
-  document.body.appendChild(t);
-  setTimeout(() => t.remove(), 3000);
-}
-
-connectSSE();
-</script>
-</body>
-</html>

services/nvidia_sidecar/webui.py

@@ -1,293 +0,0 @@
-"""
-NVIDIA Sidecar — WebUI 后端 API
-
-提供仪表盘 SSE 实时推送 + 配置热重载 API。
-"""
-
-from __future__ import annotations
-
-import asyncio
-import json
-import os
-import time
-from pathlib import Path
-from typing import Any, AsyncGenerator
-
-import structlog
-from fastapi import APIRouter, Depends, HTTPException, Request
-from fastapi.responses import HTMLResponse, JSONResponse, StreamingResponse
-from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
-from pydantic import BaseModel
-
-webui_router: APIRouter = APIRouter(prefix="/api", tags=["webui"])
-logger: structlog.stdlib.BoundLogger = structlog.get_logger("nvidia_sidecar.webui")
-
-STATIC_DIR: Path = Path(__file__).parent / "static"
-
-# dashboard.html 缓存（严维序评审 #6 / 梁思筑评审 #8：避免每次请求读磁盘）
-_dashboard_html_cache: tuple[str, float] | None = None
-_DASHBOARD_CACHE_TTL: float = 300.0  # 5 分钟
-
-# Admin API 认证（严维序评审 #1）
-_ADMIN_TOKEN: str | None = os.environ.get("SIDECAR_ADMIN_TOKEN")
-_admin_auth_scheme: HTTPBearer = HTTPBearer(auto_error=False)
-
-
-# ---------------------------------------------------------------------------
-# 配置热重载模型
-# ---------------------------------------------------------------------------
-
-class ConfigPatch(BaseModel):
-    """可在线修改的配置字段。"""
-    rate_rpm: int | None = None
-    queue_max_size: int | None = None
-    fallback_enabled_passthrough: bool | None = None
-
-
-# ---------------------------------------------------------------------------
-# 仪表盘 SSE 推送
-# ---------------------------------------------------------------------------
-
-async def _dashboard_stream(request: Request) -> StreamingResponse:
-    """SSE 实时推送 Sidecar 完整状态快照（每秒一次）。
-
-    供 dashboard.html 的 EventSource 消费。
-    """
-    async def event_generator() -> AsyncGenerator[str, None]:
-        # 首帧发送 retry 字段（严维序评审 minor）：指示客户端断连后等待 3s 重试
-        first_frame = True
-        while True:
-            if await request.is_disconnected():
-                break
-            try:
-                snapshot: dict[str, Any] = await _build_snapshot()
-                payload_sse = f"data: {json.dumps(snapshot, ensure_ascii=False)}\n\n"
-                if first_frame:
-                    payload_sse = f"retry: 3000\n{payload_sse}"
-                    first_frame = False
-                yield payload_sse
-            except Exception:
-                logger.exception("dashboard_sse_error")
-                yield f"data: {json.dumps({'error': 'internal'})}\n\n"
-            await asyncio.sleep(1.0)
-
-    return StreamingResponse(
-        event_generator(),
-        media_type="text/event-stream",
-        headers={
-            "Cache-Control": "no-cache",
-            "X-Accel-Buffering": "no",
-        },
-    )
-
-
-# SSE 首帧写入 retry 字段（严维序评审 minor），在 event_generator 首次 yield 前注入
-# 通过在 StreamingResponse 返回前手动发送 retry header 实现
-# （SSE 协议支持 retry 字段作为重建连接间隔）
-# 注：在 event_generator 的首个 yield 中加入 retry 声明
-
-
-async def _build_snapshot() -> dict[str, Any]:
-    """构建当前状态快照（从全局状态读取，含队列深度）。"""
-    # 延迟导入避免循环依赖
-    from nvidia_sidecar import server
-
-    try:
-        _stats = server._stats
-        _token_bucket = server._token_bucket
-        bucket_status = _token_bucket.get_status()
-        now = time.time()
-        uptime = int(now - _stats["start_time"]) if _stats.get("start_time") else 0
-
-        # 获取队列统计数据（含 per-priority depth）
-        queue_data: dict[str, Any] = {"current_size": 0, "per_priority": {}}
-        try:
-            queue_stats = await server._priority_queue.get_stats()
-            queue_data = {
-                "max_size": queue_stats.get("max_size", 0),
-                "current_size": queue_stats.get("current_size", 0),
-                "per_priority": queue_stats.get("depth_by_priority", {}),
-                "total_enqueued": queue_stats.get("total_enqueued", 0),
-                "total_dequeued": queue_stats.get("total_dequeued", 0),
-                "total_dropped": queue_stats.get("total_dropped", 0),
-            }
-        except Exception:
-            logger.warning("queue_stats_unavailable", message="队列统计获取失败，仪表盘队列深度可能不准确")
-
-        return {
-            "timestamp": now,
-            "uptime_seconds": uptime,
-            "token_bucket": bucket_status,
-            "queue": queue_data,
-            "retreat": {
-                "state": getattr(_token_bucket, "_retreat_state", "normal"),
-                "effective_rpm": round(getattr(_token_bucket, "get_effective_rate_rpm", lambda: 40.0)(), 1),
-                "base_rpm": round(getattr(_token_bucket, "get_base_rate_rpm", lambda: 40.0)(), 1),
-                "upstream_429_rate": round(getattr(_token_bucket, "get_429_rate", lambda: 0.0)(), 4),
-            },
-            "requests": {
-                "total": _stats.get("total_requests", 0),
-                "nvidia": _stats.get("nvidia_requests", 0),
-                "passthrough": _stats.get("passthrough_requests", 0),
-                "ratelimited": _stats.get("ratelimited_requests", 0),
-            },
-            "errors": {
-                "queue_full_rejects": _stats.get("queue_full_rejects", 0),
-                "upstream_errors": _stats.get("upstream_errors", 0),
-            },
-        }
-    except Exception:
-        logger.exception("snapshot_build_error")
-        return {"error": "snapshot_unavailable", "timestamp": time.time()}
-
-
-# ---------------------------------------------------------------------------
-# 配置热重载
-# ---------------------------------------------------------------------------
-
-async def get_config() -> dict[str, Any]:
-    """获取当前完整配置。"""
-    from nvidia_sidecar import server
-
-    cfg = server._config
-    return {
-        "listen_host": cfg.listen_host,
-        "listen_port": cfg.listen_port,
-        "metrics_port": cfg.metrics_port,
-        "upstream_url": cfg.upstream_url,
-        "upstream_api_key": _mask_api_key(cfg.upstream_api_key),
-        "rate_rpm": _get_current_rate(server),
-        "bucket_capacity": cfg.bucket_capacity,
-        "request_timeout": cfg.request_timeout,
-        "queue_max_size": cfg.queue_max_size,
-        "low_priority_timeout": cfg.low_priority_timeout,
-        "fallback_enabled_passthrough": cfg.fallback_enabled_passthrough,
-        "log_level": cfg.log_level,
-    }
-
-
-async def update_config(body: ConfigPatch) -> JSONResponse:
-    """在线修改配置项并即时生效。"""
-    from nvidia_sidecar import server
-
-    cfg = server._config
-    changed: list[str] = []
-
-    if body.rate_rpm is not None:
-        if body.rate_rpm <= 0:
-            raise HTTPException(status_code=400, detail="rate_rpm must be > 0")
-        cfg.rate_rpm = body.rate_rpm
-        server._token_bucket.set_rate(body.rate_rpm / 60.0)
-        changed.append("rate_rpm")
-
-    if body.queue_max_size is not None:
-        if body.queue_max_size <= 0:
-            raise HTTPException(status_code=400, detail="queue_max_size must be > 0")
-        ok, msg = server._priority_queue.set_max_size(body.queue_max_size)
-        if not ok:
-            raise HTTPException(status_code=400, detail=msg)
-        cfg.queue_max_size = body.queue_max_size
-        changed.append("queue_max_size")
-        logger.info("queue_max_size_updated", detail=msg)
-
-    if body.fallback_enabled_passthrough is not None:
-        cfg.fallback_enabled_passthrough = body.fallback_enabled_passthrough
-        changed.append("fallback_enabled_passthrough")
-
-    logger.info("config_updated", changed=changed)
-    return JSONResponse(
-        content={"status": "ok", "changed": changed},
-    )
-
-
-def _mask_api_key(key: str) -> str:
-    """对 API Key 进行脱敏处理，仅保留前 4 位以供识别。
-
-    严维序评审 #2 / 沈路明评审 #3：防止 API Key 明文泄露。
-    """
-    if not key:
-        return ""
-    if len(key) <= 4:
-        return key[:2] + "****"
-    return key[:4] + "****"
-
-
-def _get_current_rate(server_module: Any) -> float:
-    """获取当前实际速率（避退调整后），兼容 AdaptiveTokenBucket。"""
-    tb = server_module._token_bucket
-    if hasattr(tb, "get_effective_rate_rpm"):
-        return float(round(tb.get_effective_rate_rpm(), 1))
-    return float(tb.rate * 60.0)
-
-
-# ---------------------------------------------------------------------------
-# 路由注册
-# ---------------------------------------------------------------------------
-
-@webui_router.get("/dashboard/stream")
-async def dashboard_stream(request: Request) -> StreamingResponse:
-    """SSE 仪表盘实时推送端点。"""
-    return await _dashboard_stream(request)
-
-
-async def _verify_admin_auth(
-    credentials: HTTPAuthorizationCredentials | None = Depends(_admin_auth_scheme),
-) -> None:
-    """Admin API Bearer Token 认证（严维序评审 #1）。
-
-    若设置了 SIDECAR_ADMIN_TOKEN 环境变量，则要求请求携带匹配的 Bearer Token。
-    未设置时跳过认证（开发/测试环境）。
-    """
-    if _ADMIN_TOKEN is None:
-        return  # 未配置认证 token，允许无认证访问
-    if credentials is None:
-        raise HTTPException(status_code=401, detail="需要 Bearer Token 认证（Admin API）")
-    if credentials.credentials != _ADMIN_TOKEN:
-        raise HTTPException(status_code=403, detail="Admin Token 无效")
-
-
-@webui_router.get("/admin/config")
-async def admin_get_config(
-    _auth: None = Depends(_verify_admin_auth),
-) -> JSONResponse:
-    """获取当前配置（需要 Admin 认证）。"""
-    return JSONResponse(content=await get_config())
-
-
-@webui_router.post("/admin/config")
-async def admin_update_config(
-    body: ConfigPatch,
-    _auth: None = Depends(_verify_admin_auth),
-) -> JSONResponse:
-    """在线修改配置（热重载，需要 Admin 认证）。"""
-    return await update_config(body)
-
-
-# ---------------------------------------------------------------------------
-# 仪表盘静态页面
-# ---------------------------------------------------------------------------
-
-def _get_dashboard_html() -> str:
-    """获取仪表盘 HTML（带缓存，严维序评审 #6 / 梁思筑评审 #8）。
-
-    首次加载后缓存 5 分钟，避免每次请求读磁盘。
-    """
-    global _dashboard_html_cache
-    now = time.monotonic()
-    if _dashboard_html_cache is not None:
-        cached_content, cached_at = _dashboard_html_cache
-        if now - cached_at < _DASHBOARD_CACHE_TTL:
-            return cached_content
-
-    dashboard_path = STATIC_DIR / "dashboard.html"
-    if dashboard_path.is_file():
-        content = dashboard_path.read_text(encoding="utf-8")
-        _dashboard_html_cache = (content, now)
-        return content
-    return "<h1>dashboard.html not found</h1>"
-
-
-@webui_router.get("/dashboard", include_in_schema=False)
-async def dashboard_page() -> HTMLResponse:
-    """仪表盘 HTML 页面（含缓存策略）。"""
-    return HTMLResponse(content=_get_dashboard_html())