BIZ-46 Phase3: 7项 follow-up 开发完成
1. 架构解耦 — SidecarContext + FastAPI Depends 注入 - 新增 context.py: SidecarContext dataclass 收敛全部全局状态 - server.py: 移除模块级全局变量,lifespan 创建 ctx → app.state.sidecar - webui.py: 移除反向导入 server,改用 Depends(get_context) 2. Prometheus 标签基数治理 — model_id → provider - upstream_latency_seconds / upstream_errors_total label 收敛为 provider - 模型级信息保留在 structlog JSON 日志 3. SSE 快照共享缓存 - 1s TTL 共享 snapshot cache + double-check locking - 多客户端不重复构建快照 4. 部署支撑 - Dockerfile (python:3.12-slim, 非 root 用户, HEALTHCHECK) - systemd service (安全加固, 资源限制) - .env.example (完整环境变量清单) 5. Readiness HTTP Client 复用 - check_upstream() 注入主 http_client,不再每次创建新 client 6. Retreat 并发回归测试 - 5 个测试用例全部通过(死锁检测 + 状态转换 + 并发安全) 7. Dashboard UX 优化 - 队列柱状图 300ms 平滑动画 - SSE 断连 5s 半透明遮罩 - 队列图标题显示总排队数 - 页面加载同步配置 验证: mypy strict 通过 (0 errors), pytest 5/5 通过, server 导入正常 (13 routes) Co-authored-by: multica-agent <github@multica.ai>
This commit is contained in:
@@ -0,0 +1,31 @@
|
||||
# NVIDIA Sidecar 环境变量清单 (BIZ-46 Phase3 §4)
|
||||
# 复制为 .env 后按需修改,供 Docker / systemd 使用。
|
||||
|
||||
# 网络
|
||||
SIDECAR_HOST=127.0.0.1
|
||||
SIDECAR_PORT=9190
|
||||
SIDECAR_METRICS_PORT=9191
|
||||
|
||||
# 上游 API(必填)
|
||||
SIDECAR_UPSTREAM=https://integrate.api.nvidia.com/v1
|
||||
SIDECAR_API_KEY=nvapi-your-key-here
|
||||
|
||||
# 限流
|
||||
SIDECAR_RATE_RPM=40
|
||||
SIDECAR_BUCKET_CAPACITY=40
|
||||
|
||||
# 超时
|
||||
SIDECAR_TIMEOUT=60
|
||||
|
||||
# 队列
|
||||
SIDECAR_QUEUE_MAX=500
|
||||
SIDECAR_LOW_TIMEOUT=2
|
||||
|
||||
# 降级
|
||||
SIDECAR_FALLBACK_PASSTHROUGH=true
|
||||
|
||||
# 日志
|
||||
SIDECAR_LOG_LEVEL=INFO
|
||||
|
||||
# Admin API 认证(可选,不设置则跳过认证)
|
||||
# SIDECAR_ADMIN_TOKEN=your-admin-token-here
|
||||
@@ -0,0 +1,49 @@
|
||||
# NVIDIA Sidecar 限流代理 — systemd service (BIZ-46 Phase3 §4)
|
||||
#
|
||||
# 安装:
|
||||
# sudo cp deploy/nvidia-sidecar.service /etc/systemd/system/
|
||||
# sudo systemctl daemon-reload
|
||||
# sudo systemctl enable nvidia-sidecar
|
||||
# sudo systemctl start nvidia-sidecar
|
||||
#
|
||||
# 运维:
|
||||
# sudo systemctl status nvidia-sidecar
|
||||
# sudo journalctl -u nvidia-sidecar -f
|
||||
|
||||
[Unit]
|
||||
Description=NVIDIA Sidecar Rate-Limiting Proxy
|
||||
Documentation=https://github.com/bizwings/nvidia-sidecar
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=sidecar
|
||||
Group=sidecar
|
||||
WorkingDirectory=/opt/nvidia-sidecar
|
||||
ExecStart=/opt/nvidia-sidecar/.venv/bin/uvicorn nvidia_sidecar.server:app \
|
||||
--host 127.0.0.1 \
|
||||
--port 9190 \
|
||||
--log-level info
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
# 环境变量
|
||||
EnvironmentFile=/opt/nvidia-sidecar/.env
|
||||
|
||||
# 安全加固
|
||||
NoNewPrivileges=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
ReadWritePaths=/opt/nvidia-sidecar/logs
|
||||
|
||||
# 资源限制
|
||||
LimitNOFILE=65536
|
||||
MemoryMax=512M
|
||||
|
||||
# 启动延迟(等待网络就绪)
|
||||
ExecStartPre=/bin/sleep 1
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
Reference in New Issue
Block a user