05LynShenRust service · React consoleRust 服务 · React 控制台

LynShen GatewayLynShen 网关

A Rust gateway that translates between three AI API dialects by meaning, not field names, and routes each request with retries and health checks.一个 Rust 网关：在三种 AI API 协议之间按含义转换，而不是按字段名；每条请求都有重试和健康检查的路由。

My role我的角色
Lead developer of LynShen’s fork, built on an open-source MIT gateway core. I shipped the console, the store and billing UI, CDN-resilient delivery, routing and failover, phone plans, the search API and the release pipeline.LynShen 分支的主力开发者，基于一个 MIT 协议的开源网关内核。我交付了控制台、商店和计费界面、抗 CDN 故障的分发、路由与故障切换、手机端套餐、搜索 API，以及发布流水线。
Status状态
Private repo私有仓库
Gateway console request logs with demo data: model, channel, latency and tokens per request.
3 → 63 → 6client protocols → upstream channel types客户端协议 → 上游渠道类型
1,500+1500+Rust tests added in this codebase这个代码库里新增的 Rust 测试
44docs locales, 240 MDX pages种文档语言，240 个 MDX 页面

01The problem问题

Three APIs that almost agree.三套几乎一样、又不太一样的 API。

Different shapes结构不同

OpenAI Responses, Chat Completions and Anthropic Messages model reasoning, tool calls, usage and streaming in different ways.OpenAI Responses、Chat Completions 和 Anthropic Messages 对推理、工具调用、用量和流式输出的建模各不相同。

Lossy converters转换会丢东西

Field-by-field mapping drops encrypted reasoning, breaks stream lifecycles, or quietly switches upstream halfway through a reply.逐字段映射会丢掉加密的推理内容、打乱流的生命周期，或者在回复中途悄悄切换上游。

Failure is normal失败是常态

Upstreams time out, rate-limit and fail mid-stream. Retrying must never send a client two half-answers stitched together.上游会超时、限流，也会在流到一半时出错。重试绝不能把两个半截回答拼在一起发给客户端。

02What I built我做了什么

Convert the meaning, not the field names.转换的是含义，不是字段名。

Every request is decoded into URP, a typed internal format, then encoded for whichever upstream serves it. Streams are re-emitted as events in the client’s own dialect.每条请求先解码成带类型的内部格式 URP，再按负责它的上游编码。流式回复按客户端自己的协议重新发出事件。

  • Provider waterfall routing with bounded retries, circuit breakers, passive and active health probes, cooldowns and model affinity.按顺序的服务商瀑布路由：有限重试、熔断、被动和主动健康探测、冷却和模型亲和。
  • 43 transform modules: prompt-cache breakpoints, reasoning mapping, role repair, SSE frame splitting, image compression (JPEG, PNG, WebP, JPEG XL).43 个变换模块：提示缓存断点、推理映射、角色修复、SSE 分帧、图片压缩（JPEG、PNG、WebP、JPEG XL）。
  • Usage is counted per request in integer units and rounded once, so totals never drift.用量按请求以整数单位计量，只舍入一次，合计不会漂移。
  • A React console (keys, request logs with a capture viewer, a Playground, provider health) in 4 UI languages.React 控制台（密钥、带抓包查看器的请求日志、Playground、服务商健康状态），界面有 4 种语言。
  • Public docs: a Next.js static site with 240 MDX pages in 4 locales, including one page per transform.公开文档：Next.js 静态站，4 种语言、240 个 MDX 页面，每个变换都有单独一页。

03How it works怎么做的

One binary, one request path.一个程序，一条请求路径。

  1. STEP 01

    Your app你的应用

    Any SDK: Responses, Chat or Messages.任意 SDK：Responses、Chat 或 Messages。

  2. STEP 02

    Decode解码

    Into URP, a typed internal format.转成带类型的内部格式 URP。

  3. STEP 03

    Transform变换

    Cache hints, reasoning, role repair.缓存提示、推理映射、角色修复。

  4. STEP 04

    Route路由

    Waterfall with retries and breakers.瀑布路由，带重试和熔断。

  5. STEP 05

    Stream back流式返回

    In the client’s own dialect.按客户端自己的协议。

04Interface and details界面与细节

Make the invisible path visible.把看不见的路径摆出来。

  1. Request logs with model, channel and latency (demo data).
    01 / 03

    Every request, accounted for每条请求都有账可查

    Request logs show the model, the channel that actually served it, time to first token and tokens in and out. A capture viewer opens the exact request and response for debugging.请求日志显示模型、实际服务的渠道、首字延迟和输入输出 Token。抓包查看器能打开完整的请求和响应，方便排查。

  2. Playground chat showing a reasoning summary (demo data).
    02 / 03

    A Playground that speaks every dialect能讲所有协议的 Playground

    The Playground sends real requests through the gateway and renders reasoning summaries, tool calls and streamed output, so you can see what a client would see.Playground 通过网关发真实请求，渲染推理摘要、工具调用和流式输出，让你看到客户端会看到的东西。

  3. The public docs site: the Streaming guide.
    03 / 03

    Docs that state the rules把规则写清楚的文档

    The docs explain the contract precisely, for example: once the first byte is sent, the gateway will not switch to another upstream; errors after that arrive as stream events.文档把约定写得很精确，比如：第一个字节发出后，网关不会再换上游；之后的错误以流事件的形式返回。

Video视频

One task, in the logs.一个任务，落在日志里。

The request log as the desktop agent works: each subagent call lands as a row with its model, latency and tokens. Recorded with demo data.桌面智能体干活时的请求日志：每次子智能体调用都变成一行，带模型、延迟和 Token。用演示数据录制。

05Tech used用到的技术

What it’s made of.用什么做的。

Server服务端
  • Rust 2024
  • axum 0.8
  • tokio
  • reqwest + rustls
  • tokio-tungstenite
  • tower-http
Data数据
  • SeaORM
  • sqlx
  • SQLite
  • PostgreSQL
  • rust_decimal
  • Prometheus
Console控制台
  • React 19
  • Vite 7
  • TypeScript
  • Tailwind
  • shadcn/ui
  • SWR
  • i18next
  • Recharts
Docs文档
  • Next.js 16 (static export)
  • Fumadocs
  • MDX
  • KaTeX
Quality质量
  • 1,500+ Rust tests added新增 1500+ Rust 测试
  • ~330 front-end tests约 330 个前端测试
  • Playwright browser scriptsPlaywright 浏览器脚本
  • PostgreSQL migration tests in CICI 中的 PostgreSQL 迁移测试

06What I learned我学到了什么

Lessons I’ll keep.会一直记得的几件事。

  1. Write the spec before the code. Routing and usage accounting each have a written spec; the tests follow the spec, so edge cases are decided once, on paper.先写规格，再写代码。路由和用量统计都有成文的规格；测试照规格写，边界情况只在纸上决定一次。
  2. Round once. Storing counts as integers and rounding at a single, named step removed a whole class of off-by-one totals.只舍入一次。用整数存计数、只在一个明确的步骤舍入，一整类「差一点点」的合计问题就消失了。
  3. Streams are contracts. The hardest bugs were not in parsing but in lifecycle: when a stream may still fail over, and when it may not.流就是约定。最难的问题不在解析，而在生命周期：流什么时候还能切换，什么时候不能。