02Rust library · open sourceRust 库 · 开源

llm-provider-kit

A Rust crate that speaks three LLM wire protocols and turns their streams into one event type, with OAuth and device-code logins.一个 Rust 库：讲三种大模型通信协议，把它们的流统一成一种事件，并支持 OAuth 和设备码登录。

My role我的角色
Co-author of a shared Rust library. My part: the transport, retries, thinking parameters and client headers.共同作者，这是一个共享的 Rust 库。我负责的部分：传输层、重试、思考参数和客户端请求头。
Status状态
Public · Apache-2.0公开 · Apache-2.0
pub enum WireEvent {
    Delta(String),            // visible text
    ReasoningDelta(String),   // reasoning summary
    ResponseItem(Value),      // canonical output item
    Usage(Usage),             // tokens in / out
}
33wire protocols, one event type种协议，一种事件
9595unit tests个单元测试
00async runtime dependencies个异步运行时依赖

01The problem问题

Every caller was writing its own HTTP client.每个调用方都在自己写 HTTP 客户端。

Three streams, three shapes三种流，三种结构

OpenAI Responses, Anthropic Messages and Chat Completions each stream server-sent events with different event names, item types and usage fields.OpenAI Responses、Anthropic Messages 和 Chat Completions 都用 SSE 流式输出，但事件名、条目类型和用量字段都不一样。

Logins are protocols too登录也是协议

Providers sign in with OAuth authorization codes, device codes or plain API keys, each with its own refresh rules.服务商的登录方式有 OAuth 授权码、设备码和普通 API Key，每种都有自己的刷新规则。

Small binaries二进制要小

The CLI engine should not pull in an async runtime just to read one stream at a time.CLI 引擎一次只读一条流，不该为此引入一整个异步运行时。

02What I built我做了什么

One crate that owns the wire.一个专门负责通信协议的库。

Callers get a Client, pick a Protocol, and read unified WireEvents. Responses-style output items are the canonical format they store, whatever the provider spoke.调用方拿到一个 Client，选一种 Protocol，读出统一的 WireEvent。无论服务商讲的是哪种协议，存下来的都是 Responses 风格的输出条目。

  • Parsers for the three SSE dialects, each with its own module and tests.三种 SSE 方言的解析器，各有独立模块和测试。
  • A blocking transport (ureq + threads) that retries transport failures, 429s and 5xx, and honours Retry-After up to a cap.阻塞式传输（ureq + 线程）：对传输失败、429 和 5xx 重试，并在上限内遵守 Retry-After。
  • Login flows: OAuth authorization code with PKCE, device code, and API key, plus token refresh. Credentials are returned; storing them is the caller’s job.登录流程：带 PKCE 的 OAuth 授权码、设备码和 API Key，以及令牌刷新。凭据只返回给调用方，存储由调用方负责。
  • A provider catalog parser (models, dialects, routes) and provider templates.服务商目录解析（模型、协议方言、路由）和服务商模板。
  • Tool execution, approvals and the agent loop deliberately stay outside the crate.工具执行、审批和智能体循环被刻意留在库外面。

03How it works怎么做的

From bytes to one event type.从字节到同一种事件。

  1. STEP 01

    Caller调用方

    Builds a request with a Protocol and a base URL.用 Protocol 和 base URL 构造请求。

  2. STEP 02

    Transport传输

    Sends over blocking HTTP; retries with Retry-After.通过阻塞式 HTTP 发送；按 Retry-After 重试。

  3. STEP 03

    Dialect parser方言解析

    Responses, Messages or Chat SSE frames.解析 Responses、Messages 或 Chat 的 SSE 帧。

  4. STEP 04

    WireEventWireEvent

    Text, reasoning, output items and usage, unified.文字、推理、输出条目和用量，统一成一种。

05Tech used用到的技术

What it’s made of.用什么做的。

Language语言
  • Rust 2021
  • ~7,000 lines约 7000 行
Dependencies依赖
  • ureq
  • serde_json
  • sha2
  • base64
  • getrandom
Quality质量

06What I learned我学到了什么

Lessons I’ll keep.会一直记得的几件事。

  1. Pick one canonical format. Storing every provider’s output as the same item type removed a whole layer of conversions from the callers.选一种标准格式。把所有服务商的输出存成同一种条目，调用方就少了一整层转换。
  2. Boundaries are features. Saying “credentials are returned, not stored” and “no agent loop here” kept the crate small and easy to test.边界本身就是功能。「凭据只返回、不存储」和「这里没有智能体循环」让这个库保持小而好测。
  3. Blocking is fine. One stream at a time over threads was simpler than async and fast enough for an interactive tool.阻塞也没问题。用线程一次处理一条流，比异步简单，对交互式工具来说也足够快。