docs: design postgres admin state module

This commit is contained in:
youfak 2026-07-29 18:17:26 +08:00
parent 63bc56be27
commit 08800cb7a5
3 changed files with 303 additions and 0 deletions

View File

@ -0,0 +1,142 @@
# ADR-006PostgreSQL 管理面采用事务深模块
## 状态
接受2026-07-29。
## 背景
Admin HTTP Handler 已定义 Upstream 启停、Routing 手工切换、配置重载和状态查询,
但权威管理状态、审计与 Outbox 还没有生产存储实现。PostgreSQL 只属于控制面;
Proxy 地址、凭据、生命周期、Worker 所有权、逐次提取记录和短期幂等结果都不能
进入 PostgreSQL。
管理写入存在一个不可拆分的不变量业务状态、Admin 审计和 Outbox 必须在同一
PostgreSQL 事务内提交。若把它们暴露成多个 Repository调用方容易漏写审计、
漏发事件或在失败时留下部分状态。
## 决策
### 模块与 seam
新增 `domain/adminstate` 公用契约。调用方按用途依赖窄接口,一个 Adapter 可以
同时实现全部接口:
```go
type Mutator interface {
SetUpstreamEnabled(context.Context, SetUpstreamCommand) (MutationResult, error)
SwitchRouting(context.Context, SwitchRoutingCommand) (MutationResult, error)
CommitConfig(context.Context, CommitConfigCommand) (MutationResult, error)
}
type SnapshotReader interface {
Snapshot(context.Context) (Snapshot, error)
}
type Outbox interface {
Claim(context.Context, ClaimCommand) ([]Event, error)
Acknowledge(context.Context, AcknowledgeCommand) error
}
```
三个 mutation 方法保留现有 Admin 语义事务、修订号、CAS、审计和 Outbox 都
隐藏在模块实现内部。不会暴露 `BeginTx`、SQL executor 或五个可被错误组合的浅
Repository。
`MemoryStore` 是并发安全参考 AdapterPostgreSQL Adapter 必须运行同一套公用
行为契约。Controller Admin 应用层只负责 DTO 映射、配置解析/校验和运行态发布,
不自行拼装数据库事务。
### 全局修订
每次产生管理状态变化时分配单调递增的全局 `revision`
- Upstream 目标状态已满足时返回 `changed=false`,不增加修订。
- Routing 的 `expectedCurrent` 不匹配时返回冲突,不写任何记录。
- 相同配置版本和校验和再次提交时返回 `changed=false`
- 相同配置版本对应不同校验和时返回冲突。
- `MutationResult.version` 是已提交的全局管理修订,不是 YAML 格式版本或 Worker
Snapshot 版本。
每次合法 Admin mutation 都写审计,包括 `changed=false`;只有真实状态变化写
Outbox。状态变化、审计和 Outbox 在同一事务提交。
### 配置修订
配置提交只保存管理面恢复所需的非敏感事实配置版本、SHA-256 校验和、来源、
Upstream 启用状态和 Routing 候选/当前选择。已解析 Secret、Provider Token、
Proxy 凭据和完整运行时对象不进入 PostgreSQL。
配置重载以一个事务替换管理快照。新 Routing 的当前 Upstream 必须属于其候选集,
所有引用的 Upstream 必须存在,名称与列表必须非空且唯一。校验失败发生在事务前,
旧修订继续生效。
### Routing CAS
`SwitchRouting` 在单条事务中锁定目标 Routing并同时校验
1. Routing 存在且启用;
2. `expectedCurrent` 等于权威当前值;
3. `target` 属于该 Routing 的候选 Upstream
4. 目标与当前值不同。
并发使用同一 expected 值时最多一个请求成功。目标等于当前值且 expected 匹配时
返回 `changed=false`,仍写审计但不写 Outbox。
### Outbox 消费
Outbox 使用有界 claim/ack而不是无界全表扫描
- `Claim` 要求稳定 consumer ID、当前时间、正租期和有界 limit。
- 未发布且未被有效租约占用的事件按序 claim。
- `Acknowledge` 只允许当前 consumer 在租期内确认自己 claim 的事件。
- 发布失败不 ACK租期过后可由其他 consumer 重试。
事件 payload 只包含管理资源名、目标状态、修订和必要原因,不包含 Secret、
Proxy、Client 或完整配置正文。
### Schema 边界
首个迁移只创建:
```text
control_revisions
config_revisions
upstream_admin_state
routing_admin_state
admin_audit_log
admin_outbox
```
迁移和集成测试必须断言不存在 Proxy 明细、Worker ownership、提取记录或短期
幂等表。PostgreSQL 故障只使管理写入失败关闭,不改变 Redis Extract 的可用性。
## 测试与验收
MemoryStore 与 PostgreSQL Adapter 运行相同契约,至少覆盖:
- 配置首次提交、相同重放、校验和冲突和非法引用零写入。
- Upstream enable/disable 幂等、修订单调、审计必写、Outbox 仅在变更时写。
- Routing CAS、目标校验和 100 个并发请求最多一个成功。
- 任一审计/Outbox 写故障导致状态完全回滚。
- Outbox 有界 claim、租约到期重试、错误 consumer ACK 拒绝和顺序稳定。
- 上下文取消、错误脱敏和 Snapshot 防止调用方修改内部状态。
- 真实 PostgreSQL 重复迁移、事务回滚和禁止数据表边界。
所有测试命令最长 60 秒。100,000 QPS 属于 Gateway 集群目标,不以管理面数据库
测试推导吞吐结论。
## 后果
收益调用方无法绕过事务不变量Memory/PostgreSQL 行为一致;管理数据边界可
通过 Schema 自动验证Outbox 重试有界且可观测。
代价PostgreSQL Adapter 内部实现较深配置提交需要完整管理快照Outbox
dispatcher 需要租约续期或确保单批发布时间小于 claim TTL。
## 不采用的方案
- 五个公开 Repository 加公开事务管理器:接口浅,调用方容易产生部分提交。
- 将所有命令塞入一个弱类型 `Command`:入口最少,但 Go 调用方需要运行时判断
联合字段,错误更晚暴露。
- 把 Proxy 或提取事实写入 PostgreSQL违反短效活动池与数据最小化边界。

View File

@ -47,3 +47,15 @@ Provider 入池、健康状态、Distribution 独占提取、Worker 所有权、
完整决策、键空间、原子操作和测试门禁见 完整决策、键空间、原子操作和测试门禁见
[ADR-005](005-redis-activity-pool.md)。 [ADR-005](005-redis-activity-pool.md)。
## ADR-006PostgreSQL 管理面采用事务深模块
**状态:** 接受。
Upstream/Routing/配置修订、Admin 审计和 Outbox 由一个深 Adapter 在同一事务
提交。调用方只依赖 mutation、snapshot 和有界 outbox 接口,不接触数据库事务
或细粒度 Repository。PostgreSQL Schema 明确排除 Proxy、Worker ownership、
逐次提取记录和短期幂等结果。
完整接口、事务不变量、Schema 与公用契约见
[ADR-006](006-postgresql-admin-state.md)。

View File

@ -0,0 +1,149 @@
# PostgreSQL Admin State Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use
> superpowers:subagent-driven-development (recommended) or superpowers:executing-plans
> to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the authoritative PostgreSQL management-state module for config revisions,
Upstream/Routing mutations, Admin audit and reliable Outbox without storing Proxy or
per-extraction data.
**Architecture:** `domain/adminstate` defines narrow capability interfaces implemented by a
concurrency-safe MemoryStore and one deep PostgreSQL Adapter. Every mutation hides a single
transaction that commits state, audit and Outbox together. Both adapters run the same public
contract; Controller Admin maps its existing typed commands onto this seam.
**Tech Stack:** Go 1.26, standard library, pgx/v5, PostgreSQL 18, Docker Compose.
---
### Task 1: Domain Contracts and Immutable Values
**Files:**
- Create: `internal/domain/adminstate/adminstate.go`
- Create: `internal/domain/adminstate/validation_test.go`
- [ ] Define `Mutator`, `SnapshotReader` and `Outbox` interfaces from ADR-006.
- [ ] Define typed config, Upstream, Routing, actor, audit, event and mutation values.
- [ ] Define stable errors for invalid input, missing resources, CAS conflict and unavailable
storage.
- [ ] Validate non-empty bounded identifiers, UTC timestamps, unique lists, config checksum,
Routing references, claim limits and claim TTL.
- [ ] Clone every slice/map/JSON value at the seam so callers cannot mutate stored state.
- [ ] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate` and verify the tests
fail before implementation, then pass after implementation.
- [ ] Commit with `feat: define admin state transaction contracts`.
### Task 2: Memory Reference Adapter and Shared Contract
**Files:**
- Create: `internal/domain/adminstate/memory.go`
- Create: `internal/domain/adminstate/contracttest/contract.go`
- Create: `internal/domain/adminstate/contract_external_test.go`
- [ ] Write a public contract factory that can create an isolated `Mutator + SnapshotReader +
Outbox` implementation.
- [ ] Cover config commit/replay/conflict/invalid references and zero-write rollback.
- [ ] Cover Upstream idempotency, monotonic revisions, mandatory audit and changed-only events.
- [ ] Cover Routing CAS and 100 concurrent switches with at most one success.
- [ ] Cover bounded claim, exclusive claim, lease expiry, ACK ownership and stable ordering.
- [ ] Cover context cancellation and immutable Snapshot/output values.
- [ ] Implement MemoryStore behind the seam with one mutex per atomic management state.
- [ ] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate/...`.
- [ ] Commit with `feat: add transactional admin state reference store`.
### Task 3: Schema Migration and Static Data-Boundary Test
**Files:**
- Create: `internal/adapters/postgresadmin/migrations/0001_admin_state.sql`
- Create: `internal/adapters/postgresadmin/migrations.go`
- Create: `internal/adapters/postgresadmin/migrations_test.go`
- [ ] Embed ordered migrations and expose one `Migrations() []Migration` read-only accessor.
- [ ] Create only the six ADR-006 tables with primary/foreign keys, UTC timestamps, indexes,
outbox claim fields and bounded checks.
- [ ] Add a parser-backed/static test that asserts required tables/columns are present and
forbidden Proxy/extraction/ownership/idempotency tables or columns are absent.
- [ ] Test migration IDs are unique, strictly ordered and statements are transactional.
- [ ] Run `go test -count=1 -timeout 60s ./internal/adapters/postgresadmin`.
- [ ] Commit with `feat: add postgres admin state schema`.
### Task 4: PostgreSQL Deep Adapter
**Files:**
- Create: `internal/adapters/postgresadmin/adapter.go`
- Create: `internal/adapters/postgresadmin/mutate.go`
- Create: `internal/adapters/postgresadmin/snapshot.go`
- Create: `internal/adapters/postgresadmin/outbox.go`
- Create: `internal/adapters/postgresadmin/codec.go`
- Modify: `go.mod`
- Modify: `go.sum`
- [ ] Add the approved pinned pgx/v5 dependency without changing unrelated modules.
- [ ] Accept a narrow pgx pool interface and options; reject nil pools and invalid namespaces.
- [ ] Implement Config/Upstream/Routing mutations with SQL validation, row locks/CAS and one
transaction for revision, state, audit and Outbox.
- [ ] Map PostgreSQL constraint/CAS/connection errors to domain errors without leaking DSNs,
SQL or values.
- [ ] Implement immutable Snapshot reads from one repeatable-read transaction.
- [ ] Implement bounded Outbox claim using `FOR UPDATE SKIP LOCKED` and guarded ACK.
- [ ] Keep SQL, tx retries, codecs and driver types private to the Adapter.
- [ ] Run focused unit tests and `go vet ./internal/adapters/postgresadmin/...`.
- [ ] Commit with `feat: implement postgres admin state adapter`.
### Task 5: Real PostgreSQL Contract Fixture
**Files:**
- Create: `deploy/docker-compose.postgres-test.yml`
- Create: `scripts/test-postgres.ps1`
- Create: `internal/adapters/postgresadmin/testpostgres_test.go`
- Create: `internal/adapters/postgresadmin/contract_integration_test.go`
- Create: `internal/adapters/postgresadmin/rollback_integration_test.go`
- [ ] Start an isolated PostgreSQL 18 fixture on a dedicated loopback port and database.
- [ ] Apply migrations through the same migration runner used by the production Adapter.
- [ ] Run the public adminstate contract against a unique schema per test.
- [ ] Inject audit and Outbox constraint failures and prove state/revision rollback.
- [ ] Query `information_schema` and prove no Proxy, extraction, ownership or idempotency
detail tables/columns exist.
- [ ] Clean only the unique test schema; do not drop shared databases or use broad cleanup.
- [ ] Run `.\scripts\test-postgres.ps1` with every Go test timeout set to 60 seconds.
- [ ] Commit with `test: add postgres admin state contract fixture`.
### Task 6: Admin Application Integration
**Files:**
- Create: `internal/controller/admin/service.go`
- Create: `internal/controller/admin/service_test.go`
- Modify: `internal/controller/admin/handler.go`
- Modify: `internal/controller/admin/handler_test.go`
- [ ] Change Admin protection to resolve `httpsecurity.Identity` once and add actor/source IP
to mutation commands without exposing credentials.
- [ ] Map typed Handler commands to `adminstate.Mutator`; map domain conflict/not-found/
invalid/unavailable errors to the existing HTTP contract.
- [ ] Build Status from one adminstate Snapshot plus injected activity/worker aggregate readers.
- [ ] Reload configuration with existing strict loader/validator, submit a secret-free management
snapshot, then atomically publish runtime config only after persistence succeeds.
- [ ] Test persistence failure leaves the current runtime config unchanged and Redis Extract is
not referenced by the Admin module.
- [ ] Run `go test -count=1 -timeout 60s ./internal/controller/admin/...`.
- [ ] Commit with `feat: connect admin API to management state`.
### Task 7: Documentation and Delivery Verification
**Files:**
- Modify: `docs/development/implementation-plan.md`
- Modify: `docs/requirements/completion-audit.md`
- Modify: `docs/testing/test-strategy.md`
- Modify: `docs/operations/runbook.md`
- Modify: `progress.md`
- [ ] Mark only verified PostgreSQL capabilities complete and retain command/runtime/load gaps.
- [ ] Document migration, backup, Outbox backlog/replay and data-boundary checks.
- [ ] Run `.\scripts\verify.ps1`, `.\scripts\test-redis.ps1`,
`.\scripts\test-postgres.ps1` and `git diff --check`.
- [ ] Audit that Gateway has no PostgreSQL/Redis dependency and Distribution has no PostgreSQL
dependency.
- [ ] Confirm the user-owned deletion of `proxy-pool-docs-v1.0.zip` is not staged.
- [ ] Commit with `docs: record postgres admin state delivery` and push the feature branch.