docs: design postgres admin state module
This commit is contained in:
parent
63bc56be27
commit
08800cb7a5
142
docs/adr/006-postgresql-admin-state.md
Normal file
142
docs/adr/006-postgresql-admin-state.md
Normal file
@ -0,0 +1,142 @@
|
|||||||
|
# ADR-006:PostgreSQL 管理面采用事务深模块
|
||||||
|
|
||||||
|
## 状态
|
||||||
|
|
||||||
|
接受,2026-07-29。
|
||||||
|
|
||||||
|
## 背景
|
||||||
|
|
||||||
|
Admin HTTP Handler 已定义 Upstream 启停、Routing 手工切换、配置重载和状态查询,
|
||||||
|
但权威管理状态、审计与 Outbox 还没有生产存储实现。PostgreSQL 只属于控制面;
|
||||||
|
Proxy 地址、凭据、生命周期、Worker 所有权、逐次提取记录和短期幂等结果都不能
|
||||||
|
进入 PostgreSQL。
|
||||||
|
|
||||||
|
管理写入存在一个不可拆分的不变量:业务状态、Admin 审计和 Outbox 必须在同一
|
||||||
|
PostgreSQL 事务内提交。若把它们暴露成多个 Repository,调用方容易漏写审计、
|
||||||
|
漏发事件或在失败时留下部分状态。
|
||||||
|
|
||||||
|
## 决策
|
||||||
|
|
||||||
|
### 模块与 seam
|
||||||
|
|
||||||
|
新增 `domain/adminstate` 公用契约。调用方按用途依赖窄接口,一个 Adapter 可以
|
||||||
|
同时实现全部接口:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Mutator interface {
|
||||||
|
SetUpstreamEnabled(context.Context, SetUpstreamCommand) (MutationResult, error)
|
||||||
|
SwitchRouting(context.Context, SwitchRoutingCommand) (MutationResult, error)
|
||||||
|
CommitConfig(context.Context, CommitConfigCommand) (MutationResult, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type SnapshotReader interface {
|
||||||
|
Snapshot(context.Context) (Snapshot, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type Outbox interface {
|
||||||
|
Claim(context.Context, ClaimCommand) ([]Event, error)
|
||||||
|
Acknowledge(context.Context, AcknowledgeCommand) error
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
三个 mutation 方法保留现有 Admin 语义,事务、修订号、CAS、审计和 Outbox 都
|
||||||
|
隐藏在模块实现内部。不会暴露 `BeginTx`、SQL executor 或五个可被错误组合的浅
|
||||||
|
Repository。
|
||||||
|
|
||||||
|
`MemoryStore` 是并发安全参考 Adapter;PostgreSQL Adapter 必须运行同一套公用
|
||||||
|
行为契约。Controller Admin 应用层只负责 DTO 映射、配置解析/校验和运行态发布,
|
||||||
|
不自行拼装数据库事务。
|
||||||
|
|
||||||
|
### 全局修订
|
||||||
|
|
||||||
|
每次产生管理状态变化时分配单调递增的全局 `revision`:
|
||||||
|
|
||||||
|
- Upstream 目标状态已满足时返回 `changed=false`,不增加修订。
|
||||||
|
- Routing 的 `expectedCurrent` 不匹配时返回冲突,不写任何记录。
|
||||||
|
- 相同配置版本和校验和再次提交时返回 `changed=false`。
|
||||||
|
- 相同配置版本对应不同校验和时返回冲突。
|
||||||
|
- `MutationResult.version` 是已提交的全局管理修订,不是 YAML 格式版本或 Worker
|
||||||
|
Snapshot 版本。
|
||||||
|
|
||||||
|
每次合法 Admin mutation 都写审计,包括 `changed=false`;只有真实状态变化写
|
||||||
|
Outbox。状态变化、审计和 Outbox 在同一事务提交。
|
||||||
|
|
||||||
|
### 配置修订
|
||||||
|
|
||||||
|
配置提交只保存管理面恢复所需的非敏感事实:配置版本、SHA-256 校验和、来源、
|
||||||
|
Upstream 启用状态和 Routing 候选/当前选择。已解析 Secret、Provider Token、
|
||||||
|
Proxy 凭据和完整运行时对象不进入 PostgreSQL。
|
||||||
|
|
||||||
|
配置重载以一个事务替换管理快照。新 Routing 的当前 Upstream 必须属于其候选集,
|
||||||
|
所有引用的 Upstream 必须存在,名称与列表必须非空且唯一。校验失败发生在事务前,
|
||||||
|
旧修订继续生效。
|
||||||
|
|
||||||
|
### Routing CAS
|
||||||
|
|
||||||
|
`SwitchRouting` 在单条事务中锁定目标 Routing,并同时校验:
|
||||||
|
|
||||||
|
1. Routing 存在且启用;
|
||||||
|
2. `expectedCurrent` 等于权威当前值;
|
||||||
|
3. `target` 属于该 Routing 的候选 Upstream;
|
||||||
|
4. 目标与当前值不同。
|
||||||
|
|
||||||
|
并发使用同一 expected 值时最多一个请求成功。目标等于当前值且 expected 匹配时
|
||||||
|
返回 `changed=false`,仍写审计但不写 Outbox。
|
||||||
|
|
||||||
|
### Outbox 消费
|
||||||
|
|
||||||
|
Outbox 使用有界 claim/ack,而不是无界全表扫描:
|
||||||
|
|
||||||
|
- `Claim` 要求稳定 consumer ID、当前时间、正租期和有界 limit。
|
||||||
|
- 未发布且未被有效租约占用的事件按序 claim。
|
||||||
|
- `Acknowledge` 只允许当前 consumer 在租期内确认自己 claim 的事件。
|
||||||
|
- 发布失败不 ACK,租期过后可由其他 consumer 重试。
|
||||||
|
|
||||||
|
事件 payload 只包含管理资源名、目标状态、修订和必要原因,不包含 Secret、
|
||||||
|
Proxy、Client 或完整配置正文。
|
||||||
|
|
||||||
|
### Schema 边界
|
||||||
|
|
||||||
|
首个迁移只创建:
|
||||||
|
|
||||||
|
```text
|
||||||
|
control_revisions
|
||||||
|
config_revisions
|
||||||
|
upstream_admin_state
|
||||||
|
routing_admin_state
|
||||||
|
admin_audit_log
|
||||||
|
admin_outbox
|
||||||
|
```
|
||||||
|
|
||||||
|
迁移和集成测试必须断言不存在 Proxy 明细、Worker ownership、提取记录或短期
|
||||||
|
幂等表。PostgreSQL 故障只使管理写入失败关闭,不改变 Redis Extract 的可用性。
|
||||||
|
|
||||||
|
## 测试与验收
|
||||||
|
|
||||||
|
MemoryStore 与 PostgreSQL Adapter 运行相同契约,至少覆盖:
|
||||||
|
|
||||||
|
- 配置首次提交、相同重放、校验和冲突和非法引用零写入。
|
||||||
|
- Upstream enable/disable 幂等、修订单调、审计必写、Outbox 仅在变更时写。
|
||||||
|
- Routing CAS、目标校验和 100 个并发请求最多一个成功。
|
||||||
|
- 任一审计/Outbox 写故障导致状态完全回滚。
|
||||||
|
- Outbox 有界 claim、租约到期重试、错误 consumer ACK 拒绝和顺序稳定。
|
||||||
|
- 上下文取消、错误脱敏和 Snapshot 防止调用方修改内部状态。
|
||||||
|
- 真实 PostgreSQL 重复迁移、事务回滚和禁止数据表边界。
|
||||||
|
|
||||||
|
所有测试命令最长 60 秒。100,000 QPS 属于 Gateway 集群目标,不以管理面数据库
|
||||||
|
测试推导吞吐结论。
|
||||||
|
|
||||||
|
## 后果
|
||||||
|
|
||||||
|
收益:调用方无法绕过事务不变量;Memory/PostgreSQL 行为一致;管理数据边界可
|
||||||
|
通过 Schema 自动验证;Outbox 重试有界且可观测。
|
||||||
|
|
||||||
|
代价:PostgreSQL Adapter 内部实现较深;配置提交需要完整管理快照;Outbox
|
||||||
|
dispatcher 需要租约续期或确保单批发布时间小于 claim TTL。
|
||||||
|
|
||||||
|
## 不采用的方案
|
||||||
|
|
||||||
|
- 五个公开 Repository 加公开事务管理器:接口浅,调用方容易产生部分提交。
|
||||||
|
- 将所有命令塞入一个弱类型 `Command`:入口最少,但 Go 调用方需要运行时判断
|
||||||
|
联合字段,错误更晚暴露。
|
||||||
|
- 把 Proxy 或提取事实写入 PostgreSQL:违反短效活动池与数据最小化边界。
|
||||||
@ -47,3 +47,15 @@ Provider 入池、健康状态、Distribution 独占提取、Worker 所有权、
|
|||||||
|
|
||||||
完整决策、键空间、原子操作和测试门禁见
|
完整决策、键空间、原子操作和测试门禁见
|
||||||
[ADR-005](005-redis-activity-pool.md)。
|
[ADR-005](005-redis-activity-pool.md)。
|
||||||
|
|
||||||
|
## ADR-006:PostgreSQL 管理面采用事务深模块
|
||||||
|
|
||||||
|
**状态:** 接受。
|
||||||
|
|
||||||
|
Upstream/Routing/配置修订、Admin 审计和 Outbox 由一个深 Adapter 在同一事务
|
||||||
|
提交。调用方只依赖 mutation、snapshot 和有界 outbox 接口,不接触数据库事务
|
||||||
|
或细粒度 Repository。PostgreSQL Schema 明确排除 Proxy、Worker ownership、
|
||||||
|
逐次提取记录和短期幂等结果。
|
||||||
|
|
||||||
|
完整接口、事务不变量、Schema 与公用契约见
|
||||||
|
[ADR-006](006-postgresql-admin-state.md)。
|
||||||
|
|||||||
149
docs/superpowers/plans/2026-07-29-postgresql-admin-state.md
Normal file
149
docs/superpowers/plans/2026-07-29-postgresql-admin-state.md
Normal file
@ -0,0 +1,149 @@
|
|||||||
|
# PostgreSQL Admin State Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
||||||
|
> superpowers:subagent-driven-development (recommended) or superpowers:executing-plans
|
||||||
|
> to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Build the authoritative PostgreSQL management-state module for config revisions,
|
||||||
|
Upstream/Routing mutations, Admin audit and reliable Outbox without storing Proxy or
|
||||||
|
per-extraction data.
|
||||||
|
|
||||||
|
**Architecture:** `domain/adminstate` defines narrow capability interfaces implemented by a
|
||||||
|
concurrency-safe MemoryStore and one deep PostgreSQL Adapter. Every mutation hides a single
|
||||||
|
transaction that commits state, audit and Outbox together. Both adapters run the same public
|
||||||
|
contract; Controller Admin maps its existing typed commands onto this seam.
|
||||||
|
|
||||||
|
**Tech Stack:** Go 1.26, standard library, pgx/v5, PostgreSQL 18, Docker Compose.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Domain Contracts and Immutable Values
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `internal/domain/adminstate/adminstate.go`
|
||||||
|
- Create: `internal/domain/adminstate/validation_test.go`
|
||||||
|
|
||||||
|
- [ ] Define `Mutator`, `SnapshotReader` and `Outbox` interfaces from ADR-006.
|
||||||
|
- [ ] Define typed config, Upstream, Routing, actor, audit, event and mutation values.
|
||||||
|
- [ ] Define stable errors for invalid input, missing resources, CAS conflict and unavailable
|
||||||
|
storage.
|
||||||
|
- [ ] Validate non-empty bounded identifiers, UTC timestamps, unique lists, config checksum,
|
||||||
|
Routing references, claim limits and claim TTL.
|
||||||
|
- [ ] Clone every slice/map/JSON value at the seam so callers cannot mutate stored state.
|
||||||
|
- [ ] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate` and verify the tests
|
||||||
|
fail before implementation, then pass after implementation.
|
||||||
|
- [ ] Commit with `feat: define admin state transaction contracts`.
|
||||||
|
|
||||||
|
### Task 2: Memory Reference Adapter and Shared Contract
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `internal/domain/adminstate/memory.go`
|
||||||
|
- Create: `internal/domain/adminstate/contracttest/contract.go`
|
||||||
|
- Create: `internal/domain/adminstate/contract_external_test.go`
|
||||||
|
|
||||||
|
- [ ] Write a public contract factory that can create an isolated `Mutator + SnapshotReader +
|
||||||
|
Outbox` implementation.
|
||||||
|
- [ ] Cover config commit/replay/conflict/invalid references and zero-write rollback.
|
||||||
|
- [ ] Cover Upstream idempotency, monotonic revisions, mandatory audit and changed-only events.
|
||||||
|
- [ ] Cover Routing CAS and 100 concurrent switches with at most one success.
|
||||||
|
- [ ] Cover bounded claim, exclusive claim, lease expiry, ACK ownership and stable ordering.
|
||||||
|
- [ ] Cover context cancellation and immutable Snapshot/output values.
|
||||||
|
- [ ] Implement MemoryStore behind the seam with one mutex per atomic management state.
|
||||||
|
- [ ] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate/...`.
|
||||||
|
- [ ] Commit with `feat: add transactional admin state reference store`.
|
||||||
|
|
||||||
|
### Task 3: Schema Migration and Static Data-Boundary Test
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `internal/adapters/postgresadmin/migrations/0001_admin_state.sql`
|
||||||
|
- Create: `internal/adapters/postgresadmin/migrations.go`
|
||||||
|
- Create: `internal/adapters/postgresadmin/migrations_test.go`
|
||||||
|
|
||||||
|
- [ ] Embed ordered migrations and expose one `Migrations() []Migration` read-only accessor.
|
||||||
|
- [ ] Create only the six ADR-006 tables with primary/foreign keys, UTC timestamps, indexes,
|
||||||
|
outbox claim fields and bounded checks.
|
||||||
|
- [ ] Add a parser-backed/static test that asserts required tables/columns are present and
|
||||||
|
forbidden Proxy/extraction/ownership/idempotency tables or columns are absent.
|
||||||
|
- [ ] Test migration IDs are unique, strictly ordered and statements are transactional.
|
||||||
|
- [ ] Run `go test -count=1 -timeout 60s ./internal/adapters/postgresadmin`.
|
||||||
|
- [ ] Commit with `feat: add postgres admin state schema`.
|
||||||
|
|
||||||
|
### Task 4: PostgreSQL Deep Adapter
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `internal/adapters/postgresadmin/adapter.go`
|
||||||
|
- Create: `internal/adapters/postgresadmin/mutate.go`
|
||||||
|
- Create: `internal/adapters/postgresadmin/snapshot.go`
|
||||||
|
- Create: `internal/adapters/postgresadmin/outbox.go`
|
||||||
|
- Create: `internal/adapters/postgresadmin/codec.go`
|
||||||
|
- Modify: `go.mod`
|
||||||
|
- Modify: `go.sum`
|
||||||
|
|
||||||
|
- [ ] Add the approved pinned pgx/v5 dependency without changing unrelated modules.
|
||||||
|
- [ ] Accept a narrow pgx pool interface and options; reject nil pools and invalid namespaces.
|
||||||
|
- [ ] Implement Config/Upstream/Routing mutations with SQL validation, row locks/CAS and one
|
||||||
|
transaction for revision, state, audit and Outbox.
|
||||||
|
- [ ] Map PostgreSQL constraint/CAS/connection errors to domain errors without leaking DSNs,
|
||||||
|
SQL or values.
|
||||||
|
- [ ] Implement immutable Snapshot reads from one repeatable-read transaction.
|
||||||
|
- [ ] Implement bounded Outbox claim using `FOR UPDATE SKIP LOCKED` and guarded ACK.
|
||||||
|
- [ ] Keep SQL, tx retries, codecs and driver types private to the Adapter.
|
||||||
|
- [ ] Run focused unit tests and `go vet ./internal/adapters/postgresadmin/...`.
|
||||||
|
- [ ] Commit with `feat: implement postgres admin state adapter`.
|
||||||
|
|
||||||
|
### Task 5: Real PostgreSQL Contract Fixture
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `deploy/docker-compose.postgres-test.yml`
|
||||||
|
- Create: `scripts/test-postgres.ps1`
|
||||||
|
- Create: `internal/adapters/postgresadmin/testpostgres_test.go`
|
||||||
|
- Create: `internal/adapters/postgresadmin/contract_integration_test.go`
|
||||||
|
- Create: `internal/adapters/postgresadmin/rollback_integration_test.go`
|
||||||
|
|
||||||
|
- [ ] Start an isolated PostgreSQL 18 fixture on a dedicated loopback port and database.
|
||||||
|
- [ ] Apply migrations through the same migration runner used by the production Adapter.
|
||||||
|
- [ ] Run the public adminstate contract against a unique schema per test.
|
||||||
|
- [ ] Inject audit and Outbox constraint failures and prove state/revision rollback.
|
||||||
|
- [ ] Query `information_schema` and prove no Proxy, extraction, ownership or idempotency
|
||||||
|
detail tables/columns exist.
|
||||||
|
- [ ] Clean only the unique test schema; do not drop shared databases or use broad cleanup.
|
||||||
|
- [ ] Run `.\scripts\test-postgres.ps1` with every Go test timeout set to 60 seconds.
|
||||||
|
- [ ] Commit with `test: add postgres admin state contract fixture`.
|
||||||
|
|
||||||
|
### Task 6: Admin Application Integration
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `internal/controller/admin/service.go`
|
||||||
|
- Create: `internal/controller/admin/service_test.go`
|
||||||
|
- Modify: `internal/controller/admin/handler.go`
|
||||||
|
- Modify: `internal/controller/admin/handler_test.go`
|
||||||
|
|
||||||
|
- [ ] Change Admin protection to resolve `httpsecurity.Identity` once and add actor/source IP
|
||||||
|
to mutation commands without exposing credentials.
|
||||||
|
- [ ] Map typed Handler commands to `adminstate.Mutator`; map domain conflict/not-found/
|
||||||
|
invalid/unavailable errors to the existing HTTP contract.
|
||||||
|
- [ ] Build Status from one adminstate Snapshot plus injected activity/worker aggregate readers.
|
||||||
|
- [ ] Reload configuration with existing strict loader/validator, submit a secret-free management
|
||||||
|
snapshot, then atomically publish runtime config only after persistence succeeds.
|
||||||
|
- [ ] Test persistence failure leaves the current runtime config unchanged and Redis Extract is
|
||||||
|
not referenced by the Admin module.
|
||||||
|
- [ ] Run `go test -count=1 -timeout 60s ./internal/controller/admin/...`.
|
||||||
|
- [ ] Commit with `feat: connect admin API to management state`.
|
||||||
|
|
||||||
|
### Task 7: Documentation and Delivery Verification
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `docs/development/implementation-plan.md`
|
||||||
|
- Modify: `docs/requirements/completion-audit.md`
|
||||||
|
- Modify: `docs/testing/test-strategy.md`
|
||||||
|
- Modify: `docs/operations/runbook.md`
|
||||||
|
- Modify: `progress.md`
|
||||||
|
|
||||||
|
- [ ] Mark only verified PostgreSQL capabilities complete and retain command/runtime/load gaps.
|
||||||
|
- [ ] Document migration, backup, Outbox backlog/replay and data-boundary checks.
|
||||||
|
- [ ] Run `.\scripts\verify.ps1`, `.\scripts\test-redis.ps1`,
|
||||||
|
`.\scripts\test-postgres.ps1` and `git diff --check`.
|
||||||
|
- [ ] Audit that Gateway has no PostgreSQL/Redis dependency and Distribution has no PostgreSQL
|
||||||
|
dependency.
|
||||||
|
- [ ] Confirm the user-owned deletion of `proxy-pool-docs-v1.0.zip` is not staged.
|
||||||
|
- [ ] Commit with `docs: record postgres admin state delivery` and push the feature branch.
|
||||||
Loading…
Reference in New Issue
Block a user