diff --git a/docs/adr/006-postgresql-admin-state.md b/docs/adr/006-postgresql-admin-state.md new file mode 100644 index 0000000..d482044 --- /dev/null +++ b/docs/adr/006-postgresql-admin-state.md @@ -0,0 +1,142 @@ +# ADR-006:PostgreSQL 管理面采用事务深模块 + +## 状态 + +接受,2026-07-29。 + +## 背景 + +Admin HTTP Handler 已定义 Upstream 启停、Routing 手工切换、配置重载和状态查询, +但权威管理状态、审计与 Outbox 还没有生产存储实现。PostgreSQL 只属于控制面; +Proxy 地址、凭据、生命周期、Worker 所有权、逐次提取记录和短期幂等结果都不能 +进入 PostgreSQL。 + +管理写入存在一个不可拆分的不变量:业务状态、Admin 审计和 Outbox 必须在同一 +PostgreSQL 事务内提交。若把它们暴露成多个 Repository,调用方容易漏写审计、 +漏发事件或在失败时留下部分状态。 + +## 决策 + +### 模块与 seam + +新增 `domain/adminstate` 公用契约。调用方按用途依赖窄接口,一个 Adapter 可以 +同时实现全部接口: + +```go +type Mutator interface { + SetUpstreamEnabled(context.Context, SetUpstreamCommand) (MutationResult, error) + SwitchRouting(context.Context, SwitchRoutingCommand) (MutationResult, error) + CommitConfig(context.Context, CommitConfigCommand) (MutationResult, error) +} + +type SnapshotReader interface { + Snapshot(context.Context) (Snapshot, error) +} + +type Outbox interface { + Claim(context.Context, ClaimCommand) ([]Event, error) + Acknowledge(context.Context, AcknowledgeCommand) error +} +``` + +三个 mutation 方法保留现有 Admin 语义,事务、修订号、CAS、审计和 Outbox 都 +隐藏在模块实现内部。不会暴露 `BeginTx`、SQL executor 或五个可被错误组合的浅 +Repository。 + +`MemoryStore` 是并发安全参考 Adapter;PostgreSQL Adapter 必须运行同一套公用 +行为契约。Controller Admin 应用层只负责 DTO 映射、配置解析/校验和运行态发布, +不自行拼装数据库事务。 + +### 全局修订 + +每次产生管理状态变化时分配单调递增的全局 `revision`: + +- Upstream 目标状态已满足时返回 `changed=false`,不增加修订。 +- Routing 的 `expectedCurrent` 不匹配时返回冲突,不写任何记录。 +- 相同配置版本和校验和再次提交时返回 `changed=false`。 +- 相同配置版本对应不同校验和时返回冲突。 +- `MutationResult.version` 是已提交的全局管理修订,不是 YAML 格式版本或 Worker + Snapshot 版本。 + +每次合法 Admin mutation 都写审计,包括 `changed=false`;只有真实状态变化写 +Outbox。状态变化、审计和 Outbox 在同一事务提交。 + +### 配置修订 + +配置提交只保存管理面恢复所需的非敏感事实:配置版本、SHA-256 校验和、来源、 +Upstream 启用状态和 Routing 候选/当前选择。已解析 Secret、Provider Token、 +Proxy 凭据和完整运行时对象不进入 PostgreSQL。 + +配置重载以一个事务替换管理快照。新 Routing 的当前 Upstream 必须属于其候选集, +所有引用的 Upstream 必须存在,名称与列表必须非空且唯一。校验失败发生在事务前, +旧修订继续生效。 + +### Routing CAS + +`SwitchRouting` 在单条事务中锁定目标 Routing,并同时校验: + +1. Routing 存在且启用; +2. `expectedCurrent` 等于权威当前值; +3. `target` 属于该 Routing 的候选 Upstream; +4. 目标与当前值不同。 + +并发使用同一 expected 值时最多一个请求成功。目标等于当前值且 expected 匹配时 +返回 `changed=false`,仍写审计但不写 Outbox。 + +### Outbox 消费 + +Outbox 使用有界 claim/ack,而不是无界全表扫描: + +- `Claim` 要求稳定 consumer ID、当前时间、正租期和有界 limit。 +- 未发布且未被有效租约占用的事件按序 claim。 +- `Acknowledge` 只允许当前 consumer 在租期内确认自己 claim 的事件。 +- 发布失败不 ACK,租期过后可由其他 consumer 重试。 + +事件 payload 只包含管理资源名、目标状态、修订和必要原因,不包含 Secret、 +Proxy、Client 或完整配置正文。 + +### Schema 边界 + +首个迁移只创建: + +```text +control_revisions +config_revisions +upstream_admin_state +routing_admin_state +admin_audit_log +admin_outbox +``` + +迁移和集成测试必须断言不存在 Proxy 明细、Worker ownership、提取记录或短期 +幂等表。PostgreSQL 故障只使管理写入失败关闭,不改变 Redis Extract 的可用性。 + +## 测试与验收 + +MemoryStore 与 PostgreSQL Adapter 运行相同契约,至少覆盖: + +- 配置首次提交、相同重放、校验和冲突和非法引用零写入。 +- Upstream enable/disable 幂等、修订单调、审计必写、Outbox 仅在变更时写。 +- Routing CAS、目标校验和 100 个并发请求最多一个成功。 +- 任一审计/Outbox 写故障导致状态完全回滚。 +- Outbox 有界 claim、租约到期重试、错误 consumer ACK 拒绝和顺序稳定。 +- 上下文取消、错误脱敏和 Snapshot 防止调用方修改内部状态。 +- 真实 PostgreSQL 重复迁移、事务回滚和禁止数据表边界。 + +所有测试命令最长 60 秒。100,000 QPS 属于 Gateway 集群目标,不以管理面数据库 +测试推导吞吐结论。 + +## 后果 + +收益:调用方无法绕过事务不变量;Memory/PostgreSQL 行为一致;管理数据边界可 +通过 Schema 自动验证;Outbox 重试有界且可观测。 + +代价:PostgreSQL Adapter 内部实现较深;配置提交需要完整管理快照;Outbox +dispatcher 需要租约续期或确保单批发布时间小于 claim TTL。 + +## 不采用的方案 + +- 五个公开 Repository 加公开事务管理器:接口浅,调用方容易产生部分提交。 +- 将所有命令塞入一个弱类型 `Command`:入口最少,但 Go 调用方需要运行时判断 + 联合字段,错误更晚暴露。 +- 把 Proxy 或提取事实写入 PostgreSQL:违反短效活动池与数据最小化边界。 diff --git a/docs/adr/README.md b/docs/adr/README.md index 2173889..243966e 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -47,3 +47,15 @@ Provider 入池、健康状态、Distribution 独占提取、Worker 所有权、 完整决策、键空间、原子操作和测试门禁见 [ADR-005](005-redis-activity-pool.md)。 + +## ADR-006:PostgreSQL 管理面采用事务深模块 + +**状态:** 接受。 + +Upstream/Routing/配置修订、Admin 审计和 Outbox 由一个深 Adapter 在同一事务 +提交。调用方只依赖 mutation、snapshot 和有界 outbox 接口,不接触数据库事务 +或细粒度 Repository。PostgreSQL Schema 明确排除 Proxy、Worker ownership、 +逐次提取记录和短期幂等结果。 + +完整接口、事务不变量、Schema 与公用契约见 +[ADR-006](006-postgresql-admin-state.md)。 diff --git a/docs/superpowers/plans/2026-07-29-postgresql-admin-state.md b/docs/superpowers/plans/2026-07-29-postgresql-admin-state.md new file mode 100644 index 0000000..0d42384 --- /dev/null +++ b/docs/superpowers/plans/2026-07-29-postgresql-admin-state.md @@ -0,0 +1,149 @@ +# PostgreSQL Admin State Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use +> superpowers:subagent-driven-development (recommended) or superpowers:executing-plans +> to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build the authoritative PostgreSQL management-state module for config revisions, +Upstream/Routing mutations, Admin audit and reliable Outbox without storing Proxy or +per-extraction data. + +**Architecture:** `domain/adminstate` defines narrow capability interfaces implemented by a +concurrency-safe MemoryStore and one deep PostgreSQL Adapter. Every mutation hides a single +transaction that commits state, audit and Outbox together. Both adapters run the same public +contract; Controller Admin maps its existing typed commands onto this seam. + +**Tech Stack:** Go 1.26, standard library, pgx/v5, PostgreSQL 18, Docker Compose. + +--- + +### Task 1: Domain Contracts and Immutable Values + +**Files:** +- Create: `internal/domain/adminstate/adminstate.go` +- Create: `internal/domain/adminstate/validation_test.go` + +- [ ] Define `Mutator`, `SnapshotReader` and `Outbox` interfaces from ADR-006. +- [ ] Define typed config, Upstream, Routing, actor, audit, event and mutation values. +- [ ] Define stable errors for invalid input, missing resources, CAS conflict and unavailable + storage. +- [ ] Validate non-empty bounded identifiers, UTC timestamps, unique lists, config checksum, + Routing references, claim limits and claim TTL. +- [ ] Clone every slice/map/JSON value at the seam so callers cannot mutate stored state. +- [ ] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate` and verify the tests + fail before implementation, then pass after implementation. +- [ ] Commit with `feat: define admin state transaction contracts`. + +### Task 2: Memory Reference Adapter and Shared Contract + +**Files:** +- Create: `internal/domain/adminstate/memory.go` +- Create: `internal/domain/adminstate/contracttest/contract.go` +- Create: `internal/domain/adminstate/contract_external_test.go` + +- [ ] Write a public contract factory that can create an isolated `Mutator + SnapshotReader + + Outbox` implementation. +- [ ] Cover config commit/replay/conflict/invalid references and zero-write rollback. +- [ ] Cover Upstream idempotency, monotonic revisions, mandatory audit and changed-only events. +- [ ] Cover Routing CAS and 100 concurrent switches with at most one success. +- [ ] Cover bounded claim, exclusive claim, lease expiry, ACK ownership and stable ordering. +- [ ] Cover context cancellation and immutable Snapshot/output values. +- [ ] Implement MemoryStore behind the seam with one mutex per atomic management state. +- [ ] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate/...`. +- [ ] Commit with `feat: add transactional admin state reference store`. + +### Task 3: Schema Migration and Static Data-Boundary Test + +**Files:** +- Create: `internal/adapters/postgresadmin/migrations/0001_admin_state.sql` +- Create: `internal/adapters/postgresadmin/migrations.go` +- Create: `internal/adapters/postgresadmin/migrations_test.go` + +- [ ] Embed ordered migrations and expose one `Migrations() []Migration` read-only accessor. +- [ ] Create only the six ADR-006 tables with primary/foreign keys, UTC timestamps, indexes, + outbox claim fields and bounded checks. +- [ ] Add a parser-backed/static test that asserts required tables/columns are present and + forbidden Proxy/extraction/ownership/idempotency tables or columns are absent. +- [ ] Test migration IDs are unique, strictly ordered and statements are transactional. +- [ ] Run `go test -count=1 -timeout 60s ./internal/adapters/postgresadmin`. +- [ ] Commit with `feat: add postgres admin state schema`. + +### Task 4: PostgreSQL Deep Adapter + +**Files:** +- Create: `internal/adapters/postgresadmin/adapter.go` +- Create: `internal/adapters/postgresadmin/mutate.go` +- Create: `internal/adapters/postgresadmin/snapshot.go` +- Create: `internal/adapters/postgresadmin/outbox.go` +- Create: `internal/adapters/postgresadmin/codec.go` +- Modify: `go.mod` +- Modify: `go.sum` + +- [ ] Add the approved pinned pgx/v5 dependency without changing unrelated modules. +- [ ] Accept a narrow pgx pool interface and options; reject nil pools and invalid namespaces. +- [ ] Implement Config/Upstream/Routing mutations with SQL validation, row locks/CAS and one + transaction for revision, state, audit and Outbox. +- [ ] Map PostgreSQL constraint/CAS/connection errors to domain errors without leaking DSNs, + SQL or values. +- [ ] Implement immutable Snapshot reads from one repeatable-read transaction. +- [ ] Implement bounded Outbox claim using `FOR UPDATE SKIP LOCKED` and guarded ACK. +- [ ] Keep SQL, tx retries, codecs and driver types private to the Adapter. +- [ ] Run focused unit tests and `go vet ./internal/adapters/postgresadmin/...`. +- [ ] Commit with `feat: implement postgres admin state adapter`. + +### Task 5: Real PostgreSQL Contract Fixture + +**Files:** +- Create: `deploy/docker-compose.postgres-test.yml` +- Create: `scripts/test-postgres.ps1` +- Create: `internal/adapters/postgresadmin/testpostgres_test.go` +- Create: `internal/adapters/postgresadmin/contract_integration_test.go` +- Create: `internal/adapters/postgresadmin/rollback_integration_test.go` + +- [ ] Start an isolated PostgreSQL 18 fixture on a dedicated loopback port and database. +- [ ] Apply migrations through the same migration runner used by the production Adapter. +- [ ] Run the public adminstate contract against a unique schema per test. +- [ ] Inject audit and Outbox constraint failures and prove state/revision rollback. +- [ ] Query `information_schema` and prove no Proxy, extraction, ownership or idempotency + detail tables/columns exist. +- [ ] Clean only the unique test schema; do not drop shared databases or use broad cleanup. +- [ ] Run `.\scripts\test-postgres.ps1` with every Go test timeout set to 60 seconds. +- [ ] Commit with `test: add postgres admin state contract fixture`. + +### Task 6: Admin Application Integration + +**Files:** +- Create: `internal/controller/admin/service.go` +- Create: `internal/controller/admin/service_test.go` +- Modify: `internal/controller/admin/handler.go` +- Modify: `internal/controller/admin/handler_test.go` + +- [ ] Change Admin protection to resolve `httpsecurity.Identity` once and add actor/source IP + to mutation commands without exposing credentials. +- [ ] Map typed Handler commands to `adminstate.Mutator`; map domain conflict/not-found/ + invalid/unavailable errors to the existing HTTP contract. +- [ ] Build Status from one adminstate Snapshot plus injected activity/worker aggregate readers. +- [ ] Reload configuration with existing strict loader/validator, submit a secret-free management + snapshot, then atomically publish runtime config only after persistence succeeds. +- [ ] Test persistence failure leaves the current runtime config unchanged and Redis Extract is + not referenced by the Admin module. +- [ ] Run `go test -count=1 -timeout 60s ./internal/controller/admin/...`. +- [ ] Commit with `feat: connect admin API to management state`. + +### Task 7: Documentation and Delivery Verification + +**Files:** +- Modify: `docs/development/implementation-plan.md` +- Modify: `docs/requirements/completion-audit.md` +- Modify: `docs/testing/test-strategy.md` +- Modify: `docs/operations/runbook.md` +- Modify: `progress.md` + +- [ ] Mark only verified PostgreSQL capabilities complete and retain command/runtime/load gaps. +- [ ] Document migration, backup, Outbox backlog/replay and data-boundary checks. +- [ ] Run `.\scripts\verify.ps1`, `.\scripts\test-redis.ps1`, + `.\scripts\test-postgres.ps1` and `git diff --check`. +- [ ] Audit that Gateway has no PostgreSQL/Redis dependency and Distribution has no PostgreSQL + dependency. +- [ ] Confirm the user-owned deletion of `proxy-pool-docs-v1.0.zip` is not staged. +- [ ] Commit with `docs: record postgres admin state delivery` and push the feature branch.