diff --git a/docs/adr/006-postgresql-admin-state.md b/docs/adr/006-postgresql-admin-state.md index d482044..6520953 100644 --- a/docs/adr/006-postgresql-admin-state.md +++ b/docs/adr/006-postgresql-admin-state.md @@ -33,6 +33,10 @@ type SnapshotReader interface { Snapshot(context.Context) (Snapshot, error) } +type AuditReader interface { + ReadAudit(context.Context, AuditQuery) ([]AuditRecord, error) +} + type Outbox interface { Claim(context.Context, ClaimCommand) ([]Event, error) Acknowledge(context.Context, AcknowledgeCommand) error diff --git a/docs/adr/README.md b/docs/adr/README.md index 243966e..e3ff36c 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -38,7 +38,7 @@ PostgreSQL 只持久化配置版本、Upstream/Routing 管理状态、Admin 审 ## ADR-005:Redis 活动池采用单实例原子深模块 -**状态:** 接受。 +**状态:** 接受并已实现。 首版 Redis 活动池部署在单实例或 Sentinel 主节点,通过一个深 Adapter 统一实现 Provider 入池、健康状态、Distribution 独占提取、Worker 所有权、库存读取和 diff --git a/docs/api/admin.md b/docs/api/admin.md index 36040ba..8667033 100644 --- a/docs/api/admin.md +++ b/docs/api/admin.md @@ -25,8 +25,10 @@ Admin API 使用独立监听器与权限,契约位于 `api/openapi/admin.yaml` 不能混用配置格式版本或单 Worker Snapshot 版本。 严格 JSON、请求体上限、Request ID、JSON/Problem 响应由 -`platform/httpapi` 公用实现提供。Admin Handler 必须注入 `Authorizer`,标准 -装配使用 `httpsecurity.Protection`,并在路由匹配前完成保护。网关使用的 +`platform/httpapi` 公用实现提供。Admin Handler 必须注入 `IdentityResolver`, +标准装配使用 `httpsecurity.Protection`,并在路由匹配前完成保护。解析出的 +Actor ID 与可信 SourceIP 会进入所有 mutation 命令,但认证材料不会进入审计。 +网关使用的 `Proxy-Authorization`/407 语义不得复用到 Admin 的 `Authorization`/401 语义。 `controller/runtime` 将 Admin 与 Distribution 放在不同 `net.Listener`,任一 监听器异常会触发同组端点的有界优雅停机。 diff --git a/docs/development/implementation-plan.md b/docs/development/implementation-plan.md index 2f680fd..0963c65 100644 --- a/docs/development/implementation-plan.md +++ b/docs/development/implementation-plan.md @@ -158,9 +158,10 @@ test/{fixtures,integration,e2e,load}/ **Files:** `internal/controller/distribution/*.go`, `admin/*.go`, `internal/adapters/postgres/*.go`, `internal/adapters/redis/*.go`, migrations, tests -- [ ] Define PostgreSQL ports for ConfigVersion, Upstream/Routing management state, - AdminAudit, Outbox, and optional aggregate metrics; never persist Proxy details or - per-extraction records. +- [x] Define the PostgreSQL management seam for ConfigVersion, Upstream/Routing state, + AdminAudit and leased Outbox; the public contract has no Proxy or extraction detail types. +- [x] Add the six-table PostgreSQL management migration with static data-boundary checks. +- [ ] Implement the pgx PostgreSQL Adapter and run the public contract against PostgreSQL 18. - [x] Implement the Redis TTL activity pool and one atomic extraction operation covering candidate eligibility, Gateway reserve, ownership, removal, and short-lived idempotency. - [x] Implement Redis Worker ownership, drain/ACK, expiry reclaim, inventory and bounded @@ -181,8 +182,9 @@ extract/live/ready Handler 与 Admin status/enable/disable/switch/reload Handler Bearer/CIDR、可信代理、Client ID、本地准入和 API 401/Gateway 407 差异,并作为 Admin/Distribution 必需依赖。共享 `platform/httpserver` 与 `controller/runtime` 已完成 Distribution/Admin 独立监听器、首错联动关闭和 -有界优雅停机。生产命令入口、PostgreSQL 管理面 Adapter 及其 Compose 集成测试 -仍待实现。 +有界优雅停机。Admin mutation 已携带认证 Actor/SourceIP;公用 `adminstate` +事务契约、MemoryStore、100 并发 Routing CAS、租约 Outbox 和六表管理 Schema +已完成。生产命令入口、pgx Adapter 及 PostgreSQL 18 集成测试仍待实现。 已新增公用 `domain/activitypool` 契约及并发安全内存参考实现,Provider Reconciler 通过 `UpsertFetched` 写入带供应商 TTL 和分配安全余量的批次;已覆盖 diff --git a/docs/requirements/completion-audit.md b/docs/requirements/completion-audit.md index d0d16c5..82c605b 100644 --- a/docs/requirements/completion-audit.md +++ b/docs/requirements/completion-audit.md @@ -39,6 +39,12 @@ - `DIST/Admin HTTP`:严格 JSON、Request ID、Problem 响应及 Distribution/Admin Handler 已实现;共享认证、CIDR、可信代理、Client ID 与本地准入保护链已接入, Controller Runtime 已将二者装配到独立监听器并支持联动优雅停机。 +- `Redis Activity Adapter`:真实 Redis 8.2 已覆盖 Provider Upsert、健康更新、 + 原子独占提取、短期幂等、Worker ownership、库存和有界过期清理,Memory/Redis + 运行同一公用契约。 +- `PostgreSQL 管理面基础`:已定义 `adminstate` 事务 seam、并发安全 MemoryStore、 + 公用契约、100 并发 Routing CAS、租约 Outbox 和只含六张管理表的 Schema; + Admin Handler 已向 mutation 传播 Actor/SourceIP。 ## 2. 已执行验证 @@ -64,13 +70,11 @@ CI 已配置 Linux race job。Docker/Kubernetes 仅完成静态验证,没有 1. `cmd/proxy-gateway/controller/checker/loadgen` 进程装配。 2. Gateway 进程装配、生产连接池调优与代表性流量压测。 3. Provider 分布式 singleflight/Leader、长期凭据回收和累计额度执行器。 -4. PostgreSQL 配置版本、Upstream/Routing 管理状态、Admin 审计/Outbox - repository 和迁移;可选聚合指标不包含 Proxy 明细。 -5. Redis TTL 活动池、原子独占提取、短期幂等结果、Leader、速率限制、心跳与 - 所有权适配器;活动池可由 Provider 重建。接线时必须验证进入 - `allocationSafetyMargin` 的不可分配条目不会长期占用补池额度,同时不得 - 突破 `pool.maxSize` 硬上限。 -6. Worker ownership drain/ACK/过期回收和网络快照流。 +4. PostgreSQL pgx 管理面 Adapter、真实 PostgreSQL 18 公用契约、Admin 应用层 + 接线和可选聚合指标;Schema/领域 seam/Memory 参考实现已经完成。 +5. Redis Provider Leader、分布式速率与 Client 限制、Worker 心跳和自动重建; + TTL 活动池、原子提取和 Worker ownership 已完成。 +6. Worker 网络快照流;Redis ownership drain/ACK/过期回收已完成。 7. Checker 调度、探测器和健康 reducer。 8. Admin/Distribution 细粒度授权、分布式限流和审计查询。 9. 真实 Compose/Kubernetes 集成、故障演练和代表性集群负载测试。 diff --git a/docs/requirements/traceability.md b/docs/requirements/traceability.md index d1914f1..6f8b80a 100644 --- a/docs/requirements/traceability.md +++ b/docs/requirements/traceability.md @@ -85,5 +85,6 @@ | SEC-002 | 非回环监听无保护时严格模式启动失败 | 8112-8441 | 配置校验测试 | | OPS-001 | 配置校验后构建不可变快照并原子替换 | 8959-8999 | 100k 索引、版本/epoch 与并发 Apply/Acquire 测试 | | OPS-002 | 优雅停机停止新请求/Fetch,等待现有流量后超时关闭 | 8981-9000 | Provider Run 收敛与 `Handler.Shutdown` HTTP 排空、Hijacked CONNECT 超时关闭测试 | +| OPS-003 | PostgreSQL 只保存管理修订、Upstream/Routing 状态、Admin 审计与 Outbox | 当前会话 | ADR-006、`adminstate` 公用契约和六表 Schema 边界测试;真实 PostgreSQL 契约待完成 | | OBS-001 | 指标禁止 Proxy IP、session、Client、完整 URL 高基数标签 | 9001-9029 | 指标描述符测试 | | TEST-001 | 覆盖对话中列出的 11 个关键并发与故障场景 | 9030-9082 | 测试清单;Redis 活动池由 Memory/Redis 公用契约覆盖,跨进程故障场景仍按清单推进 | diff --git a/docs/superpowers/plans/2026-07-29-postgresql-admin-state.md b/docs/superpowers/plans/2026-07-29-postgresql-admin-state.md index 0d42384..5ef50e6 100644 --- a/docs/superpowers/plans/2026-07-29-postgresql-admin-state.md +++ b/docs/superpowers/plans/2026-07-29-postgresql-admin-state.md @@ -23,16 +23,16 @@ contract; Controller Admin maps its existing typed commands onto this seam. - Create: `internal/domain/adminstate/adminstate.go` - Create: `internal/domain/adminstate/validation_test.go` -- [ ] Define `Mutator`, `SnapshotReader` and `Outbox` interfaces from ADR-006. -- [ ] Define typed config, Upstream, Routing, actor, audit, event and mutation values. -- [ ] Define stable errors for invalid input, missing resources, CAS conflict and unavailable +- [x] Define `Mutator`, `SnapshotReader`, `AuditReader` and `Outbox` interfaces from ADR-006. +- [x] Define typed config, Upstream, Routing, actor, audit, event and mutation values. +- [x] Define stable errors for invalid input, missing resources, CAS conflict and unavailable storage. -- [ ] Validate non-empty bounded identifiers, UTC timestamps, unique lists, config checksum, +- [x] Validate non-empty bounded identifiers, UTC timestamps, unique lists, config checksum, Routing references, claim limits and claim TTL. -- [ ] Clone every slice/map/JSON value at the seam so callers cannot mutate stored state. -- [ ] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate` and verify the tests +- [x] Clone every slice/map/JSON value at the seam so callers cannot mutate stored state. +- [x] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate` and verify the tests fail before implementation, then pass after implementation. -- [ ] Commit with `feat: define admin state transaction contracts`. +- [x] Commit with `feat: define admin state transaction contracts`. ### Task 2: Memory Reference Adapter and Shared Contract @@ -41,16 +41,16 @@ contract; Controller Admin maps its existing typed commands onto this seam. - Create: `internal/domain/adminstate/contracttest/contract.go` - Create: `internal/domain/adminstate/contract_external_test.go` -- [ ] Write a public contract factory that can create an isolated `Mutator + SnapshotReader + - Outbox` implementation. -- [ ] Cover config commit/replay/conflict/invalid references and zero-write rollback. -- [ ] Cover Upstream idempotency, monotonic revisions, mandatory audit and changed-only events. -- [ ] Cover Routing CAS and 100 concurrent switches with at most one success. -- [ ] Cover bounded claim, exclusive claim, lease expiry, ACK ownership and stable ordering. -- [ ] Cover context cancellation and immutable Snapshot/output values. -- [ ] Implement MemoryStore behind the seam with one mutex per atomic management state. -- [ ] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate/...`. -- [ ] Commit with `feat: add transactional admin state reference store`. +- [x] Write a public contract factory that can create an isolated `adminstate.Store` + implementation over the four narrow capability interfaces. +- [x] Cover config commit/replay/conflict/invalid references and zero-write rollback. +- [x] Cover Upstream idempotency, monotonic revisions, mandatory audit and changed-only events. +- [x] Cover Routing CAS and 100 concurrent switches with at most one success. +- [x] Cover bounded claim, exclusive claim, lease expiry, ACK ownership and stable ordering. +- [x] Cover context cancellation and immutable Snapshot/output values. +- [x] Implement MemoryStore behind the seam with one mutex per atomic management state. +- [x] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate/...`. +- [x] Commit with `feat: add transactional admin state reference store`. ### Task 3: Schema Migration and Static Data-Boundary Test @@ -59,14 +59,14 @@ contract; Controller Admin maps its existing typed commands onto this seam. - Create: `internal/adapters/postgresadmin/migrations.go` - Create: `internal/adapters/postgresadmin/migrations_test.go` -- [ ] Embed ordered migrations and expose one `Migrations() []Migration` read-only accessor. -- [ ] Create only the six ADR-006 tables with primary/foreign keys, UTC timestamps, indexes, +- [x] Embed ordered migrations and expose one `Migrations() []Migration` read-only accessor. +- [x] Create only the six ADR-006 tables with primary/foreign keys, UTC timestamps, indexes, outbox claim fields and bounded checks. -- [ ] Add a parser-backed/static test that asserts required tables/columns are present and +- [x] Add a static structure test that asserts required tables/columns are present and forbidden Proxy/extraction/ownership/idempotency tables or columns are absent. -- [ ] Test migration IDs are unique, strictly ordered and statements are transactional. -- [ ] Run `go test -count=1 -timeout 60s ./internal/adapters/postgresadmin`. -- [ ] Commit with `feat: add postgres admin state schema`. +- [x] Test migration IDs are unique, strictly ordered and statements are transactional. +- [x] Run `go test -count=1 -timeout 60s ./internal/adapters/postgresadmin`. +- [x] Commit with `feat: add postgres admin state schema`. ### Task 4: PostgreSQL Deep Adapter @@ -118,7 +118,7 @@ contract; Controller Admin maps its existing typed commands onto this seam. - Modify: `internal/controller/admin/handler.go` - Modify: `internal/controller/admin/handler_test.go` -- [ ] Change Admin protection to resolve `httpsecurity.Identity` once and add actor/source IP +- [x] Change Admin protection to resolve `httpsecurity.Identity` once and add actor/source IP to mutation commands without exposing credentials. - [ ] Map typed Handler commands to `adminstate.Mutator`; map domain conflict/not-found/ invalid/unavailable errors to the existing HTTP contract. diff --git a/docs/testing/test-strategy.md b/docs/testing/test-strategy.md index 75ee495..66c0947 100644 --- a/docs/testing/test-strategy.md +++ b/docs/testing/test-strategy.md @@ -88,6 +88,11 @@ fixture 的执行命令是: 100 轮并发提取和 100 轮所有权竞争。fixture 使用唯一命名空间,不执行 `FLUSHDB`;本地 Redis 关闭 AOF、RDB 和数据卷,避免短效 Proxy 与凭据落盘。 +PostgreSQL 管理面使用 `adminstate/contracttest` 作为 Memory/PostgreSQL 公用 +行为契约。当前 MemoryStore 已覆盖配置提交、Upstream 幂等、100 并发 Routing +CAS、审计和租约 Outbox;Schema 静态测试证明只声明六张管理表。只有 pgx +Adapter 在真实 PostgreSQL 18 上运行同一契约后,才标记生产持久化完成。 + 需要 PostgreSQL/Redis 的测试使用独立实例和短生命周期容器,不复用开发数据。 测试结束后验证没有残留 Worker ownership、Leader 租约、活动池条目或幂等键, 并检查 PostgreSQL 中不存在 Proxy 明细和逐次提取记录。 diff --git a/progress.md b/progress.md index da140f8..3723458 100644 --- a/progress.md +++ b/progress.md @@ -20,6 +20,14 @@ 余量停止新分配。 - PostgreSQL 管理面 Adapter、Provider Leader/分布式限流与心跳装配、生产命令 入口、Redis 故障转移验证和代表性 100,000 QPS 集群压测仍待实现。 +- 已新增 ADR-006 与公用 `adminstate` 事务 seam;MemoryStore 在同一锁内提交管理 + 状态、Admin 审计与 Outbox,覆盖配置冲突、Upstream 幂等、100 并发 Routing + CAS、Outbox claim/ack 租约、上下文取消和不可变快照。 +- PostgreSQL 管理 Schema 已限制为六张配置/状态/审计/Outbox 表,并由静态测试 + 拒绝 Proxy、凭据、逐次提取、ownership 和短期幂等明细;pgx Adapter 和真实 + PostgreSQL 18 契约仍待实现。 +- Admin Handler 现在复用 `httpsecurity.Identity`,把 Actor ID 与可信 SourceIP + 传给 enable/disable/switch/reload mutation,供持久化审计直接使用。 - 本轮 `.\scripts\verify.ps1`、`.\scripts\test-redis.ps1`、Compose 静态展开和 Compose 非持久化策略测试通过;Windows `CGO_ENABLED=0`,race 继续由 Linux CI 执行。 diff --git a/task_plan.md b/task_plan.md index 5753c7c..0b1fc5f 100644 --- a/task_plan.md +++ b/task_plan.md @@ -28,6 +28,8 @@ TTL 活动池契约;PostgreSQL 退出代理数据路径 10. [已完成] 实现生产 Redis Activity Adapter、原子 Lua、公用行为契约和 Redis 8.2 集成 fixture;本地 Redis 禁止短效代理数据持久化 +11. [进行中] 实现 PostgreSQL 管理面;ADR、领域事务契约、MemoryStore、公用 + 契约、六表 Schema 和 Admin Actor 传播已完成,pgx Adapter/集成测试待完成 ## 串并行关系