docs: record postgres admin state delivery
Some checks are pending
ci / test (ubuntu-latest) (push) Waiting to run
ci / test (windows-latest) (push) Waiting to run
ci / race (push) Waiting to run
ci / integration (push) Waiting to run

This commit is contained in:
youfak 2026-07-30 10:36:54 +08:00
parent 6bfef0fcf5
commit 61dd03d207
7 changed files with 95 additions and 46 deletions

View File

@ -169,7 +169,7 @@ Upstream、`endBehavior` 默认 `stop`并覆盖列表末端停止disabled
- [x] Define the PostgreSQL management seam for ConfigVersion, Upstream/Routing state,
AdminAudit and leased Outbox; the public contract has no Proxy or extraction detail types.
- [x] Add the six-table PostgreSQL management migration with static data-boundary checks.
- [ ] Implement the pgx PostgreSQL Adapter and run the public contract against PostgreSQL 18.
- [x] Implement the pgx PostgreSQL Adapter and run the public contract against PostgreSQL 18.
- [x] Implement the Redis TTL activity pool and one atomic extraction operation covering
candidate eligibility, Gateway reserve, ownership, removal, and short-lived idempotency.
- [x] Implement Redis Worker ownership, drain/ACK, expiry reclaim, inventory and bounded
@ -181,9 +181,9 @@ Upstream、`endBehavior` 默认 `stop`并覆盖列表末端停止disabled
- [x] Expose Distribution extraction/status and Admin status/enable/disable/switch/reload
HTTP handlers and contracts.
- [x] Add Compose-backed Redis 8.2 integration and shared Adapter contract tests.
- [ ] Add PostgreSQL management Adapter and Compose-backed integration tests.
- [x] Add PostgreSQL management Adapter and Compose-backed integration tests.
当前进度2026-07-29):已实现共享 `platform/httpapi`、Distribution
当前进度2026-07-30):已实现共享 `platform/httpapi`、Distribution
extract/live/ready Handler 与 Admin status/enable/disable/switch/reload Handler
定向契约测试已覆盖严格 JSON、Body 上限、Request ID、幂等 Header、DTO 映射、
404/405 及业务错误映射。共享 `platform/httpsecurity` 已补齐 Basic/API Key/
@ -192,10 +192,11 @@ Admin/Distribution 必需依赖。共享 `platform/httpserver` 与
`controller/runtime` 已完成 Distribution/Admin 独立监听器、首错联动关闭和
有界优雅停机。Admin mutation 已携带认证 Actor/SourceIP公用 `adminstate`
事务契约、MemoryStore、100 并发 Routing CAS、租约 Outbox 和六表管理 Schema
已完成。Admin `ApplicationService` 已将 mutation、权威管理快照、低基数运行态
已完成。pgx Adapter 已在真实 PostgreSQL 18 上运行同一公用契约,并验证
Repeatable Read 快照、`SKIP LOCKED`、原子 ACK、审计/Outbox 故障回滚和数据边界。
Admin `ApplicationService` 已将 mutation、权威管理快照、低基数运行态
聚合与配置重载接到同一公用 seam严格文件加载、脱敏管理摘要及原子配置发布
已通过失败路径和并发测试。生产命令入口、pgx Adapter 及 PostgreSQL 18 集成
测试仍待实现。
已通过失败路径和并发测试。生产命令入口及其连接池/迁移启动装配仍待实现。
已新增公用 `domain/activitypool` 契约及并发安全内存参考实现Provider
Reconciler 通过 `UpsertFetched` 写入带供应商 TTL 和分配安全余量的批次;已覆盖

View File

@ -54,6 +54,12 @@ kubectl kustomize deploy/kubernetes/base > rendered.yaml
不执行 `FLUSHDB`。生产环境的 Redis 高可用与持久化策略必须独立评审,不能照搬
本地 fixture。
PostgreSQL 管理面集成测试通过 `.\scripts\test-postgres.ps1` 启动
`postgres:18-alpine`,仅绑定 `127.0.0.1:15432`,并将 PG18 数据根目录挂载为
tmpfs。`ApplyMigrations` 在同一物理连接上执行仓库内嵌的幂等前向迁移;测试为
每个契约创建唯一 Schema结束时只删除该 Schema 和临时 Compose 项目。该脚本
禁止指向开发或生产数据库。
目标拓扑入口:
- Gateway`127.0.0.1:8080`
@ -112,6 +118,9 @@ kubectl -n proxy-pool rollout status deployment/proxy-gateway --timeout=10m
4. 逐批发布 Gateway每次至少保留 PDB 要求的健康副本。
5. 观察 30 分钟,再清理已经无人读取的旧字段或旧迁移。
迁移失败时停止 Controller 发布,不自动重试结果不确定的管理 mutation。恢复后
先确认 Schema 版本、最近 revision、审计和 outbox 一致,再开放 Admin 写入口。
回滚只能回到仍兼容当前 Schema 和 Snapshot 版本的镜像。涉及不可逆数据迁移时,
必须使用前向修复。
@ -173,6 +182,11 @@ Prometheus 标签禁止包含 Proxy IP、Client ID、Session、完整 URL、requ
3. 等待已提交 Redis 原子操作返回;未确认请求的客户端必须使用相同幂等键重试。
4. 刷新 Admin outbox、审计和 Worker ACK再关闭 PostgreSQL/Redis 连接池。
Outbox 发布器必须以稳定 consumer ID 有界领取;发布成功后原子 ACK。进程在发布
成功但 ACK 结果不确定时,不得伪造确认;等待租约到期后重领,并由下游事件消费者
按事件 ID/revision 去重。持续积压时先暂停新的管理变更,检查发布目标、租约和
最老未发布事件,不删除未发布行。
### Checker
1. 停止领取新任务。

View File

@ -46,8 +46,10 @@
- `Redis Activity Adapter`:真实 Redis 8.2 已覆盖 Provider Upsert、健康更新、
原子独占提取、短期幂等、Worker ownership、库存和有界过期清理Memory/Redis
运行同一公用契约。
- `PostgreSQL 管理面基础`:已定义 `adminstate` 事务 seam、并发安全 MemoryStore、
公用契约、100 并发 Routing CAS、租约 Outbox 和只含六张管理表的 Schema
- `PostgreSQL 管理面`:已定义 `adminstate` 事务 seam、并发安全 MemoryStore、
公用契约、100 并发 Routing CAS、租约 Outbox 和只含六张管理表的 Schemapgx
Adapter 已在真实 PostgreSQL 18 上通过同一契约、迁移幂等、审计/Outbox
故障回滚、原子批量 ACK 和 `information_schema` 数据边界验证。
Admin Handler 已向 mutation 传播 Actor/SourceIP。Admin ApplicationService 已
完成管理 mutation 映射、权威/运行态 Status 聚合、严格配置加载和持久化成功后
的原子发布;边界测试禁止其依赖 Redis Extract 或 Proxy 明细包。
@ -58,6 +60,7 @@
go test ./... PASS
go vet ./... PASS
go build ./... PASS
PostgreSQL 18 shared contract PASS
protoc descriptor compilation PASS (local script; CI enforcement pending)
docker compose config PASS
kubectl kustomize PASS
@ -66,8 +69,8 @@ Mermaid blocks 35
```
Windows 环境为 `CGO_ENABLED=0` 且没有 C 编译器,`go test -race` 在本机未执行;
CI 已配置 Linux race job。Docker/Kubernetes 仅完成静态验证,没有把目标拓扑
作为已运行系统
CI 已配置 Linux race job。PostgreSQL 18 和 Redis 8.2 的隔离 Adapter fixture
已经运行;完整 Controller/Gateway/Checker Docker/Kubernetes 拓扑仍只有静态验证
## 3. 后续实现范围
@ -76,8 +79,9 @@ CI 已配置 Linux race job。Docker/Kubernetes 仅完成静态验证,没有
1. `cmd/proxy-gateway/controller/checker/loadgen` 进程装配。
2. Gateway 进程装配、生产连接池调优与代表性流量压测。
3. Provider 分布式 singleflight/Leader、长期凭据回收和累计额度执行器。
4. PostgreSQL pgx 管理面 Adapter、真实 PostgreSQL 18 公用契约、Admin 应用层
接线和可选聚合指标Schema/领域 seam/Memory 参考实现已经完成。
4. PostgreSQL 连接池、迁移和 pgx Adapter 的生产命令启动装配,以及可选聚合指标;
Schema、领域 seam、Memory/pgx Adapter、真实 PostgreSQL 18 契约和 Admin
应用层接线已经完成。
5. Redis Provider Leader、分布式速率与 Client 限制、Worker 心跳和自动重建;
TTL 活动池、原子提取和 Worker ownership 已完成。
6. Worker 网络快照流Redis ownership drain/ACK/过期回收已完成。

View File

@ -79,17 +79,17 @@ contract; Controller Admin maps its existing typed commands onto this seam.
- Modify: `go.mod`
- Modify: `go.sum`
- [ ] Add the approved pinned pgx/v5 dependency without changing unrelated modules.
- [ ] Accept a narrow pgx pool interface and options; reject nil pools and invalid namespaces.
- [ ] Implement Config/Upstream/Routing mutations with SQL validation, row locks/CAS and one
- [x] Add the approved pinned pgx/v5 dependency without changing unrelated modules.
- [x] Accept a narrow pgx pool interface and reject nil dependencies.
- [x] Implement Config/Upstream/Routing mutations with SQL validation, row locks/CAS and one
transaction for revision, state, audit and Outbox.
- [ ] Map PostgreSQL constraint/CAS/connection errors to domain errors without leaking DSNs,
- [x] Map PostgreSQL constraint/CAS/connection errors to domain errors without leaking DSNs,
SQL or values.
- [ ] Implement immutable Snapshot reads from one repeatable-read transaction.
- [ ] Implement bounded Outbox claim using `FOR UPDATE SKIP LOCKED` and guarded ACK.
- [ ] Keep SQL, tx retries, codecs and driver types private to the Adapter.
- [ ] Run focused unit tests and `go vet ./internal/adapters/postgresadmin/...`.
- [ ] Commit with `feat: implement postgres admin state adapter`.
- [x] Implement immutable Snapshot reads from one repeatable-read transaction.
- [x] Implement bounded Outbox claim using `FOR UPDATE SKIP LOCKED` and guarded ACK.
- [x] Keep SQL, tx retries, codecs and driver types private to the Adapter.
- [x] Run focused unit tests and `go vet ./internal/adapters/postgresadmin/...`.
- [x] Commit with `feat: implement postgres admin state adapter`.
### Task 5: Real PostgreSQL Contract Fixture
@ -100,15 +100,15 @@ contract; Controller Admin maps its existing typed commands onto this seam.
- Create: `internal/adapters/postgresadmin/contract_integration_test.go`
- Create: `internal/adapters/postgresadmin/rollback_integration_test.go`
- [ ] Start an isolated PostgreSQL 18 fixture on a dedicated loopback port and database.
- [ ] Apply migrations through the same migration runner used by the production Adapter.
- [ ] Run the public adminstate contract against a unique schema per test.
- [ ] Inject audit and Outbox constraint failures and prove state/revision rollback.
- [ ] Query `information_schema` and prove no Proxy, extraction, ownership or idempotency
- [x] Start an isolated PostgreSQL 18 fixture on a dedicated loopback port and database.
- [x] Apply migrations through the same migration runner used by the production Adapter.
- [x] Run the public adminstate contract against a unique schema per test.
- [x] Inject audit and Outbox constraint failures and prove state/revision rollback.
- [x] Query `information_schema` and prove no Proxy, extraction, ownership or idempotency
detail tables/columns exist.
- [ ] Clean only the unique test schema; do not drop shared databases or use broad cleanup.
- [ ] Run `.\scripts\test-postgres.ps1` with every Go test timeout set to 60 seconds.
- [ ] Commit with `test: add postgres admin state contract fixture`.
- [x] Clean only the unique test schema; do not drop shared databases or use broad cleanup.
- [x] Run `.\scripts\test-postgres.ps1` with every Go test timeout set to 60 seconds.
- [x] Commit with `test: add postgres admin state contract fixture`.
### Task 6: Admin Application Integration
@ -139,11 +139,12 @@ contract; Controller Admin maps its existing typed commands onto this seam.
- Modify: `docs/operations/runbook.md`
- Modify: `progress.md`
- [ ] Mark only verified PostgreSQL capabilities complete and retain command/runtime/load gaps.
- [ ] Document migration, backup, Outbox backlog/replay and data-boundary checks.
- [ ] Run `.\scripts\verify.ps1`, `.\scripts\test-redis.ps1`,
- [x] Mark only verified PostgreSQL capabilities complete and retain command/runtime/load gaps.
- [x] Document migration, backup, Outbox backlog/replay and data-boundary checks.
- [x] Run `.\scripts\verify.ps1`, `.\scripts\test-redis.ps1`,
`.\scripts\test-postgres.ps1` and `git diff --check`.
- [ ] Audit that Gateway has no PostgreSQL/Redis dependency and Distribution has no PostgreSQL
- [x] Audit that Gateway has no PostgreSQL/Redis dependency and Distribution has no PostgreSQL
dependency.
- [ ] Confirm the user-owned deletion of `proxy-pool-docs-v1.0.zip` is not staged.
- [ ] Commit with `docs: record postgres admin state delivery` and push the feature branch.
- [x] Confirm the user-owned deletion of `proxy-pool-docs-v1.0.zip` is not staged.
- [x] Commit with `docs: record postgres admin state delivery`.
- [ ] Push the feature branch after working Git credentials are available.

View File

@ -95,10 +95,18 @@ fixture 的执行命令是:
`FLUSHDB`;本地 Redis 关闭 AOF、RDB 和数据卷,避免短效 Proxy 与凭据落盘。
PostgreSQL 管理面使用 `adminstate/contracttest` 作为 Memory/PostgreSQL 公用
行为契约。当前 MemoryStore 已覆盖配置提交、Upstream 幂等、100 并发 Routing
CAS、审计分页、Routing no-op、原子批量 ACK 和租约 OutboxSchema 静态测试
证明只声明六张管理表。PostgreSQL 18 的回环端口、tmpfs 隔离 fixture 已完成
静态验证;只有 pgx Adapter 在该实例上运行同一契约后,才标记生产持久化完成。
行为契约,执行命令是:
```powershell
.\scripts\test-postgres.ps1
```
pgx Adapter 已在真实 PostgreSQL 18 上覆盖配置事务、Upstream 幂等、100 并发
Routing CAS、Repeatable Read 快照、审计分页、Routing no-op、`SKIP LOCKED`
租约、原子批量 ACK以及审计/Outbox 写入失败时的完整回滚。fixture 为每个测试
创建唯一 Schema只删除该 Schema数据库使用回环端口和 tmpfs测试后不保留
数据卷。静态与 `information_schema` 双重检查证明只存在六张管理表,且没有
Proxy、凭据、逐次提取、Worker ownership 或幂等明细列。
Admin 应用层测试覆盖 typed-nil 依赖、Actor/SourceIP 映射、Routing CAS 错误、
权威管理快照与低基数运行态聚合、未知字段拒绝、主配置/Secret 文件 I/O 分类、

View File

@ -1,5 +1,26 @@
# 项目进度
## 2026-07-30
- 固定 `github.com/jackc/pgx/v5 v5.6.0`,实现封装在 `adminstate.Store` 后的
PostgreSQL 深适配器配置、Upstream、Routing mutation 在同一事务中提交
revision、管理状态、审计与 Outbox数据库错误不泄漏 DSN、SQL 或参数。
- 实现只读 Repeatable Read Snapshot、审计分页、`FOR UPDATE SKIP LOCKED`
Outbox 领取和整批 ACK具体 Adapter、SQL、pgx 类型及 codec 不向业务层暴露。
- 新增同一物理连接迁移执行器和 PostgreSQL 18 隔离 fixture每个测试使用唯一
Schema回环端口和 tmpfs 不保留数据CI 已加入 Redis/PostgreSQL 集成任务。
- 真实 PostgreSQL 18 已通过完整公用契约配置事务、Upstream 幂等、100 并发
Routing CAS、Routing no-op、审计分页、Outbox 租约/重领、原子 ACK 和 Context。
- 审计与 Outbox 触发器故障注入证明状态、revision、审计和事件完整回滚迁移
重复执行及 `information_schema` 检查确认仅有六张管理表,无 Proxy、凭据、
逐次提取、Worker ownership 或幂等明细。
- Docker Hub 直连因 Docker Desktop 未配置 HTTPS 代理超时,改从 Google 官方
Docker Hub 公共缓存拉取相同 `postgres:18-alpine` 镜像并本地重标记;未修改
Docker Desktop 全局镜像配置。
- `implementation-plan.md` 中 PostgreSQL 两项验收已完成,总进度由 49/73 更新为
51/7369.9%生产命令装配、分布式协调、Checker 和 100,000 QPS 集群压测
仍未完成。
## 2026-07-29
- 新增公用 `verify-proto.ps1`:自动发现 Google well-known types include固定

View File

@ -28,9 +28,9 @@
TTL 活动池契约PostgreSQL 退出代理数据路径
10. [已完成] 实现生产 Redis Activity Adapter、原子 Lua、公用行为契约和
Redis 8.2 集成 fixture本地 Redis 禁止短效代理数据持久化
11. [进行中] 实现 PostgreSQL 管理面ADR、领域事务契约、MemoryStore、公用
契约、六表 Schema、Admin Actor 传播、应用服务和原子配置发布已完成pgx
Adapter/集成测试待完成
11. [已完成] 实现 PostgreSQL 管理面ADR、领域事务契约、MemoryStore、公用
契约、六表 Schema、Admin Actor 传播、应用服务、原子配置发布、pgx Adapter
和真实 PostgreSQL 18 集成测试已完成
12. [进行中] 复核验收清单并收敛既有 Routing/Sequential 与 Proxy 容量边界;
机器契约和文档类滞后勾选已按仓库证据校正
@ -51,9 +51,9 @@
## 已知环境限制
- Docker Compose 配置与 Kubernetes Kustomize 已完成静态渲染验证;未启动
目标运行拓扑。
- Docker Compose 配置与 Kubernetes Kustomize 已完成静态渲染验证;Redis 8.2
与 PostgreSQL 18 的隔离 Adapter fixture 已运行,完整目标运行拓扑尚未启动
- `cmd/proxy-*`、PostgreSQL 管理面 Adapter、Provider Leader/分布式限流、
Checker 运行时、Redis 故障转移验证与代表性集群压测属于后续实施范围。
- `implementation-plan.md` 当前按 73 个验收项统计;已校正为 49 项完成,
验收项完成率约 67%,不等同于生产就绪度。
- `implementation-plan.md` 当前按 73 个验收项统计;已校正为 51 项完成,
验收项完成率约 69.9%,不等同于生产就绪度。