docs: record postgres admin state delivery
This commit is contained in:
parent
6bfef0fcf5
commit
61dd03d207
@ -169,7 +169,7 @@ Upstream、`endBehavior` 默认 `stop`,并覆盖列表末端停止;disabled
|
||||
- [x] Define the PostgreSQL management seam for ConfigVersion, Upstream/Routing state,
|
||||
AdminAudit and leased Outbox; the public contract has no Proxy or extraction detail types.
|
||||
- [x] Add the six-table PostgreSQL management migration with static data-boundary checks.
|
||||
- [ ] Implement the pgx PostgreSQL Adapter and run the public contract against PostgreSQL 18.
|
||||
- [x] Implement the pgx PostgreSQL Adapter and run the public contract against PostgreSQL 18.
|
||||
- [x] Implement the Redis TTL activity pool and one atomic extraction operation covering
|
||||
candidate eligibility, Gateway reserve, ownership, removal, and short-lived idempotency.
|
||||
- [x] Implement Redis Worker ownership, drain/ACK, expiry reclaim, inventory and bounded
|
||||
@ -181,9 +181,9 @@ Upstream、`endBehavior` 默认 `stop`,并覆盖列表末端停止;disabled
|
||||
- [x] Expose Distribution extraction/status and Admin status/enable/disable/switch/reload
|
||||
HTTP handlers and contracts.
|
||||
- [x] Add Compose-backed Redis 8.2 integration and shared Adapter contract tests.
|
||||
- [ ] Add PostgreSQL management Adapter and Compose-backed integration tests.
|
||||
- [x] Add PostgreSQL management Adapter and Compose-backed integration tests.
|
||||
|
||||
当前进度(2026-07-29):已实现共享 `platform/httpapi`、Distribution
|
||||
当前进度(2026-07-30):已实现共享 `platform/httpapi`、Distribution
|
||||
extract/live/ready Handler 与 Admin status/enable/disable/switch/reload Handler;
|
||||
定向契约测试已覆盖严格 JSON、Body 上限、Request ID、幂等 Header、DTO 映射、
|
||||
404/405 及业务错误映射。共享 `platform/httpsecurity` 已补齐 Basic/API Key/
|
||||
@ -192,10 +192,11 @@ Admin/Distribution 必需依赖。共享 `platform/httpserver` 与
|
||||
`controller/runtime` 已完成 Distribution/Admin 独立监听器、首错联动关闭和
|
||||
有界优雅停机。Admin mutation 已携带认证 Actor/SourceIP;公用 `adminstate`
|
||||
事务契约、MemoryStore、100 并发 Routing CAS、租约 Outbox 和六表管理 Schema
|
||||
已完成。Admin `ApplicationService` 已将 mutation、权威管理快照、低基数运行态
|
||||
已完成。pgx Adapter 已在真实 PostgreSQL 18 上运行同一公用契约,并验证
|
||||
Repeatable Read 快照、`SKIP LOCKED`、原子 ACK、审计/Outbox 故障回滚和数据边界。
|
||||
Admin `ApplicationService` 已将 mutation、权威管理快照、低基数运行态
|
||||
聚合与配置重载接到同一公用 seam;严格文件加载、脱敏管理摘要及原子配置发布
|
||||
已通过失败路径和并发测试。生产命令入口、pgx Adapter 及 PostgreSQL 18 集成
|
||||
测试仍待实现。
|
||||
已通过失败路径和并发测试。生产命令入口及其连接池/迁移启动装配仍待实现。
|
||||
|
||||
已新增公用 `domain/activitypool` 契约及并发安全内存参考实现,Provider
|
||||
Reconciler 通过 `UpsertFetched` 写入带供应商 TTL 和分配安全余量的批次;已覆盖
|
||||
|
||||
@ -54,6 +54,12 @@ kubectl kustomize deploy/kubernetes/base > rendered.yaml
|
||||
不执行 `FLUSHDB`。生产环境的 Redis 高可用与持久化策略必须独立评审,不能照搬
|
||||
本地 fixture。
|
||||
|
||||
PostgreSQL 管理面集成测试通过 `.\scripts\test-postgres.ps1` 启动
|
||||
`postgres:18-alpine`,仅绑定 `127.0.0.1:15432`,并将 PG18 数据根目录挂载为
|
||||
tmpfs。`ApplyMigrations` 在同一物理连接上执行仓库内嵌的幂等前向迁移;测试为
|
||||
每个契约创建唯一 Schema,结束时只删除该 Schema 和临时 Compose 项目。该脚本
|
||||
禁止指向开发或生产数据库。
|
||||
|
||||
目标拓扑入口:
|
||||
|
||||
- Gateway:`127.0.0.1:8080`
|
||||
@ -112,6 +118,9 @@ kubectl -n proxy-pool rollout status deployment/proxy-gateway --timeout=10m
|
||||
4. 逐批发布 Gateway;每次至少保留 PDB 要求的健康副本。
|
||||
5. 观察 30 分钟,再清理已经无人读取的旧字段或旧迁移。
|
||||
|
||||
迁移失败时停止 Controller 发布,不自动重试结果不确定的管理 mutation。恢复后
|
||||
先确认 Schema 版本、最近 revision、审计和 outbox 一致,再开放 Admin 写入口。
|
||||
|
||||
回滚只能回到仍兼容当前 Schema 和 Snapshot 版本的镜像。涉及不可逆数据迁移时,
|
||||
必须使用前向修复。
|
||||
|
||||
@ -173,6 +182,11 @@ Prometheus 标签禁止包含 Proxy IP、Client ID、Session、完整 URL、requ
|
||||
3. 等待已提交 Redis 原子操作返回;未确认请求的客户端必须使用相同幂等键重试。
|
||||
4. 刷新 Admin outbox、审计和 Worker ACK,再关闭 PostgreSQL/Redis 连接池。
|
||||
|
||||
Outbox 发布器必须以稳定 consumer ID 有界领取;发布成功后原子 ACK。进程在发布
|
||||
成功但 ACK 结果不确定时,不得伪造确认;等待租约到期后重领,并由下游事件消费者
|
||||
按事件 ID/revision 去重。持续积压时先暂停新的管理变更,检查发布目标、租约和
|
||||
最老未发布事件,不删除未发布行。
|
||||
|
||||
### Checker
|
||||
|
||||
1. 停止领取新任务。
|
||||
|
||||
@ -46,8 +46,10 @@
|
||||
- `Redis Activity Adapter`:真实 Redis 8.2 已覆盖 Provider Upsert、健康更新、
|
||||
原子独占提取、短期幂等、Worker ownership、库存和有界过期清理,Memory/Redis
|
||||
运行同一公用契约。
|
||||
- `PostgreSQL 管理面基础`:已定义 `adminstate` 事务 seam、并发安全 MemoryStore、
|
||||
公用契约、100 并发 Routing CAS、租约 Outbox 和只含六张管理表的 Schema;
|
||||
- `PostgreSQL 管理面`:已定义 `adminstate` 事务 seam、并发安全 MemoryStore、
|
||||
公用契约、100 并发 Routing CAS、租约 Outbox 和只含六张管理表的 Schema;pgx
|
||||
Adapter 已在真实 PostgreSQL 18 上通过同一契约、迁移幂等、审计/Outbox
|
||||
故障回滚、原子批量 ACK 和 `information_schema` 数据边界验证。
|
||||
Admin Handler 已向 mutation 传播 Actor/SourceIP。Admin ApplicationService 已
|
||||
完成管理 mutation 映射、权威/运行态 Status 聚合、严格配置加载和持久化成功后
|
||||
的原子发布;边界测试禁止其依赖 Redis Extract 或 Proxy 明细包。
|
||||
@ -58,6 +60,7 @@
|
||||
go test ./... PASS
|
||||
go vet ./... PASS
|
||||
go build ./... PASS
|
||||
PostgreSQL 18 shared contract PASS
|
||||
protoc descriptor compilation PASS (local script; CI enforcement pending)
|
||||
docker compose config PASS
|
||||
kubectl kustomize PASS
|
||||
@ -66,8 +69,8 @@ Mermaid blocks 35
|
||||
```
|
||||
|
||||
Windows 环境为 `CGO_ENABLED=0` 且没有 C 编译器,`go test -race` 在本机未执行;
|
||||
CI 已配置 Linux race job。Docker/Kubernetes 仅完成静态验证,没有把目标拓扑
|
||||
作为已运行系统。
|
||||
CI 已配置 Linux race job。PostgreSQL 18 和 Redis 8.2 的隔离 Adapter fixture
|
||||
已经运行;完整 Controller/Gateway/Checker Docker/Kubernetes 拓扑仍只有静态验证。
|
||||
|
||||
## 3. 后续实现范围
|
||||
|
||||
@ -76,8 +79,9 @@ CI 已配置 Linux race job。Docker/Kubernetes 仅完成静态验证,没有
|
||||
1. `cmd/proxy-gateway/controller/checker/loadgen` 进程装配。
|
||||
2. Gateway 进程装配、生产连接池调优与代表性流量压测。
|
||||
3. Provider 分布式 singleflight/Leader、长期凭据回收和累计额度执行器。
|
||||
4. PostgreSQL pgx 管理面 Adapter、真实 PostgreSQL 18 公用契约、Admin 应用层
|
||||
接线和可选聚合指标;Schema/领域 seam/Memory 参考实现已经完成。
|
||||
4. PostgreSQL 连接池、迁移和 pgx Adapter 的生产命令启动装配,以及可选聚合指标;
|
||||
Schema、领域 seam、Memory/pgx Adapter、真实 PostgreSQL 18 契约和 Admin
|
||||
应用层接线已经完成。
|
||||
5. Redis Provider Leader、分布式速率与 Client 限制、Worker 心跳和自动重建;
|
||||
TTL 活动池、原子提取和 Worker ownership 已完成。
|
||||
6. Worker 网络快照流;Redis ownership drain/ACK/过期回收已完成。
|
||||
|
||||
@ -79,17 +79,17 @@ contract; Controller Admin maps its existing typed commands onto this seam.
|
||||
- Modify: `go.mod`
|
||||
- Modify: `go.sum`
|
||||
|
||||
- [ ] Add the approved pinned pgx/v5 dependency without changing unrelated modules.
|
||||
- [ ] Accept a narrow pgx pool interface and options; reject nil pools and invalid namespaces.
|
||||
- [ ] Implement Config/Upstream/Routing mutations with SQL validation, row locks/CAS and one
|
||||
- [x] Add the approved pinned pgx/v5 dependency without changing unrelated modules.
|
||||
- [x] Accept a narrow pgx pool interface and reject nil dependencies.
|
||||
- [x] Implement Config/Upstream/Routing mutations with SQL validation, row locks/CAS and one
|
||||
transaction for revision, state, audit and Outbox.
|
||||
- [ ] Map PostgreSQL constraint/CAS/connection errors to domain errors without leaking DSNs,
|
||||
- [x] Map PostgreSQL constraint/CAS/connection errors to domain errors without leaking DSNs,
|
||||
SQL or values.
|
||||
- [ ] Implement immutable Snapshot reads from one repeatable-read transaction.
|
||||
- [ ] Implement bounded Outbox claim using `FOR UPDATE SKIP LOCKED` and guarded ACK.
|
||||
- [ ] Keep SQL, tx retries, codecs and driver types private to the Adapter.
|
||||
- [ ] Run focused unit tests and `go vet ./internal/adapters/postgresadmin/...`.
|
||||
- [ ] Commit with `feat: implement postgres admin state adapter`.
|
||||
- [x] Implement immutable Snapshot reads from one repeatable-read transaction.
|
||||
- [x] Implement bounded Outbox claim using `FOR UPDATE SKIP LOCKED` and guarded ACK.
|
||||
- [x] Keep SQL, tx retries, codecs and driver types private to the Adapter.
|
||||
- [x] Run focused unit tests and `go vet ./internal/adapters/postgresadmin/...`.
|
||||
- [x] Commit with `feat: implement postgres admin state adapter`.
|
||||
|
||||
### Task 5: Real PostgreSQL Contract Fixture
|
||||
|
||||
@ -100,15 +100,15 @@ contract; Controller Admin maps its existing typed commands onto this seam.
|
||||
- Create: `internal/adapters/postgresadmin/contract_integration_test.go`
|
||||
- Create: `internal/adapters/postgresadmin/rollback_integration_test.go`
|
||||
|
||||
- [ ] Start an isolated PostgreSQL 18 fixture on a dedicated loopback port and database.
|
||||
- [ ] Apply migrations through the same migration runner used by the production Adapter.
|
||||
- [ ] Run the public adminstate contract against a unique schema per test.
|
||||
- [ ] Inject audit and Outbox constraint failures and prove state/revision rollback.
|
||||
- [ ] Query `information_schema` and prove no Proxy, extraction, ownership or idempotency
|
||||
- [x] Start an isolated PostgreSQL 18 fixture on a dedicated loopback port and database.
|
||||
- [x] Apply migrations through the same migration runner used by the production Adapter.
|
||||
- [x] Run the public adminstate contract against a unique schema per test.
|
||||
- [x] Inject audit and Outbox constraint failures and prove state/revision rollback.
|
||||
- [x] Query `information_schema` and prove no Proxy, extraction, ownership or idempotency
|
||||
detail tables/columns exist.
|
||||
- [ ] Clean only the unique test schema; do not drop shared databases or use broad cleanup.
|
||||
- [ ] Run `.\scripts\test-postgres.ps1` with every Go test timeout set to 60 seconds.
|
||||
- [ ] Commit with `test: add postgres admin state contract fixture`.
|
||||
- [x] Clean only the unique test schema; do not drop shared databases or use broad cleanup.
|
||||
- [x] Run `.\scripts\test-postgres.ps1` with every Go test timeout set to 60 seconds.
|
||||
- [x] Commit with `test: add postgres admin state contract fixture`.
|
||||
|
||||
### Task 6: Admin Application Integration
|
||||
|
||||
@ -139,11 +139,12 @@ contract; Controller Admin maps its existing typed commands onto this seam.
|
||||
- Modify: `docs/operations/runbook.md`
|
||||
- Modify: `progress.md`
|
||||
|
||||
- [ ] Mark only verified PostgreSQL capabilities complete and retain command/runtime/load gaps.
|
||||
- [ ] Document migration, backup, Outbox backlog/replay and data-boundary checks.
|
||||
- [ ] Run `.\scripts\verify.ps1`, `.\scripts\test-redis.ps1`,
|
||||
- [x] Mark only verified PostgreSQL capabilities complete and retain command/runtime/load gaps.
|
||||
- [x] Document migration, backup, Outbox backlog/replay and data-boundary checks.
|
||||
- [x] Run `.\scripts\verify.ps1`, `.\scripts\test-redis.ps1`,
|
||||
`.\scripts\test-postgres.ps1` and `git diff --check`.
|
||||
- [ ] Audit that Gateway has no PostgreSQL/Redis dependency and Distribution has no PostgreSQL
|
||||
- [x] Audit that Gateway has no PostgreSQL/Redis dependency and Distribution has no PostgreSQL
|
||||
dependency.
|
||||
- [ ] Confirm the user-owned deletion of `proxy-pool-docs-v1.0.zip` is not staged.
|
||||
- [ ] Commit with `docs: record postgres admin state delivery` and push the feature branch.
|
||||
- [x] Confirm the user-owned deletion of `proxy-pool-docs-v1.0.zip` is not staged.
|
||||
- [x] Commit with `docs: record postgres admin state delivery`.
|
||||
- [ ] Push the feature branch after working Git credentials are available.
|
||||
|
||||
@ -95,10 +95,18 @@ fixture 的执行命令是:
|
||||
`FLUSHDB`;本地 Redis 关闭 AOF、RDB 和数据卷,避免短效 Proxy 与凭据落盘。
|
||||
|
||||
PostgreSQL 管理面使用 `adminstate/contracttest` 作为 Memory/PostgreSQL 公用
|
||||
行为契约。当前 MemoryStore 已覆盖配置提交、Upstream 幂等、100 并发 Routing
|
||||
CAS、审计分页、Routing no-op、原子批量 ACK 和租约 Outbox;Schema 静态测试
|
||||
证明只声明六张管理表。PostgreSQL 18 的回环端口、tmpfs 隔离 fixture 已完成
|
||||
静态验证;只有 pgx Adapter 在该实例上运行同一契约后,才标记生产持久化完成。
|
||||
行为契约,执行命令是:
|
||||
|
||||
```powershell
|
||||
.\scripts\test-postgres.ps1
|
||||
```
|
||||
|
||||
pgx Adapter 已在真实 PostgreSQL 18 上覆盖配置事务、Upstream 幂等、100 并发
|
||||
Routing CAS、Repeatable Read 快照、审计分页、Routing no-op、`SKIP LOCKED`
|
||||
租约、原子批量 ACK,以及审计/Outbox 写入失败时的完整回滚。fixture 为每个测试
|
||||
创建唯一 Schema,只删除该 Schema;数据库使用回环端口和 tmpfs,测试后不保留
|
||||
数据卷。静态与 `information_schema` 双重检查证明只存在六张管理表,且没有
|
||||
Proxy、凭据、逐次提取、Worker ownership 或幂等明细列。
|
||||
|
||||
Admin 应用层测试覆盖 typed-nil 依赖、Actor/SourceIP 映射、Routing CAS 错误、
|
||||
权威管理快照与低基数运行态聚合、未知字段拒绝、主配置/Secret 文件 I/O 分类、
|
||||
|
||||
21
progress.md
21
progress.md
@ -1,5 +1,26 @@
|
||||
# 项目进度
|
||||
|
||||
## 2026-07-30
|
||||
|
||||
- 固定 `github.com/jackc/pgx/v5 v5.6.0`,实现封装在 `adminstate.Store` 后的
|
||||
PostgreSQL 深适配器;配置、Upstream、Routing mutation 在同一事务中提交
|
||||
revision、管理状态、审计与 Outbox,数据库错误不泄漏 DSN、SQL 或参数。
|
||||
- 实现只读 Repeatable Read Snapshot、审计分页、`FOR UPDATE SKIP LOCKED`
|
||||
Outbox 领取和整批 ACK;具体 Adapter、SQL、pgx 类型及 codec 不向业务层暴露。
|
||||
- 新增同一物理连接迁移执行器和 PostgreSQL 18 隔离 fixture;每个测试使用唯一
|
||||
Schema,回环端口和 tmpfs 不保留数据,CI 已加入 Redis/PostgreSQL 集成任务。
|
||||
- 真实 PostgreSQL 18 已通过完整公用契约:配置事务、Upstream 幂等、100 并发
|
||||
Routing CAS、Routing no-op、审计分页、Outbox 租约/重领、原子 ACK 和 Context。
|
||||
- 审计与 Outbox 触发器故障注入证明状态、revision、审计和事件完整回滚;迁移
|
||||
重复执行及 `information_schema` 检查确认仅有六张管理表,无 Proxy、凭据、
|
||||
逐次提取、Worker ownership 或幂等明细。
|
||||
- Docker Hub 直连因 Docker Desktop 未配置 HTTPS 代理超时,改从 Google 官方
|
||||
Docker Hub 公共缓存拉取相同 `postgres:18-alpine` 镜像并本地重标记;未修改
|
||||
Docker Desktop 全局镜像配置。
|
||||
- `implementation-plan.md` 中 PostgreSQL 两项验收已完成,总进度由 49/73 更新为
|
||||
51/73(69.9%);生产命令装配、分布式协调、Checker 和 100,000 QPS 集群压测
|
||||
仍未完成。
|
||||
|
||||
## 2026-07-29
|
||||
|
||||
- 新增公用 `verify-proto.ps1`:自动发现 Google well-known types include,固定
|
||||
|
||||
14
task_plan.md
14
task_plan.md
@ -28,9 +28,9 @@
|
||||
TTL 活动池契约;PostgreSQL 退出代理数据路径
|
||||
10. [已完成] 实现生产 Redis Activity Adapter、原子 Lua、公用行为契约和
|
||||
Redis 8.2 集成 fixture;本地 Redis 禁止短效代理数据持久化
|
||||
11. [进行中] 实现 PostgreSQL 管理面;ADR、领域事务契约、MemoryStore、公用
|
||||
契约、六表 Schema、Admin Actor 传播、应用服务和原子配置发布已完成,pgx
|
||||
Adapter/集成测试待完成
|
||||
11. [已完成] 实现 PostgreSQL 管理面;ADR、领域事务契约、MemoryStore、公用
|
||||
契约、六表 Schema、Admin Actor 传播、应用服务、原子配置发布、pgx Adapter
|
||||
和真实 PostgreSQL 18 集成测试已完成
|
||||
12. [进行中] 复核验收清单并收敛既有 Routing/Sequential 与 Proxy 容量边界;
|
||||
机器契约和文档类滞后勾选已按仓库证据校正
|
||||
|
||||
@ -51,9 +51,9 @@
|
||||
|
||||
## 已知环境限制
|
||||
|
||||
- Docker Compose 配置与 Kubernetes Kustomize 已完成静态渲染验证;未启动
|
||||
目标运行拓扑。
|
||||
- Docker Compose 配置与 Kubernetes Kustomize 已完成静态渲染验证;Redis 8.2
|
||||
与 PostgreSQL 18 的隔离 Adapter fixture 已运行,完整目标运行拓扑尚未启动。
|
||||
- `cmd/proxy-*`、PostgreSQL 管理面 Adapter、Provider Leader/分布式限流、
|
||||
Checker 运行时、Redis 故障转移验证与代表性集群压测属于后续实施范围。
|
||||
- `implementation-plan.md` 当前按 73 个验收项统计;已校正为 49 项完成,
|
||||
验收项完成率约 67%,不等同于生产就绪度。
|
||||
- `implementation-plan.md` 当前按 73 个验收项统计;已校正为 51 项完成,
|
||||
验收项完成率约 69.9%,不等同于生产就绪度。
|
||||
|
||||
Loading…
Reference in New Issue
Block a user