docs: record admin state foundation
This commit is contained in:
parent
beaf4c9c38
commit
7951c292d3
@ -33,6 +33,10 @@ type SnapshotReader interface {
|
||||
Snapshot(context.Context) (Snapshot, error)
|
||||
}
|
||||
|
||||
type AuditReader interface {
|
||||
ReadAudit(context.Context, AuditQuery) ([]AuditRecord, error)
|
||||
}
|
||||
|
||||
type Outbox interface {
|
||||
Claim(context.Context, ClaimCommand) ([]Event, error)
|
||||
Acknowledge(context.Context, AcknowledgeCommand) error
|
||||
|
||||
@ -38,7 +38,7 @@ PostgreSQL 只持久化配置版本、Upstream/Routing 管理状态、Admin 审
|
||||
|
||||
## ADR-005:Redis 活动池采用单实例原子深模块
|
||||
|
||||
**状态:** 接受。
|
||||
**状态:** 接受并已实现。
|
||||
|
||||
首版 Redis 活动池部署在单实例或 Sentinel 主节点,通过一个深 Adapter 统一实现
|
||||
Provider 入池、健康状态、Distribution 独占提取、Worker 所有权、库存读取和
|
||||
|
||||
@ -25,8 +25,10 @@ Admin API 使用独立监听器与权限,契约位于 `api/openapi/admin.yaml`
|
||||
不能混用配置格式版本或单 Worker Snapshot 版本。
|
||||
|
||||
严格 JSON、请求体上限、Request ID、JSON/Problem 响应由
|
||||
`platform/httpapi` 公用实现提供。Admin Handler 必须注入 `Authorizer`,标准
|
||||
装配使用 `httpsecurity.Protection`,并在路由匹配前完成保护。网关使用的
|
||||
`platform/httpapi` 公用实现提供。Admin Handler 必须注入 `IdentityResolver`,
|
||||
标准装配使用 `httpsecurity.Protection`,并在路由匹配前完成保护。解析出的
|
||||
Actor ID 与可信 SourceIP 会进入所有 mutation 命令,但认证材料不会进入审计。
|
||||
网关使用的
|
||||
`Proxy-Authorization`/407 语义不得复用到 Admin 的 `Authorization`/401 语义。
|
||||
`controller/runtime` 将 Admin 与 Distribution 放在不同 `net.Listener`,任一
|
||||
监听器异常会触发同组端点的有界优雅停机。
|
||||
|
||||
@ -158,9 +158,10 @@ test/{fixtures,integration,e2e,load}/
|
||||
**Files:** `internal/controller/distribution/*.go`, `admin/*.go`,
|
||||
`internal/adapters/postgres/*.go`, `internal/adapters/redis/*.go`, migrations, tests
|
||||
|
||||
- [ ] Define PostgreSQL ports for ConfigVersion, Upstream/Routing management state,
|
||||
AdminAudit, Outbox, and optional aggregate metrics; never persist Proxy details or
|
||||
per-extraction records.
|
||||
- [x] Define the PostgreSQL management seam for ConfigVersion, Upstream/Routing state,
|
||||
AdminAudit and leased Outbox; the public contract has no Proxy or extraction detail types.
|
||||
- [x] Add the six-table PostgreSQL management migration with static data-boundary checks.
|
||||
- [ ] Implement the pgx PostgreSQL Adapter and run the public contract against PostgreSQL 18.
|
||||
- [x] Implement the Redis TTL activity pool and one atomic extraction operation covering
|
||||
candidate eligibility, Gateway reserve, ownership, removal, and short-lived idempotency.
|
||||
- [x] Implement Redis Worker ownership, drain/ACK, expiry reclaim, inventory and bounded
|
||||
@ -181,8 +182,9 @@ extract/live/ready Handler 与 Admin status/enable/disable/switch/reload Handler
|
||||
Bearer/CIDR、可信代理、Client ID、本地准入和 API 401/Gateway 407 差异,并作为
|
||||
Admin/Distribution 必需依赖。共享 `platform/httpserver` 与
|
||||
`controller/runtime` 已完成 Distribution/Admin 独立监听器、首错联动关闭和
|
||||
有界优雅停机。生产命令入口、PostgreSQL 管理面 Adapter 及其 Compose 集成测试
|
||||
仍待实现。
|
||||
有界优雅停机。Admin mutation 已携带认证 Actor/SourceIP;公用 `adminstate`
|
||||
事务契约、MemoryStore、100 并发 Routing CAS、租约 Outbox 和六表管理 Schema
|
||||
已完成。生产命令入口、pgx Adapter 及 PostgreSQL 18 集成测试仍待实现。
|
||||
|
||||
已新增公用 `domain/activitypool` 契约及并发安全内存参考实现,Provider
|
||||
Reconciler 通过 `UpsertFetched` 写入带供应商 TTL 和分配安全余量的批次;已覆盖
|
||||
|
||||
@ -39,6 +39,12 @@
|
||||
- `DIST/Admin HTTP`:严格 JSON、Request ID、Problem 响应及 Distribution/Admin
|
||||
Handler 已实现;共享认证、CIDR、可信代理、Client ID 与本地准入保护链已接入,
|
||||
Controller Runtime 已将二者装配到独立监听器并支持联动优雅停机。
|
||||
- `Redis Activity Adapter`:真实 Redis 8.2 已覆盖 Provider Upsert、健康更新、
|
||||
原子独占提取、短期幂等、Worker ownership、库存和有界过期清理,Memory/Redis
|
||||
运行同一公用契约。
|
||||
- `PostgreSQL 管理面基础`:已定义 `adminstate` 事务 seam、并发安全 MemoryStore、
|
||||
公用契约、100 并发 Routing CAS、租约 Outbox 和只含六张管理表的 Schema;
|
||||
Admin Handler 已向 mutation 传播 Actor/SourceIP。
|
||||
|
||||
## 2. 已执行验证
|
||||
|
||||
@ -64,13 +70,11 @@ CI 已配置 Linux race job。Docker/Kubernetes 仅完成静态验证,没有
|
||||
1. `cmd/proxy-gateway/controller/checker/loadgen` 进程装配。
|
||||
2. Gateway 进程装配、生产连接池调优与代表性流量压测。
|
||||
3. Provider 分布式 singleflight/Leader、长期凭据回收和累计额度执行器。
|
||||
4. PostgreSQL 配置版本、Upstream/Routing 管理状态、Admin 审计/Outbox
|
||||
repository 和迁移;可选聚合指标不包含 Proxy 明细。
|
||||
5. Redis TTL 活动池、原子独占提取、短期幂等结果、Leader、速率限制、心跳与
|
||||
所有权适配器;活动池可由 Provider 重建。接线时必须验证进入
|
||||
`allocationSafetyMargin` 的不可分配条目不会长期占用补池额度,同时不得
|
||||
突破 `pool.maxSize` 硬上限。
|
||||
6. Worker ownership drain/ACK/过期回收和网络快照流。
|
||||
4. PostgreSQL pgx 管理面 Adapter、真实 PostgreSQL 18 公用契约、Admin 应用层
|
||||
接线和可选聚合指标;Schema/领域 seam/Memory 参考实现已经完成。
|
||||
5. Redis Provider Leader、分布式速率与 Client 限制、Worker 心跳和自动重建;
|
||||
TTL 活动池、原子提取和 Worker ownership 已完成。
|
||||
6. Worker 网络快照流;Redis ownership drain/ACK/过期回收已完成。
|
||||
7. Checker 调度、探测器和健康 reducer。
|
||||
8. Admin/Distribution 细粒度授权、分布式限流和审计查询。
|
||||
9. 真实 Compose/Kubernetes 集成、故障演练和代表性集群负载测试。
|
||||
|
||||
@ -85,5 +85,6 @@
|
||||
| SEC-002 | 非回环监听无保护时严格模式启动失败 | 8112-8441 | 配置校验测试 |
|
||||
| OPS-001 | 配置校验后构建不可变快照并原子替换 | 8959-8999 | 100k 索引、版本/epoch 与并发 Apply/Acquire 测试 |
|
||||
| OPS-002 | 优雅停机停止新请求/Fetch,等待现有流量后超时关闭 | 8981-9000 | Provider Run 收敛与 `Handler.Shutdown` HTTP 排空、Hijacked CONNECT 超时关闭测试 |
|
||||
| OPS-003 | PostgreSQL 只保存管理修订、Upstream/Routing 状态、Admin 审计与 Outbox | 当前会话 | ADR-006、`adminstate` 公用契约和六表 Schema 边界测试;真实 PostgreSQL 契约待完成 |
|
||||
| OBS-001 | 指标禁止 Proxy IP、session、Client、完整 URL 高基数标签 | 9001-9029 | 指标描述符测试 |
|
||||
| TEST-001 | 覆盖对话中列出的 11 个关键并发与故障场景 | 9030-9082 | 测试清单;Redis 活动池由 Memory/Redis 公用契约覆盖,跨进程故障场景仍按清单推进 |
|
||||
|
||||
@ -23,16 +23,16 @@ contract; Controller Admin maps its existing typed commands onto this seam.
|
||||
- Create: `internal/domain/adminstate/adminstate.go`
|
||||
- Create: `internal/domain/adminstate/validation_test.go`
|
||||
|
||||
- [ ] Define `Mutator`, `SnapshotReader` and `Outbox` interfaces from ADR-006.
|
||||
- [ ] Define typed config, Upstream, Routing, actor, audit, event and mutation values.
|
||||
- [ ] Define stable errors for invalid input, missing resources, CAS conflict and unavailable
|
||||
- [x] Define `Mutator`, `SnapshotReader`, `AuditReader` and `Outbox` interfaces from ADR-006.
|
||||
- [x] Define typed config, Upstream, Routing, actor, audit, event and mutation values.
|
||||
- [x] Define stable errors for invalid input, missing resources, CAS conflict and unavailable
|
||||
storage.
|
||||
- [ ] Validate non-empty bounded identifiers, UTC timestamps, unique lists, config checksum,
|
||||
- [x] Validate non-empty bounded identifiers, UTC timestamps, unique lists, config checksum,
|
||||
Routing references, claim limits and claim TTL.
|
||||
- [ ] Clone every slice/map/JSON value at the seam so callers cannot mutate stored state.
|
||||
- [ ] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate` and verify the tests
|
||||
- [x] Clone every slice/map/JSON value at the seam so callers cannot mutate stored state.
|
||||
- [x] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate` and verify the tests
|
||||
fail before implementation, then pass after implementation.
|
||||
- [ ] Commit with `feat: define admin state transaction contracts`.
|
||||
- [x] Commit with `feat: define admin state transaction contracts`.
|
||||
|
||||
### Task 2: Memory Reference Adapter and Shared Contract
|
||||
|
||||
@ -41,16 +41,16 @@ contract; Controller Admin maps its existing typed commands onto this seam.
|
||||
- Create: `internal/domain/adminstate/contracttest/contract.go`
|
||||
- Create: `internal/domain/adminstate/contract_external_test.go`
|
||||
|
||||
- [ ] Write a public contract factory that can create an isolated `Mutator + SnapshotReader +
|
||||
Outbox` implementation.
|
||||
- [ ] Cover config commit/replay/conflict/invalid references and zero-write rollback.
|
||||
- [ ] Cover Upstream idempotency, monotonic revisions, mandatory audit and changed-only events.
|
||||
- [ ] Cover Routing CAS and 100 concurrent switches with at most one success.
|
||||
- [ ] Cover bounded claim, exclusive claim, lease expiry, ACK ownership and stable ordering.
|
||||
- [ ] Cover context cancellation and immutable Snapshot/output values.
|
||||
- [ ] Implement MemoryStore behind the seam with one mutex per atomic management state.
|
||||
- [ ] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate/...`.
|
||||
- [ ] Commit with `feat: add transactional admin state reference store`.
|
||||
- [x] Write a public contract factory that can create an isolated `adminstate.Store`
|
||||
implementation over the four narrow capability interfaces.
|
||||
- [x] Cover config commit/replay/conflict/invalid references and zero-write rollback.
|
||||
- [x] Cover Upstream idempotency, monotonic revisions, mandatory audit and changed-only events.
|
||||
- [x] Cover Routing CAS and 100 concurrent switches with at most one success.
|
||||
- [x] Cover bounded claim, exclusive claim, lease expiry, ACK ownership and stable ordering.
|
||||
- [x] Cover context cancellation and immutable Snapshot/output values.
|
||||
- [x] Implement MemoryStore behind the seam with one mutex per atomic management state.
|
||||
- [x] Run `go test -count=1 -timeout 60s ./internal/domain/adminstate/...`.
|
||||
- [x] Commit with `feat: add transactional admin state reference store`.
|
||||
|
||||
### Task 3: Schema Migration and Static Data-Boundary Test
|
||||
|
||||
@ -59,14 +59,14 @@ contract; Controller Admin maps its existing typed commands onto this seam.
|
||||
- Create: `internal/adapters/postgresadmin/migrations.go`
|
||||
- Create: `internal/adapters/postgresadmin/migrations_test.go`
|
||||
|
||||
- [ ] Embed ordered migrations and expose one `Migrations() []Migration` read-only accessor.
|
||||
- [ ] Create only the six ADR-006 tables with primary/foreign keys, UTC timestamps, indexes,
|
||||
- [x] Embed ordered migrations and expose one `Migrations() []Migration` read-only accessor.
|
||||
- [x] Create only the six ADR-006 tables with primary/foreign keys, UTC timestamps, indexes,
|
||||
outbox claim fields and bounded checks.
|
||||
- [ ] Add a parser-backed/static test that asserts required tables/columns are present and
|
||||
- [x] Add a static structure test that asserts required tables/columns are present and
|
||||
forbidden Proxy/extraction/ownership/idempotency tables or columns are absent.
|
||||
- [ ] Test migration IDs are unique, strictly ordered and statements are transactional.
|
||||
- [ ] Run `go test -count=1 -timeout 60s ./internal/adapters/postgresadmin`.
|
||||
- [ ] Commit with `feat: add postgres admin state schema`.
|
||||
- [x] Test migration IDs are unique, strictly ordered and statements are transactional.
|
||||
- [x] Run `go test -count=1 -timeout 60s ./internal/adapters/postgresadmin`.
|
||||
- [x] Commit with `feat: add postgres admin state schema`.
|
||||
|
||||
### Task 4: PostgreSQL Deep Adapter
|
||||
|
||||
@ -118,7 +118,7 @@ contract; Controller Admin maps its existing typed commands onto this seam.
|
||||
- Modify: `internal/controller/admin/handler.go`
|
||||
- Modify: `internal/controller/admin/handler_test.go`
|
||||
|
||||
- [ ] Change Admin protection to resolve `httpsecurity.Identity` once and add actor/source IP
|
||||
- [x] Change Admin protection to resolve `httpsecurity.Identity` once and add actor/source IP
|
||||
to mutation commands without exposing credentials.
|
||||
- [ ] Map typed Handler commands to `adminstate.Mutator`; map domain conflict/not-found/
|
||||
invalid/unavailable errors to the existing HTTP contract.
|
||||
|
||||
@ -88,6 +88,11 @@ fixture 的执行命令是:
|
||||
100 轮并发提取和 100 轮所有权竞争。fixture 使用唯一命名空间,不执行
|
||||
`FLUSHDB`;本地 Redis 关闭 AOF、RDB 和数据卷,避免短效 Proxy 与凭据落盘。
|
||||
|
||||
PostgreSQL 管理面使用 `adminstate/contracttest` 作为 Memory/PostgreSQL 公用
|
||||
行为契约。当前 MemoryStore 已覆盖配置提交、Upstream 幂等、100 并发 Routing
|
||||
CAS、审计和租约 Outbox;Schema 静态测试证明只声明六张管理表。只有 pgx
|
||||
Adapter 在真实 PostgreSQL 18 上运行同一契约后,才标记生产持久化完成。
|
||||
|
||||
需要 PostgreSQL/Redis 的测试使用独立实例和短生命周期容器,不复用开发数据。
|
||||
测试结束后验证没有残留 Worker ownership、Leader 租约、活动池条目或幂等键,
|
||||
并检查 PostgreSQL 中不存在 Proxy 明细和逐次提取记录。
|
||||
|
||||
@ -20,6 +20,14 @@
|
||||
余量停止新分配。
|
||||
- PostgreSQL 管理面 Adapter、Provider Leader/分布式限流与心跳装配、生产命令
|
||||
入口、Redis 故障转移验证和代表性 100,000 QPS 集群压测仍待实现。
|
||||
- 已新增 ADR-006 与公用 `adminstate` 事务 seam;MemoryStore 在同一锁内提交管理
|
||||
状态、Admin 审计与 Outbox,覆盖配置冲突、Upstream 幂等、100 并发 Routing
|
||||
CAS、Outbox claim/ack 租约、上下文取消和不可变快照。
|
||||
- PostgreSQL 管理 Schema 已限制为六张配置/状态/审计/Outbox 表,并由静态测试
|
||||
拒绝 Proxy、凭据、逐次提取、ownership 和短期幂等明细;pgx Adapter 和真实
|
||||
PostgreSQL 18 契约仍待实现。
|
||||
- Admin Handler 现在复用 `httpsecurity.Identity`,把 Actor ID 与可信 SourceIP
|
||||
传给 enable/disable/switch/reload mutation,供持久化审计直接使用。
|
||||
- 本轮 `.\scripts\verify.ps1`、`.\scripts\test-redis.ps1`、Compose 静态展开和
|
||||
Compose 非持久化策略测试通过;Windows `CGO_ENABLED=0`,race 继续由 Linux
|
||||
CI 执行。
|
||||
|
||||
@ -28,6 +28,8 @@
|
||||
TTL 活动池契约;PostgreSQL 退出代理数据路径
|
||||
10. [已完成] 实现生产 Redis Activity Adapter、原子 Lua、公用行为契约和
|
||||
Redis 8.2 集成 fixture;本地 Redis 禁止短效代理数据持久化
|
||||
11. [进行中] 实现 PostgreSQL 管理面;ADR、领域事务契约、MemoryStore、公用
|
||||
契约、六表 Schema 和 Admin Actor 传播已完成,pgx Adapter/集成测试待完成
|
||||
|
||||
## 串并行关系
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user