proxy-pool/docs/development/implementation-plan.md

Proxy Pool Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development or superpowers:executing-plans. Every step is tracked with checkbox syntax and must preserve the requirement IDs in docs/requirements/traceability.md.

Goal: Build a production-oriented Go repository whose domain behavior, interfaces, configuration, contracts, documentation, and deployment layout implement the final semantics in 对话内容.md.

Architecture: Separate Gateway, Controller, Checker, and Loadgen commands. Domain packages remain transport-free. Gateway reads immutable local snapshots. Controller owns provider fetch, pool lifecycle, routing state, extraction, persistence, and worker distribution. PostgreSQL persists management state and Admin audit/outbox. Redis owns the rebuildable TTL Proxy activity pool and atomic extraction; Gateway hot paths remain memory-only.

Tech Stack: Go 1.26, go.yaml.in/yaml/v4, pgx/v5, go-redis/v9, gRPC/Protobuf, Prometheus, PostgreSQL, Redis, Docker Compose, Kubernetes.

---

File Structure

cmd/
  proxy-gateway/main.go
  proxy-controller/main.go
  proxy-checker/main.go
  proxy-loadgen/main.go
internal/
  config/{config.go,load.go,validate.go}
  domain/proxy/{proxy.go,state.go,capacity.go}
  domain/routing/{rule.go,strategy.go,sequential.go}
  domain/upstream/{upstream.go,fetch_result.go,pool.go}
  domain/extraction/{extraction.go,store.go}
  domain/client/client.go
  gateway/{server,dispatch,snapshot,transport}/
  controller/{provider,pool,routing,extraction,health,distribution,operations,runtime,bootstrap}/
  adapters/{memory,postgres,redis,providerapi}/
  platform/{logging,metrics,shutdown}/
api/{openapi,proto}/
configs/
deploy/{compose,kubernetes,haproxy,prometheus,grafana}/
docs/{design,development,configuration,api,operations,testing,adr,requirements}/
diagrams/
examples/
test/{fixtures,integration,e2e,load}/

Task 1: Repository and Build Baseline

Files: go.mod, .golangci.yml, README.md, scripts/verify.ps1, .github/workflows/ci.yml

• Create module github.com/proxy-pool/proxy-pool with Go 1.26.
• Pin YAML v4, pgx/v5, go-redis/v9, gRPC, protobuf, Prometheus, and x/sync.
• Add scripts/verify.ps1 that runs format check, go vet, unit tests, race tests, and builds all commands, each test command bounded to 60 seconds.
• Add CI for Windows and Linux with unit/race/build jobs.
• Verify go mod tidy, go test ./..., and go build ./cmd/... succeed.

Task 2: Strict Configuration

Files: internal/config/config.go, load.go, validate.go, corresponding tests, configs/default.yaml, docs/configuration/reference.md

• Define versioned types for security, gateway, distribution, admin, metrics, storage, routing, upstream/provider/api/proxyAuth/pool/capacity/lifecycle/fetch/check.
• Decode one YAML document with known fields enabled and resolve ${ENV} plus secret file references without logging values.
• Validate listener protection, routing references/order, regexes, strategy fields, positive limits, TTL margins, pool/fetch limits, auth modes, and exposure modes.
• Add table tests for every invalid condition in CFG requirements.

Task 3: Proxy Domain and Capacity

Files: internal/domain/proxy/*.go, corresponding tests

• Implement Proxy fields, UTC TTL precedence, canonical host/port, and unique key.
• Implement state transitions and reject illegal transitions.
• Implement sharded runtime counters with CAS Reserve, Commit, Cancel, Release.
• Prove with 1,000 concurrent goroutines that effective capacity is never exceeded.
• Add race coverage and duplicate-release invariant metrics hook.

当前进度（2026-07-29）：固定 Max 下的每 Proxy 打包 CAS、Cancel/Commit/Release 生命周期、重复终结、错误顺序和同一 Reservation 并发终结已通过领域测试；已退出 Proxy 会停止新预留、保留非零 Drain 计数，并在归零后由后续 Snapshot Apply 回收。 动态降容契约、低基数不变量指标和 Linux race 证据仍待完成。

Task 4: Routing and Sequential Switching

Files: internal/domain/routing/*.go, corresponding tests

• Compile first-match host/method/path/header rules into an immutable RuleSet.
• Implement random, round-robin, weighted, least-connections, and sequential.
• Model upstream empty counters separately from per-routing current indexes.
• Implement versioned CAS switch so simultaneous threshold observers advance once.
• Cover four-empty-then-success, five-empty, A-to-B-only, disabled references, end behavior, and explicit onUnavailable.

当前进度（2026-07-29）：领域构造器与严格配置已统一 Sequential 至少两个 Upstream、endBehavior 默认 stop，并覆盖列表末端停止；disabled candidate、 跨实例恢复和 onUnavailable 运行链仍待完成。

Task 5: Provider Fetch Classification and Scheduling

Files: internal/controller/provider/*.go, internal/domain/upstream/*.go, tests

• Implement Valid, Empty, DuplicateOnly, and Error result classes exactly as the traceability matrix defines.
• Implement one coalesced reconcile signal per Upstream using singleflight.
• Enforce requestInterval, maxInFlight, maxSize, maxTotal, timeout, retry, exponential backoff, jitter, and Retry-After.
• Define ProviderAdapter and safe TemplateParser ports; add fixture adapters.
• Test that 100 concurrent capacity signals do not fan out 100 Provider calls.

Task 6: Pool Reconciliation and Ownership

Files: internal/controller/pool/*.go, internal/domain/upstream/pool.go, tests

• Compute Available Slots from eligible Proxy capacity, Active, Reserved, TTL, health, ownership, pending expected fetch, and gateway reserve.
• Implement pool.maxSize and fetch.maxTotal as distinct counters.
• Allocate each Proxy to one Worker with epoch/version/expiry ownership.
• Implement revoke -> drain -> ACK -> unowned transition.
• Test Worker crash expiry and prevent simultaneous dual ownership.

Task 7: Exclusive Extraction

Files: internal/domain/extraction/*.go, internal/controller/extraction/*.go, internal/adapters/memory/extraction.go, tests

• Implement POST extraction command with protocol/region/carrier/upstream filters.
• Enforce minRemainingTTL, maxHealthCheckAge, maxCount, client limits, and reserveForGateway.
• Atomically remove selected AVAILABLE entries from the allocatable set and return the result; MemoryPool and the production Redis Adapter run the same shared contract.
• Implement partial and allOrNothing without Lease, release, or renewal concepts.
• Run 1,000 concurrent claim attempts and prove every Proxy ID appears at most once.

Task 8: Immutable Snapshot and Dispatch

Files: internal/gateway/snapshot/*.go, internal/gateway/dispatch/*.go, tests

• Define cluster/worker/epoch/version/checksum snapshot envelopes.
• Build indexes before publication and atomically swap complete snapshots.
• Reject version gaps and wrong epochs; request full resync.
• Implement Dispatch Acquire/Commit/Release over local owned Proxy runtime.
• Benchmark 100k Proxy snapshots and record allocations and latency.

Task 9: Gateway Transport

Files: internal/gateway/server/*.go, internal/gateway/transport/*.go, tests

• Implement HTTP forward proxy and HTTPS CONNECT through an upstream proxy.
• Add Client auth/access/admission and destination policy checks before routing.
• Implement safe retry commit points and prevent non-idempotent/established tunnel replay.
• Use bounded buffers, deadlines, connection pools, and graceful shutdown.
• Add local fake upstream end-to-end tests for success, 407, timeout, cancel, half close, retry, and blocked private destinations.

Task 10: Controller APIs and Persistence Ports

Files: internal/controller/distribution/*.go, admin/*.go, internal/adapters/postgres/*.go, internal/adapters/redis/*.go, migrations, tests

• Define the PostgreSQL management seam for ConfigVersion, Upstream/Routing state, AdminAudit and leased Outbox; the public contract has no Proxy or extraction detail types.
• Add the six-table PostgreSQL management migration with static data-boundary checks.
• Implement the pgx PostgreSQL Adapter and run the public contract against PostgreSQL 18.
• Implement the Redis TTL activity pool and one atomic extraction operation covering candidate eligibility, Gateway reserve, ownership, removal, and short-lived idempotency.
• Implement Redis Worker ownership, drain/ACK, expiry reclaim, inventory and bounded sweep primitives with a monotonic global epoch.
• Implement Redis Provider leader, distributed request quota, Client limit and automatic Provider inventory rebuild after Redis state loss.
• Implement the Worker heartbeat receiving path and session lifecycle.
• Keep Provider output in Redis TTL activity state and node memory only; keep the Gateway request path on immutable local snapshots with no Redis/PostgreSQL calls.
• Expose Distribution extraction/status and Admin status/enable/disable/switch/reload HTTP handlers and contracts.
• Add Compose-backed Redis 8.2 integration and shared Adapter contract tests.
• Add PostgreSQL management Adapter and Compose-backed integration tests.

当前进度（2026-07-31）：已实现共享 platform/httpapi、Distribution extract/live/ready Handler 与 Admin status/enable/disable/switch/reload Handler； 定向契约测试已覆盖严格 JSON、Body 上限、Request ID、幂等 Header、DTO 映射、 404/405 及业务错误映射。共享 platform/httpsecurity 已补齐 Basic/API Key/ Bearer/CIDR、可信代理、Client ID、本地准入和 API 401/Gateway 407 差异，并作为 Admin/Distribution 必需依赖。共享 platform/httpserver 与 controller/runtime 已完成 Distribution/Admin 独立监听器、首错联动关闭和 有界优雅停机。Admin mutation 已携带认证 Actor/SourceIP；公用 adminstate 事务契约、MemoryStore、100 并发 Routing CAS、租约 Outbox 和六表管理 Schema 已完成。pgx Adapter 已在真实 PostgreSQL 18 上运行同一公用契约，并验证 Repeatable Read 快照、SKIP LOCKED、原子 ACK、审计/Outbox 故障回滚和数据边界。 Admin ApplicationService 已将 mutation、权威管理快照、低基数运行态 聚合与配置重载接到同一公用 seam；严格文件加载、外部密钥 HMAC 管理指纹及 revision 单调配置发布已通过失败路径和确定性并发测试。cmd/proxy-controller 与公用 controller/bootstrap 已完成配置单次加载、PostgreSQL 连接/迁移、Redis 活动池、状态聚合、 Distribution/Admin 服务构造、错误合并和资源关闭。生产 Provider Supervisor 已按 权威管理状态动态装配 Upstream，并与 HTTP Runtime 通过公用 lifecycle Group 联动 停机；Admin disable 会取消 Runtime，reload 在提交前预检并在发布后替换运行实例。 组合 fixture 已验证隔离 Redis namespace 下的选主、Provider HTTP 调用、模板解析 和活动池写入。Controller Metrics 独立入口现已提供 /livez、/readyz 与基础 Prometheus 运行时指标，三监听器隔离已通过测试；业务指标仍待 实现。双存储 bootstrap 已通过 PostgreSQL 18 + Redis 8.2 组合 fixture，覆盖 迁移、启动配置提交、Readiness、Admin Status 和 Metrics 探针。

WorkerControlPlane 现已接入 Controller 生命周期：Register、ACK 和 Runtime 报告均经 Redis 服务端 TTL 的 session/issued-snapshot/ACK 栅栏校验；mTLS SPIFFE 身份、消息/流限制和有界停机已实现。WatchSnapshots 会发送当前 epoch 的基础完整 Snapshot 并保持连接；Gateway 已具备 Register/Watch/ACK/Runtime 会话协调组件与 proxy-gateway 进程装配。 按 Worker 的可下发 ownership 索引已进入 Redis 原子脚本，并可构建无凭据引用的 已归属 Proxy payload。Snapshot 签发与 session 匹配在同一 Redis Lua 操作中完成， 重注册会清除旧引用，避免迟到 Stream 覆盖新 session。Worker 服务端会在最近完整 Snapshot 的 valid_until 到达时结束流；公用 SessionSupervisor 已为 Gateway 调用方 提供可恢复错误的有界指数退避重连，并在参数/认证/协议错误时停止。Gateway Routing payload 已按配置顺序和 Admin revision/current 状态发布并覆盖 checksum；Gateway 已将其与 Proxy 原子编译为同版本 View，动态 Router 只匹配该未过期 View。派发器的五种上游选择已 接入该 View，并在容量耗尽时在同版本候选中回退；onUnavailable 的 reject、wait 与 direct 已接入 Gateway；proxy-gateway 已装配本地 HTTP/Metrics 监听、快照就绪探针和 控制面重连，凭据分发与 Outcome 上报仍未实现。

已新增公用 domain/activitypool 契约及并发安全内存参考实现，Provider Reconciler 通过 UpsertFetched 写入带供应商 TTL 和分配安全余量的批次；已覆盖 usableUntil 向 Worker Snapshot 的传播与 Gateway 本地截止过滤、 重复刷新、过期淘汰、独占提取、短期幂等及 Worker ownership 互斥。生产 Redis Adapter 已通过真实 Redis 8.2 运行同一套公用契约；原子 Lua 覆盖提取、所有权和 有界清理。新增低基数 StateInventory Hash，五类写脚本在同一原子边界维护状态 计数，读取不扫描 Proxy 明细；过期清理积压或负计数时 fail-closed。Redis Sentinel/故障转移验证与代表性多节点压测仍待实施。

Provider 分布式协调已新增公用 Coordinator.RunLeader / LeaderSession seam 与 独立 redisprovider Adapter。真实 Redis 8.2 已验证同 Upstream 双实例互斥、 generation + epoch fence、全局 requestInterval、全局 maxInFlight Permit、TTL 回收及 Redis 状态丢失后的新 generation 自动重建；Redis 异常期间不发放请求。 补池配置新增必填 refill 双水位和 fetch.estimatedIPsPerCall，Pool Reconciler 已实现迟滞与 pending 槽位折算，FetchBudget 仅在无 pending 时同步 Redis 权威 Managed。Gateway 已增加打包原子 Active/Reserved 读取与完整稀疏运行态快照； 公用 workerruntime session/report/read seam 同时提供并发安全内存参考实现和 生产 Redis Adapter。Redis 以服务端时间、Worker session、已 ACK snapshot/epoch、 单调 report sequence 和报告 TTL 原子隔离旧实例，并由 pool.InventoryReader 汇总 权威 Managed/Available Slots；真实 Redis 8.2 已覆盖空报告、幂等重放、倒序、 冲突、超前 epoch、过期、单 Upstream 扫描隔离和预算耗尽的 fail-closed 行为。 Redis Provider Permit 现已把 requestInterval、maxInFlight 与 maxTotal 放在同一 原子边界，按 expected 预留、实际合法数量结算，并支持失败保守计费、换主后结算和 过期回收。响应型代理凭据使用独立、有界 lease，在 Redis Upsert、候选截断或解析 失败后按版本释放；Provider Empty/Error 低基数计数已接入 Admin Status，配置删除 时回收历史统计容量。Redis inventory 扫描上限固定覆盖配置允许的最大池，支持小池 启动后动态扩容。Supervisor 以 PostgreSQL 权威 HMAC 指纹和 revision 栅栏协调 多副本 reload：管理库瞬断沿用 last-known 状态，本地共享源落后时停止旧 Provider， 源匹配并预检后自动替换，迟到旧 revision 不覆盖新配置。 Distribution 现通过公用 admission.Admitter 接入独立 redisadmission Adapter； 全局和单 Client 分钟额度使用 Redis 服务端时间并在单个 Lua 原子边界内检查、递增， Controller 多副本共享同一计数。Client 身份只以 SHA-256 摘要进入 Redis，窗口切换 原子回收历史字段；Redis 异常 fail-closed 并返回 503，真实额度耗尽返回 429。 Gateway 请求热路径仍只使用本地准入，不增加 Redis/PostgreSQL 调用。 WorkerControlPlane gRPC 接收端、session 签发/心跳、Snapshot ACK 账本、基础 Snapshot 流和 Gateway 会话客户端已完成；权威 Proxy/Routing 发布、凭据分发、 Outcome 和健康执行链仍待完成，因此 Task 10 尚未全部完成。

Task 11: Checker and Health Reducer

Files: internal/controller/health/*.go, cmd/proxy-checker/main.go, tests

• Schedule global and route health with jitter and bounded maxInFlight.
• Implement FETCHED -> CHECKING -> AVAILABLE and SUSPECT/UNHEALTHY transitions.
• Ensure target failures affect only the target profile.
• Add fixture target server and deterministic clock/scheduler tests.

Task 12: Machine-readable Contracts

Files: api/openapi/proxy-pool.yaml, api/proto/controlplane/v1/controlplane.proto, docs/api/*.md

• Specify Distribution/Admin REST schemas, status codes, authentication, examples, and idempotency behavior.
• Specify Worker register, snapshot, delta, ACK, report, heartbeat, ownership drain, and resync messages.
• Validate OpenAPI and compile protobuf descriptors in CI.

当前进度（2026-07-29）：Distribution/Admin OpenAPI 已由 Go 测试在双平台 CI 校验本地 $ref 闭合、operationId 唯一、响应存在及 security scheme 引用； scripts/verify-proto.ps1 已可复现编译包含 imports/source info 的 descriptor， 并在本地存在 protoc 时进入完整验证；完整 OAS 工具验证与 CI 强制安装/执行 protoc 仍待完成。

Task 13: Deployment and Observability

Files: deploy/**, internal/platform/**, docs/operations/**

• Add Compose for local Controller/Gateway/Checker/PostgreSQL/Redis/Prometheus/ Grafana/HAProxy.
• Add Kubernetes Deployments, Services, PDBs, HPA, NetworkPolicy, Secrets examples, probes, resource limits, topology spread, and graceful termination.
• Add low-cardinality Prometheus metrics and structured secret-safe logs.
• Document backup, recovery, rollout, rollback, capacity, kernel, file descriptor, NAT/conntrack, and incident runbooks.

Task 14: Documentation, Examples, and Diagrams

Files: docs/**, examples/**, diagrams/**

• Complete README navigation, design document, developer guide, configuration reference, API guide, deployment guide, security model, testing guide, and roadmap.
• Provide at least 20 validated configuration examples.
• Provide at least 30 Mermaid architecture, flow, sequence, state, and failure diagrams.
• Generate proxy-pool-docs-v1.0.zip from versioned documentation assets.

Task 15: Completion Audit

• Map every requirement ID to code, test, contract, document, or verified runtime evidence.
• Run gofmt, go vet, unit tests, race tests, builds, contract validation, and documentation link/example validation.
• Run bounded local performance benchmarks; label 100k QPS as unverified until a representative cluster load run exists.
• Confirm no TODO/TBD/placeholders, secrets, unbounded queues, high-cardinality metric labels, extraction Lease APIs, or conflicting maxSize semantics remain.