docs: plan redis activity pool implementation
This commit is contained in:
parent
2fdc37e8d7
commit
179e4d5a0c
761
docs/superpowers/plans/2026-07-29-redis-activity-pool.md
Normal file
761
docs/superpowers/plans/2026-07-29-redis-activity-pool.md
Normal file
@ -0,0 +1,761 @@
|
||||
# Redis Activity Pool Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build the production Redis activity pool that atomically owns short-lived Proxy inventory, health availability, exclusive extraction, Worker ownership, bounded idempotency, and expiry cleanup without putting Redis on the Gateway request path.
|
||||
|
||||
**Architecture:** One deep `redisactivity.Adapter` implements the existing narrow domain ports plus health, inventory, and maintenance ports. The first release targets Redis standalone/Sentinel; all keys share `{activity}` for future single-slot Cluster compatibility, and bounded Lua scripts serialize every state transition that can race.
|
||||
|
||||
**Tech Stack:** Go 1.26, `github.com/redis/go-redis/v9`, embedded Redis Lua scripts, Redis 8.2 integration tests, Docker Compose, PowerShell verification.
|
||||
|
||||
---
|
||||
|
||||
## Public Test Seams
|
||||
|
||||
The approved public seams are:
|
||||
|
||||
- `activitypool.Upserter.UpsertFetched`
|
||||
- `activitypool.HealthStore.ApplyHealth`
|
||||
- `activitypool.InventoryReader.Inventory`
|
||||
- `extraction.Store.Extract`
|
||||
- `ownership.Repository` context-aware methods
|
||||
- `activitypool.Maintainer.SweepExpired`
|
||||
|
||||
Tests assert behavior only through these seams. Lua source, Redis key contents, codec fields,
|
||||
script SHA values, and private runner calls are not test seams.
|
||||
|
||||
## File Map
|
||||
|
||||
- Modify `internal/domain/activitypool/pool.go`: public activity contracts and memory reference implementation.
|
||||
- Modify `internal/domain/activitypool/pool_test.go`: shared behavioral expectations for max size, health, inventory, and maintenance.
|
||||
- Modify `internal/domain/ownership/ownership.go`: context-aware production repository contract.
|
||||
- Modify `internal/controller/pool/ownership.go`: pass contexts and return storage failures.
|
||||
- Modify `internal/controller/pool/ownership_test.go`: public manager contract after the signature migration.
|
||||
- Modify `internal/controller/provider/reconciler.go`: pass `pool.maxSize` into authoritative Upsert.
|
||||
- Modify `internal/controller/provider/reconciler_test.go`: verify max-size propagation.
|
||||
- Modify `internal/controller/extraction/service.go`: classify Redis availability failures as 503-safe service errors.
|
||||
- Modify `internal/controller/extraction/service_test.go`: verify error classification.
|
||||
- Create `internal/adapters/redisactivity/adapter.go`: constructor, dependencies, options, interface assertions.
|
||||
- Create `internal/adapters/redisactivity/keys.go`: normalized key construction and hash tag enforcement.
|
||||
- Create `internal/adapters/redisactivity/codec.go`: Proxy, ownership, result, and script reply encoding.
|
||||
- Create `internal/adapters/redisactivity/scripts.go`: embedded script declarations.
|
||||
- Create `internal/adapters/redisactivity/upsert.go`: credential materialization and bounded Upsert calls.
|
||||
- Create `internal/adapters/redisactivity/health.go`: health transition adapter.
|
||||
- Create `internal/adapters/redisactivity/extract.go`: extraction command digest, reply mapping, and URL construction.
|
||||
- Create `internal/adapters/redisactivity/ownership.go`: context-aware ownership methods.
|
||||
- Create `internal/adapters/redisactivity/maintenance.go`: bounded expiry cleanup and inventory reads.
|
||||
- Create `internal/adapters/redisactivity/scripts/*.lua`: atomic Redis state transitions.
|
||||
- Create `internal/adapters/redisactivity/contract_test.go`: real Redis public-seam contract suite.
|
||||
- Create `internal/adapters/redisactivity/testredis_test.go`: isolated integration client and namespace helpers.
|
||||
- Create `deploy/docker-compose.test.yml`: non-persistent Redis 8.2 test fixture.
|
||||
- Create `deploy/compose_test.go`: structured local Redis persistence-policy assertion.
|
||||
- Create `scripts/test-redis.ps1`: bounded integration-test runner.
|
||||
- Modify `deploy/docker-compose.yml`: make local runtime Redis explicitly non-persistent.
|
||||
- Modify `docs/development/implementation-plan.md`: record the delivered Adapter boundary.
|
||||
- Modify `docs/testing/test-strategy.md`: record Redis contract and failure tests.
|
||||
|
||||
### Task 1: Make Ownership Storage Context-Aware
|
||||
|
||||
**Files:**
|
||||
- Modify: `internal/domain/ownership/ownership.go`
|
||||
- Modify: `internal/controller/pool/ownership.go`
|
||||
- Modify: `internal/controller/pool/ownership_test.go`
|
||||
- Modify: `internal/domain/activitypool/pool.go`
|
||||
- Modify: `internal/domain/activitypool/pool_test.go`
|
||||
|
||||
- [ ] **Step 1: Write failing manager tests for cancellation, storage errors, and bounded expiry**
|
||||
|
||||
Add a recording repository implementing the new intended contract and tests with these calls:
|
||||
|
||||
```go
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
|
||||
if _, err := manager.Assign(ctx, now, "proxy-1", "worker-1", time.Minute); !errors.Is(err, context.Canceled) {
|
||||
t.Fatalf("Assign() error = %v, want context.Canceled", err)
|
||||
}
|
||||
|
||||
expired, err := manager.Expire(context.Background(), now, 32)
|
||||
if err != nil {
|
||||
t.Fatalf("Expire(): %v", err)
|
||||
}
|
||||
if repository.expireLimit != 32 || len(expired) != 1 {
|
||||
t.Fatalf("Expire() = %+v, limit=%d", expired, repository.expireLimit)
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the focused tests and verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
go test -timeout 60s ./internal/controller/pool ./internal/domain/activitypool
|
||||
```
|
||||
|
||||
Expected: compilation fails because the current manager methods do not accept a context and
|
||||
`Expire` has neither a limit nor an error result.
|
||||
|
||||
- [ ] **Step 3: Replace the ownership port and manager signatures**
|
||||
|
||||
Use this exact repository contract:
|
||||
|
||||
```go
|
||||
type Repository interface {
|
||||
Assign(context.Context, time.Time, string, string, time.Duration) (Assignment, error)
|
||||
Renew(context.Context, time.Time, string, string, uint64, time.Duration) (Assignment, error)
|
||||
BeginDrain(context.Context, string, string, uint64) (Assignment, error)
|
||||
AcknowledgeDrain(context.Context, string, string, uint64, int64, int64) error
|
||||
Get(context.Context, string) (Assignment, bool, error)
|
||||
Expire(context.Context, time.Time, int) ([]Assignment, error)
|
||||
}
|
||||
```
|
||||
|
||||
Update `OwnershipManager` to validate `ctx != nil`, pass the context unchanged, require
|
||||
`limit > 0`, and return repository errors without swallowing them. Update `MemoryPool` to
|
||||
check `ctx.Err()` before and after locking, return `(Assignment, bool, error)` from `Get`,
|
||||
and stop `Expire` after `limit` assignments.
|
||||
|
||||
- [ ] **Step 4: Update all ownership call sites and run GREEN**
|
||||
|
||||
Pass `context.Background()` from existing tests that do not exercise cancellation. Run:
|
||||
|
||||
```powershell
|
||||
go test -timeout 60s ./internal/controller/pool ./internal/domain/activitypool
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit the interface migration**
|
||||
|
||||
```powershell
|
||||
git add internal/domain/ownership internal/domain/activitypool internal/controller/pool
|
||||
git commit -m "refactor: make ownership repository context aware"
|
||||
```
|
||||
|
||||
### Task 2: Add Health, Inventory, MaxSize, and Maintenance Contracts
|
||||
|
||||
**Files:**
|
||||
- Modify: `internal/domain/activitypool/pool.go`
|
||||
- Modify: `internal/domain/activitypool/pool_test.go`
|
||||
- Modify: `internal/controller/provider/reconciler.go`
|
||||
- Modify: `internal/controller/provider/reconciler_test.go`
|
||||
|
||||
- [ ] **Step 1: Write failing public-seam tests**
|
||||
|
||||
Add tests covering a Fetched -> Checking -> Available transition, stale health rejection,
|
||||
per-upstream MaxSize, current inventory, and bounded expiry:
|
||||
|
||||
```go
|
||||
updated, err := pool.ApplyHealth(context.Background(), HealthUpdate{
|
||||
ProxyID: proxyID, CheckedAt: now.Add(time.Second), NextState: proxyDomain.StateChecking,
|
||||
})
|
||||
if err != nil || updated.State != proxyDomain.StateChecking {
|
||||
t.Fatalf("ApplyHealth(checking) = %+v, %v", updated, err)
|
||||
}
|
||||
updated, err = pool.ApplyHealth(context.Background(), HealthUpdate{
|
||||
ProxyID: proxyID, CheckedAt: now.Add(2 * time.Second),
|
||||
NextState: proxyDomain.StateAvailable, Latency: 25 * time.Millisecond,
|
||||
})
|
||||
if err != nil || updated.State != proxyDomain.StateAvailable {
|
||||
t.Fatalf("ApplyHealth(available) = %+v, %v", updated, err)
|
||||
}
|
||||
|
||||
inventory, err := pool.Inventory(context.Background(), "provider-a", now.Add(2*time.Second))
|
||||
if err != nil || inventory.Managed != 1 {
|
||||
t.Fatalf("Inventory() = %+v, %v", inventory, err)
|
||||
}
|
||||
```
|
||||
|
||||
Insert two distinct candidates with `FetchedBatch.MaxSize = 1` and assert one insert plus one
|
||||
capacity drop. Call `SweepExpired(ctx, afterExpiry, 1)` twice and assert each call removes at
|
||||
most one record.
|
||||
|
||||
- [ ] **Step 2: Run the tests and verify RED**
|
||||
|
||||
```powershell
|
||||
go test -timeout 60s ./internal/domain/activitypool ./internal/controller/provider
|
||||
```
|
||||
|
||||
Expected: compilation fails because the new ports and `MaxSize` do not exist.
|
||||
|
||||
- [ ] **Step 3: Add the approved activity contracts**
|
||||
|
||||
Add these public types:
|
||||
|
||||
```go
|
||||
type HealthUpdate struct {
|
||||
ProxyID string
|
||||
CheckedAt time.Time
|
||||
NextState proxyDomain.State
|
||||
Latency time.Duration
|
||||
}
|
||||
|
||||
type Inventory struct {
|
||||
UpstreamID string
|
||||
Managed int
|
||||
}
|
||||
|
||||
type HealthStore interface {
|
||||
ApplyHealth(context.Context, HealthUpdate) (Entry, error)
|
||||
}
|
||||
|
||||
type InventoryReader interface {
|
||||
Inventory(context.Context, string, time.Time) (Inventory, error)
|
||||
}
|
||||
|
||||
type Maintainer interface {
|
||||
SweepExpired(context.Context, time.Time, int) (int, error)
|
||||
}
|
||||
```
|
||||
|
||||
Add `MaxSize int` to `FetchedBatch`. Reject non-positive MaxSize as an invalid batch. Count
|
||||
only managed states belonging to the incumbent upstream. Capacity-rejected candidates count
|
||||
as `Dropped` and never create unique-key mappings.
|
||||
|
||||
`ApplyHealth` must reject missing IDs, zero time, negative latency, missing entries, invalid
|
||||
state transitions, and observations older than `LastCheckedAt`. Replaying the same state and
|
||||
timestamp is idempotent. A transition to AVAILABLE updates `LastSuccessAt`.
|
||||
|
||||
- [ ] **Step 4: Pass MaxSize from Provider Reconciler**
|
||||
|
||||
Add `MaxSize int` to `provider.Config`, require it to be positive, and populate:
|
||||
|
||||
```go
|
||||
activitypool.FetchedBatch{
|
||||
ObservedAt: r.runtime.Clock.Now().UTC(),
|
||||
ConfiguredTTL: r.config.TTL,
|
||||
AllocationSafetyMargin: r.config.AllocationSafetyMargin,
|
||||
MaxSize: r.config.MaxSize,
|
||||
Proxies: retained,
|
||||
}
|
||||
```
|
||||
|
||||
Update constructor tests to use a positive MaxSize and assert the activity sink receives it.
|
||||
|
||||
- [ ] **Step 5: Run GREEN and commit**
|
||||
|
||||
```powershell
|
||||
go test -timeout 60s ./internal/domain/activitypool ./internal/controller/provider ./internal/controller/pool
|
||||
git add internal/domain/activitypool internal/controller/provider
|
||||
git commit -m "feat: add activity health and inventory contracts"
|
||||
```
|
||||
|
||||
Expected: PASS, then a commit containing only this slice.
|
||||
|
||||
### Task 3: Classify Activity Store Availability Failures
|
||||
|
||||
**Files:**
|
||||
- Modify: `internal/domain/extraction/extraction.go`
|
||||
- Modify: `internal/controller/extraction/service.go`
|
||||
- Modify: `internal/controller/extraction/service_test.go`
|
||||
- Modify: `internal/controller/distribution/handler_test.go`
|
||||
|
||||
- [ ] **Step 1: Write failing service and HTTP error tests**
|
||||
|
||||
Configure a recording Store to return `domain.ErrStoreUnavailable`. Assert:
|
||||
|
||||
```go
|
||||
_, err := service.Extract(context.Background(), validRequest)
|
||||
if !errors.Is(err, ErrUnavailable) || !errors.Is(err, domain.ErrStoreUnavailable) {
|
||||
t.Fatalf("Extract() error = %v, want unavailable classification", err)
|
||||
}
|
||||
```
|
||||
|
||||
At the Handler seam, assert the same error becomes HTTP 503 with code
|
||||
`SERVICE_UNAVAILABLE` and no underlying Redis text in the response body.
|
||||
|
||||
- [ ] **Step 2: Run RED**
|
||||
|
||||
```powershell
|
||||
go test -timeout 60s ./internal/controller/extraction ./internal/controller/distribution
|
||||
```
|
||||
|
||||
Expected: compilation fails because `ErrStoreUnavailable` is not defined.
|
||||
|
||||
- [ ] **Step 3: Add the stable storage error and mapping**
|
||||
|
||||
Add:
|
||||
|
||||
```go
|
||||
var ErrStoreUnavailable = errors.New("extraction store unavailable")
|
||||
```
|
||||
|
||||
In `Service.Extract`, preserve `ErrInsufficientProxies`, `ErrIdempotencyConflict`, and
|
||||
`ErrInvalidCommand`; wrap any error matching `ErrStoreUnavailable` with `ErrUnavailable`:
|
||||
|
||||
```go
|
||||
if err != nil {
|
||||
if errors.Is(err, domain.ErrStoreUnavailable) {
|
||||
return response, errors.Join(ErrUnavailable, err)
|
||||
}
|
||||
return response, err
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run GREEN and commit**
|
||||
|
||||
```powershell
|
||||
go test -timeout 60s ./internal/controller/extraction ./internal/controller/distribution
|
||||
git add internal/domain/extraction internal/controller/extraction internal/controller/distribution
|
||||
git commit -m "feat: classify extraction store failures"
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 4: Add Redis Adapter Foundation
|
||||
|
||||
**Files:**
|
||||
- Modify: `go.mod`
|
||||
- Modify: `go.sum`
|
||||
- Create: `internal/adapters/redisactivity/adapter.go`
|
||||
- Create: `internal/adapters/redisactivity/keys.go`
|
||||
- Create: `internal/adapters/redisactivity/codec.go`
|
||||
- Create: `internal/adapters/redisactivity/scripts.go`
|
||||
- Create: `internal/adapters/redisactivity/adapter_test.go`
|
||||
- Create: `internal/adapters/redisactivity/testredis_test.go`
|
||||
- Create: `deploy/docker-compose.test.yml`
|
||||
- Create: `scripts/test-redis.ps1`
|
||||
|
||||
- [ ] **Step 1: Add failing constructor and key-safety tests**
|
||||
|
||||
Test that nil clients, nil credential stores, empty namespaces, braces in namespaces, zero
|
||||
operation TTL, and non-positive scan/cleanup limits fail. Test that valid options build keys
|
||||
with exactly one fixed hash tag:
|
||||
|
||||
```go
|
||||
adapter, err := New(client, Options{
|
||||
Namespace: "test-a", Credentials: credentialStore,
|
||||
OperationTTL: time.Minute, MaxCandidateScan: 2048, CleanupLimit: 128,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("New(): %v", err)
|
||||
}
|
||||
if got := adapter.keys.records; got != "pp:{activity}:test-a:records" {
|
||||
t.Fatalf("records key = %q", got)
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run RED**
|
||||
|
||||
```powershell
|
||||
go test -timeout 60s ./internal/adapters/redisactivity
|
||||
```
|
||||
|
||||
Expected: package does not exist.
|
||||
|
||||
- [ ] **Step 3: Pin go-redis and implement the narrow constructor**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
go get github.com/redis/go-redis/v9@v9.19.0
|
||||
```
|
||||
|
||||
Use `redis.Scripter` rather than exposing `redis.UniversalClient` throughout the package:
|
||||
|
||||
```go
|
||||
type Options struct {
|
||||
Namespace string
|
||||
Credentials credentials.Store
|
||||
OperationTTL time.Duration
|
||||
MaxCandidateScan int
|
||||
CleanupLimit int
|
||||
}
|
||||
|
||||
type Adapter struct {
|
||||
client redis.Scripter
|
||||
credentials credentials.Store
|
||||
keys keyspace
|
||||
options Options
|
||||
}
|
||||
|
||||
func New(client redis.Scripter, options Options) (*Adapter, error)
|
||||
```
|
||||
|
||||
Embed immutable scripts once and reuse `redis.Script`, which provides EVALSHA with EVAL
|
||||
fallback when the server script cache is empty:
|
||||
|
||||
```go
|
||||
//go:embed scripts/upsert.lua
|
||||
var upsertSource string
|
||||
|
||||
var upsertScript = redis.NewScript(upsertSource)
|
||||
```
|
||||
|
||||
Normalize the namespace with a strict `[A-Za-z0-9._-]+` policy. Hash user-controlled
|
||||
Client IDs, idempotency keys, proxy unique keys, and operation IDs with SHA-256 before using
|
||||
them in Redis key names or fields.
|
||||
|
||||
- [ ] **Step 4: Implement deterministic codecs with redacted formatting**
|
||||
|
||||
Use JSON only at the Redis boundary. Define private `proxyRecord`, `ownershipRecord`,
|
||||
`idempotencyRecord`, and typed script reply structures. Store times as Unix milliseconds and
|
||||
durations as integer nanoseconds. Decode with strict state and integer validation. Any
|
||||
formatting method for structures containing passwords writes `<redacted>`.
|
||||
|
||||
- [ ] **Step 5: Run GREEN and commit**
|
||||
|
||||
Create the Redis 8.2 test fixture before the first integration slice. It binds only
|
||||
`127.0.0.1:16379`, has no data volume, runs `--appendonly no --save ""`, and exposes a PING
|
||||
health check. Add `testredis_test.go` that skips integration-tagged tests when
|
||||
`PROXY_POOL_TEST_REDIS_URL` is absent and otherwise creates a unique namespace without
|
||||
flushing shared databases. Add `scripts/test-redis.ps1` with `try/finally` fixture shutdown.
|
||||
|
||||
```powershell
|
||||
gofmt -w internal/adapters/redisactivity
|
||||
go test -timeout 60s ./internal/adapters/redisactivity
|
||||
git add go.mod go.sum internal/adapters/redisactivity deploy/docker-compose.test.yml scripts/test-redis.ps1
|
||||
git commit -m "feat: add redis activity adapter foundation"
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 5: Implement Atomic Upsert and Health Transitions
|
||||
|
||||
**Files:**
|
||||
- Create: `internal/adapters/redisactivity/upsert.go`
|
||||
- Create: `internal/adapters/redisactivity/health.go`
|
||||
- Create: `internal/adapters/redisactivity/scripts/upsert.lua`
|
||||
- Create: `internal/adapters/redisactivity/scripts/health.lua`
|
||||
- Create: `internal/adapters/redisactivity/upsert_integration_test.go`
|
||||
|
||||
- [ ] **Step 1: Write failing real-Redis tests at the approved seams**
|
||||
|
||||
Put `//go:build integration` at the top of every real-Redis test file and use a unique
|
||||
namespace per test. Cover:
|
||||
|
||||
```go
|
||||
result, err := store.UpsertFetched(ctx, "provider-a", activitypool.FetchedBatch{
|
||||
ObservedAt: now, ConfiguredTTL: 30 * time.Second,
|
||||
AllocationSafetyMargin: 3 * time.Second, MaxSize: 1,
|
||||
Proxies: []proxyDomain.Proxy{first, second},
|
||||
})
|
||||
if err != nil || result.Inserted != 1 || result.Dropped != 1 {
|
||||
t.Fatalf("UpsertFetched() = %+v, %v", result, err)
|
||||
}
|
||||
```
|
||||
|
||||
Also verify cross-provider incumbent preservation, refresh preserving runtime health, expired
|
||||
incumbent replacement, credential resolution before commit, Fetched -> Checking -> Available,
|
||||
and stale health observations not replacing newer state.
|
||||
|
||||
- [ ] **Step 2: Start Redis and verify RED**
|
||||
|
||||
```powershell
|
||||
docker compose -f deploy/docker-compose.test.yml up -d --wait
|
||||
$env:PROXY_POOL_TEST_REDIS_URL='redis://127.0.0.1:16379/15'
|
||||
go test -tags=integration -timeout 60s ./internal/adapters/redisactivity -run 'TestRedis.*(Upsert|Health)'
|
||||
```
|
||||
|
||||
Expected: tests fail because Upsert and Health return no production behavior.
|
||||
|
||||
- [ ] **Step 3: Implement bounded Upsert**
|
||||
|
||||
Resolve every non-empty credential reference before running Lua:
|
||||
|
||||
```go
|
||||
value, err := a.credentials.Resolve(ctx, credentials.Reference{
|
||||
SecretRef: candidate.SecretRef, CredentialVersion: candidate.CredentialVersion,
|
||||
})
|
||||
if err != nil {
|
||||
return activitypool.UpsertResult{}, fmt.Errorf("resolve proxy credential: %w", err)
|
||||
}
|
||||
```
|
||||
|
||||
The script receives validated records and atomically maintains `records`, `unique`, `idkeys`,
|
||||
`expiry`, `available`, facet indexes, and `inventory`. It performs a cleanup batch first,
|
||||
preserves incumbent upstream and runtime state, refuses EXTRACTED refresh, enforces MaxSize,
|
||||
and stores an operation result keyed by one generated operation ID so EVALSHA retry returns
|
||||
the original counters.
|
||||
|
||||
- [ ] **Step 4: Implement Health state changes**
|
||||
|
||||
The health script loads one record, rejects missing/expired entries, rejects invalid or stale
|
||||
transitions, updates health fields, and adds or removes every availability index in the same
|
||||
atomic call. Replaying the same `CheckedAt` and state returns the stored result without
|
||||
changing indexes.
|
||||
|
||||
- [ ] **Step 5: Run GREEN and commit**
|
||||
|
||||
```powershell
|
||||
go test -tags=integration -timeout 60s ./internal/adapters/redisactivity -run 'TestRedis.*(Upsert|Health)'
|
||||
git add internal/adapters/redisactivity
|
||||
git commit -m "feat: add redis activity upsert and health"
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 6: Implement Atomic Exclusive Extraction
|
||||
|
||||
**Files:**
|
||||
- Create: `internal/adapters/redisactivity/extract.go`
|
||||
- Create: `internal/adapters/redisactivity/scripts/extract.lua`
|
||||
- Create: `internal/adapters/redisactivity/extract_integration_test.go`
|
||||
|
||||
- [ ] **Step 1: Write failing extraction contract tests**
|
||||
|
||||
Seed AVAILABLE records through Upsert/ApplyHealth and test partial, allOrNothing, every filter,
|
||||
MinRemainingTTL, MaxHealthCheckAge, ReserveForGateway, same-key replay, conflicting digest,
|
||||
and bounded idempotency expiry. Add a 100-round concurrent race:
|
||||
|
||||
```go
|
||||
for iteration := 0; iteration < 100; iteration++ {
|
||||
// Two goroutines call Extract for the same one-item namespace.
|
||||
// Exactly one returned result must contain the Proxy ID.
|
||||
}
|
||||
```
|
||||
|
||||
Test that a selective query exhausting `MaxCandidateScan` returns
|
||||
`extraction.ErrStoreUnavailable` and leaves every Proxy AVAILABLE.
|
||||
|
||||
- [ ] **Step 2: Run RED against Redis 8.2**
|
||||
|
||||
```powershell
|
||||
go test -tags=integration -timeout 60s ./internal/adapters/redisactivity -run 'TestRedis.*Extract'
|
||||
```
|
||||
|
||||
Expected: FAIL because `Adapter.Extract` is absent.
|
||||
|
||||
- [ ] **Step 3: Implement request digest and typed reply mapping**
|
||||
|
||||
Canonicalize each filter as a sorted, duplicate-free list and hash this payload:
|
||||
|
||||
```go
|
||||
type digestInput struct {
|
||||
Requested int `json:"requested"`
|
||||
Fulfillment extraction.Fulfillment `json:"fulfillment"`
|
||||
Protocols []string `json:"protocols"`
|
||||
Regions []string `json:"regions"`
|
||||
Carriers []string `json:"carriers"`
|
||||
Upstreams []string `json:"upstreams"`
|
||||
}
|
||||
```
|
||||
|
||||
Exclude RequestID, SourceIP, Now, and operational TTLs from the business digest. Use Client ID
|
||||
plus Idempotency-Key for business replay; use RequestID only for an internal operation key.
|
||||
|
||||
- [ ] **Step 4: Implement one bounded extraction script**
|
||||
|
||||
The script must:
|
||||
|
||||
1. Return a committed result when the idempotency digest matches.
|
||||
2. Return conflict without mutation when it differs.
|
||||
3. Choose the smallest `ZCARD` among filter dimensions containing exactly one requested
|
||||
value; otherwise use global available. This keeps the driver a complete superset when a
|
||||
dimension contains OR values.
|
||||
4. Scan from longest `usableUntil`, validate complete records, and stop after
|
||||
`requested + reserveForGateway` matches or `MaxCandidateScan` records.
|
||||
5. Return scan-budget exhaustion without mutation when the result cannot be decided.
|
||||
6. Return insufficient without mutation for allOrNothing.
|
||||
7. Mark selected records EXTRACTED, remove every availability index, decrement inventory,
|
||||
and store the response in the same script.
|
||||
8. Set idempotency TTL to the earlier of configured TTL and earliest selected hard expiry.
|
||||
|
||||
Build URL values in Go with `net/url.URL` and `net.JoinHostPort`; never concatenate user or
|
||||
password fields manually.
|
||||
|
||||
- [ ] **Step 5: Run GREEN and commit**
|
||||
|
||||
```powershell
|
||||
go test -tags=integration -timeout 60s ./internal/adapters/redisactivity -run 'TestRedis.*Extract'
|
||||
git add internal/adapters/redisactivity
|
||||
git commit -m "feat: add atomic redis proxy extraction"
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 7: Implement Ownership, Inventory, and Expiry Maintenance
|
||||
|
||||
**Files:**
|
||||
- Create: `internal/adapters/redisactivity/ownership.go`
|
||||
- Create: `internal/adapters/redisactivity/maintenance.go`
|
||||
- Create: `internal/adapters/redisactivity/scripts/ownership.lua`
|
||||
- Create: `internal/adapters/redisactivity/scripts/sweep.lua`
|
||||
- Create: `internal/adapters/redisactivity/ownership_integration_test.go`
|
||||
|
||||
- [ ] **Step 1: Write failing public-seam tests**
|
||||
|
||||
Cover Assign, expired-lease takeover, Renew version increments, stale epoch rejection,
|
||||
BeginDrain idempotency, ACK active/reserved rejection, successful ACK, Get, limited Expire,
|
||||
Inventory decrement, and limited SweepExpired. Add 100 Assign-vs-Extract races and assert
|
||||
exactly one winner each time.
|
||||
|
||||
- [ ] **Step 2: Run RED**
|
||||
|
||||
```powershell
|
||||
go test -tags=integration -timeout 60s ./internal/adapters/redisactivity -run 'TestRedis.*(Ownership|Inventory|Sweep)'
|
||||
```
|
||||
|
||||
Expected: FAIL because these Adapter methods are absent.
|
||||
|
||||
- [ ] **Step 3: Implement constant-work ownership scripts**
|
||||
|
||||
Use one operation selector with separate validated argument shapes:
|
||||
|
||||
```text
|
||||
assign | renew | begin_drain | acknowledge_drain | get | expire
|
||||
```
|
||||
|
||||
Assign removes the Proxy from AVAILABLE indexes. Renew caps lease expiry at `usableUntil`.
|
||||
ACK restores indexes only when the Proxy remains AVAILABLE and usable. Each mutating call
|
||||
uses an internal operation ID so a transport retry does not increment epoch/version twice.
|
||||
Expire processes no more than the caller-provided limit.
|
||||
|
||||
- [ ] **Step 4: Implement inventory and bounded sweep**
|
||||
|
||||
`Inventory` first runs the same small cleanup budget, then returns the upstream counter.
|
||||
`SweepExpired` pops no more than `limit` hard-expired Proxy IDs, removes record/unique/id/facet/
|
||||
ownership indexes, and decrements only the incumbent upstream counter. Repeated sweep calls
|
||||
are idempotent and counters never go below zero.
|
||||
|
||||
- [ ] **Step 5: Run GREEN and commit**
|
||||
|
||||
```powershell
|
||||
go test -tags=integration -timeout 60s ./internal/adapters/redisactivity -run 'TestRedis.*(Ownership|Inventory|Sweep)'
|
||||
git add internal/adapters/redisactivity
|
||||
git commit -m "feat: add redis ownership and expiry maintenance"
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 8: Reuse One Contract Suite for Memory and Redis
|
||||
|
||||
**Files:**
|
||||
- Create: `internal/domain/activitypool/contracttest/contract.go`
|
||||
- Create: `internal/domain/activitypool/contract_external_test.go`
|
||||
- Create: `internal/adapters/redisactivity/contract_integration_test.go`
|
||||
- Modify: `internal/adapters/redisactivity/testredis_test.go`
|
||||
- Modify: `scripts/test-redis.ps1`
|
||||
|
||||
- [ ] **Step 1: Extract behavior-only contract cases**
|
||||
|
||||
Define a factory that returns the approved narrow seams and cleanup:
|
||||
|
||||
```go
|
||||
type Store interface {
|
||||
activitypool.Upserter
|
||||
activitypool.HealthStore
|
||||
activitypool.InventoryReader
|
||||
activitypool.Maintainer
|
||||
extraction.Store
|
||||
ownership.Repository
|
||||
}
|
||||
|
||||
type Factory func(*testing.T) (Store, func())
|
||||
|
||||
func Run(t *testing.T, factory Factory)
|
||||
```
|
||||
|
||||
Create `contract_external_test.go` with package `activitypool_test` and run the shared suite
|
||||
against `activitypool.NewMemoryPool`. Retain the existing package-internal Snapshot tests in
|
||||
`pool_test.go`; do not create an import cycle and do not inspect Redis keys from the shared
|
||||
contract.
|
||||
|
||||
- [ ] **Step 2: Run the contract against MemoryPool**
|
||||
|
||||
```powershell
|
||||
go test -timeout 60s ./internal/domain/activitypool/...
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 3: Complete the isolated Redis fixture runner**
|
||||
|
||||
Extend `scripts/test-redis.ps1` to run the complete integration contract. It must use
|
||||
`try/finally`, wait for health, set `PROXY_POOL_TEST_REDIS_URL`, run only integration-tagged
|
||||
tests with a 60-second Go timeout, and stop the fixture in `finally`.
|
||||
|
||||
- [ ] **Step 4: Run the same contract against Redis**
|
||||
|
||||
```powershell
|
||||
.\scripts\test-redis.ps1
|
||||
```
|
||||
|
||||
Expected: Redis 8.2 becomes healthy and all integration tests PASS in at most 60 seconds per
|
||||
Go test invocation.
|
||||
|
||||
- [ ] **Step 5: Commit the shared contract and fixture**
|
||||
|
||||
```powershell
|
||||
git add internal/domain/activitypool internal/adapters/redisactivity scripts/test-redis.ps1
|
||||
git commit -m "test: add redis activity pool contract suite"
|
||||
```
|
||||
|
||||
### Task 9: Finalize Local Runtime Policy and Documentation
|
||||
|
||||
**Files:**
|
||||
- Modify: `deploy/docker-compose.yml`
|
||||
- Create: `deploy/compose_test.go`
|
||||
- Modify: `docs/development/implementation-plan.md`
|
||||
- Modify: `docs/testing/test-strategy.md`
|
||||
- Modify: `docs/operations/runbook.md`
|
||||
- Modify: `docs/requirements/traceability.md`
|
||||
- Modify: `progress.md`
|
||||
|
||||
- [ ] **Step 1: Write a failing Compose policy check**
|
||||
|
||||
Add `deploy/compose_test.go` using `go.yaml.in/yaml/v4` to parse `docker-compose.yml` and assert
|
||||
that the local Redis service uses `--appendonly no`, `--save ""`, and has no persistent data
|
||||
volume. Run `go test -timeout 60s ./deploy` before changing Compose and confirm it fails.
|
||||
|
||||
- [ ] **Step 2: Make local Redis explicitly ephemeral**
|
||||
|
||||
Change only the Redis service command and volume mount. Keep health checks and the backend
|
||||
network unchanged. Do not remove or manipulate any existing Docker volume on the machine.
|
||||
|
||||
- [ ] **Step 3: Update authoritative documents**
|
||||
|
||||
Record:
|
||||
|
||||
- Redis Adapter and shared contract as complete.
|
||||
- Gateway hot path still has no Redis access.
|
||||
- PostgreSQL still contains no Proxy details or extraction records.
|
||||
- Redis Activity Pool is runtime truth and local Redis is intentionally non-persistent.
|
||||
- `docs/testing/strategy.md` legacy PostgreSQL extraction wording is superseded.
|
||||
- `docs/requirements/traceability.md` no longer claims per-proxy extraction audit storage.
|
||||
- 100,000 QPS remains an unverified end-to-end target.
|
||||
|
||||
- [ ] **Step 4: Run full verification**
|
||||
|
||||
```powershell
|
||||
.\scripts\verify.ps1
|
||||
.\scripts\test-redis.ps1
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: gofmt, go vet, all unit tests, build, and Redis integration tests PASS. Race tests
|
||||
run only when `CGO_ENABLED=1`; otherwise the script prints the existing explicit skip reason.
|
||||
|
||||
- [ ] **Step 5: Perform independent review**
|
||||
|
||||
Review public contracts, Lua atomicity, retry behavior, TTL bounds, credential redaction,
|
||||
inventory counters, and the absence of Redis calls in `internal/gateway`. Fix every blocking
|
||||
finding and rerun both verification scripts.
|
||||
|
||||
- [ ] **Step 6: Commit the runtime policy and documentation**
|
||||
|
||||
```powershell
|
||||
git add deploy/docker-compose.yml deploy/compose_test.go docs/development/implementation-plan.md docs/testing/test-strategy.md docs/operations/runbook.md docs/requirements/traceability.md progress.md
|
||||
git commit -m "docs: finalize redis activity pool delivery"
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Push the completed branch**
|
||||
|
||||
Verify the diff contains no user-owned deletion, then push:
|
||||
|
||||
```powershell
|
||||
git status --short
|
||||
git push origin build/proxy-pool-architecture
|
||||
```
|
||||
|
||||
Expected: the remote branch advances through all Redis Activity Pool commits while
|
||||
`proxy-pool-docs-v1.0.zip` remains untracked by the commits.
|
||||
|
||||
## Self-Review Result
|
||||
|
||||
- ADR-005 deployment, module, keyspace, Upsert, Health, Extract, Ownership, cleanup,
|
||||
credentials, inventory, failure, test, and persistence sections each map to a task above.
|
||||
- Every public method used by later tasks is defined in Tasks 1-4.
|
||||
- Memory and Redis implementations share behavior tests only through approved public seams.
|
||||
- No PostgreSQL Proxy storage, Gateway Redis hot-path access, Cluster sharding, or production
|
||||
throughput claim is introduced.
|
||||
Loading…
Reference in New Issue
Block a user