# DFP Bootstrap Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Build a tested dry-run DFP bootstrap layer that consumes `DeviceProfile` and emits signed DFP/unifiedId request specs without importing from `out/`.

**Architecture:** Migrate reusable protocol builders into focused `core` modules: `dfp_sq0` for protobuf wire bytes, `dfp_knn` for lite/full kNN maps, and `dfp_forms` for 10400 `deviceInfo` plus signed request forms. Extend `tools/new_device.py` only after core builders pass focused tests.

**Tech Stack:** Python 3.13, standard library, existing `core.enc_data`, existing `core.dfp_sign`, existing `unittest`.

---

## File Structure

- Create `core/dfp_sq0.py`
  - Encodes and decodes DFP sq0 string fields.
- Create `core/dfp_knn.py`
  - Builds lite/full kNN maps from `DeviceProfile`.
- Create `core/dfp_forms.py`
  - Builds signed request specs for unifiedId and gdfp report.
- Modify `core/__init__.py`
  - Adds module names without removing existing exports.
- Modify `tools/new_device.py`
  - Adds `--dfp-dry-run` and `--profile`.
- Create `tests/test_dfp_sq0.py`
  - Tests protobuf encoding and decoding.
- Create `tests/test_dfp_knn.py`
  - Tests kNN identity field mapping and CRC.
- Create `tests/test_dfp_forms.py`
  - Tests form order and request spec construction.

---

### Task 1: Migrate sq0 protobuf encoder

**Files:**
- Create: `tests/test_dfp_sq0.py`
- Create: `core/dfp_sq0.py`

- [ ] **Step 1: Write failing sq0 tests**

```python
import unittest

from core.dfp_sq0 import decode_sq0_string_fields, encode_sq0_device_info


class DfpSq0Tests(unittest.TestCase):
    def test_lite_encoding_preserves_known_tag_order(self):
        raw = encode_sq0_device_info({"k5": "a", "k14": "bc", "k113": "z"}, mode="lite")
        self.assertEqual(raw.hex(), "2a0161720262638a07017a")

        fields = decode_sq0_string_fields(raw)
        self.assertEqual(
            [(field["proto_tag"], field["value"]) for field in fields],
            [(5, "a"), (14, "bc"), (113, "z")],
        )

    def test_empty_values_are_not_encoded(self):
        raw = encode_sq0_device_info({"k5": "a", "k14": "", "k113": "z"}, mode="lite")
        fields = decode_sq0_string_fields(raw)
        self.assertEqual(
            [(field["proto_tag"], field["value"]) for field in fields],
            [(5, "a"), (113, "z")],
        )

    def test_unknown_key_is_rejected(self):
        with self.assertRaises(KeyError):
            encode_sq0_device_info({"k999": "x"}, mode="lite")


if __name__ == "__main__":
    unittest.main()
```

- [ ] **Step 2: Run RED**

Run:

```powershell
uv run python -m unittest tests.test_dfp_sq0 -v
```

Expected:

```text
ModuleNotFoundError: No module named 'core.dfp_sq0'
```

- [ ] **Step 3: Implement `core/dfp_sq0.py`**

Implementation requirements:

```python
LITE_TAGS = {
    "k5": 5,
    "k14": 14,
    "k22": 22,
    "k23": 23,
    "k27": 27,
    "k29": 29,
    "k31": 31,
    "k34": 34,
    "k35": 35,
    "k36": 36,
    "k39": 39,
    "k40": 40,
    "k46": 46,
    "k57": 57,
    "k61": 61,
    "k64": 64,
    "k66": 66,
    "k68": 68,
    "k83": 83,
    "k86": 86,
    "k93": 93,
    "k97": 97,
    "k101": 101,
    "k102": 102,
    "k105": 105,
    "k106": 106,
    "k107": 107,
    "k108": 108,
    "k109": 109,
    "k110": 110,
    "k111": 111,
    "k112": 112,
    "k113": 113,
}
```

The module must expose:

```python
def encode_sq0_device_info(values: dict[str, str], mode: str) -> bytes: ...
def decode_sq0_string_fields(raw: bytes) -> list[dict[str, object]]: ...
```

Full mode can initially use tags `k1..k119` mapped to matching numeric tags.

- [ ] **Step 4: Run GREEN**

Run:

```powershell
uv run python -m unittest tests.test_dfp_sq0 -v
```

Expected:

```text
Ran 3 tests
OK
```

---

### Task 2: Build lite kNN from DeviceProfile

**Files:**
- Create: `tests/test_dfp_knn.py`
- Create: `core/dfp_knn.py`

- [ ] **Step 1: Write failing kNN tests**

```python
import json
import unittest

from core.device_profile import DeviceProfileGenerator
from core.dfp_knn import LITE_KEYS, build_lite_knn, recompute_k14_crc


class DfpKnnTests(unittest.TestCase):
    def test_lite_knn_uses_device_profile_identity_fields(self):
        profile = DeviceProfileGenerator(seed=1234).new_profile()
        knn = build_lite_knn(profile)

        self.assertEqual(list(knn), LITE_KEYS)
        self.assertEqual(knn["k31"], profile.android_id)
        self.assertEqual(knn["k66"], profile.o_did.removeprefix("ANDROID_"))
        self.assertEqual(knn["k107"], str(profile.cdid_tag))
        self.assertIn(profile.g_rdi2, json.loads(knn["k93"])["28"])

    def test_k14_crc_changes_when_identity_changes(self):
        profile = DeviceProfileGenerator(seed=1234).new_profile()
        knn = build_lite_knn(profile)
        original = knn["k14"]

        changed = dict(knn)
        changed["k31"] = "0000000000000000"
        changed["k14"] = recompute_k14_crc(changed, LITE_KEYS)

        self.assertNotEqual(changed["k14"], original)


if __name__ == "__main__":
    unittest.main()
```

- [ ] **Step 2: Run RED**

Run:

```powershell
uv run python -m unittest tests.test_dfp_knn -v
```

Expected:

```text
ModuleNotFoundError: No module named 'core.dfp_knn'
```

- [ ] **Step 3: Implement `core/dfp_knn.py`**

Implementation requirements:

```python
LITE_KEYS = [
    "k5", "k14", "k22", "k23", "k27", "k29", "k31", "k34",
    "k35", "k36", "k39", "k40", "k46", "k57", "k61", "k64",
    "k66", "k68", "k83", "k86", "k93", "k97", "k101", "k102",
    "k105", "k106", "k107", "k108", "k109", "k110", "k111",
    "k112", "k113",
]
```

The module must expose:

```python
def build_lite_knn(profile: DeviceProfile, overrides: dict[str, str] | None = None) -> dict[str, str]: ...
def build_full_knn(profile: DeviceProfile, overrides: dict[str, str] | None = None) -> dict[str, str]: ...
def recompute_k14_crc(values: dict[str, str], ordered_keys: list[str]) -> str: ...
```

Minimum identity mapping:

```text
k31 = profile.android_id
k66 = profile.o_did without ANDROID_
k83 = profile.egid
k107 = profile.cdid_tag
k93["28"] = profile.g_rdi2
k14 = AND:<crc32>
```

- [ ] **Step 4: Run GREEN**

Run:

```powershell
uv run python -m unittest tests.test_dfp_knn -v
```

Expected:

```text
Ran 2 tests
OK
```

---

### Task 3: Build signed DFP form specs

**Files:**
- Create: `tests/test_dfp_forms.py`
- Create: `core/dfp_forms.py`

- [ ] **Step 1: Write failing form tests**

```python
import unittest
from urllib.parse import parse_qs

from core.device_profile import DeviceProfileGenerator
from core.dfp_forms import (
    GDFP_REPORT_FORM_ORDER,
    UNIFIED_FETCH_FORM_ORDER,
    build_gdfp_report_request,
    build_unified_fetch_request,
)


class DfpFormsTests(unittest.TestCase):
    def test_unified_fetch_preserves_form_order(self):
        profile = DeviceProfileGenerator(seed=1234).new_profile()
        request = build_unified_fetch_request(
            profile,
            counter=1,
            unix_time=1783749817,
            session_seed=0x5D7E742B,
            ts_millis="1783749817000",
            epoch_seconds=1783749817,
        )

        self.assertEqual(request.form_order, UNIFIED_FETCH_FORM_ORDER)
        self.assertEqual(request.form["did"], profile.did)
        self.assertEqual(request.form["rdid"], profile.rdid)
        self.assertIn("sign", request.form)

    def test_gdfp_report_request_body_order(self):
        profile = DeviceProfileGenerator(seed=1234).new_profile()
        request = build_gdfp_report_request(
            profile,
            counter=2,
            unix_time=1783749817,
            session_seed=0x5D7E742B,
            ts_millis="1783749817000",
            epoch_seconds=1783749817,
        )

        self.assertEqual(request.form_order, GDFP_REPORT_FORM_ORDER)
        self.assertTrue(request.body.startswith("productName=NEBULA&ts=1783749817000&deviceInfo="))
        parsed = parse_qs(request.body)
        self.assertEqual(parsed["rdid"], [profile.rdid])
        self.assertEqual(parsed["didtag"], [str(profile.cdid_tag)])


if __name__ == "__main__":
    unittest.main()
```

- [ ] **Step 2: Run RED**

Run:

```powershell
uv run python -m unittest tests.test_dfp_forms -v
```

Expected:

```text
ModuleNotFoundError: No module named 'core.dfp_forms'
```

- [ ] **Step 3: Implement `core/dfp_forms.py`**

Implementation requirements:

```python
@dataclass(frozen=True)
class DfpRequestSpec:
    method: str
    url: str
    headers: dict[str, str]
    form_order: list[str]
    form: dict[str, str]
    body: str
```

Expose:

```python
def build_unified_fetch_request(profile: DeviceProfile, *, counter: int, unix_time: int, session_seed: int, ts_millis: str | None = None, epoch_seconds: int | None = None) -> DfpRequestSpec: ...
def build_gdfp_report_request(profile: DeviceProfile, *, counter: int, unix_time: int, session_seed: int, ts_millis: str | None = None, epoch_seconds: int | None = None) -> DfpRequestSpec: ...
```

Required constants:

```python
UNIFIED_FETCH_FORM_ORDER = [
    "aegon", "appVersion", "deviceInfo", "did", "didTag",
    "hgidReportId", "platform", "productName", "rdid",
    "requestId", "sdkVersion", "sv", "ts", "sign",
]

GDFP_REPORT_FORM_ORDER = [
    "productName", "ts", "deviceInfo", "sign", "sv", "rdid", "didtag",
]
```

Use:

- `core.dfp_knn.build_lite_knn()`
- `core.dfp_knn.build_full_knn()`
- `core.dfp_sq0.encode_sq0_device_info()`
- `core.enc_data.kwsg_10400_raw()`
- `core.dfp_sign.sign_dfp_form()`

- [ ] **Step 4: Run GREEN**

Run:

```powershell
uv run python -m unittest tests.test_dfp_forms -v
```

Expected:

```text
Ran 2 tests
OK
```

---

### Task 4: Export modules and add CLI dry-run

**Files:**
- Modify: `core/__init__.py`
- Modify: `tools/new_device.py`
- Create: `tests/test_new_device_dfp_cli.py`

- [ ] **Step 1: Write failing CLI dry-run test**

```python
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path


class NewDeviceDfpCliTests(unittest.TestCase):
    def test_cli_writes_dfp_dry_run_requests(self):
        with tempfile.TemporaryDirectory() as tmp:
            out_dir = Path(tmp) / "devices"
            result = subprocess.run(
                [
                    sys.executable,
                    "tools/new_device.py",
                    "--count",
                    "1",
                    "--out-dir",
                    str(out_dir),
                    "--seed",
                    "1",
                    "--env",
                    "--dfp-dry-run",
                    "--force",
                ],
                check=False,
                cwd=Path(__file__).resolve().parents[1],
                text=True,
                capture_output=True,
            )

            self.assertEqual(result.returncode, 0, result.stderr)
            request_path = out_dir / "device_001_dfp_requests.json"
            self.assertTrue(request_path.exists())
            data = json.loads(request_path.read_text(encoding="utf-8"))
            self.assertIn("unified_fetch", data)
            self.assertIn("gdfp_report", data)


if __name__ == "__main__":
    unittest.main()
```

- [ ] **Step 2: Run RED**

Run:

```powershell
uv run python -m unittest tests.test_new_device_dfp_cli -v
```

Expected:

```text
error: unrecognized arguments: --dfp-dry-run
```

- [ ] **Step 3: Update `tools/new_device.py`**

Add parser option:

```python
parser.add_argument("--dfp-dry-run", action="store_true", help="write DFP request material")
```

After saving each profile, when `args.dfp_dry_run` is true:

```python
from core.dfp_forms import build_gdfp_report_request, build_unified_fetch_request

dfp_requests = {
    "unified_fetch": build_unified_fetch_request(
        profile,
        counter=1,
        unix_time=int(profile.cold_launch_time_ms // 1000),
        session_seed=0x5D7E742B,
        ts_millis=str(profile.cold_launch_time_ms),
        epoch_seconds=int(profile.cold_launch_time_ms // 1000),
    ).to_dict(),
    "gdfp_report": build_gdfp_report_request(
        profile,
        counter=2,
        unix_time=int(profile.cold_launch_time_ms // 1000),
        session_seed=0x5D7E742B,
        ts_millis=str(profile.cold_launch_time_ms),
        epoch_seconds=int(profile.cold_launch_time_ms // 1000),
    ).to_dict(),
}
(out_dir / f"{stem}_dfp_requests.json").write_text(
    json.dumps(dfp_requests, ensure_ascii=False, indent=2) + "\n",
    encoding="utf-8",
)
```

- [ ] **Step 4: Run GREEN**

Run:

```powershell
uv run python -m unittest tests.test_new_device_dfp_cli -v
```

Expected:

```text
Ran 1 test
OK
```

---

### Task 5: Verification

**Files:**
- Verify: `core/dfp_sq0.py`
- Verify: `core/dfp_knn.py`
- Verify: `core/dfp_forms.py`
- Verify: `tools/new_device.py`

- [ ] **Step 1: Run focused DFP tests**

Run:

```powershell
uv run python -m unittest tests.test_dfp_sq0 tests.test_dfp_knn tests.test_dfp_forms tests.test_new_device_dfp_cli -v
```

Expected:

```text
Ran 8 tests
OK
```

- [ ] **Step 2: Run phase1 tests**

Run:

```powershell
uv run python -m unittest tests.test_device_profile tests.test_new_device_cli -v
```

Expected:

```text
Ran 6 tests
OK
```

- [ ] **Step 3: Compile core/tools/tests**

Run:

```powershell
uv run python -m compileall core tools tests
```

Expected:

```text
Listing 'core'...
Listing 'tools'...
Listing 'tests'...
```

- [ ] **Step 4: Generate dry-run sample**

Run:

```powershell
uv run python tools/new_device.py --count 1 --out-dir out/devices_dfp_sample --seed 20260711 --env --dfp-dry-run --force
```

Expected files:

```text
out/devices_dfp_sample/device_001.json
out/devices_dfp_sample/device_001.env
out/devices_dfp_sample/device_001_dfp_requests.json
```

---

## Self-Review

- Spec coverage: dry-run sq0/kNN/forms and CLI integration are covered.
- Online POST is intentionally not included in this first implementation plan.
- Placeholder scan: no placeholder tasks remain.
- Type consistency: module and function names match across tests and implementation tasks.
- Known global blocker: `tests/test_main.py` currently imports old `main.BuiltRequest`; final verification must report that separately instead of claiming full suite success.