ksjsb/docs/superpowers/plans/2026-07-11-dfp-bootstrap.md

DFP Bootstrap Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Build a tested dry-run DFP bootstrap layer that consumes DeviceProfile and emits signed DFP/unifiedId request specs without importing from out/.

Architecture: Migrate reusable protocol builders into focused core modules: dfp_sq0 for protobuf wire bytes, dfp_knn for lite/full kNN maps, and dfp_forms for 10400 deviceInfo plus signed request forms. Extend tools/new_device.py only after core builders pass focused tests.

Tech Stack: Python 3.13, standard library, existing core.enc_data, existing core.dfp_sign, existing unittest.

---

File Structure

• Create core/dfp_sq0.py
  • Encodes and decodes DFP sq0 string fields.
• Create core/dfp_knn.py
  • Builds lite/full kNN maps from DeviceProfile.
• Create core/dfp_forms.py
  • Builds signed request specs for unifiedId and gdfp report.
• Modify core/__init__.py
  • Adds module names without removing existing exports.
• Modify tools/new_device.py
  • Adds --dfp-dry-run and --profile.
• Create tests/test_dfp_sq0.py
  • Tests protobuf encoding and decoding.
• Create tests/test_dfp_knn.py
  • Tests kNN identity field mapping and CRC.
• Create tests/test_dfp_forms.py
  • Tests form order and request spec construction.

---

Task 1: Migrate sq0 protobuf encoder

Files:

• Create: tests/test_dfp_sq0.py

• Create: core/dfp_sq0.py

• Step 1: Write failing sq0 tests

import unittest

from core.dfp_sq0 import decode_sq0_string_fields, encode_sq0_device_info


class DfpSq0Tests(unittest.TestCase):
    def test_lite_encoding_preserves_known_tag_order(self):
        raw = encode_sq0_device_info({"k5": "a", "k14": "bc", "k113": "z"}, mode="lite")
        self.assertEqual(raw.hex(), "2a0161720262638a07017a")

        fields = decode_sq0_string_fields(raw)
        self.assertEqual(
            [(field["proto_tag"], field["value"]) for field in fields],
            [(5, "a"), (14, "bc"), (113, "z")],
        )

    def test_empty_values_are_not_encoded(self):
        raw = encode_sq0_device_info({"k5": "a", "k14": "", "k113": "z"}, mode="lite")
        fields = decode_sq0_string_fields(raw)
        self.assertEqual(
            [(field["proto_tag"], field["value"]) for field in fields],
            [(5, "a"), (113, "z")],
        )

    def test_unknown_key_is_rejected(self):
        with self.assertRaises(KeyError):
            encode_sq0_device_info({"k999": "x"}, mode="lite")


if __name__ == "__main__":
    unittest.main()

• Step 2: Run RED

Run:

uv run python -m unittest tests.test_dfp_sq0 -v

Expected:

ModuleNotFoundError: No module named 'core.dfp_sq0'

• Step 3: Implement core/dfp_sq0.py

Implementation requirements:

LITE_TAGS = {
    "k5": 5,
    "k14": 14,
    "k22": 22,
    "k23": 23,
    "k27": 27,
    "k29": 29,
    "k31": 31,
    "k34": 34,
    "k35": 35,
    "k36": 36,
    "k39": 39,
    "k40": 40,
    "k46": 46,
    "k57": 57,
    "k61": 61,
    "k64": 64,
    "k66": 66,
    "k68": 68,
    "k83": 83,
    "k86": 86,
    "k93": 93,
    "k97": 97,
    "k101": 101,
    "k102": 102,
    "k105": 105,
    "k106": 106,
    "k107": 107,
    "k108": 108,
    "k109": 109,
    "k110": 110,
    "k111": 111,
    "k112": 112,
    "k113": 113,
}

The module must expose:

def encode_sq0_device_info(values: dict[str, str], mode: str) -> bytes: ...
def decode_sq0_string_fields(raw: bytes) -> list[dict[str, object]]: ...

Full mode can initially use tags k1..k119 mapped to matching numeric tags.

• Step 4: Run GREEN

Run:

uv run python -m unittest tests.test_dfp_sq0 -v

Expected:

Ran 3 tests
OK

---

Task 2: Build lite kNN from DeviceProfile

Files:

• Create: tests/test_dfp_knn.py

• Create: core/dfp_knn.py

• Step 1: Write failing kNN tests

import json
import unittest

from core.device_profile import DeviceProfileGenerator
from core.dfp_knn import LITE_KEYS, build_lite_knn, recompute_k14_crc


class DfpKnnTests(unittest.TestCase):
    def test_lite_knn_uses_device_profile_identity_fields(self):
        profile = DeviceProfileGenerator(seed=1234).new_profile()
        knn = build_lite_knn(profile)

        self.assertEqual(list(knn), LITE_KEYS)
        self.assertEqual(knn["k31"], profile.android_id)
        self.assertEqual(knn["k66"], profile.o_did.removeprefix("ANDROID_"))
        self.assertEqual(knn["k107"], str(profile.cdid_tag))
        self.assertIn(profile.g_rdi2, json.loads(knn["k93"])["28"])

    def test_k14_crc_changes_when_identity_changes(self):
        profile = DeviceProfileGenerator(seed=1234).new_profile()
        knn = build_lite_knn(profile)
        original = knn["k14"]

        changed = dict(knn)
        changed["k31"] = "0000000000000000"
        changed["k14"] = recompute_k14_crc(changed, LITE_KEYS)

        self.assertNotEqual(changed["k14"], original)


if __name__ == "__main__":
    unittest.main()

• Step 2: Run RED

Run:

uv run python -m unittest tests.test_dfp_knn -v

Expected:

ModuleNotFoundError: No module named 'core.dfp_knn'

• Step 3: Implement core/dfp_knn.py

Implementation requirements:

LITE_KEYS = [
    "k5", "k14", "k22", "k23", "k27", "k29", "k31", "k34",
    "k35", "k36", "k39", "k40", "k46", "k57", "k61", "k64",
    "k66", "k68", "k83", "k86", "k93", "k97", "k101", "k102",
    "k105", "k106", "k107", "k108", "k109", "k110", "k111",
    "k112", "k113",
]

The module must expose:

def build_lite_knn(profile: DeviceProfile, overrides: dict[str, str] | None = None) -> dict[str, str]: ...
def build_full_knn(profile: DeviceProfile, overrides: dict[str, str] | None = None) -> dict[str, str]: ...
def recompute_k14_crc(values: dict[str, str], ordered_keys: list[str]) -> str: ...

Minimum identity mapping:

k31 = profile.android_id
k66 = profile.o_did without ANDROID_
k83 = profile.egid
k107 = profile.cdid_tag
k93["28"] = profile.g_rdi2
k14 = AND:<crc32>

• Step 4: Run GREEN

Run:

uv run python -m unittest tests.test_dfp_knn -v

Expected:

Ran 2 tests
OK

---

Task 3: Build signed DFP form specs

Files:

• Create: tests/test_dfp_forms.py

• Create: core/dfp_forms.py

• Step 1: Write failing form tests

import unittest
from urllib.parse import parse_qs

from core.device_profile import DeviceProfileGenerator
from core.dfp_forms import (
    GDFP_REPORT_FORM_ORDER,
    UNIFIED_FETCH_FORM_ORDER,
    build_gdfp_report_request,
    build_unified_fetch_request,
)


class DfpFormsTests(unittest.TestCase):
    def test_unified_fetch_preserves_form_order(self):
        profile = DeviceProfileGenerator(seed=1234).new_profile()
        request = build_unified_fetch_request(
            profile,
            counter=1,
            unix_time=1783749817,
            session_seed=0x5D7E742B,
            ts_millis="1783749817000",
            epoch_seconds=1783749817,
        )

        self.assertEqual(request.form_order, UNIFIED_FETCH_FORM_ORDER)
        self.assertEqual(request.form["did"], profile.did)
        self.assertEqual(request.form["rdid"], profile.rdid)
        self.assertIn("sign", request.form)

    def test_gdfp_report_request_body_order(self):
        profile = DeviceProfileGenerator(seed=1234).new_profile()
        request = build_gdfp_report_request(
            profile,
            counter=2,
            unix_time=1783749817,
            session_seed=0x5D7E742B,
            ts_millis="1783749817000",
            epoch_seconds=1783749817,
        )

        self.assertEqual(request.form_order, GDFP_REPORT_FORM_ORDER)
        self.assertTrue(request.body.startswith("productName=NEBULA&ts=1783749817000&deviceInfo="))
        parsed = parse_qs(request.body)
        self.assertEqual(parsed["rdid"], [profile.rdid])
        self.assertEqual(parsed["didtag"], [str(profile.cdid_tag)])


if __name__ == "__main__":
    unittest.main()

• Step 2: Run RED

Run:

uv run python -m unittest tests.test_dfp_forms -v

Expected:

ModuleNotFoundError: No module named 'core.dfp_forms'

• Step 3: Implement core/dfp_forms.py

Implementation requirements:

@dataclass(frozen=True)
class DfpRequestSpec:
    method: str
    url: str
    headers: dict[str, str]
    form_order: list[str]
    form: dict[str, str]
    body: str

Expose:

def build_unified_fetch_request(profile: DeviceProfile, *, counter: int, unix_time: int, session_seed: int, ts_millis: str | None = None, epoch_seconds: int | None = None) -> DfpRequestSpec: ...
def build_gdfp_report_request(profile: DeviceProfile, *, counter: int, unix_time: int, session_seed: int, ts_millis: str | None = None, epoch_seconds: int | None = None) -> DfpRequestSpec: ...

Required constants:

UNIFIED_FETCH_FORM_ORDER = [
    "aegon", "appVersion", "deviceInfo", "did", "didTag",
    "hgidReportId", "platform", "productName", "rdid",
    "requestId", "sdkVersion", "sv", "ts", "sign",
]

GDFP_REPORT_FORM_ORDER = [
    "productName", "ts", "deviceInfo", "sign", "sv", "rdid", "didtag",
]

Use:

• core.dfp_knn.build_lite_knn()

• core.dfp_knn.build_full_knn()

• core.dfp_sq0.encode_sq0_device_info()

• core.enc_data.kwsg_10400_raw()

• core.dfp_sign.sign_dfp_form()

• Step 4: Run GREEN

Run:

uv run python -m unittest tests.test_dfp_forms -v

Expected:

Ran 2 tests
OK

---

Task 4: Export modules and add CLI dry-run

Files:

• Modify: core/__init__.py

• Modify: tools/new_device.py

• Create: tests/test_new_device_dfp_cli.py

• Step 1: Write failing CLI dry-run test

import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path


class NewDeviceDfpCliTests(unittest.TestCase):
    def test_cli_writes_dfp_dry_run_requests(self):
        with tempfile.TemporaryDirectory() as tmp:
            out_dir = Path(tmp) / "devices"
            result = subprocess.run(
                [
                    sys.executable,
                    "tools/new_device.py",
                    "--count",
                    "1",
                    "--out-dir",
                    str(out_dir),
                    "--seed",
                    "1",
                    "--env",
                    "--dfp-dry-run",
                    "--force",
                ],
                check=False,
                cwd=Path(__file__).resolve().parents[1],
                text=True,
                capture_output=True,
            )

            self.assertEqual(result.returncode, 0, result.stderr)
            request_path = out_dir / "device_001_dfp_requests.json"
            self.assertTrue(request_path.exists())
            data = json.loads(request_path.read_text(encoding="utf-8"))
            self.assertIn("unified_fetch", data)
            self.assertIn("gdfp_report", data)


if __name__ == "__main__":
    unittest.main()

• Step 2: Run RED

Run:

uv run python -m unittest tests.test_new_device_dfp_cli -v

Expected:

error: unrecognized arguments: --dfp-dry-run

• Step 3: Update tools/new_device.py

Add parser option:

parser.add_argument("--dfp-dry-run", action="store_true", help="write DFP request material")

After saving each profile, when args.dfp_dry_run is true:

from core.dfp_forms import build_gdfp_report_request, build_unified_fetch_request

dfp_requests = {
    "unified_fetch": build_unified_fetch_request(
        profile,
        counter=1,
        unix_time=int(profile.cold_launch_time_ms // 1000),
        session_seed=0x5D7E742B,
        ts_millis=str(profile.cold_launch_time_ms),
        epoch_seconds=int(profile.cold_launch_time_ms // 1000),
    ).to_dict(),
    "gdfp_report": build_gdfp_report_request(
        profile,
        counter=2,
        unix_time=int(profile.cold_launch_time_ms // 1000),
        session_seed=0x5D7E742B,
        ts_millis=str(profile.cold_launch_time_ms),
        epoch_seconds=int(profile.cold_launch_time_ms // 1000),
    ).to_dict(),
}
(out_dir / f"{stem}_dfp_requests.json").write_text(
    json.dumps(dfp_requests, ensure_ascii=False, indent=2) + "\n",
    encoding="utf-8",
)

• Step 4: Run GREEN

Run:

uv run python -m unittest tests.test_new_device_dfp_cli -v

Expected:

Ran 1 test
OK

---

Task 5: Verification

Files:

• Verify: core/dfp_sq0.py

• Verify: core/dfp_knn.py

• Verify: core/dfp_forms.py

• Verify: tools/new_device.py

• Step 1: Run focused DFP tests

Run:

uv run python -m unittest tests.test_dfp_sq0 tests.test_dfp_knn tests.test_dfp_forms tests.test_new_device_dfp_cli -v

Expected:

Ran 8 tests
OK

• Step 2: Run phase1 tests

Run:

uv run python -m unittest tests.test_device_profile tests.test_new_device_cli -v

Expected:

Ran 6 tests
OK

• Step 3: Compile core/tools/tests

Run:

uv run python -m compileall core tools tests

Expected:

Listing 'core'...
Listing 'tools'...
Listing 'tests'...

• Step 4: Generate dry-run sample

Run:

uv run python tools/new_device.py --count 1 --out-dir out/devices_dfp_sample --seed 20260711 --env --dfp-dry-run --force

Expected files:

out/devices_dfp_sample/device_001.json
out/devices_dfp_sample/device_001.env
out/devices_dfp_sample/device_001_dfp_requests.json

---

Self-Review

• Spec coverage: dry-run sq0/kNN/forms and CLI integration are covered.
• Online POST is intentionally not included in this first implementation plan.
• Placeholder scan: no placeholder tasks remain.
• Type consistency: module and function names match across tests and implementation tasks.
• Known global blocker: tests/test_main.py currently imports old main.BuiltRequest; final verification must report that separately instead of claiming full suite success.