151 lines
No EOL
13 KiB
Markdown
151 lines
No EOL
13 KiB
Markdown
# PSX Art-Binding Recovery
|
|
|
|
## Scope
|
|
|
|
- Active target: retail PlayStation `SLUS_002.68` feeding the renderer-local cache pipeline in `Crusader-Map-Viewer/map_renderer`.
|
|
- Goal of this pass: stop treating the current unreadable PSX output as a renderer-only problem and measure whether placeholders are mainly caused by extraction-time art binding failures.
|
|
- This note records the first pragmatic recovery pass that replaces large placeholder bands with real bundle-backed art while keeping the result auditable in exported scene metadata.
|
|
|
|
## Root Cause Summary
|
|
|
|
- The current unreadable output was primarily an extraction-side problem, not a front-end draw bug.
|
|
- Before this pass, the built `.cache/scene-cache/psx-remorse` set carried `58,262` fallback placeholders versus only `1,714` bundle-mapped items.
|
|
- The fallback concentration was overwhelmingly in `section0_constructor_placements` (`94.5%` of fallback items), not in the smaller `section0_dispatch_roots` family.
|
|
- The hottest unresolved types were `0x0042`, `0x0041`, `0x0047`, `0x0045`, `0x003f`, `0x004b`, `0x0046`, `0x0044`, `0x0048`, and `0x004a`.
|
|
- Those types already appeared in the exported `DAT_800758d8` art-template layer in many maps, but almost all of their rows were `blockSize = 0` and therefore had no direct payload dwords to match against bundle offsets.
|
|
- The neighboring state banks were still populated: `DAT_800758cc` carried valid script tables for the same types, while `DAT_800758d0` stayed empty for this family and `DAT_800758d4` remained on the runtime-bounds side.
|
|
- That combination strongly suggests inherited or aliased presentation for these constructor-placement families rather than a literal absence of art.
|
|
|
|
## Implemented Recovery Rule
|
|
|
|
The cache builder now resolves PSX art in three stages instead of dropping directly from "no direct `DAT_800758d8` payload" to a placeholder:
|
|
|
|
1. Use the direct non-zero `DAT_800758d8` row when a type has a real local template payload and bundle match.
|
|
2. If the type lives in the unresolved zero-block constructor-placement family, look for a same-map donor type whose `DAT_800758cc` script blob is identical and whose `DAT_800758d8` row resolves to a real bundle.
|
|
3. If no exact script-signature donor exists, fall back to the nearest resolved same-map donor inside the current constructor-placement family band (`0x003e..0x0064`).
|
|
|
|
The exporter keeps this explicit instead of pretending the rule is fully solved:
|
|
|
|
- `mappingSource` now records whether the art came from a direct template, a `cc-signature-donor:xxxx`, or a `generic-family-donor:xxxx` path.
|
|
- `templateTypeId` and `donorTypeId` are preserved in exported `mapSource` rows.
|
|
- The unresolved type still uses its own `DAT_800758cc` selector-to-frame map; only the bundle source is borrowed.
|
|
|
|
## Measured Effect
|
|
|
|
Two measured rebuilds mattered in this pass.
|
|
|
|
### First donor pass
|
|
|
|
- Added donor reuse for the original generic family band.
|
|
- Totals moved from `58,262` fallback / `1,714` bundle-mapped to `37,054` fallback / `22,922` bundle-mapped.
|
|
|
|
Representative maps:
|
|
|
|
- `map 0`: `1078 / 111` -> `340 / 849`
|
|
- `map 9`: `664 / 111` -> `197 / 578`
|
|
- `map 43`: `707 / 97` -> `196 / 608`
|
|
- `map 64`: `1194 / 161` -> `736 / 619`
|
|
- `map 104`: `909 / 93` -> `894 / 108`
|
|
|
|
### Extended donor band
|
|
|
|
- Extended the same heuristic through the next zero-block constructor-placement band up to `0x0064`.
|
|
- Final measured totals after rebuild: `25,038` fallback / `34,938` bundle-mapped.
|
|
|
|
Representative maps after the extended pass:
|
|
|
|
- `map 0`: `66` fallback / `1123` bundle-mapped
|
|
- `map 9`: `42` fallback / `733` bundle-mapped
|
|
- `map 43`: `9` fallback / `795` bundle-mapped
|
|
- `map 64`: `160` fallback / `1195` bundle-mapped
|
|
- `map 104`: `866` fallback / `136` bundle-mapped
|
|
|
|
So the first practical conclusion is straightforward: the exporter was the main blocker for most maps, and a constrained donor heuristic can replace a large fraction of placeholders with real graphics without touching the viewer runtime.
|
|
|
|
## No-Placeholder Cohort Pass
|
|
|
|
- A second follow-up pass replaced synthetic fallback atlas generation with cohort-aware donor resolution keyed by authored family plus raw `u5` lane before the older coarse type default is consulted.
|
|
- The exporter still prefers direct local `DAT_800758d8` payload matches first, but when a type bucket is mixed-role or otherwise unresolved it now resolves each cohort separately instead of collapsing the whole `map:type` bucket into one donor or one placeholder.
|
|
- Focused validation on `map 104` now rebuilds as `1002` real-art items with `0` fallback items, `1` atlas, and `136` shape definitions. Scene `sourceCounts` are now `52` `section0_dispatch_roots-art` plus `950` `section0_constructor_placements-art`.
|
|
- The scene JSON also now preserves `mappingSource` and optional `artCohort` on each `mapSource` row, so provisional cohort and emergency donor matches stay auditable after placeholders are removed.
|
|
- This is still not the final executable-proof binding rule. The focused cache is now placeholder-free, but many rows still resolve through `cohort-*` or `emergency-global-donor:*` provenance and should be treated as provisional PSX art recovery rather than as final family/resource closure.
|
|
|
|
## Remaining Unresolved Mass
|
|
|
|
- The cache is still not fully closed. `25,038` fallback items remain across `62` maps.
|
|
- The remaining fallback distribution is still concentrated in `section0_constructor_placements`.
|
|
- Current top fallback-heavy types after the donor pass are `0x0042`, `0x005a`, `0x005b`, `0x005c`, `0x0047`, `0x0059`, `0x005d`, `0x005e`, `0x0062`, and `0x0063`.
|
|
- `0x0042` remains the single largest unresolved type even after the heuristic pass.
|
|
- `map 104` is the most obvious current outlier: it still sits at `866` fallback items versus only `136` bundle-mapped items, so the next recovery pass should treat it as the best stress case rather than relying only on the now-mostly-readable early maps.
|
|
|
|
Those counts are now historical rather than current for the focused `map 104` export. They remain useful as the baseline that motivated the cohort pass, but the immediate blocker has shifted from “remove placeholders” to “replace provisional donor provenance with tighter executable-backed cohort/resource rules.”
|
|
|
|
## Practical Interpretation
|
|
|
|
- The donor heuristic is good enough to prove that many placeholders were caused by missing extraction-time inheritance logic.
|
|
- It is not strong enough to count as the final executable-backed rule for the unresolved families.
|
|
- The remaining gap still sits where the earlier Ghidra work already pointed: somewhere between the constructor-side art/resource creation lane and the live post-spawn state/resource/frame reselection path.
|
|
|
|
## Palette Lock-In Rule
|
|
|
|
- The PSX exporter must not treat `mode 1` bundle header palette index `+0x14` as the rendered 256-color selector.
|
|
- The current dump-grounded rule is narrower and stronger: the known-good visible family decodes against one contiguous 256-entry lookup table equivalent to live VRAM row `0xF0`, `x=0`.
|
|
- In extractor terms, that behaves like the first `16` adjacent `16-color` CLUTs flattened into one 256-entry palette. That is the rule now encoded in `map_renderer/src/lib/psx-cache.js` for `mode 1` bundles.
|
|
- The old bundle-header palette index is still preserved in exported scene metadata as `defaultPaletteIndex`, but it is diagnostic provenance, not the primary rendered selector.
|
|
- `mode 2` stays on the earlier per-bundle/per-usage palette-index path. This lock-in applies specifically to the current `mode 1` family proven by the VRAM dump.
|
|
- If a future pass proves a wider runtime CLUT-row formula, update this note and the exporter together in one change. Do not silently reintroduce `mode 1 -> bundle header palette index` as a fallback rule.
|
|
|
|
## Static Export Follow-Up
|
|
|
|
- The processed PSX catalog already carried `62` maps during this pass, so the "single map" symptom was not a cache-build enumeration failure.
|
|
- The immediate export-side blocker was config: `psx-remorse` was excluded from static export even though the prebuilt catalog/build-manager path already supports multi-map PSX scenes.
|
|
- The renderer config now includes `psx-remorse` in static export so full PSX exports can surface the full processed map set instead of dropping the version entirely.
|
|
|
|
## Live MCP Follow-Up (2026-04-12): Wall-Family Split Failure Mode
|
|
|
|
- Scope for this focused pass: live `SLUS_002.68` around the wall-heavy generic family band (`0x003e..0x004f`) and exporter donor-heavy bundles (`0x0008b48c`, `0x00085c40`).
|
|
- Constructor-side bind is now explicit and consistent across this family: both `psx_object_create_simple_record` (`0x800249f4`) and `psx_object_create_compound_record` (`0x80024eec`) read the per-type active art header from `DAT_800758d8[type]` before any lane-orientation updates.
|
|
- That means cross-family visual collapse is unlikely to be caused by a missing pre-constructor resource-bank split in this band; divergence happens after bind.
|
|
- Post-bind state/frame path is also explicit: `psx_object_select_state_script` (`0x800260e8`) installs selector state (`obj+0x9e` and script cursor `obj+0x90`), then `psx_object_advance_state_script` (`0x80025d68`) latches live frame token `obj+0x94` from the script stream.
|
|
- Draw submission consumes that latched token, not raw authored selector: stage-1 geometry (`0x80040d44`) and stage-2 geometry (`0x80040f78`) both query frame geometry from `obj+0x94`, and submitters read the same token in `psx_draw_main_visible_object` (`0x80041458`) / `psx_draw_special_visible_queue` (`0x80041144`).
|
|
- The stage-1/stage-2 route split matters materially: main-visible injects authored palette token only for type bands `>=0x003e`, while special-visible does not. If exporter cohorts collapse route outcome, families can appear as repeated art with wrong face/palette behavior.
|
|
- Practical exporter implication: the strongest missing discriminator is runtime route+latch state (effective selector/latch outcome), not a new type-level resource bank key.
|
|
|
|
Conservative live-artifact updates applied in Ghidra for this pass:
|
|
|
|
- `0x80046038`: comment clarifying `DAT_800758d8[type]` install timing and post-bind divergence expectation.
|
|
- `0x80026100`: comment clarifying pre-latch selector install and risk of authored-`u4`-only grouping.
|
|
- `0x80041554`: comment clarifying main-visible-only authored palette-token lane for `>=0x003e`.
|
|
- `0x80040f88`: comment clarifying stage-2 queue still consumes `obj+0x94` but has different route semantics.
|
|
|
|
## Live MCP Follow-Up (2026-04-12): CLUT Override Routing Closure
|
|
|
|
- Scope for this focused pass: world-object draw path and submitters at `0x80041458`, `0x80041144`, `0x80044bdc`, and `0x80044e9c`, with CLUT table symbols at `0x800a9f48` and `0x800a9f66`.
|
|
- `0x800a9f48` remains `psx_clut_table_by_resource_bank` and is both load-populated (`level_palette_upload_cluts`) and draw-consumed.
|
|
- `0x800a9f66` remains `psx_clut_override_table_by_palette_token` and is draw-consumed by both submitters.
|
|
- Main-visible (`0x80041458`) injects authored high-byte palette token into submit flags for type bands `>=0x003e`; special-visible (`0x80041144`) does not inject authored token and only forwards `obj_flags & 0x0002`.
|
|
- Submitter override gate is explicit in both submitters: override is used only when `(submit_flags & 0xfffffff0) != 0`.
|
|
- For this world-object path, token `0` behaves as "no override": with only low flag bits present, submitters fall through to default bank CLUT lookup.
|
|
- `psx_image_table_submit_frame` uses token as a key into `psx_clut_override_table_by_palette_token[token]` when override is active.
|
|
- `psx_sprite_resource_submit_frame` has two override lanes based on resource format field (`resource+4`):
|
|
- format `== 2`: token keys `psx_clut_override_table_by_palette_token[token]`.
|
|
- format `!= 2`: token is used as a row key into `psx_clut_table_by_resource_bank` (`token << 4` halfword index).
|
|
|
|
Exporter-facing implication from executable evidence:
|
|
|
|
- The current flattening bug matches a real semantic split in executable code: authored token injection is lane-dependent (main-visible only) and CLUT resolution is submitter/resource-format dependent.
|
|
- A single flattened token->CLUT mapping in exporter code will miscolor mixed cohorts where route lane or submitter/resource format differs.
|
|
|
|
Conservative live-artifact updates applied in Ghidra for this pass:
|
|
|
|
- `0x800415b0`: comment clarifying main-visible token injection source and band split.
|
|
- `0x800412d0`: comment clarifying special-visible non-injection behavior.
|
|
- `0x80044e10`: comment clarifying sprite submit override gate and token-0 fallthrough.
|
|
- `0x80044eb8`: comment clarifying image-table override keying behavior.
|
|
|
|
## Next Steps
|
|
|
|
1. Trace the remaining high-volume band `0x0055..0x0063` in Ghidra with the same question used for `0x0042`: why does `DAT_800758d8` stay zero-sized while visible art still exists at runtime?
|
|
2. Use `map 104` as the primary regression target and dump its remaining fallback type/state/lane distribution before doing any broader heuristic expansion.
|
|
3. Compare unresolved zero-block types against nearby resolved donor types at the constructor/resource level, not only at the script-signature level, so borrowed bundles can be replaced with an executable-backed alias rule.
|
|
4. Keep the `DAT_800758d4` work on the bounds side unless a family-specific caller proves otherwise; this pass did not reopen that conclusion. |