Crusader_Decomp/docs/psx/map-viewer-plan.md

PSX Map Viewer And JL-9 Investigation Plan

Scope

• Active target: retail PlayStation SLUS_002.68 already loaded in Ghidra.
• Keep all PSX documentation in docs/psx/.
• Primary objective: get PSX maps loading into the existing map viewer coherently.
• Secondary objective: make PSX graphics export with the correct palette automatically instead of by partial heuristics.
• Tertiary objective: determine whether JL-9 is a real weapon in the PSX build, how it is unlocked or granted, and which sprite/bundle represents it.

Current State

• docs/psx/psx.md already closes the boot executable, the broad LSET*.WDL layout, and the likely split between map-like regions and graphics-like regions.
• The earlier region00-first viewer export is now known to be based on a bad assumption: the ~45..59 records it exposes per map are only the small top-level WDL descriptor stream, not the full level content.
• The stronger current model is a multi-section bundle layout: a top-level 0x18-byte dispatch-record table, typed subordinate resource tables rooted at DAT_800758cc/d0/d4/d8, and at least one separate compressed level-state blob that is inflated into DAT_8006769c by FUN_8003b00c(..., 0x3e00, 0x3e00).
• The strongest current graphics source remains post_audio_region_04.
• A first PSX debug scene has already been exported experimentally, but the active workflow is now the renderer-local .cache pipeline rather than site output.
• The active live probe now builds provisional real-art atlases in map_renderer/src/build-psx-cache.js from map_renderer/STATIC_PSX into .cache/psx, .cache/reference-data/psx-remorse, and .cache/scene-cache/psx-remorse/....
• The current verified processed build exposes 62 PSX maps in the live renderer catalog under the runtime-record scene format (4032 atlas-backed shapes, 1925 packed shared atlases after the latest atlas pass).
• The exporter root cause is now clearer: the old five-region post-audio carve was still masking the real visible payload. Loader-sized post_audio_section_00 contains both the small 0x18 root descriptor rows and the dense 24-byte bulk placement rows, so the cache builder now recovers both visible families from that first real section instead of from the guessed region00/region01 split.
• A verified full rebuild now carries region00 + region01 across all 62 maps. LSET1/L0.WDL now emits 1189 items, LSET1/L1.WDL emits 754, and every rebuilt map now reports uniqueZCount > 1 instead of the earlier mostly-flat z = 0 export.
• The next subordinate layers are now structurally split too: DAT_800758d8 is the per-type art/template bank, DAT_800758d0 feeds the simple constructor's local component payload, and DAT_800758cc/d4 feed the compound constructor's state/variant tables. The executable model is solid, but the generic raw-file export for DAT_800758cc/d0/d4 is not currently landing in the live scene cache, so that serialization path stays open work.
• The late LSET template bank is now less speculative too. The currently working map-local DAT_800758d8 candidate is not the old "small typed section" guess; on retail LSET1/L9.WDL it decodes cleanly only when the parser treats the late large section as a bank with an embedded +0x38 start, which is now enough to recover real bundle-backed mappings for a first subset of map types.
• The main visible bulk layer is no longer flat. The accepted region01 placements now use the constructor-backed +0x06 byte as provisional z, and LSET1/L0.WDL currently exports 11 distinct structured elevation levels instead of one forced z = 0 plane.
• One renderer-side mismatch is now closed: PSX sprites use authored item.screen rectangles, and the bounding/highlight overlay path now uses those same authored rectangles instead of recomputing a DOS-style wireframe from provisional world coordinates.
• The executable now closes the last projection stage: authored object coordinates land in object fields +0x3c/+0x40/+0x44 as 16.16 fixed-point values, and FUN_80040d44 / FUN_80040f78 project them with screen_x = y - x and screen_y = 2*z - (x + y)/2 before writing the final screen rectangle at +0x20..+0x2e.
• Palette handling is partially grounded by runtime VRAM evidence, but the per-placement override rule is still missing.
• The scene/cache naming now uses executable-backed family names (section0_dispatch_roots, section0_constructor_placements) with the old region00/region01 labels kept only as legacy aliases.
• The offline FUN_8003b00c path now exists in the renderer-local exporter and serializes one candidate on-disk compressed source plus the decoded 0x3e00 state buffer into the cache for each map.
• The type-to-art pass is still open. The exporter now scans parsed per-type template-bank payloads for bundle references, and it no longer promotes the disproven scan-order bundle fallback into visible map art. Unverified types stay on placeholders until the executable state/type path yields a real art binding.
• That loader-shaped bank selection is now already paying off in the live cache: map 9 moved from 0 resolved bundle-mapped items to 111 after the template pass switched to the embedded late-section parse, even though unresolved root-dispatch families such as 0x0042 and 0x0049 still need the downstream state/variant path before they can stop using placeholders.
• The old fallback art binding is now positively disproven for map rendering, not just "still unverified": in the live cache, early section0_dispatch_roots types 0x0042 and 0x0049 repeatedly bind to portrait/talk-animation bundles (for example map 0 offsets 0x000B2970 and 0x000D84F4), which confirms the section-0 dispatch rows are generic runtime-object descriptors whose visible art still depends on downstream per-type state/variant selection.
• The executable-side type path is now clearer and named in the live PSX Ghidra database. psx_object_create_simple_record and psx_object_create_compound_record both index the same per-type banks rooted at DAT_800758d8/d0/cc/d4; psx_object_select_state_script selects an active state script from DAT_800758cc, psx_object_advance_state_script at 0x80025d68 interprets sentinel-driven script records, psx_object_lookup_variant_entry resolves a companion entry from DAT_800758d4, and psx_reset_type_runtime_banks_from at 0x80025ce8 is the nearby bank-reset helper that had been misnamed earlier. So the missing map-render rule is not one flat type -> bundle table but a multi-stage runtime selection path.
• The visible render pass is less opaque now too. FUN_80041378 draws in three stages: the sorted visible-object list through FUN_80041458, a second special-visible list through FUN_80041144, and then HUD/overlay/icon primitives through FUN_800416cc. That means the remaining map-viewer gap is still mainly in world-object and special-object families, not in the HUD pass.
• The stage-2 path is now strong enough to affect renderer planning directly. FUN_80040f78 is the queue-builder for the FUN_80041144 pass: it projects an object just like the main FUN_80040d44 path but appends it to DAT_80078b70 / DAT_80067472 instead of the main DAT_8006ad5c visible list. So a renderer that only models the stage-1 visible list will still miss a real world-facing object lane.
• Palette override provenance is tighter too: object field +0xa0 is the original authored source-record pointer written by both constructors, so the current override path in FUN_80041458 is reading authored record bytes directly rather than a hidden runtime side table.
• One narrow renderer-side consequence is now verified in output, not just in notes: the cache builder now applies the executable-backed 0x0050 selector map (0..3 -> frame 0..3) as a temporary fallback, and retail map 9 now exports type=80 state_selector=1 chosen_frame=1 instead of forcing frame 0.
• JL-9 already appears in recovered PSX weapon-name tables, but gameplay availability and sprite identity are not yet closed.

Success Criteria

Map-viewer success

• At least one PSX map loads in the existing viewer with stable world placement, defensible draw order, and recognizable room/layout structure.
• The PSX path reuses the existing viewer pipeline instead of creating a separate one-off viewer.
• Exported scene data preserves enough raw metadata to keep later decomp passes reversible.

Palette success

• Bundle export chooses the same palette family the runtime uses for that placement class.
• At least one tile-heavy scene and one object-heavy scene render with mostly correct colors without manual palette swapping.
• Palette selection logic is encoded in exporter metadata or viewer-side decode rules, not only in prose notes.

JL-9 success

• JL-9 is classified as one of: fully usable weapon, cut/incomplete leftover, menu-only string, or debug-only grant.
• The unlock or acquisition path is identified from executable logic, data tables, or authored content.
• The weapon's sprite or best candidate art bundle is identified and documented.

Workstreams

1. Close the PSX map record format

Purpose: replace the invalid small top-level record stream == whole level assumption with a renderer-fed scene that includes the real bulk map substrate.

Tasks:

1. Revisit the executable loader chain around the LSET*.WDL stream consumer and name the section families loaded into DAT_800678f4, DAT_80067720, DAT_800758cc/d0/d4/d8, DAT_800675f8, and DAT_8006769c.
2. Prove which loaded section is the small top-level object/dispatch list and which section holds the actual bulk map substrate.
3. Recover the format and semantics of the compressed blob that FUN_8003b00c inflates into the 0x3e00 level buffer.
4. Tie one concrete subordinate record family to the constructor inputs that feed object +0x3c/+0x40/+0x44 as 16.16 fixed-point coordinates.
5. Recover the bundle/frame binding rule for map placements well enough to stop relying on broad candidate pairing.
6. Recover the draw-order or layer rule used when multiple map records overlap.
7. Validate the corrected multi-section schema on at least L0.WDL and L1.WDL so the decode is not overfit to one level.

Expected output:

• a stable PSX placement schema recorded in docs/psx/
• one exporter that emits scene JSON in the same broad shape as the existing viewer pipeline
• one known-good reference map whose structure is visually recognizable

2. Close palette selection instead of guessing it

Purpose: make exported graphics match the runtime palette path automatically.

Tasks:

1. Continue from the already identified texture draw helpers and the caller path that reads palette override metadata from the object field currently described as +0xa0 in the notes.
2. Determine whether the placement record itself, a second-stage runtime header, or a side table supplies the override palette index.
3. Reconcile the live VRAM row 0xF0 / x=0 success case against the on-disk palette blob so the export path can reproduce the runtime source instead of only matching dumps.
4. Identify whether different bundle modes or resource classes use different CLUT selection rules.
5. Add exporter-side palette metadata that preserves both bundle default palette and resolved placement palette.
6. Validate against at least three anchor assets: one wall/floor-heavy tile set, one object sprite with obvious color identity, and one UI or portrait-like asset.

Expected output:

• a documented palette-selection rule in docs/psx/
• exported PSX atlases or frame PNGs that no longer require manual palette picking for the common solved families
• a short unresolved list only for genuinely exceptional palette cases

3. Integrate the PSX decode into the existing map viewer

Purpose: stop treating PSX as a disconnected experiment and make it a first-class renderer source.

Tasks:

1. Define one PSX scene format version that keeps raw decode fields visible while still fitting the current viewer's atlas-plus-scene model.
2. Export one minimal but real PSX map scene from the solved map schema and load it through the existing viewer path.
3. Compare the rendered result against in-game screenshots, captured VRAM/framebuffer evidence, or clearly identifiable room geometry.
4. Tighten the exporter until one map reads coherently before trying to bulk-export the entire disc.
5. Only after a coherent single-map success, generalize to more LSET maps and add any PSX-specific catalog or loader toggles the viewer needs.

Expected output:

• one coherent PSX map visible in the existing viewer
• one stable exporter path that can be iterated on without forking the viewer architecture

4. Investigate JL-9 as data, logic, and art

Purpose: close the question of whether JL-9 is real and what it corresponds to visually.

Tasks:

1. Locate the PSX weapon-name table and the code/data structure that indexes into it.
2. Identify the item or weapon definition row for JL-9, including ammo type, flags, and any inventory/equipability markers.
3. Trace all code and data references to that row: mission rewards, cheats, debug grants, pickups, shop/loadout flow, or scripted usecode equivalents if present.
4. Check whether JL-9 appears in the pre-alpha build under the same index and whether its surrounding data differs from retail.
5. Identify the sprite by following the weapon/item definition to the bundle/frame or icon resource it uses.
6. Classify the result clearly: shipped and obtainable, shipped but gated/unused, or string/data leftover only.

Expected output:

• a short docs/psx/ note or section that states whether JL-9 is real
• the acquisition or unlock path if one exists
• the best supported sprite or bundle match

Recommended Execution Order

1. Finish map-record closure enough to bind placements to the right art.
2. Replace the current .cache runtime-record probe premise with the corrected multi-section WDL model, then recover the runtime type/resource lookup that can replace the still-provisional u0 -> bundle index rule with real art binding.
3. Get one map loading coherently in the existing viewer.
4. After the viewer path is grounded, use the now-stronger bundle identification flow to close JL-9 sprite identity and availability.

Immediate Next Batch

1. In Ghidra, tighten the section-family naming around DAT_800678f4, DAT_80067720, and the candidate DAT_8006b5d8 source so the current section0_* labels can be promoted from exporter-safe names to exact loader names.
2. Record which helpers read DAT_80067720 versus which helpers read the decompressed DAT_8006769c buffer now that the offline decode path is present in the cache.
3. Compare the rebuilt all-map exports against recognizable rooms and decide whether the remaining missing structure now lives mainly in the decoded DAT_8006769c buffer or in still-unrendered subordinate tables.
4. Tighten the raw file mappings for the newly exported runtime-bank layers (DAT_800758d8, DAT_800758d0, DAT_800758cc, DAT_800758d4) so their current section selection is proven rather than heuristic.
5. Recover an actual bundle/frame reference from the per-type template payloads or their consumers so the exporter can replace the now-disproven scan-order bundle fallback with a verified type-to-art rule. Current delta: the template bank selection is now stronger and already recovers real art for a first subset, but the still-missing families need the stage-1/stage-2 object draw path plus DAT_800758cc/d4 state interpretation, not more HUD/overlay decoding. Current delta: stage 2 is no longer hypothetical. The next renderer-improvement candidate is to expose/export the queued-object lane that feeds FUN_80041144, because the executable now clearly maintains it separately from the main visible list.
6. Split section-0 placements into at least three executable-backed render classes: world-facing geometry/object placements, animated runtime-only objects, and clearly non-map-facing UI/talk assets such as the portrait bundles currently surfacing through fallback art matching.
7. Decode the psx_object_advance_state_script sentinel opcodes (ffff, fffe, fffd, fffc, fffb) well enough to tell when a placement loops, jumps into a subsidiary script, or fires a side-effect helper, because that state-machine branch is now the main discriminator between map-facing art and non-map runtime assets. Current delta: fffe is now closed as an audio/effect dispatch through FUN_8004061c, so the next sentinel work should focus on the remaining control-flow opcodes.
8. In parallel with the map pass, trace the palette-override read path from the known draw helper caller and document which source field feeds the resolved CLUT.
9. Locate the JL-9 weapon entry in the PSX executable tables and log its table index, surrounding weapon names, and all code/data xrefs.
10. Create a short follow-up note in docs/psx/ after the batch rather than burying the result only in Ghidra comments.

Documentation Rule For This Track

• Keep long-form findings in docs/psx/psx.md or another dedicated file under docs/psx/.
• Keep this file as the active plan and update it when a major blocker closes or the execution order changes.
• When JL-9 closes cleanly, give it its own short note under docs/psx/ instead of leaving it as one bullet in a larger map note.