164 lines
No EOL
17 KiB
Markdown
164 lines
No EOL
17 KiB
Markdown
# PSX Map Viewer And JL-9 Investigation Plan
|
|
|
|
## Scope
|
|
|
|
- Active target: retail PlayStation `SLUS_002.68` already loaded in Ghidra.
|
|
- Keep all PSX documentation in `docs/psx/`.
|
|
- Primary objective: get PSX maps loading into the existing map viewer coherently.
|
|
- Secondary objective: make PSX graphics export with the correct palette automatically instead of by partial heuristics.
|
|
- Tertiary objective: determine whether `JL-9` is a real weapon in the PSX build, how it is unlocked or granted, and which sprite/bundle represents it.
|
|
|
|
## Current State
|
|
|
|
- `docs/psx/psx.md` already closes the boot executable, the broad `LSET*.WDL` layout, and the likely split between map-like regions and graphics-like regions.
|
|
- The earlier `region00-first` viewer export is now known to be based on a bad assumption: the `~45..59` records it exposes per map are only the small top-level WDL descriptor stream, not the full level content.
|
|
- The stronger current model is a multi-section bundle layout: a top-level `0x18`-byte dispatch-record table, typed subordinate resource tables rooted at `DAT_800758cc/d0/d4/d8`, and at least one separate compressed level-state blob that is inflated into `DAT_8006769c` by `FUN_8003b00c(..., 0x3e00, 0x3e00)`.
|
|
- The strongest current graphics source remains `post_audio_region_04`.
|
|
- A first PSX debug scene has already been exported experimentally, but the active workflow is now the renderer-local `.cache` pipeline rather than `site` output.
|
|
- The active live probe now builds provisional real-art atlases in `map_renderer/src/build-psx-cache.js` from `map_renderer/STATIC_PSX` into `.cache/psx`, `.cache/reference-data/psx-remorse`, and `.cache/scene-cache/psx-remorse/...`.
|
|
- The current verified processed build exposes `62` PSX maps in the live renderer catalog under the runtime-record scene format (`4032` atlas-backed shapes, `1925` packed shared atlases after the latest atlas pass).
|
|
- The exporter root cause is now clearer: the old five-region post-audio carve was still masking the real visible payload. Loader-sized `post_audio_section_00` contains both the small `0x18` root descriptor rows and the dense 24-byte bulk placement rows, so the cache builder now recovers both visible families from that first real section instead of from the guessed `region00/region01` split.
|
|
- A verified full rebuild now carries `region00 + region01` across all `62` maps. `LSET1/L0.WDL` now emits `1189` items, `LSET1/L1.WDL` emits `754`, and every rebuilt map now reports `uniqueZCount > 1` instead of the earlier mostly-flat `z = 0` export.
|
|
- The next subordinate layers are now structurally split too: `DAT_800758d8` is the per-type art/template bank, `DAT_800758d0` feeds the simple constructor's local component payload, and `DAT_800758cc/d4` feed the compound constructor's state/variant tables. The executable model is solid, but the generic raw-file export for `DAT_800758cc/d0/d4` is not currently landing in the live scene cache, so that serialization path stays open work.
|
|
- The late LSET template bank is now less speculative too. The currently working map-local `DAT_800758d8` candidate is not the old "small typed section" guess; on retail `LSET1/L9.WDL` it decodes cleanly only when the parser treats the late large section as a bank with an embedded `+0x38` start, which is now enough to recover real bundle-backed mappings for a first subset of map types.
|
|
- The main visible bulk layer is no longer flat. The accepted `region01` placements now use the constructor-backed `+0x06` byte as provisional `z`, and `LSET1/L0.WDL` currently exports `11` distinct structured elevation levels instead of one forced `z = 0` plane.
|
|
- One renderer-side mismatch is now closed: PSX sprites use authored `item.screen` rectangles, and the bounding/highlight overlay path now uses those same authored rectangles instead of recomputing a DOS-style wireframe from provisional `world` coordinates.
|
|
- The executable now closes the last projection stage: authored object coordinates land in object fields `+0x3c/+0x40/+0x44` as `16.16` fixed-point values, and `FUN_80040d44` / `FUN_80040f78` project them with `screen_x = y - x` and `screen_y = 2*z - (x + y)/2` before writing the final screen rectangle at `+0x20..+0x2e`.
|
|
- Palette handling is partially grounded by runtime VRAM evidence, but the per-placement override rule is still missing.
|
|
- The scene/cache naming now uses executable-backed family names (`section0_dispatch_roots`, `section0_constructor_placements`) with the old `region00/region01` labels kept only as legacy aliases.
|
|
- The offline `FUN_8003b00c` path now exists in the renderer-local exporter and serializes one candidate on-disk compressed source plus the decoded `0x3e00` state buffer into the cache for each map.
|
|
- The type-to-art pass is still open. The exporter now scans parsed per-type template-bank payloads for bundle references, and it no longer promotes the disproven scan-order bundle fallback into visible map art. Unverified types stay on placeholders until the executable state/type path yields a real art binding.
|
|
- That loader-shaped bank selection is now already paying off in the live cache: map `9` moved from `0` resolved bundle-mapped items to `111` after the template pass switched to the embedded late-section parse, even though unresolved root-dispatch families such as `0x0042` and `0x0049` still need the downstream state/variant path before they can stop using placeholders.
|
|
- The old fallback art binding is now positively disproven for map rendering, not just "still unverified": in the live cache, early `section0_dispatch_roots` types `0x0042` and `0x0049` repeatedly bind to portrait/talk-animation bundles (for example map `0` offsets `0x000B2970` and `0x000D84F4`), which confirms the section-0 dispatch rows are generic runtime-object descriptors whose visible art still depends on downstream per-type state/variant selection.
|
|
- The executable-side type path is now clearer and named in the live PSX Ghidra database. `psx_object_create_simple_record` and `psx_object_create_compound_record` both index the same per-type banks rooted at `DAT_800758d8/d0/cc/d4`; `psx_object_select_state_script` selects an active state script from `DAT_800758cc`, `psx_object_advance_state_script` at `0x80025d68` interprets sentinel-driven script records, `psx_object_lookup_variant_entry` resolves a companion entry from `DAT_800758d4`, and `psx_reset_type_runtime_banks_from` at `0x80025ce8` is the nearby bank-reset helper that had been misnamed earlier. So the missing map-render rule is not one flat `type -> bundle` table but a multi-stage runtime selection path.
|
|
- The visible render pass is less opaque now too. `FUN_80041378` draws in three stages: the sorted visible-object list through `FUN_80041458`, a second special-visible list through `FUN_80041144`, and then HUD/overlay/icon primitives through `FUN_800416cc`. That means the remaining map-viewer gap is still mainly in world-object and special-object families, not in the HUD pass.
|
|
- The stage-2 path is now strong enough to affect renderer planning directly. `FUN_80040f78` is the queue-builder for the `FUN_80041144` pass: it projects an object just like the main `FUN_80040d44` path but appends it to `DAT_80078b70` / `DAT_80067472` instead of the main `DAT_8006ad5c` visible list. So a renderer that only models the stage-1 visible list will still miss a real world-facing object lane.
|
|
- Palette override provenance is tighter too: object field `+0xa0` is the original authored source-record pointer written by both constructors, so the current override path in `FUN_80041458` is reading authored record bytes directly rather than a hidden runtime side table.
|
|
- One narrow renderer-side consequence is now verified in output, not just in notes: the cache builder now applies the executable-backed `0x0050` selector map (`0..3 -> frame 0..3`) as a temporary fallback, and retail map `9` now exports `type=80 state_selector=1 chosen_frame=1` instead of forcing frame `0`.
|
|
- `JL-9` already appears in recovered PSX weapon-name tables, but gameplay availability and sprite identity are not yet closed.
|
|
|
|
## Success Criteria
|
|
|
|
### Map-viewer success
|
|
|
|
- At least one PSX map loads in the existing viewer with stable world placement, defensible draw order, and recognizable room/layout structure.
|
|
- The PSX path reuses the existing viewer pipeline instead of creating a separate one-off viewer.
|
|
- Exported scene data preserves enough raw metadata to keep later decomp passes reversible.
|
|
|
|
### Palette success
|
|
|
|
- Bundle export chooses the same palette family the runtime uses for that placement class.
|
|
- At least one tile-heavy scene and one object-heavy scene render with mostly correct colors without manual palette swapping.
|
|
- Palette selection logic is encoded in exporter metadata or viewer-side decode rules, not only in prose notes.
|
|
|
|
### JL-9 success
|
|
|
|
- `JL-9` is classified as one of: fully usable weapon, cut/incomplete leftover, menu-only string, or debug-only grant.
|
|
- The unlock or acquisition path is identified from executable logic, data tables, or authored content.
|
|
- The weapon's sprite or best candidate art bundle is identified and documented.
|
|
|
|
## Workstreams
|
|
|
|
## 1. Close the PSX map record format
|
|
|
|
Purpose: replace the invalid `small top-level record stream == whole level` assumption with a renderer-fed scene that includes the real bulk map substrate.
|
|
|
|
Tasks:
|
|
|
|
1. Revisit the executable loader chain around the `LSET*.WDL` stream consumer and name the section families loaded into `DAT_800678f4`, `DAT_80067720`, `DAT_800758cc/d0/d4/d8`, `DAT_800675f8`, and `DAT_8006769c`.
|
|
2. Prove which loaded section is the small top-level object/dispatch list and which section holds the actual bulk map substrate.
|
|
3. Recover the format and semantics of the compressed blob that `FUN_8003b00c` inflates into the `0x3e00` level buffer.
|
|
4. Tie one concrete subordinate record family to the constructor inputs that feed object `+0x3c/+0x40/+0x44` as `16.16` fixed-point coordinates.
|
|
5. Recover the bundle/frame binding rule for map placements well enough to stop relying on broad candidate pairing.
|
|
6. Recover the draw-order or layer rule used when multiple map records overlap.
|
|
7. Validate the corrected multi-section schema on at least `L0.WDL` and `L1.WDL` so the decode is not overfit to one level.
|
|
|
|
Expected output:
|
|
|
|
- a stable PSX placement schema recorded in `docs/psx/`
|
|
- one exporter that emits scene JSON in the same broad shape as the existing viewer pipeline
|
|
- one known-good reference map whose structure is visually recognizable
|
|
|
|
## 2. Close palette selection instead of guessing it
|
|
|
|
Purpose: make exported graphics match the runtime palette path automatically.
|
|
|
|
Tasks:
|
|
|
|
1. Continue from the already identified texture draw helpers and the caller path that reads palette override metadata from the object field currently described as `+0xa0` in the notes.
|
|
2. Determine whether the placement record itself, a second-stage runtime header, or a side table supplies the override palette index.
|
|
3. Reconcile the live VRAM `row 0xF0 / x=0` success case against the on-disk palette blob so the export path can reproduce the runtime source instead of only matching dumps.
|
|
4. Identify whether different bundle modes or resource classes use different CLUT selection rules.
|
|
5. Add exporter-side palette metadata that preserves both bundle default palette and resolved placement palette.
|
|
6. Validate against at least three anchor assets: one wall/floor-heavy tile set, one object sprite with obvious color identity, and one UI or portrait-like asset.
|
|
|
|
Expected output:
|
|
|
|
- a documented palette-selection rule in `docs/psx/`
|
|
- exported PSX atlases or frame PNGs that no longer require manual palette picking for the common solved families
|
|
- a short unresolved list only for genuinely exceptional palette cases
|
|
|
|
## 3. Integrate the PSX decode into the existing map viewer
|
|
|
|
Purpose: stop treating PSX as a disconnected experiment and make it a first-class renderer source.
|
|
|
|
Tasks:
|
|
|
|
1. Define one PSX scene format version that keeps raw decode fields visible while still fitting the current viewer's atlas-plus-scene model.
|
|
2. Export one minimal but real PSX map scene from the solved map schema and load it through the existing viewer path.
|
|
3. Compare the rendered result against in-game screenshots, captured VRAM/framebuffer evidence, or clearly identifiable room geometry.
|
|
4. Tighten the exporter until one map reads coherently before trying to bulk-export the entire disc.
|
|
5. Only after a coherent single-map success, generalize to more `LSET` maps and add any PSX-specific catalog or loader toggles the viewer needs.
|
|
|
|
Expected output:
|
|
|
|
- one coherent PSX map visible in the existing viewer
|
|
- one stable exporter path that can be iterated on without forking the viewer architecture
|
|
|
|
## 4. Investigate JL-9 as data, logic, and art
|
|
|
|
Purpose: close the question of whether `JL-9` is real and what it corresponds to visually.
|
|
|
|
Tasks:
|
|
|
|
1. Locate the PSX weapon-name table and the code/data structure that indexes into it.
|
|
2. Identify the item or weapon definition row for `JL-9`, including ammo type, flags, and any inventory/equipability markers.
|
|
3. Trace all code and data references to that row: mission rewards, cheats, debug grants, pickups, shop/loadout flow, or scripted usecode equivalents if present.
|
|
4. Check whether `JL-9` appears in the pre-alpha build under the same index and whether its surrounding data differs from retail.
|
|
5. Identify the sprite by following the weapon/item definition to the bundle/frame or icon resource it uses.
|
|
6. Classify the result clearly: shipped and obtainable, shipped but gated/unused, or string/data leftover only.
|
|
|
|
Expected output:
|
|
|
|
- a short `docs/psx/` note or section that states whether `JL-9` is real
|
|
- the acquisition or unlock path if one exists
|
|
- the best supported sprite or bundle match
|
|
|
|
## Recommended Execution Order
|
|
|
|
1. Finish map-record closure enough to bind placements to the right art.
|
|
2. Replace the current `.cache` runtime-record probe premise with the corrected multi-section WDL model, then recover the runtime type/resource lookup that can replace the still-provisional `u0 -> bundle index` rule with real art binding.
|
|
3. Get one map loading coherently in the existing viewer.
|
|
4. After the viewer path is grounded, use the now-stronger bundle identification flow to close `JL-9` sprite identity and availability.
|
|
|
|
## Immediate Next Batch
|
|
|
|
1. In Ghidra, tighten the section-family naming around `DAT_800678f4`, `DAT_80067720`, and the candidate `DAT_8006b5d8` source so the current `section0_*` labels can be promoted from exporter-safe names to exact loader names.
|
|
2. Record which helpers read `DAT_80067720` versus which helpers read the decompressed `DAT_8006769c` buffer now that the offline decode path is present in the cache.
|
|
3. Compare the rebuilt all-map exports against recognizable rooms and decide whether the remaining missing structure now lives mainly in the decoded `DAT_8006769c` buffer or in still-unrendered subordinate tables.
|
|
4. Tighten the raw file mappings for the newly exported runtime-bank layers (`DAT_800758d8`, `DAT_800758d0`, `DAT_800758cc`, `DAT_800758d4`) so their current section selection is proven rather than heuristic.
|
|
5. Recover an actual bundle/frame reference from the per-type template payloads or their consumers so the exporter can replace the now-disproven scan-order bundle fallback with a verified type-to-art rule.
|
|
Current delta: the template bank selection is now stronger and already recovers real art for a first subset, but the still-missing families need the stage-1/stage-2 object draw path plus `DAT_800758cc/d4` state interpretation, not more HUD/overlay decoding.
|
|
Current delta: stage 2 is no longer hypothetical. The next renderer-improvement candidate is to expose/export the queued-object lane that feeds `FUN_80041144`, because the executable now clearly maintains it separately from the main visible list.
|
|
6. Split section-0 placements into at least three executable-backed render classes: world-facing geometry/object placements, animated runtime-only objects, and clearly non-map-facing UI/talk assets such as the portrait bundles currently surfacing through fallback art matching.
|
|
7. Decode the `psx_object_advance_state_script` sentinel opcodes (`ffff`, `fffe`, `fffd`, `fffc`, `fffb`) well enough to tell when a placement loops, jumps into a subsidiary script, or fires a side-effect helper, because that state-machine branch is now the main discriminator between map-facing art and non-map runtime assets.
|
|
Current delta: `fffe` is now closed as an audio/effect dispatch through `FUN_8004061c`, so the next sentinel work should focus on the remaining control-flow opcodes.
|
|
8. In parallel with the map pass, trace the palette-override read path from the known draw helper caller and document which source field feeds the resolved CLUT.
|
|
9. Locate the `JL-9` weapon entry in the PSX executable tables and log its table index, surrounding weapon names, and all code/data xrefs.
|
|
10. Create a short follow-up note in `docs/psx/` after the batch rather than burying the result only in Ghidra comments.
|
|
|
|
## Documentation Rule For This Track
|
|
|
|
- Keep long-form findings in `docs/psx/psx.md` or another dedicated file under `docs/psx/`.
|
|
- Keep this file as the active plan and update it when a major blocker closes or the execution order changes.
|
|
- When `JL-9` closes cleanly, give it its own short note under `docs/psx/` instead of leaving it as one bullet in a larger map note. |