psx map standalone exporter
This commit is contained in:
Marco
parent
a9153546ae
commit
2f243976b6
16 changed files with 3254 additions and 5 deletions
256
psx-map-exporter/docs/spec.md
Normal file
256
psx-map-exporter/docs/spec.md
Normal file
|
|
@ -0,0 +1,256 @@
|
|||
# PSX Map Exporter Spec
|
||||
|
||||
## Goal
|
||||
|
||||
`psx-map-exporter` is a standalone Node.js probe for Crusader PSX map extraction.
|
||||
|
||||
It exists to prove a fresh end-to-end path from raw `LSET*.WDL` input to:
|
||||
|
||||
- extracted intermediate sprite assets under `.cache`
|
||||
- a rendered map PNG under `.output`
|
||||
|
||||
This project does not reuse `Crusader-Map-Viewer` code, scene caches, donor mappings, or sidecar summaries as binding inputs. It only consumes raw PSX assets plus the documented executable-backed findings from `docs/psx` and the live Ghidra session.
|
||||
|
||||
## Scope
|
||||
|
||||
Version `v0` is intentionally narrow.
|
||||
|
||||
It will:
|
||||
|
||||
- read one PSX `LSET*.WDL` file
|
||||
- parse the documented `0x38`-byte top-level header
|
||||
- carve the post-audio map/art regions from header-derived boundaries
|
||||
- parse the loader-sized post-audio sections as a second, higher-value view of the file layout
|
||||
- extract the dense constructor-placement family from `post_audio_section_00`
|
||||
- keep the smaller root-dispatch family available as a comparison probe
|
||||
- render a layered authored probe that can combine constructor placements with the smaller root-dispatch lane
|
||||
- scan `post_audio_region_04` for type-4/type-5 sprite bundles
|
||||
- decode bundle frames directly from the raw WDL
|
||||
- write extracted frame PNGs to `.cache`
|
||||
- compose a probe map PNG to `.output`
|
||||
|
||||
It will not claim full runtime parity yet.
|
||||
|
||||
Known non-goals for `v0`:
|
||||
|
||||
- exact `DAT_800758d8/d0/cc/d4` parity
|
||||
- exact CLUT reproduction
|
||||
- full stage-1 dependency-graph ordering
|
||||
- exact type-to-resource binding for unresolved families
|
||||
- full `post_audio_region_01` / `post_audio_region_02` semantic decode
|
||||
|
||||
## Evidence Constraints
|
||||
|
||||
The implementation is grounded in these current facts from the docs and Ghidra:
|
||||
|
||||
- `LSET*.WDL` uses a fixed `0x38`-byte top-level header.
|
||||
- The second dword is the audio/SPU blob size.
|
||||
- The old region-only carve is not sufficient on its own for visible-object recovery; loader-sized `post_audio_section_00` contains both the small root-dispatch rows and the dense constructor-placement rows.
|
||||
- The file contains a post-audio area with four high-confidence absolute boundaries that split:
|
||||
- `post_audio_region_00`
|
||||
- `post_audio_region_01`
|
||||
- `post_audio_region_02`
|
||||
- `post_audio_region_03`
|
||||
- `post_audio_region_04`
|
||||
- The small count-prefixed section-0 root-dispatch rows are real, but they are not the whole map object set.
|
||||
- The dense constructor-placement records recovered from loader-sized `post_audio_section_00` are currently the best standalone live-object seed source, not a proven final visible-map layer.
|
||||
- Current strongest standalone layout read: the constructor-placement lane is a count-prefixed `12`-byte substream inside the loader-sized section-0 span rather than a whole-section `24`-byte row grid. For `LSET1/L0.WDL`, the best current candidate has a section-relative header at `0x38`, a record start at `0x3c`, and a reported count of `1182` records.
|
||||
- The constructor-placement stream can extend slightly past the nominal `post_audio_section_00` slice, so standalone parsing must follow the detected stream count from the section-0 base instead of truncating strictly at the section object boundary.
|
||||
- `post_audio_region_04` is the strongest current graphics bank candidate.
|
||||
- The direct `typeWord -> bundle slot` scan-order binding is disproven as a final art rule and is retained only as a diagnostic bundle-family probe.
|
||||
- The real art/template lane is `DAT_800758d8`, but the executable now shows two distinct late art feeds per WDL pass rather than one monolithic bank:
|
||||
- an earlier art-install blob that builds resources and temporarily mirrors them into `DAT_800758d8`
|
||||
- a later `8`-byte header-only override blob that restores raw active-header pointers into `DAT_800758d8`
|
||||
- The later header-only override is the safer standalone parser target: constructors branch on first dword `0x58` and then reuse `DAT_800758c8[type]`, so the final post-load `DAT_800758d8` state is a raw-header lane, not a permanently built-resource lane.
|
||||
- Type-4/type-5 drawable bundles expose width, height, palette mode/index, frame count, frame table offset, and data offset in the raw bundle header.
|
||||
- Bundle frame entries use a `20`-byte row with size, relative data offset, width, height, origin x/y, and flags.
|
||||
- `sprite_rle_decode_rows` uses row-local control bytes:
|
||||
- positive: repeat next byte N times
|
||||
- negative: copy next `abs(N)` literal bytes
|
||||
- zero: end row
|
||||
- The executable projection basis is:
|
||||
|
||||
$$
|
||||
screen_x = y - x
|
||||
$$
|
||||
|
||||
$$
|
||||
screen_y = 2z - \frac{x + y}{2}
|
||||
$$
|
||||
|
||||
## Input Model
|
||||
|
||||
The exporter accepts either:
|
||||
|
||||
- a direct `--wdl` path
|
||||
- or a `--source` path relative to a PSX disc root
|
||||
|
||||
Default disc root for local workspace runs:
|
||||
|
||||
- `d:/Ghidra/Crusader-Map-Viewer/map_renderer/STATIC_PSX`
|
||||
|
||||
Expected source examples:
|
||||
|
||||
- `LSET1/L0.WDL`
|
||||
- `LSET4/L37.WDL`
|
||||
|
||||
## Output Layout
|
||||
|
||||
### `.cache`
|
||||
|
||||
Per-run cache path:
|
||||
|
||||
- `.cache/<map-stem>/`
|
||||
|
||||
Contents:
|
||||
|
||||
- `wdl-summary.json`
|
||||
- `records.json`
|
||||
- `bundles.json`
|
||||
- `frame-manifest.json`
|
||||
- `active-header-overrides.json`
|
||||
- `sprites/<bundle-offset>/frame_<n>.png`
|
||||
|
||||
The cache is disposable. It exists to preserve intermediate evidence and make re-runs inspectable.
|
||||
|
||||
`records.json` now also records constructor-stream detection metadata when available: stream header offset, record start offset, reported count, and the initial structured-prefix run.
|
||||
|
||||
The cache also records candidate late `DAT_800758d8` header-only override blobs as a standalone diagnostic. Those candidates are not used as final art binding yet.
|
||||
|
||||
`wdl-summary.json` now also emits `sceneInterpretation`, which is an explicit warning-bearing classification of what the current export most likely represents. For constructor-placement exports this should currently read as a constructor-fed live-object seed lane rather than a final visible-world reconstruction.
|
||||
|
||||
### `.output`
|
||||
|
||||
Per-run final outputs:
|
||||
|
||||
- `.output/<map-stem>.png`
|
||||
- `.output/<map-stem>.json`
|
||||
- `.output/<map-stem>_<layer>.png` for each rendered authored layer when layered mode is active
|
||||
|
||||
The JSON stores the final probe scene manifest used to draw the PNG.
|
||||
|
||||
The `.output` folder is reset at the start of each export so evaluation only sees artifacts from the current run.
|
||||
|
||||
The `.output/<map-stem>.json` manifest inherits `sceneInterpretation` from `wdl-summary.json` so consumers do not need to infer that warning from prose docs alone.
|
||||
|
||||
## Record Extraction Rules
|
||||
|
||||
`v0` now uses the loader-sized `post_audio_section_00` extraction paths as the primary scene source.
|
||||
|
||||
Current interpretation constraint:
|
||||
|
||||
- `section0_constructor_placements` should currently be treated as constructor-fed world-object seed records.
|
||||
- They preserve meaningful layout and projection structure, but current evidence does not support treating them as the complete visible map or static architecture layer.
|
||||
- If a render shows coherent room layout with globally wrong or repeated art, the exporter is currently visualizing one runtime object lane without the downstream per-type bind/state path and without the separate static-world substrate.
|
||||
|
||||
Record extraction rule:
|
||||
|
||||
- `auto` / `combined` / `layered` mode merges both authored section-0 families into one layered probe:
|
||||
- constructor placements provide the dense live-object seed lane
|
||||
- root-dispatch rows provide the smaller comparison and auxiliary authored lane
|
||||
- `constructors` / `region01` mode first searches the section-0 span for a count-prefixed `12`-byte constructor stream and, when found, treats each record as six little-endian `u16` words:
|
||||
- `typeWord`
|
||||
- `xWord`
|
||||
- `yWord`
|
||||
- `zWord`
|
||||
- `selectorWord`
|
||||
- `laneWord`
|
||||
- If a count-prefixed constructor stream is not found, the exporter falls back to the older whole-section `24`-byte paired-record scan as a compatibility probe.
|
||||
- `roots` / `region00` mode keeps the small count-prefixed root-dispatch probe for comparison and negative-evidence checks
|
||||
|
||||
Plausibility filter:
|
||||
|
||||
- `typeWord` in a conservative visible-family range
|
||||
- not all coordinate words are zero
|
||||
- `laneWord` is non-zero and within the current conservative control-word range
|
||||
|
||||
This is explicitly a probe schema, not a final loader-faithful schema.
|
||||
|
||||
Current negative result:
|
||||
|
||||
- Correcting the constructor stream start/count for `LSET1/L0.WDL` only changes the standalone constructor probe slightly (`1130 -> 1135` records, `1090 -> 1095` rendered items) and does not materially change the repeated wrong-art output. Current evidence therefore points to unresolved art/runtime binding as the primary blocker, not a missed constructor-tail decode.
|
||||
|
||||
## Art Binding Rule
|
||||
|
||||
`v0` uses one explicit diagnostic binding rule:
|
||||
|
||||
- `typeWord -> bundle slot index`
|
||||
|
||||
That means the sorted bundle list from `post_audio_region_04` is indexed directly by `typeWord` when the slot exists.
|
||||
|
||||
This rule is explicitly not claimed as final executable truth. Current docs and Ghidra evidence show the final art path goes through the late `DAT_800758d8` art bank plus downstream state-script/runtime selection. The slot rule remains useful only as a clean standalone negative-evidence probe.
|
||||
|
||||
For the generic family band now dominating `LSET1/L0` failures (`0x003e`, `0x0042`, `0x0044`, `0x0045`, `0x004f`, `0x0059`, `0x005b`), repeated wrong art is now understood as both a binding failure and a semantic-layer failure: the exporter is currently visualizing constructor-fed runtime object seeds as though they were the final visible world.
|
||||
|
||||
The chosen bundle and clamped frame index, plus binding-diversity metrics, are preserved in output metadata so failures stay auditable.
|
||||
|
||||
When debug labels are enabled for a map render, labels now identify unique rendered resources rather than per-instance placements. The stable label key is currently `bundle offset + clamped frame + resolved palette`. Validation atlas sheets still use progressive cell indices.
|
||||
|
||||
## Rendering Rule
|
||||
|
||||
For each record:
|
||||
|
||||
- compute `screenX` and `screenY` from the documented projection basis
|
||||
- select frame index from `selectorWord`, clamped to available frames
|
||||
- place sprite top-left at:
|
||||
- `screenX - originX`
|
||||
- `screenY - originY`
|
||||
|
||||
Current draw order is conservative:
|
||||
|
||||
- main-visible before special-visible
|
||||
- then ascending `screenY`
|
||||
- then ascending `screenX`
|
||||
|
||||
This is a probe approximation. The later graph-based stage-1 ordering still belongs to a future pass.
|
||||
|
||||
The rendered PNG uses a neutral opaque background by default so probe silhouettes are legible without relying on transparency.
|
||||
|
||||
## Color Rule
|
||||
|
||||
`v0` emits grayscale art from raw pixel indices.
|
||||
|
||||
Reason:
|
||||
|
||||
- bundle frame decode is already well constrained
|
||||
- full CLUT parity is not
|
||||
- grayscale preserves shape/variant evidence without pretending the palette problem is solved
|
||||
|
||||
Transparent index `0` stays transparent.
|
||||
|
||||
## CLI
|
||||
|
||||
Primary command:
|
||||
|
||||
```powershell
|
||||
node src/cli.js --source LSET1/L0.WDL
|
||||
```
|
||||
|
||||
Supported options:
|
||||
|
||||
- `--source <relative-path>`
|
||||
- `--wdl <absolute-or-relative-file>`
|
||||
- `--disc-root <path>`
|
||||
- `--map-source <auto|combined|layered|constructors|roots|region01|region00>`
|
||||
- `--out-name <stem>`
|
||||
|
||||
## Success Criteria
|
||||
|
||||
`v0` is successful if it can:
|
||||
|
||||
- parse a raw `LSET*.WDL`
|
||||
- recover the loader-sized section view alongside the region carve
|
||||
- scan bundles directly from `post_audio_region_04`
|
||||
- decode at least one frame from raw data
|
||||
- extract a stable constructor-placement record set from `post_audio_section_00`
|
||||
- write extracted sprite PNGs into `.cache`
|
||||
- write a readable diagnostic probe PNG into `.output`
|
||||
|
||||
## Planned Follow-Ups
|
||||
|
||||
- replace diagnostic slot binding with a direct parser for the late header-only `DAT_800758d8` override stream and bundle match path
|
||||
- recover the exact raw on-disk encoding of the earlier built-resource art-install blob so the two late art feeds are modeled separately instead of flattened into one guessed bank
|
||||
- identify and parse the separate static-world or subordinate level substrate that complements the constructor-fed live-object lane, instead of treating section-0 constructor placements as the whole map
|
||||
- add palette/CLUT reconstruction
|
||||
- add stage-1 graph ordering recovery
|
||||
- compare the probe scene against fixed live samples such as `map 104` without reintroducing viewer-side donor assumptions
|
||||
Loading…
Add table
Add a link
Reference in a new issue