PSX Decompilation
This commit is contained in:
parent
56f6099820
commit
bbd29b1f10
25 changed files with 1921 additions and 701 deletions
|
|
@ -231,19 +231,106 @@ Verified first batch landed in the live `CRUSADER.EXE` session on 2026-04-05.
|
|||
- Exercised the new storage-aware prototype route against the two known 16-bit repair cases (`1000:42e2` and `1420:1499`) through the active MCP session. The checked-in source has the new route wiring, but the live GUI plugin still answered with legacy behavior: `/set_function_prototype_storage` returned the old `set_function_prototype` failure body, and `/set_storage_aware_prototype` returned `404 No context found for request`. That confirms the remaining issue is live deployment parity, not endpoint design.
|
||||
- Rechecked the direct callers of `CreateFromSlotIndex`: `Usecode_ItemCallEvent` plus two `Interpreter_NextUsecodeOp` call sites. The `Usecode_ItemCallEvent` path explicitly calls `CreateFromSlotIndex((EntityVmContext *)0x0,0,...)` as an allocate-and-return factory, and the current caller-side uses immediately consume only base `Process`-style fields such as `procid` and termination flags. The two interpreter call sites likewise just store the returned far pointer in `DX:AX` scratch pairs before later base-process handling.
|
||||
- That caller evidence is enough to keep the current conservative return type for now: `CreateFromSlotIndex` is clearly manufacturing an `EntityVmContext`, but promoting the return to `EntityVmContext *` before the inheritance/base-process datatype story is explicit would probably make current caller decompilation less clear rather than more clear.
|
||||
- Verified seventh live batch landed on 2026-04-06 after the refreshed MCP build came up.
|
||||
- Re-exercised `set_function_prototype_storage(...)` in-session on the two known 16-bit repair cases. The route now reaches the real storage-aware implementation and can preserve the explicit `AX:DX` return storage in-session, but two live issues remain: stack offsets at `10` and above currently need `0x` prefixes to avoid landing at `0x10`/`0x12`/`0x14`/`0x16`, and `calling_convention='__cdecl16far'` still normalizes the repaired functions to plain `__cdecl`.
|
||||
- Updated `/Remorse/EntityVmSlotEntry` one step deeper from the `InitSlotOwnerBuffers` and `EnsureSlotChunkLoaded` evidence:
|
||||
- `+0x00 match_key_farptr`
|
||||
- `+0x0a owner_chunk_count`
|
||||
- `+0x12 owner_data_base`
|
||||
- retained the earlier `+0x1e..+0x24` owner-buffer and chunk-state pointer pairs
|
||||
- Updated local variable typing so `AcquireSlotForEntity` now carries `EntityVmSlotEntry *` locals for the current slot cursor/free-slot candidate lane, and `InitSlotOwnerBuffers` now carries an `EntityVmSlotEntry *` local for the owner-metadata scratch object.
|
||||
- The decompiler payoff is immediate: `InitSlotOwnerBuffers` now shows `owner_chunk_count`, `owner_buffer_*`, and `chunk_state_*` directly, and `EnsureSlotChunkLoaded` now shows `owner_data_base` where the slot metadata seeds the later owner-data window.
|
||||
- Tried the stronger storage-aware `Create(this: /Remorse/EntityVmRuntime * @ stack:0x4:4, ...)` model through the new endpoint, but it still fails with `Storage size does not match data type size: 2`. That makes the remaining blocker more precise again: the live MCP route is now good enough to express the desired 4-byte storage, but the current `EntityVmRuntime *` datatype in this 16-bit NE session still resolves to a 2-byte pointer type.
|
||||
- Verified eighth live batch landed on 2026-04-06.
|
||||
- Reloaded the live plugin and re-verified `set_function_prototype_storage(...)` on the two known 16-bit proof cases. The route now works in-session and preserves explicit `AX:DX` return storage cleanly, but `calling_convention='__cdecl16far'` still normalizes both `1000:42e2` and `1420:1499` to plain `__cdecl`.
|
||||
- Renamed `1420:1d72` to `entity_vm_runtime_get_slot_chunk_ptr_at_offset` after confirming from `CreateFromSlotIndex`, `Load`, and `FUN_1418_035f` that it is just a wrapper over `EnsureSlotChunkLoaded` plus a caller-supplied offset.
|
||||
- Renamed `1420:1d8d` to `entity_vm_runtime_release_slot_chunk_ref` after confirming from the `Interpreter_NextUsecodeOp` caller that it decrements one live chunk-state refcount and asserts if the chunk was not retained.
|
||||
- Renamed `1420:1e17` to `entity_vm_runtime_try_unload_slot_chunk` after confirming from `entity_vm_runtime_apply_to_matching_owner_rows` that it only unloads a chunk when the chunk-state count has reached zero, restoring the owner-buffer entry and freeing runtime budget during cleanup/eviction.
|
||||
- Added short decompiler comments to those three helpers so the slot-entry ownership story stays visible in the live database.
|
||||
- Verified ninth live batch landed on 2026-04-06.
|
||||
- Created provisional datatype `/Remorse/EntityVmLoadedChunkRecord` with the current stable cleanup/iterator anchors:
|
||||
- `+0x06 next_offset`
|
||||
- `+0x08 next_segment`
|
||||
- `+0x0e saved_chunk_offset`
|
||||
- `+0x10 saved_chunk_segment`
|
||||
- `+0x12 slot_index`
|
||||
- `+0x14 chunk_index`
|
||||
- Updated `1420:1e17 entity_vm_runtime_try_unload_slot_chunk` so the second parameter is now `EntityVmLoadedChunkRecord * loaded_chunk_record`, and then tightened the return to `byte __cdecl16far` with explicit `AL` storage after caller disassembly at `1420:1f50` and `1420:1fc1` showed both call sites consume only `AL`.
|
||||
- Updated the iterator local `uStack_6` in `1420:1f24 entity_vm_runtime_apply_to_matching_owner_rows` to `EntityVmLoadedChunkRecord *`, so the owner-row cleanup walk now renders `next_*`, `slot_index`, and `chunk_index` directly instead of anonymous stack-pair traffic.
|
||||
- Confirmed the interpreter-side release helper caller at `1418:3330` pushes the live chunk record's `slot_index` / `chunk_index` pair from `ES:[BX+0x32]` / `ES:[BX+0x34]` together with the runtime far pointer before calling `entity_vm_runtime_release_slot_chunk_ref`, which makes the loaded-chunk record a real shared runtime record rather than a one-off cleanup scratch blob.
|
||||
- Verified tenth live batch landed on 2026-04-06.
|
||||
- Renamed local helper `1418:003c` to `interpreter_pop_saved_farptr` after confirming from its only caller in `Interpreter_NextUsecodeOp` that it decrements a saved-farptr stack count at `+0x80` and returns the far pointer stored at the new top entry.
|
||||
- Added short decompiler comments at `1418:003c` and `1418:3330` so the interpreter-side release/restore lane stays visible without overcommitting the restored far pointer to a stronger semantic than the current evidence supports.
|
||||
- Verified eleventh live batch landed on 2026-04-06.
|
||||
- Created class owner `Remorse::EntityVmSlotEntry` in the live database and moved `1420:2040` under it as `CreateOrClear`.
|
||||
- Tightened `Remorse::EntityVmSlotEntry::CreateOrClear` so the single parameter is now named `this` and the explicit far return storage is restored to `AX` for the `EntityVmSlotEntry *` result.
|
||||
- Moved the previously global runtime cleanup helpers under `Remorse::EntityVmRuntime` as real methods:
|
||||
- `1420:1d72` -> `GetSlotChunkPtrAtOffset`
|
||||
- `1420:1d8d` -> `ReleaseSlotChunkRef`
|
||||
- `1420:1cca` -> `DebugDumpSlotMemory`
|
||||
- `1420:1e17` -> `TryUnloadSlotChunk`
|
||||
- `1420:1f24` -> `ApplyToMatchingOwnerRows`
|
||||
- Tightened the `ReleaseSlotChunkRef` parameter names to `runtime_farptr`, `slot_index`, and `chunk_index`, and renamed the `DebugDumpSlotMemory` far-pointer argument to `runtime_farptr` so the runtime-owned chunk/refcount lane reads more like method code than detached helper code.
|
||||
- Verified twelfth live batch landed on 2026-04-06.
|
||||
- Tightened `Remorse::EntityVmRuntime::GetSlotChunkPtrAtOffset` to `dword __stdcall16far GetSlotChunkPtrAtOffset(dword runtime_farptr, int slot_index, int chunk_index, dword intra_chunk_offset)` after re-checking the `CreateFromSlotIndex` and `Load` callers. The current best read is: load/ensure one slot chunk through the runtime, then add a caller-supplied intra-chunk offset pair to the returned far pointer.
|
||||
- Tightened `Remorse::EntityVmRuntime::ApplyToMatchingOwnerRows` to `byte __cdecl16far ApplyToMatchingOwnerRows(dword runtime_farptr, int slot_index_filter, int chunk_index_filter)` after re-checking the `AcquireSlotForEntity` and `EnsureSlotChunkLoaded` callers. The current best read is: iterate the runtime-owned loaded-chunk list either broadly (`-1/-1`) or for one current slot/chunk pair.
|
||||
- Restored explicit return storage after the storage-aware retype pass so `GetSlotChunkPtrAtOffset` still returns its far pointer in `DX:AX` and `ApplyToMatchingOwnerRows` still returns its boolean result in `AL`.
|
||||
- Verified thirteenth live batch landed on 2026-04-06.
|
||||
- Lifted the grouped runtime methods from split-word `runtime_farptr` parameters to explicit 4-byte `EntityVmRuntime * this` storage using `/Remorse/EntityVmRuntime *32` in-session. The live signatures now read as real methods for:
|
||||
- `Create`
|
||||
- `InitSlots`
|
||||
- `ReleaseSlots`
|
||||
- `DebugDumpSlotMemory`
|
||||
- `ReleaseSlotChunkRef`
|
||||
- `GetSlotChunkPtrAtOffset`
|
||||
- `TryUnloadSlotChunk`
|
||||
- `ApplyToMatchingOwnerRows`
|
||||
- `EnsureSlotChunkLoaded`
|
||||
- `Remorse::EntityVmRuntime::Create` is the biggest change in that batch: it no longer needs the old split-word placeholder form and now holds `dword __cdecl16far Create(EntityVmRuntime * this, word owner_type, word owner_id)` with the original `AX:DX` return preserved.
|
||||
- `EnsureSlotChunkLoaded` now also carries the clearer `EntityVmRuntime * this, short slot_index, short chunk_index` signature with the original far-pointer return preserved in `DX:AX`.
|
||||
- `AcquireSlotForEntity` and `InitSlotOwnerBuffers` are now fully over that hurdle too: `AcquireSlotForEntity` returns `EntityVmSlotEntry *32` in `DX:AX`, and `InitSlotOwnerBuffers` now carries `EntityVmSlotEntry *32 slot_entry` as its third parameter.
|
||||
- Verified fourteenth live batch landed on 2026-04-06.
|
||||
- Finished the remaining straightforward VM pointer cleanup outside the hottest runtime helper cluster:
|
||||
- `1430:0000 Remorse::EntityVmOwnerResource::Create` -> `byte __cdecl16far Create(EntityVmOwnerResource * this, dword owner_resource_spec)`
|
||||
- `1430:00fd Remorse::EntityVmOwnerResource::Destroy` -> `Destroy(EntityVmOwnerResource * this, uint destroy_flags)`
|
||||
- `1420:1601 Remorse::EntityVmRuntime::Destroy` -> `byte __cdecl16far Destroy(EntityVmRuntime * this, word destroy_flags)`
|
||||
- `1420:10b6/10da/1162/118f/1278 Remorse::EntityVmContext::{FreeBuffer, SyncGlobalValueAndDispatch, Destroy, Save, Load}` now all carry explicit `EntityVmContext *32 this`
|
||||
- That leaves `CreateFromSlotIndex` as the one clearly still-complex VM signature in this family cluster: the body still shows a far `this`, but the remaining argument pack needs a dedicated caller-side recovery pass rather than another pointer-only rewrite.
|
||||
- Verified fifteenth live batch landed on 2026-04-06.
|
||||
- Recovered the mixed caller pack on `1420:0eec Remorse::EntityVmContext::CreateFromSlotIndex` far enough to replace the old anonymous split arguments with caller-backed names:
|
||||
- `dword owner_source_farptr`
|
||||
- `dword pitemno_farptr`
|
||||
- `word mode_flags`
|
||||
- `word slot_index`
|
||||
- `word value_add_offset`
|
||||
- `word intra_chunk_offset`
|
||||
- `dword ucparam_farptr`
|
||||
- `uint ucparamsize`
|
||||
- Restored explicit far return storage on `CreateFromSlotIndex` to `AX:DX` after the storage-aware apply briefly dropped it.
|
||||
- The same live pass also made the remaining endpoint weakness more concrete again: once the caller-backed custom-storage pack is applied, the endpoint still textualizes the function as plain `dword __cdecl` instead of preserving the earlier higher-level `UsecodeProcess *` / `__stdcall16far` surface, even though the decompiler now keeps the correct argument boundaries and the return really is back in `AX:DX`.
|
||||
- Current best caller-backed read for `CreateFromSlotIndex` is now narrower and more useful than the old placeholder form:
|
||||
- `owner_source_farptr` is a real far-pointer input that is persisted to context `+0x11b/+0x11d`
|
||||
- `ucparam_farptr` is a real far-pointer input copied into the backward-growing buffer at `+0x102`
|
||||
- `slot_index`, `value_add_offset`, and `intra_chunk_offset` are distinct scalar inputs rather than one collapsed anonymous pack
|
||||
- the conservative semantic story is still `factory/setup bridge that returns a far process/context pointer`, not `final inheritance-clean constructor signature`
|
||||
|
||||
Current live datatype state:
|
||||
|
||||
- `/Remorse/EntityVmOwnerResource` is the cleanest landed class in this lane so far.
|
||||
- `/Remorse/EntityVmRuntime` currently only freezes the stable tail fields and helper pointer, not the full slot-entry schema.
|
||||
- `/Remorse/EntityVmSlotEntry` now exists as a bounded helper datatype, but only the stable tail buffer-pair fields are named so far.
|
||||
- `/Remorse/EntityVmSlotEntry` now exists both as a bounded helper datatype and as a live `Remorse` class owner. Its current authored surface is intentionally small: one constructor/clear method plus the stable `match_key_farptr`, `owner_chunk_count`, `owner_data_base`, and owner-buffer / chunk-state pointer anchors.
|
||||
- `/Remorse/EntityVmLoadedChunkRecord` now exists as the shared cleanup/iteration record for the chunk-release and conditional-unload lane, with the currently stable next-link, saved-owner-buffer, slot-index, and chunk-index fields named.
|
||||
- `/Remorse/EntityVmContext` now exists and matches the current owned lifecycle cluster, but it still only records the safest field anchors rather than the full embedded mini-VM layout.
|
||||
- `apply_class_layout` succeeded for `Remorse::EntityVmOwnerResource` but failed for `Remorse::EntityVmRuntime` when the binder tried to apply a `this` type, even though plain ownership moves worked.
|
||||
- The old `apply_class_layout` dry-run null failure for `Remorse::EntityVmContext` no longer reproduces on the current live server, but the actual write-side `this` typing path is still effectively old-build behavior: the real apply and direct `set_function_this_type` calls still fail on the existing `UsecodeProcess *` lifecycle signatures with `Storage size does not match data type size: 2`.
|
||||
- The `EntityVmContext` lifecycle signatures are now locally repaired through PyGhidra: `CreateFromSlotIndex` plus `FreeBuffer` / `SyncGlobalValueAndDispatch` / `Destroy` / `Save` / `Load` all carry `EntityVmContext * this` as their first parameter.
|
||||
- `CreateFromSlotIndex` should still keep its conservative `UsecodeProcess *` return type for the moment. The allocate-and-return behavior is clear, but the known callers currently consume it through base-process fields, and the repo does not yet have an inheritance-aware `EntityVmContext : UsecodeProcess` datatype model that would make a promoted return cleaner across the call sites.
|
||||
- `CreateFromSlotIndex` should still keep a conservative semantic return in the notes for the moment. The active live endpoint now textualizes it as `dword __cdecl` after the caller-packed custom-storage cleanup, but the allocate-and-return behavior is clear, the real return storage is back in `AX:DX`, and the known callers still consume the result through base-process fields rather than through an inheritance-aware `EntityVmContext : UsecodeProcess` datatype model.
|
||||
- The runtime lane is now split more accurately: `InitSlots` and `ReleaseSlots` can carry a direct `EntityVmRuntime * this`, while `Create` still needs the split-word custom-storage form to avoid hidden return-storage breakage.
|
||||
- The first slot-entry prototype batch is tighter now that `EnsureSlotChunkLoaded` carries a real `EntityVmSlotEntry *` local on the acquired-slot path, but the wider slot-entry model is still improved rather than finished.
|
||||
- The runtime lane is grouped more accurately too: the chunk-access, chunk-ref release, debug-dump, conditional-unload, and owner-row iterator helpers now sit under `Remorse::EntityVmRuntime` instead of remaining global free functions.
|
||||
- The runtime lane is also typed more accurately now: the chunk accessor is no longer a five-word anonymous wrapper, and the owner-row iterator no longer pretends its runtime pointer is two independent split-word parameters.
|
||||
- The authored VM lane is now much closer to a real class surface than a namespace grouping: `EntityVmRuntime`, `EntityVmOwnerResource`, `EntityVmContext`, `EntityVmSlotEntry`, and the helper `EntityVmLoadedChunkRecord` all now participate in a mostly far-pointer-correct live type model, with `CreateFromSlotIndex` as the main remaining signature outlier.
|
||||
- The slot-entry model is tighter again: beyond the earlier `owner_buffer_*` and `chunk_state_*` tails, the datatype now also exposes `owner_chunk_count` and `owner_data_base`, which makes the allocator/count path in `InitSlotOwnerBuffers` and the owner-data window math in `EnsureSlotChunkLoaded` read as object state rather than anonymous offset pairs.
|
||||
- The adjacent helper map is tighter too: the slot-entry consumer side now has one pointer-plus-offset accessor, one chunk-ref release helper, one conditional-unload helper, and one named loaded-chunk iterator record instead of a mix of anonymous `1420:` placeholders and anonymous stack-pair scratch state.
|
||||
|
||||
Current scope of that batch stayed intentionally conservative:
|
||||
|
||||
|
|
@ -258,9 +345,14 @@ Best immediate next moves after this landed:
|
|||
- inspect `EnsureSlotChunkLoaded` and adjacent `1420:` helpers again now that `AcquireSlotForEntity` returns `EntityVmSlotEntry *`, and push the slot-entry type one step deeper only where the resulting local/object read is genuinely clearer
|
||||
- decide whether `CreateFromSlotIndex` can safely promote its return type from `UsecodeProcess *` to `EntityVmContext *`, or whether it should stay a factory-style bridge that only types `this`
|
||||
- if the context/base-process inheritance story becomes explicit in datatypes, revisit `CreateFromSlotIndex` return typing then; until that point, keep the current `UsecodeProcess *` return even though the body itself clearly builds an `EntityVmContext`
|
||||
- recover a storage-aware `this`-typing path for `Create` specifically; `InitSlots` and `ReleaseSlots` no longer need to stay in the unresolved set
|
||||
- decide whether `match_key_farptr` at `+0x00` should stay as a neutral far-pointer field or can now be promoted to a stronger entity/owner key name from caller-side evidence
|
||||
- recover a storage-aware `this`-typing path for `Create` specifically; the live route now works well enough to test explicit 4-byte storage, but the remaining blocker is the 2-byte `EntityVmRuntime *` datatype itself rather than endpoint reachability
|
||||
- inspect the broader `Interpreter_NextUsecodeOp` lane around `1418:3330` now that the release call and `interpreter_pop_saved_farptr` are anchored, and decide whether the loaded-chunk record can absorb any more of the surrounding save/restore stack traffic without overfitting transient interpreter locals
|
||||
- redeploy or otherwise verify the live storage-fallback `set_function_this_type` / `apply_class_layout` build, then retry the `EntityVmContext` lifecycle typing pass in-session before dropping back to local PyGhidra
|
||||
- identify one or two additional strongly owned runtime or owner-resource helpers if the live session exposes them cleanly
|
||||
- decide whether `ApplyToMatchingOwnerRows` should keep its current generic split-word parameters under `Remorse::EntityVmRuntime` or whether the first argument pair is now well enough understood to collapse into a typed runtime `this`
|
||||
- decide whether the newly clarified `runtime_farptr` argument on `GetSlotChunkPtrAtOffset` and `ApplyToMatchingOwnerRows` is enough to justify a safe typed-`this` experiment on those methods, or whether the current `EntityVmRuntime *` pointer-size issue still makes the explicit `dword runtime_farptr` form the least misleading representation
|
||||
- use the now-recovered `CreateFromSlotIndex` caller pack as the baseline for any next cleanup, and only chase a prettier return type once the base-process inheritance story is explicit enough to make that promotion a real readability win
|
||||
- keep the masked-create hub and offset-specialized wrapper ladder outside the class until caller-side role recovery is tighter
|
||||
|
||||
## Source-Emission Guidance
|
||||
|
|
|
|||
164
docs/psx/map-viewer-plan.md
Normal file
164
docs/psx/map-viewer-plan.md
Normal file
|
|
@ -0,0 +1,164 @@
|
|||
# PSX Map Viewer And JL-9 Investigation Plan
|
||||
|
||||
## Scope
|
||||
|
||||
- Active target: retail PlayStation `SLUS_002.68` already loaded in Ghidra.
|
||||
- Keep all PSX documentation in `docs/psx/`.
|
||||
- Primary objective: get PSX maps loading into the existing map viewer coherently.
|
||||
- Secondary objective: make PSX graphics export with the correct palette automatically instead of by partial heuristics.
|
||||
- Tertiary objective: determine whether `JL-9` is a real weapon in the PSX build, how it is unlocked or granted, and which sprite/bundle represents it.
|
||||
|
||||
## Current State
|
||||
|
||||
- `docs/psx/psx.md` already closes the boot executable, the broad `LSET*.WDL` layout, and the likely split between map-like regions and graphics-like regions.
|
||||
- The earlier `region00-first` viewer export is now known to be based on a bad assumption: the `~45..59` records it exposes per map are only the small top-level WDL descriptor stream, not the full level content.
|
||||
- The stronger current model is a multi-section bundle layout: a top-level `0x18`-byte dispatch-record table, typed subordinate resource tables rooted at `DAT_800758cc/d0/d4/d8`, and at least one separate compressed level-state blob that is inflated into `DAT_8006769c` by `FUN_8003b00c(..., 0x3e00, 0x3e00)`.
|
||||
- The strongest current graphics source remains `post_audio_region_04`.
|
||||
- A first PSX debug scene has already been exported experimentally, but the active workflow is now the renderer-local `.cache` pipeline rather than `site` output.
|
||||
- The active live probe now builds provisional real-art atlases in `map_renderer/src/build-psx-cache.js` from `map_renderer/STATIC_PSX` into `.cache/psx`, `.cache/reference-data/psx-remorse`, and `.cache/scene-cache/psx-remorse/...`.
|
||||
- The current verified processed build exposes `62` PSX maps in the live renderer catalog under the runtime-record scene format (`4032` atlas-backed shapes, `1925` packed shared atlases after the latest atlas pass).
|
||||
- The exporter root cause is now clearer: the old five-region post-audio carve was still masking the real visible payload. Loader-sized `post_audio_section_00` contains both the small `0x18` root descriptor rows and the dense 24-byte bulk placement rows, so the cache builder now recovers both visible families from that first real section instead of from the guessed `region00/region01` split.
|
||||
- A verified full rebuild now carries `region00 + region01` across all `62` maps. `LSET1/L0.WDL` now emits `1189` items, `LSET1/L1.WDL` emits `754`, and every rebuilt map now reports `uniqueZCount > 1` instead of the earlier mostly-flat `z = 0` export.
|
||||
- The next subordinate layers are now structurally split too: `DAT_800758d8` is the per-type art/template bank, `DAT_800758d0` feeds the simple constructor's local component payload, and `DAT_800758cc/d4` feed the compound constructor's state/variant tables. The executable model is solid, but the generic raw-file export for `DAT_800758cc/d0/d4` is not currently landing in the live scene cache, so that serialization path stays open work.
|
||||
- The late LSET template bank is now less speculative too. The currently working map-local `DAT_800758d8` candidate is not the old "small typed section" guess; on retail `LSET1/L9.WDL` it decodes cleanly only when the parser treats the late large section as a bank with an embedded `+0x38` start, which is now enough to recover real bundle-backed mappings for a first subset of map types.
|
||||
- The main visible bulk layer is no longer flat. The accepted `region01` placements now use the constructor-backed `+0x06` byte as provisional `z`, and `LSET1/L0.WDL` currently exports `11` distinct structured elevation levels instead of one forced `z = 0` plane.
|
||||
- One renderer-side mismatch is now closed: PSX sprites use authored `item.screen` rectangles, and the bounding/highlight overlay path now uses those same authored rectangles instead of recomputing a DOS-style wireframe from provisional `world` coordinates.
|
||||
- The executable now closes the last projection stage: authored object coordinates land in object fields `+0x3c/+0x40/+0x44` as `16.16` fixed-point values, and `FUN_80040d44` / `FUN_80040f78` project them with `screen_x = y - x` and `screen_y = 2*z - (x + y)/2` before writing the final screen rectangle at `+0x20..+0x2e`.
|
||||
- Palette handling is partially grounded by runtime VRAM evidence, but the per-placement override rule is still missing.
|
||||
- The scene/cache naming now uses executable-backed family names (`section0_dispatch_roots`, `section0_constructor_placements`) with the old `region00/region01` labels kept only as legacy aliases.
|
||||
- The offline `FUN_8003b00c` path now exists in the renderer-local exporter and serializes one candidate on-disk compressed source plus the decoded `0x3e00` state buffer into the cache for each map.
|
||||
- The type-to-art pass is still open. The exporter now scans parsed per-type template-bank payloads for bundle references, and it no longer promotes the disproven scan-order bundle fallback into visible map art. Unverified types stay on placeholders until the executable state/type path yields a real art binding.
|
||||
- That loader-shaped bank selection is now already paying off in the live cache: map `9` moved from `0` resolved bundle-mapped items to `111` after the template pass switched to the embedded late-section parse, even though unresolved root-dispatch families such as `0x0042` and `0x0049` still need the downstream state/variant path before they can stop using placeholders.
|
||||
- The old fallback art binding is now positively disproven for map rendering, not just "still unverified": in the live cache, early `section0_dispatch_roots` types `0x0042` and `0x0049` repeatedly bind to portrait/talk-animation bundles (for example map `0` offsets `0x000B2970` and `0x000D84F4`), which confirms the section-0 dispatch rows are generic runtime-object descriptors whose visible art still depends on downstream per-type state/variant selection.
|
||||
- The executable-side type path is now clearer and named in the live PSX Ghidra database. `psx_object_create_simple_record` and `psx_object_create_compound_record` both index the same per-type banks rooted at `DAT_800758d8/d0/cc/d4`; `psx_object_select_state_script` selects an active state script from `DAT_800758cc`, `psx_object_advance_state_script` at `0x80025d68` interprets sentinel-driven script records, `psx_object_lookup_variant_entry` resolves a companion entry from `DAT_800758d4`, and `psx_reset_type_runtime_banks_from` at `0x80025ce8` is the nearby bank-reset helper that had been misnamed earlier. So the missing map-render rule is not one flat `type -> bundle` table but a multi-stage runtime selection path.
|
||||
- The visible render pass is less opaque now too. `FUN_80041378` draws in three stages: the sorted visible-object list through `FUN_80041458`, a second special-visible list through `FUN_80041144`, and then HUD/overlay/icon primitives through `FUN_800416cc`. That means the remaining map-viewer gap is still mainly in world-object and special-object families, not in the HUD pass.
|
||||
- The stage-2 path is now strong enough to affect renderer planning directly. `FUN_80040f78` is the queue-builder for the `FUN_80041144` pass: it projects an object just like the main `FUN_80040d44` path but appends it to `DAT_80078b70` / `DAT_80067472` instead of the main `DAT_8006ad5c` visible list. So a renderer that only models the stage-1 visible list will still miss a real world-facing object lane.
|
||||
- Palette override provenance is tighter too: object field `+0xa0` is the original authored source-record pointer written by both constructors, so the current override path in `FUN_80041458` is reading authored record bytes directly rather than a hidden runtime side table.
|
||||
- One narrow renderer-side consequence is now verified in output, not just in notes: the cache builder now applies the executable-backed `0x0050` selector map (`0..3 -> frame 0..3`) as a temporary fallback, and retail map `9` now exports `type=80 state_selector=1 chosen_frame=1` instead of forcing frame `0`.
|
||||
- `JL-9` already appears in recovered PSX weapon-name tables, but gameplay availability and sprite identity are not yet closed.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Map-viewer success
|
||||
|
||||
- At least one PSX map loads in the existing viewer with stable world placement, defensible draw order, and recognizable room/layout structure.
|
||||
- The PSX path reuses the existing viewer pipeline instead of creating a separate one-off viewer.
|
||||
- Exported scene data preserves enough raw metadata to keep later decomp passes reversible.
|
||||
|
||||
### Palette success
|
||||
|
||||
- Bundle export chooses the same palette family the runtime uses for that placement class.
|
||||
- At least one tile-heavy scene and one object-heavy scene render with mostly correct colors without manual palette swapping.
|
||||
- Palette selection logic is encoded in exporter metadata or viewer-side decode rules, not only in prose notes.
|
||||
|
||||
### JL-9 success
|
||||
|
||||
- `JL-9` is classified as one of: fully usable weapon, cut/incomplete leftover, menu-only string, or debug-only grant.
|
||||
- The unlock or acquisition path is identified from executable logic, data tables, or authored content.
|
||||
- The weapon's sprite or best candidate art bundle is identified and documented.
|
||||
|
||||
## Workstreams
|
||||
|
||||
## 1. Close the PSX map record format
|
||||
|
||||
Purpose: replace the invalid `small top-level record stream == whole level` assumption with a renderer-fed scene that includes the real bulk map substrate.
|
||||
|
||||
Tasks:
|
||||
|
||||
1. Revisit the executable loader chain around the `LSET*.WDL` stream consumer and name the section families loaded into `DAT_800678f4`, `DAT_80067720`, `DAT_800758cc/d0/d4/d8`, `DAT_800675f8`, and `DAT_8006769c`.
|
||||
2. Prove which loaded section is the small top-level object/dispatch list and which section holds the actual bulk map substrate.
|
||||
3. Recover the format and semantics of the compressed blob that `FUN_8003b00c` inflates into the `0x3e00` level buffer.
|
||||
4. Tie one concrete subordinate record family to the constructor inputs that feed object `+0x3c/+0x40/+0x44` as `16.16` fixed-point coordinates.
|
||||
5. Recover the bundle/frame binding rule for map placements well enough to stop relying on broad candidate pairing.
|
||||
6. Recover the draw-order or layer rule used when multiple map records overlap.
|
||||
7. Validate the corrected multi-section schema on at least `L0.WDL` and `L1.WDL` so the decode is not overfit to one level.
|
||||
|
||||
Expected output:
|
||||
|
||||
- a stable PSX placement schema recorded in `docs/psx/`
|
||||
- one exporter that emits scene JSON in the same broad shape as the existing viewer pipeline
|
||||
- one known-good reference map whose structure is visually recognizable
|
||||
|
||||
## 2. Close palette selection instead of guessing it
|
||||
|
||||
Purpose: make exported graphics match the runtime palette path automatically.
|
||||
|
||||
Tasks:
|
||||
|
||||
1. Continue from the already identified texture draw helpers and the caller path that reads palette override metadata from the object field currently described as `+0xa0` in the notes.
|
||||
2. Determine whether the placement record itself, a second-stage runtime header, or a side table supplies the override palette index.
|
||||
3. Reconcile the live VRAM `row 0xF0 / x=0` success case against the on-disk palette blob so the export path can reproduce the runtime source instead of only matching dumps.
|
||||
4. Identify whether different bundle modes or resource classes use different CLUT selection rules.
|
||||
5. Add exporter-side palette metadata that preserves both bundle default palette and resolved placement palette.
|
||||
6. Validate against at least three anchor assets: one wall/floor-heavy tile set, one object sprite with obvious color identity, and one UI or portrait-like asset.
|
||||
|
||||
Expected output:
|
||||
|
||||
- a documented palette-selection rule in `docs/psx/`
|
||||
- exported PSX atlases or frame PNGs that no longer require manual palette picking for the common solved families
|
||||
- a short unresolved list only for genuinely exceptional palette cases
|
||||
|
||||
## 3. Integrate the PSX decode into the existing map viewer
|
||||
|
||||
Purpose: stop treating PSX as a disconnected experiment and make it a first-class renderer source.
|
||||
|
||||
Tasks:
|
||||
|
||||
1. Define one PSX scene format version that keeps raw decode fields visible while still fitting the current viewer's atlas-plus-scene model.
|
||||
2. Export one minimal but real PSX map scene from the solved map schema and load it through the existing viewer path.
|
||||
3. Compare the rendered result against in-game screenshots, captured VRAM/framebuffer evidence, or clearly identifiable room geometry.
|
||||
4. Tighten the exporter until one map reads coherently before trying to bulk-export the entire disc.
|
||||
5. Only after a coherent single-map success, generalize to more `LSET` maps and add any PSX-specific catalog or loader toggles the viewer needs.
|
||||
|
||||
Expected output:
|
||||
|
||||
- one coherent PSX map visible in the existing viewer
|
||||
- one stable exporter path that can be iterated on without forking the viewer architecture
|
||||
|
||||
## 4. Investigate JL-9 as data, logic, and art
|
||||
|
||||
Purpose: close the question of whether `JL-9` is real and what it corresponds to visually.
|
||||
|
||||
Tasks:
|
||||
|
||||
1. Locate the PSX weapon-name table and the code/data structure that indexes into it.
|
||||
2. Identify the item or weapon definition row for `JL-9`, including ammo type, flags, and any inventory/equipability markers.
|
||||
3. Trace all code and data references to that row: mission rewards, cheats, debug grants, pickups, shop/loadout flow, or scripted usecode equivalents if present.
|
||||
4. Check whether `JL-9` appears in the pre-alpha build under the same index and whether its surrounding data differs from retail.
|
||||
5. Identify the sprite by following the weapon/item definition to the bundle/frame or icon resource it uses.
|
||||
6. Classify the result clearly: shipped and obtainable, shipped but gated/unused, or string/data leftover only.
|
||||
|
||||
Expected output:
|
||||
|
||||
- a short `docs/psx/` note or section that states whether `JL-9` is real
|
||||
- the acquisition or unlock path if one exists
|
||||
- the best supported sprite or bundle match
|
||||
|
||||
## Recommended Execution Order
|
||||
|
||||
1. Finish map-record closure enough to bind placements to the right art.
|
||||
2. Replace the current `.cache` runtime-record probe premise with the corrected multi-section WDL model, then recover the runtime type/resource lookup that can replace the still-provisional `u0 -> bundle index` rule with real art binding.
|
||||
3. Get one map loading coherently in the existing viewer.
|
||||
4. After the viewer path is grounded, use the now-stronger bundle identification flow to close `JL-9` sprite identity and availability.
|
||||
|
||||
## Immediate Next Batch
|
||||
|
||||
1. In Ghidra, tighten the section-family naming around `DAT_800678f4`, `DAT_80067720`, and the candidate `DAT_8006b5d8` source so the current `section0_*` labels can be promoted from exporter-safe names to exact loader names.
|
||||
2. Record which helpers read `DAT_80067720` versus which helpers read the decompressed `DAT_8006769c` buffer now that the offline decode path is present in the cache.
|
||||
3. Compare the rebuilt all-map exports against recognizable rooms and decide whether the remaining missing structure now lives mainly in the decoded `DAT_8006769c` buffer or in still-unrendered subordinate tables.
|
||||
4. Tighten the raw file mappings for the newly exported runtime-bank layers (`DAT_800758d8`, `DAT_800758d0`, `DAT_800758cc`, `DAT_800758d4`) so their current section selection is proven rather than heuristic.
|
||||
5. Recover an actual bundle/frame reference from the per-type template payloads or their consumers so the exporter can replace the now-disproven scan-order bundle fallback with a verified type-to-art rule.
|
||||
Current delta: the template bank selection is now stronger and already recovers real art for a first subset, but the still-missing families need the stage-1/stage-2 object draw path plus `DAT_800758cc/d4` state interpretation, not more HUD/overlay decoding.
|
||||
Current delta: stage 2 is no longer hypothetical. The next renderer-improvement candidate is to expose/export the queued-object lane that feeds `FUN_80041144`, because the executable now clearly maintains it separately from the main visible list.
|
||||
6. Split section-0 placements into at least three executable-backed render classes: world-facing geometry/object placements, animated runtime-only objects, and clearly non-map-facing UI/talk assets such as the portrait bundles currently surfacing through fallback art matching.
|
||||
7. Decode the `psx_object_advance_state_script` sentinel opcodes (`ffff`, `fffe`, `fffd`, `fffc`, `fffb`) well enough to tell when a placement loops, jumps into a subsidiary script, or fires a side-effect helper, because that state-machine branch is now the main discriminator between map-facing art and non-map runtime assets.
|
||||
Current delta: `fffe` is now closed as an audio/effect dispatch through `FUN_8004061c`, so the next sentinel work should focus on the remaining control-flow opcodes.
|
||||
8. In parallel with the map pass, trace the palette-override read path from the known draw helper caller and document which source field feeds the resolved CLUT.
|
||||
9. Locate the `JL-9` weapon entry in the PSX executable tables and log its table index, surrounding weapon names, and all code/data xrefs.
|
||||
10. Create a short follow-up note in `docs/psx/` after the batch rather than burying the result only in Ghidra comments.
|
||||
|
||||
## Documentation Rule For This Track
|
||||
|
||||
- Keep long-form findings in `docs/psx/psx.md` or another dedicated file under `docs/psx/`.
|
||||
- Keep this file as the active plan and update it when a major blocker closes or the execution order changes.
|
||||
- When `JL-9` closes cleanly, give it its own short note under `docs/psx/` instead of leaving it as one bullet in a larger map note.
|
||||
294
docs/psx/psx.md
294
docs/psx/psx.md
|
|
@ -380,6 +380,9 @@ Current color blocker:
|
|||
|
||||
- both main texture draw helpers (`FUN_80044bdc` and `FUN_80044e9c`) fall back to the bundle default palette index only when no override is present
|
||||
- the important caller path at `FUN_80041458` ORs in a high-byte palette override from object/tile metadata pointed to by object field `+0xa0`
|
||||
- that `+0xa0` pointer is now tighter too: both object constructors store the original authored source-record pointer there, so the override is not coming from a hidden runtime side table. For current solved families the draw helper reads the override straight from the authored record bytes:
|
||||
- type `0x003e..0x00ab`: high byte of source word at record `+0x06`
|
||||
- type `>= 0x00ac`: high byte of source word at record `+0x0c`
|
||||
- that means standalone bundle previews can still be wrong even when the bundle parser and raw CLUT table are both correct
|
||||
- the extractor now emits wider `u16x12` raw CSV views for `post_audio_region_01` and `post_audio_region_02` because the relevant override state appears to live beyond the first 6 words of those candidate placement records
|
||||
- the current top-ranked portrait bundle (`bundle_00064478`, default palette index `106`) is a useful color-validation anchor because the grayscale frame is obviously correct while all raw-palette candidates remain visibly wrong
|
||||
|
|
@ -497,27 +500,30 @@ Current evidence-backed next step:
|
|||
|
||||
Current renderer-compatibility result:
|
||||
|
||||
- a first PSX-compatible static real-art probe scene is now exported for the public map renderer
|
||||
- exporter script:
|
||||
- `tools/psx_export_map_debug_scene.py`
|
||||
- current generated public-report outputs:
|
||||
- `k:\ghidra\Crusader_Decomp_Public\map_renderer\site\data\maps\psx-remorse\map-0\scene.json`
|
||||
- multiple copied frame atlases such as `k:\ghidra\Crusader_Decomp_Public\map_renderer\site\data\maps\psx-remorse\map-0\bundle_0003917C_frame_000.png`
|
||||
- `k:\ghidra\Crusader_Decomp_Public\map_renderer\site\data\catalog.json`
|
||||
- `k:\ghidra\Crusader_Decomp_Public\map_renderer\site\data\catalogs\psx-remorse.csv`
|
||||
- current scene characteristics:
|
||||
- source: filtered `LSET1/L0.WDL` `post_audio_region_01` paired-record candidates
|
||||
- rendered items: `1050`
|
||||
- unique bundle-backed shape definitions: `49`
|
||||
- copied atlas/frame PNGs: `62`
|
||||
- bounds: `3896 x 8431`
|
||||
- scene format version: `psx-region01-bundle-probe-v1`
|
||||
- current probe stats: `u0` span `62..111`, fallback frame count `187`
|
||||
- the old Python/site real-art probe remains useful as discarded negative evidence, but it is no longer the active viewer workflow
|
||||
- the active integration path now lives inside `k:\ghidra\crusader_map_viewer\map_renderer` and builds live data into `.cache` from `STATIC_PSX`
|
||||
- active renderer-local scripts:
|
||||
- `src/build-psx-cache.js`
|
||||
- `src/lib/psx-cache.js`
|
||||
- build entrypoint:
|
||||
- `npm run build-psx-cache`
|
||||
- current generated live-cache outputs:
|
||||
- `k:\ghidra\crusader_map_viewer\map_renderer\.cache\psx\catalog.json`
|
||||
- `k:\ghidra\crusader_map_viewer\map_renderer\.cache\reference-data\psx-remorse\reference-data.json`
|
||||
- per-map scene files under `k:\ghidra\crusader_map_viewer\map_renderer\.cache\scene-cache\psx-remorse\map-*\<fingerprint>\scene.json`
|
||||
- `k:\ghidra\crusader_map_viewer\map_renderer\Catalogs\psx_shape_catalog_remorse.csv`
|
||||
- current processed-cache characteristics from the verified build:
|
||||
- source: `k:\ghidra\crusader_map_viewer\map_renderer\STATIC_PSX`
|
||||
- scene format version: `psx-region01-provisional-art-probe-v2`
|
||||
- processed maps: `23`
|
||||
- shared shape definitions: `313`
|
||||
- shared atlases: `313`
|
||||
- largest currently useful placement-heavy maps: `LSET1/L0` (`1050` items), `LSET4/L33` (`942` items), `LSET5/L48` (`851` items), `LSET6/L51` (`463` items), `LSET7/L63` (`315` items)
|
||||
|
||||
Current art-binding hypothesis used by this probe:
|
||||
|
||||
- region-01 `u0` is treated as a provisional direct bundle index into the extracted `sprite_bundles/` set
|
||||
- region-01 `u4` is treated as a provisional frame index within that bundle, clamped to the highest available frame when out of range
|
||||
- region-01 `u4` was originally treated as a provisional frame index within that bundle, but that interpretation is now considered wrong; the constructor chain instead points to `u4` as a state/script selector candidate
|
||||
- this is evidence-backed enough to render real PSX art in the existing map renderer, but not strong enough yet to call the binding solved
|
||||
- the strongest negative check so far is that the region-01 `u5` values (`0x20`, `0x22`, `0x30`) do not match the bundle default palette indexes, so the palette-selection/control path is still missing
|
||||
|
||||
|
|
@ -540,13 +546,19 @@ New loader/data evidence from this pass:
|
|||
- little-endian words: `0x004A, 0x1603, 0x0EE7, 0x0000, 0x0001, 0x0020`
|
||||
- that record family is a better next target than the invalidated direct bundle probe because it already exposes a small type-like word (`0x004A`) plus coordinate-like words without forcing an arbitrary raw-bundle index
|
||||
|
||||
What this first public renderer pass means:
|
||||
What this renderer pass means now:
|
||||
|
||||
- the existing renderer app can now load a PSX scene bundle from the static report without any PC `FIXED.DAT` dependency
|
||||
- this is currently a real-art probe of filtered placement candidates, not a final decoded PSX map
|
||||
- the renderer now displays extracted bundle art from `post_audio_region_04` instead of synthetic colored stand-ins
|
||||
- the current output is still useful because it shows that filtered region-01 records can drive recognizable, repeatedly used PSX art through the existing renderer pipeline
|
||||
- one bad extracted origin (`1x6` sprite with `xoff=65535`) initially blew out the fit bounds; the exporter now sanitizes implausible origins before writing scene metadata
|
||||
- the live renderer can expose PSX as an optional game only after the processed cache exists; it is no longer tied to ad hoc `site` exports
|
||||
- the current active output is now a provisional real-art probe rather than a placeholder-only type/lane scene
|
||||
- the processed-cache path is now compatible with the existing shared reference-data pipeline and PC-style catalog grouping, which keeps PSX integration inside the normal viewer architecture instead of forking it
|
||||
- the old real-art probe is still valuable as negative evidence because it proved that direct raw bundle ordering produces obviously wrong scene content
|
||||
|
||||
New renderer-grounded improvement from this pass:
|
||||
|
||||
- `src/lib/psx-cache.js` now scans `post_audio_region_04` directly from `STATIC_PSX`, parses bundle headers in JavaScript, colorizes the extracted frames with the currently available default/heuristic palette path, and writes per-map bundle atlases into `.cache/reference-data/psx-remorse`
|
||||
- the live cache no longer uses only synthetic placeholder shapes for map `0`; the current `LSET1/L0.WDL` scene references `49` real atlases and `62` real sprite frames under the still-provisional direct `u0 -> bundle index` hypothesis
|
||||
- extracted bundle origins are now sanitized on import so bad `0xFFFF` offsets do not blow out the scene bounds; `LSET1/L0.WDL` is back to a sane `3896 x 8431` footprint instead of the broken `67k`-pixel-wide intermediate result
|
||||
- PSX shape definitions now use a `1x1x1` footprint and the scene items synthesize viewer-compatible `world.x/world.y/world.z` from the final screen anchors; this keeps bounding-box and preview overlays aligned with the PSX art probe instead of projecting nonsense from the raw `u1/u2/u3` words
|
||||
|
||||
Current app compatibility notes:
|
||||
|
||||
|
|
@ -562,6 +574,242 @@ Immediate implications for the next decode pass:
|
|||
- the palette override path is still the main blocker to correct final color selection even when the bundle/frame choice is plausible
|
||||
- once the bundle key and palette control path are recovered, the same scene-export path can graduate from `real-art probe` to actual PSX map rendering
|
||||
|
||||
## PSX Provisional Real-Art Probe
|
||||
|
||||
The live renderer now prefers a smaller loader-backed record family when it can normalize that family into structured placement rows, while still preserving the older dense region-01 probe as a fallback/debugging strategy.
|
||||
|
||||
What changed in this pass:
|
||||
|
||||
- the temporary Python probe established the scene structure, but the active implementation is now renderer-local JavaScript rather than a standalone exporter
|
||||
- `src/lib/psx-cache.js` now reads `STATIC_PSX`, parses `LSET*.WDL`, prefers normalized `post_audio_region_00` count-prefixed records when they pass the existing structured-candidate filter, falls back to `post_audio_region_01` otherwise, scans `post_audio_region_04` for sprite bundles, and emits per-map atlases built from the extracted PSX frame data
|
||||
- `src/build-psx-cache.js` writes the resulting processed data into the live cache tree:
|
||||
- `k:\ghidra\crusader_map_viewer\map_renderer\.cache\psx\catalog.json`
|
||||
- `k:\ghidra\crusader_map_viewer\map_renderer\.cache\reference-data\psx-remorse\reference-data.json`
|
||||
- per-map scenes under `k:\ghidra\crusader_map_viewer\map_renderer\.cache\scene-cache\psx-remorse\...`
|
||||
- `k:\ghidra\crusader_map_viewer\map_renderer\Catalogs\psx_shape_catalog_remorse.csv`
|
||||
- the viewer now detects `psx-remorse` from the processed manifest instead of from a fake PC-style source-file heuristic
|
||||
- scene items now keep the candidate PSX `x/y` words directly in `world`, use the executable-backed projection basis `screen_x = y - x`, `screen_y = 2*z - (x + y)/2` with provisional `z = 0`, and keep `1x1x1` shape footprints so overlay boxes remain usable without pretending the old PC-style world export is solved
|
||||
|
||||
Current verified processed-cache result:
|
||||
|
||||
- scene format version: `psx-runtime-record-probe-v1`
|
||||
- processed maps: `61`
|
||||
- atlas-backed shapes: `1112`
|
||||
- atlases: `1112`
|
||||
- `LSET1/L0.WDL` preferred source family: `post_audio_region_00`
|
||||
- `LSET1/L0.WDL` rendered items from the preferred family: `59`
|
||||
- `LSET1/L0.WDL` still has `1050` dense fallback `post_audio_region_01` records preserved in scene metadata for comparison
|
||||
- `LSET1/L0.WDL` resolved real-art atlases for the preferred family: `18`
|
||||
- `LSET1/L0.WDL` resolved sprite frames for the preferred family: `26`
|
||||
- `LSET1/L0.WDL` unique `u0` types in the preferred family: `18`
|
||||
- lane split:
|
||||
- `0x0020`: `26`
|
||||
- `0x0022`: `21`
|
||||
- `0x0030`: `12`
|
||||
- `LSET1/L0.WDL` current scene bounds after the runtime-record pass: `1313 x 438`
|
||||
- `LSET1/L0.WDL` currently resolves all `59` preferred-family records to real extracted bundles with `0` placeholder fallbacks, but still clamps `15` frame requests down to the highest available extracted frame index
|
||||
- one visible viewer mismatch is now separated from the remaining map-format problem: PSX sprites already draw from authored `item.screen`, but the old highlight/bounding overlay path was still recomputing DOS-style wireframes from provisional `item.world`; `scene-presentation.js` now falls back to authored screen rectangles for PSX items instead of drawing those incorrect projected boxes
|
||||
|
||||
Why this matters:
|
||||
|
||||
- this is the first live viewer path that prefers a loader-compatible, count-prefixed record family instead of treating the huge dense region-01 stream as the only scene source
|
||||
- it keeps the strongest current working assumption narrower and more explicit:
|
||||
- normalized `post_audio_region_00` rows are now the preferred placement family when they satisfy the same structural checks as the older region-01 records
|
||||
- `post_audio_region_01` remains a dense fallback evidence source instead of being silently discarded
|
||||
- the art lookup is still unresolved and must be recovered from the real runtime resource tables rather than inferred from raw bundle ordering
|
||||
- it also moves the viewer one step closer to the executable model by applying the recovered PSX projection basis directly in the cache builder instead of plotting raw `u1/u2` values on a pseudo-screen plane
|
||||
|
||||
Immediate next consequence:
|
||||
|
||||
- the next map-format batch should treat the processed `.cache` runtime-record probe as the baseline renderer target and focus on proving exactly how the normalized `post_audio_region_00` words line up with the constructor-fed `x/y/z` fields
|
||||
- the old dense region-01 path should stay available as evidence, but it should no longer be the default scene family unless the loader-backed family fails to normalize on a given map
|
||||
- that means the remaining visual corruption should now be treated primarily as a placement/schema problem again, not as a box-overlay problem; the next pass needs to recover the authoritative height lane and the exact constructor-fed field mapping instead of spending more time on DOS-style overlay math
|
||||
|
||||
## PSX Map-System Correction
|
||||
|
||||
The current live viewer export was built on the wrong premise. The `~45..59` records currently exported per PSX map are not enough to represent a whole Crusader level, and executable tracing now shows why.
|
||||
|
||||
What the loader actually does:
|
||||
|
||||
- `wdl_resource_bundle_load_by_index` reads the selected `LSET*.WDL` into multiple section pointers, not one flat placement stream.
|
||||
- The first runtime section is a top-level table at `DAT_800678f4` whose record stride is `0x18` bytes.
|
||||
- The loader iterates that first section with:
|
||||
- `for each 0x18-byte top-level record`
|
||||
- `type = record[+0x08]`
|
||||
- `dispatch through PTR_PTR_80063118[type]`
|
||||
- Those dispatch handlers do not behave like a terrain-tile walker. They construct one runtime object or a tiny object cluster at a time through `FUN_800249f4`, `FUN_80024eec`, `FUN_8003c314`, `FUN_8003c714`, and `FUN_8003cc08`.
|
||||
|
||||
Why the current export is incoherent:
|
||||
|
||||
- the current `region00`-first exporter is effectively treating that small top-level descriptor family as if it were the whole level
|
||||
- those records are only the root nodes of the level bundle's object/resource system
|
||||
- they are too few because the bulk level content lives elsewhere in the loaded bundle state
|
||||
|
||||
New executable-backed evidence for the missing bulk content:
|
||||
|
||||
- `level_resource_stream_load` and `FUN_8003917c` populate the typed runtime resource tables rooted at `DAT_800758cc/d0/d4/d8`
|
||||
- `DAT_80067720` is a small top-level `0x18` record list used by object/event-style helpers such as `FUN_80031044` and `FUN_8002b1a8`; it is not a whole-map terrain stream
|
||||
- during bundle load, `FUN_8003b00c(DAT_8006769c, &DAT_8006b5d8, 0x3e00, 0x3e00)` inflates a separate compressed blob into a dedicated level buffer
|
||||
- that decompressed buffer is carried through save/load helpers (`FUN_8003a0f4`, `FUN_80049890`) independently of the tiny top-level descriptor list, which is exactly what a real map substrate would do
|
||||
- the two `DAT_80067720` helpers are now clearer about role too:
|
||||
- `FUN_80031044` scans the `0x18`-stride rows for `0xAAAA`-tagged entries and low-6-bit selector matches, then caches a pointer to the matched row payload
|
||||
- `FUN_8002b1a8` mutates matching rows by type/id and flag bits in place
|
||||
- both behaviors fit a small event/marker/control list and do not look like whole-map geometry submission
|
||||
- the decompressed lane is more clearly persistent substrate/state than before:
|
||||
- `FUN_8003a0f4` hands `DAT_8006769c` plus `DAT_80067528` to the save helper path
|
||||
- `FUN_80049890` repacks the `DAT_8006b5d8` / `0x3e00` state lane into the `0x4000` memory-card save block
|
||||
- this strengthens the read that `DAT_8006769c` is the saved/restored map-state substrate while `DAT_80067720` stays the tiny top-level control list
|
||||
|
||||
Current safest read:
|
||||
|
||||
- the `~59` exported records are top-level WDL nodes, not the entire PSX map
|
||||
- the real PSX level is split across:
|
||||
- a small top-level descriptor stream
|
||||
- typed subordinate resource tables
|
||||
- at least one separate decompressed level-state blob
|
||||
- the viewer looks nonsensical because it is rendering only one small layer of that system and mistaking it for the full map
|
||||
|
||||
Immediate consequence for the exporter:
|
||||
|
||||
- stop treating `post_audio_region_00` as the default whole-map scene source
|
||||
- keep `post_audio_region_00` and `post_audio_region_01` as evidence sources, but pivot the next decode pass toward the multi-section WDL model recovered from the executable
|
||||
- the next map-export target must include the decompressed bundle state and/or the subordinate placement/tile resources behind the top-level `0x18` records, not just the root records themselves
|
||||
|
||||
Exporter status after the next renderer pass:
|
||||
|
||||
- the earlier five-region post-audio carve was still wrong for visible-map recovery. The corrected loader-sized section probe shows that the first post-audio section already contains both the count-prefixed top-level descriptor rows and the dense 24-byte bulk placement rows that the flat maps were missing.
|
||||
- `map_renderer/src/lib/psx-cache.js` now recovers visible families from loader-sized `post_audio_section_00` instead of treating the old guessed `post_audio_region_01` carve as the default bulk source.
|
||||
- the exported scene metadata now records those visible families under executable-backed names instead of the old provisional labels:
|
||||
- `section0_dispatch_roots` for the top-level dispatch/root records
|
||||
- `section0_constructor_placements` for the dense constructor-fed placement records
|
||||
- a verified full rebuild now exports all `62` PSX maps with large scene volumes and non-flat `z` stats. `LSET1/L0.WDL` now emits `1189` items, `LSET1/L1.WDL` jumps from `53` items to `754`, and the rebuilt catalog reports `62/62` maps with `section0_dispatch_roots + section0_constructor_placements` coverage and `uniqueZCount > 1`.
|
||||
- the renderer-side reference payload no longer emits one atlas per resolved PSX shape. The new packed-atlas pass reduces the shared PSX reference cache from the old `4032` one-shape atlases to `1925` shared packed atlases across the same `4032` shape definitions, and a spot-check on `LSET1/L0.WDL` now exports the map scene itself with `atlasCount = 1` instead of a long per-bundle atlas list.
|
||||
- the cache export still carries the parsed `DAT_800758d8` candidate section and an offline `FUN_8003b00c` decode candidate for the compressed source feeding `DAT_8006b5d8 -> DAT_8006769c`, but the generic raw-file `DAT_800758cc/d0/d4` serialization is not currently landing in the live scene cache and should be treated as an open exporter gap rather than a closed layer.
|
||||
- this still does not mean the PSX map decode is fully solved: the viewer now has enough volume to represent whole-level candidates across the disc, but the remaining blocker is semantic decoding of the subordinate runtime banks and the separate decompressed `0x3e00` buffer, not record-count starvation.
|
||||
- the type-to-art path is only partially improved. The cache builder now scans the parsed per-type art-template payloads for bundle references, and the renderer no longer treats the disproven scan-order `u0 -> bundle` mapping as trustworthy visible art. Unverified types now stay on placeholder art instead of surfacing known-bad portrait/talk bundles as map geometry.
|
||||
- the scan-order fallback is now known to be wrong at the root, not merely incomplete. In the live `.cache` output, `section0_dispatch_roots` types `0x0042` and `0x0049` repeatedly bind to portrait/talk-animation bundles such as map `0` type `0042` -> offset `0x000B2970` and map `0` type `0049` -> offset `0x000D84F4`, with the same failure pattern continuing through early maps. Those portrait bundles are useful negative evidence: they show the top-level dispatch rows are generic object/state descriptors, not a direct map-graphics stream that can be paired to bundle order.
|
||||
|
||||
Next decoded runtime layers from the constructor pass:
|
||||
|
||||
- `DAT_800758d8` is the per-type art/template bank, not the missing whole-map substrate. `wdl_resource_bundle_load_by_index` populates it from an `8`-byte descriptor table, and both `FUN_800249f4` and `FUN_80024eec` consume it before calling `FUN_80044434` through the loader-side helper path.
|
||||
- `DAT_800758d0` is a per-type companion/component bank for the simpler constructor family. `FUN_800249f4` copies the resolved pointer from that bank into the local object payload at `obj->8->[0,4]`, so this looks like a per-type component/template block rather than a top-level placement stream.
|
||||
- `DAT_800758cc` is a per-type offset-table bank for the compound constructor family. `FUN_80024eec` stores it at `obj+0x88`, and `FUN_800260e8` later indexes it with the placement byte at `record+0x08` to resolve a state/offset subrecord into `obj+0x8c/0x90`.
|
||||
- `DAT_800758d4` is another per-type companion bank for the compound constructor family. `FUN_80024eec` stores it at `obj+0x84`, and `FUN_8002841c` queries it later using the object's `+0x94` selector, so it behaves like a variant table or companion lookup rather than raw map geometry.
|
||||
- The key functions in that chain are now renamed in the live PSX Ghidra database:
|
||||
- `FUN_800249f4` -> `psx_object_create_simple_record`
|
||||
- `FUN_80024eec` -> `psx_object_create_compound_record`
|
||||
- `FUN_80025ce8` -> `psx_reset_type_runtime_banks_from`
|
||||
- `FUN_80025d68` -> `psx_object_advance_state_script`
|
||||
- `FUN_800260e8` -> `psx_object_select_state_script`
|
||||
- `FUN_8002841c` -> `psx_object_lookup_variant_entry`
|
||||
- `FUN_8003917c` -> `psx_load_type_state_banks`
|
||||
- `FUN_80044434` -> `psx_create_image_resource_from_descriptor`
|
||||
- `FUN_80045ffc` -> `psx_cache_type_art_descriptor`
|
||||
- the constructor/runtime chain is now clearer too:
|
||||
- `psx_reset_type_runtime_banks_from` is a bank reset helper used during init/recycle paths; it clears `DAT_800758c4/c8/cc/d0/d4/d8` from the requested type index upward and is not the state interpreter itself.
|
||||
- `psx_object_create_simple_record` and `psx_object_create_compound_record` are two placement constructors for different section-0 row layouts, but both index the same per-type runtime banks by type id before any final render-facing selection is made.
|
||||
- `psx_create_image_resource_from_descriptor` turns the `DAT_800758d8` per-type descriptor into a renderable resource/header object; this is why `DAT_800758d8` should be read as an art/template descriptor bank, not as a whole-map tile layer.
|
||||
- `psx_object_select_state_script` selects a state or animation subrecord from `DAT_800758cc` using a placement byte (`record+0x08` in the compound family), storing the resolved script/state pointer at `obj+0x8c/0x90` and the selector at `obj+0x9e`.
|
||||
- `psx_object_advance_state_script` then interprets the active state script with sentinel/control values such as `0xffff`, `0xfffe`, `0xfffd`, `0xfffc`, and `0xfffb`, so the visible frame path is explicitly state-driven rather than just "type id -> one bundle".
|
||||
- The current renderer-side consequence is important: section-0 word `u4` is no longer treated as a verified sprite-frame index. It is now carried forward as a state-selector candidate in exported scene metadata until the `DAT_800758cc/d4` path is decoded far enough to pick the right animation frame from executable evidence.
|
||||
- Current strongest sentinel read:
|
||||
- `0xfffe` dispatches `FUN_8004061c`, which is an audio/effect helper rather than a visible-frame selector.
|
||||
- `0xfffd` is an in-script jump/re-anchor control that rewrites `obj+0x90` relative to the current script base.
|
||||
- `0xfffc` switches `obj+0x8c/0x90` to another subsidiary script selected through the `DAT_800758cc` offset table.
|
||||
- `0xfffb` also switches into a subsidiary script, but first scans forward to an in-script `0xfffd` marker before choosing the destination entry.
|
||||
- Current best read of those sentinels:
|
||||
- `0xffff` marks a terminal or restart control that re-anchors the script at `obj+0x8c` and raises object-state flags.
|
||||
- `0xfffe` dispatches a side-effect helper (`FUN_8004061c`) using the following word as a parameter before advancing.
|
||||
- `0xfffd`, `0xfffc`, and `0xfffb` switch into subsidiary scripts through the `DAT_800758cc` offset table rooted at `obj+0x88`.
|
||||
- `psx_object_lookup_variant_entry` finally uses `obj+0x94` to look up a companion entry in `DAT_800758d4`, which means even after construction the art-facing choice is still mediated by per-type variant/state tables.
|
||||
- This means the next PSX layers are now at least structurally separated:
|
||||
- visible root descriptors (`section0_dispatch_roots`, legacy alias `region00`)
|
||||
- visible bulk placement candidates (`section0_constructor_placements`, legacy alias `region01`)
|
||||
- per-type art/template descriptors (`DAT_800758d8`)
|
||||
- per-type simple-object component blocks (`DAT_800758d0`)
|
||||
- per-type compound state-offset tables (`DAT_800758cc`)
|
||||
- per-type compound variant tables (`DAT_800758d4`)
|
||||
- the still-separate decompressed `0x3e00` level-state buffer (`DAT_8006769c`)
|
||||
- The current renderer pass now records those banks explicitly as exported scene/state layers, while still only rendering the first two as visible scene items.
|
||||
- Immediate map-viewer consequence: the current fallback art probe should be treated only as a diagnostic overlay for candidate bundle families. A workable renderer will need to recover the per-type `DAT_800758d8` descriptor mapping and the downstream `DAT_800758cc/d4` state+variant selection path before it can decide whether a section-0 placement should show world geometry, an animated object, or something non-map-facing like a portrait/talk asset.
|
||||
- The next loader-side correction is now verified in the live cache too: the effective late `LSET*.WDL` `DAT_800758d8` candidate is not the earlier small-section heuristic, but a large late section whose working descriptor stream begins at an embedded `+0x38` offset. On retail map `9` that correction alone lifts `bundleMappedItemCount` from `0` to `111`, which is enough to restore real bundle-backed art for a first subset of types without reintroducing the disproven scan-order fallback.
|
||||
The still-unresolved root-dispatch families remain instructive rather than contradictory. `0x0042` and `0x0049` still stay on placeholders after the bank-selection fix, but the same pass now decodes their `DAT_800758cc` state rows more cleanly: type `0x0042` carries three selector-targeted scripts (`0`, `1`, `2`) that all terminate through `0xffff`, while type `0x0049` carries a single selector-`0` script. So the remaining blocker for those roots is no longer "find any late template bank at all"; it is the deeper `DAT_800758cc/d4` state-to-visible-art bridge.
|
||||
A first renderer-safe bridge landed even with that exporter gap still open: the verified `0x0050` state-script mapping (`selector 0..3 -> frame 0..3`) is now applied as a narrow fallback in the cache builder, and the rebuilt live map-9 scene now shows `type=80 state_selector=1 chosen_frame=1` instead of the old forced `chosen_frame=0`. Unresolved fallback placeholders are also now clamped to `opacity=0.45` in live scene output so the still-missing families stop visually overpowering the recovered real art. This remains intentionally scoped: the fallback frame map only covers the one family with direct executable-backed frame evidence, and the opacity clamp is diagnostic relief rather than a decoding claim.
|
||||
The current draw split is clearer too. `FUN_80041378` is a three-stage render pass:
|
||||
- stage 2: a second special-visible list drawn by `FUN_80041144`
|
||||
- stage 3: HUD/overlay/icon primitives from `FUN_800416cc`
|
||||
- That split matters for the map-viewer target: stages 1 and 2 remain relevant to missing world-facing content, while stage 3 is mostly front-end or overlay material and should not be mistaken for the missing half of the map.
|
||||
- Stage 2 is now materially better understood and is no longer just a read-side observation:
|
||||
- `FUN_80040f78` is the queue-builder for that pass. It projects an object with the same fixed-point world-to-screen math as `FUN_80040d44`, writes the final screen rectangle to `+0x20..+0x2e`, then appends the object to `DAT_80078b70` and increments `DAT_80067472`.
|
||||
- `FUN_80041144` consumes that queue directly, iterating `DAT_80078b70[0 .. DAT_80067472)` and submitting sprite primitives through the same texture draw helpers as the main object pass.
|
||||
- `FUN_80044fec` resets the queue each frame by clearing `DAT_80067472` after the top-level draw pass.
|
||||
- So the stage-2 list is not UI/HUD noise and not a duplicate of the main clipped visible list. It is a distinct world-facing queued-object lane, which is now a concrete candidate explanation for part of the still-missing map content in the viewer.
|
||||
- The immediate caller-side consequence matters too:
|
||||
- `FUN_80040d44` remains the main clipped visible-list toggle, calling the stage-1 add/remove helpers when an object enters or leaves the screen.
|
||||
- The recovered post-state-advance updater family now splits into five visible call sites: `0x80012b44`, `0x80013524`, `0x80013564`, `0x80013650`, and `0x80013778` all call `psx_object_advance_state_script`.
|
||||
- Three of those sites then feed the main stage-1 projector path through `FUN_80040d44` (`0x80012b60`, `0x8001357c`, `0x800136d4`), while two feed the stage-2 queue-builder path through `FUN_80040f78` (`0x8001352c`, `0x80013780`).
|
||||
- That exact `3` versus `2` split matters because it tightens the earlier claim: stage-2 membership is tied to a narrower runtime object/state branch after state advance, not to the decompressed substrate buffer alone and not to all state-advanced objects indiscriminately.
|
||||
- One state-script sentinel is now functionally closed too: `0xfffe` dispatches `FUN_8004061c`, which is an audio/effect helper rather than a visible-frame selector. That shrinks the unknown sentinel set for the remaining `DAT_800758cc` script work.
|
||||
- The main visible-list helpers are now also separated cleanly enough to stop treating them as a blocker:
|
||||
- `FUN_8002d240` adds an object to the stage-1 `DAT_8006ad5c` visible-list array.
|
||||
- `FUN_8002d35c` removes an object from that same array.
|
||||
- `FUN_8002d59c` returns the sorted slice that `FUN_80041378` iterates for the stage-1 world-object pass.
|
||||
- `FUN_8002d6f8` and `FUN_8002d778` act as refresh/rebucket/sort helpers over that main list.
|
||||
- This is an important scope reduction for renderer work: the remaining missing world content is now less likely to be caused by misunderstanding the main stage-1 visibility array itself, and more likely to live in the separate stage-2 queued-object pass plus the still-unresolved `DAT_800758cc/d4` state-to-art path.
|
||||
|
||||
Recovered next visible layer from the bulk placement family:
|
||||
|
||||
- The structured `section0_constructor_placements` rows are no longer height-agnostic. The `FUN_80024eec` constructor reads its authored elevation from byte `+0x06` of the input record, which corresponds to the low byte of the current exported `u3` word for the accepted bulk-placement records.
|
||||
- That byte is not just random payload on the accepted rows. Under the corrected section-0 scan, the same ladder generalizes across the whole rebuilt catalog instead of only the earlier `L0` subset. `LSET1/L0.WDL` still collapses to `11` distinct height values (`0, 2, 4, 10, 12, 14, 18, 20, 22, 24, 26`), and `LSET1/L1.WDL` now exposes `9` distinct levels with a `z` range of `0..32`.
|
||||
- The PSX cache builder now uses that recovered `z` byte for `section0_constructor_placements` projection instead of forcing the whole bulk layer onto `z = 0`, while the top-level `section0_dispatch_roots` descriptor stream stays at `z = 0` until its own constructor-backed height source is proven.
|
||||
- This is now the first PSX export pass in the viewer pipeline that produces visibly multi-layer whole-map candidates across the rebuilt retail catalog from executable-backed height data rather than from a single flattened candidate layer.
|
||||
|
||||
## PSX Coordinate Model From Executable
|
||||
|
||||
The current coordinate problem is no longer a renderer-only guess. The executable now closes the last projection step well enough to treat PSX placement as its own map-space model instead of as a PC-style direct world export.
|
||||
|
||||
Key function evidence:
|
||||
|
||||
- `FUN_800249f4` and `FUN_80024eec` are constructor paths that load authored coordinates into object fields `+0x3c`, `+0x40`, and `+0x44` as `16.16` fixed-point values.
|
||||
- For the first family, the source record shape is now strong enough to describe directly:
|
||||
- `u16` word at record `+0x08` -> object `+0x3c` as `value << 16`
|
||||
- `u16` word at record `+0x0a` -> object `+0x40` as `value << 16`
|
||||
- `u8` byte at record `+0x0c` -> object `+0x44` as `value << 16`
|
||||
- `FUN_80040d44` and `FUN_80040f78` are the projection helpers that turn those fixed-point object coordinates into the per-object screen rectangle stored at `+0x20..+0x2e`.
|
||||
- `FUN_80041458` and `FUN_80041144` then consume that already-built rectangle directly during draw submission; they do not derive screen position on the fly.
|
||||
|
||||
Recovered projection model:
|
||||
|
||||
- `+0x3e` and `+0x42` are not separate authored fields. They are the high `16`-bit halves of the fixed-point `x` and `y` values stored at `+0x3c` and `+0x40`.
|
||||
- The runtime builds an intermediate screen anchor in fixed-point at `+0x78/+0x7c` from those world coordinates:
|
||||
- `screen_anchor_x = y - x`
|
||||
- `screen_anchor_y = 2 * z - (x + y) / 2`
|
||||
- `FUN_80040d44` computes that anchor with the exact writes:
|
||||
- `obj+0x78 = ((y_hi - x_hi) << 16)`
|
||||
- `obj+0x7c = (obj_z * 2) - ((x_hi + y_hi) << 15)`
|
||||
- The projection helper then subtracts the current camera anchor from `DAT_800678d4 + 0x3c/+0x40`, subtracts sprite-frame origin/size metadata from `FUN_8004513c`, `FUN_800451d0`, `FUN_80045014`, and `FUN_800450a8`, and writes the final visible rectangle into `+0x20..+0x2e`.
|
||||
|
||||
What this means for the viewer:
|
||||
|
||||
- the PSX map does not want the PC viewer's current synthetic `world.x/world.y/world.z` guess based directly on raw candidate words
|
||||
- the most defensible renderer-side export target is now the runtime's own projected anchor or the equivalent fixed-point world tuple that reproduces the same `screen_anchor_x/screen_anchor_y` formulas
|
||||
- any importer that treats the raw authored coordinates as if they were already PC-style isometric world coordinates will bunch objects together or smear them across the map because PSX uses a different projection basis
|
||||
- the current cache builder no longer synthesizes PC-style world coordinates from final screen anchors; it now keeps the candidate PSX `x/y` words directly in exported scene items and applies the runtime projection basis separately during anchor generation
|
||||
|
||||
Open parts that still matter:
|
||||
|
||||
- this closes the final world-to-screen math, but it does not yet prove which raw `post_audio_region_01` or `post_audio_region_00` record family feeds each constructor path
|
||||
- it also does not close the type/resource lookup that selects the correct bundle/frame through `DAT_800758cc/d0/d4/d8`
|
||||
- palette override remains a separate unresolved control path layered on top of the now-understood projection math
|
||||
|
||||
Immediate consequence for the next pass:
|
||||
|
||||
- the next executable-guided decode step should map candidate authored record words directly onto constructor inputs, not onto PC-style scene coordinates
|
||||
- once the correct record family is tied to `FUN_800249f4` or `FUN_80024eec`, the renderer can export either:
|
||||
- the raw fixed-point PSX world tuple, plus a viewer-side reproduction of the runtime projection, or
|
||||
- the runtime-equivalent projected anchor/rectangle directly for debug rendering
|
||||
- the cache builder now uses the recovered projection basis and prefers the loader-backed record family, but the exact record-to-constructor link and the authoritative height lane still need proof before this can be called a solved map export
|
||||
|
||||
## PSX Script / Usecode Equivalent
|
||||
|
||||
Current status:
|
||||
|
|
|
|||
|
|
@ -43,6 +43,7 @@ That set gives the high-level target, the current candidate families, the rebuil
|
|||
- [docs/entity-class-family-split.md](docs/entity-class-family-split.md): conservative split of the large `Entity` lane into base, projectile, debris, corpse/actor, and adjacent non-entity families.
|
||||
- [docs/entity-vm-runtime-owner-resource-layout.md](docs/entity-vm-runtime-owner-resource-layout.md): current runtime/helper/context ownership model for the VM lane.
|
||||
- [docs/presentation-callback-broker-layout.md](docs/presentation-callback-broker-layout.md): current object/lifecycle/vtable evidence for the `0x4588` presentation-state callback broker family.
|
||||
- [docs/usecode-debugger-break-state-layout.md](docs/usecode-debugger-break-state-layout.md): current object/lifecycle/layout evidence for the dormant seg1408 debugger-state family.
|
||||
|
||||
### 4. Execution Checklists
|
||||
|
||||
|
|
@ -87,6 +88,34 @@ The future MCP endpoint sequence should follow the spec note rather than ad hoc
|
|||
3. Add one more dedicated note for the callback/object lane around `0x4588` only if later caller evidence supports a stronger subsystem name than `PresentationCallbackBroker`.
|
||||
4. Turn the first-class-authoring checklist into a completed execution log once the first real MCP batch lands.
|
||||
|
||||
## Current Live Authoring Snapshot
|
||||
|
||||
The live `CRUSADER.EXE` class-authoring lane is no longer just a plan.
|
||||
|
||||
Current authored `Remorse` classes in the active database are:
|
||||
|
||||
- `EntityVmOwnerResource`
|
||||
- `EntityVmRuntime`
|
||||
- `EntityVmContext`
|
||||
- `EntityVmSlotEntry`
|
||||
|
||||
The VM lane is still the furthest along in actual Ghidra authoring. Recent live batches added the bounded `EntityVmSlotEntry` class owner plus more owned `EntityVmRuntime` methods (`GetSlotChunkPtrAtOffset`, `ReleaseSlotChunkRef`, `TryUnloadSlotChunk`, `DebugDumpSlotMemory`, `ApplyToMatchingOwnerRows`) rather than stopping at free-function naming.
|
||||
|
||||
The latest signature-recovery pass also tightened two of those runtime methods materially:
|
||||
|
||||
- `GetSlotChunkPtrAtOffset(runtime_farptr, slot_index, chunk_index, intra_chunk_offset)` now reads as a real slot-chunk accessor instead of a five-word anonymous wrapper.
|
||||
- `ApplyToMatchingOwnerRows(runtime_farptr, slot_index_filter, chunk_index_filter)` now reads as a real iterator/filter helper instead of a split-word scratch signature.
|
||||
|
||||
The next live batch pushed that further still: most of the `EntityVmRuntime` method cluster now carries an explicit 4-byte `EntityVmRuntime * this` in-session, including `Create`. The main remaining type gap inside that class is no longer the runtime object itself, but the exact far slot-entry pointer positions on `AcquireSlotForEntity` and `InitSlotOwnerBuffers`.
|
||||
|
||||
That VM-side gap is now closed too: `AcquireSlotForEntity` returns `EntityVmSlotEntry *32` in `DX:AX`, `InitSlotOwnerBuffers` now accepts `EntityVmSlotEntry *32`, `EntityVmOwnerResource::{Create,Destroy}` now carry explicit 4-byte `this`, and the simple `EntityVmContext` lifecycle methods now do the same.
|
||||
|
||||
The next family switch has also landed in the live database: `Remorse::UsecodeDebuggerBreakState` now exists as a real class owner with a provisional `0x2f2` datatype and a stronger method batch (`Create`, `MaybeBreakOnCurrentLine`, `BreakpointInsertSorted`, `BreakpointRemove`, `HasBreakpoint`, `CallstackPushFrame`, `CallstackPushEntry`, `CallstackPopEntry`, `EnableSingleStep`, `ClearStepState`, `CurrentEntryGetUnitName`).
|
||||
|
||||
That debugger family is no longer just a top-level shell. The internal record shapes are now recovered and applied live well enough to treat the two tables as real fixed-size arrays in-session: breakpoint entries are `0x0b` bytes with `unit_name_inline[9] + line_number`, and callstack entries are `0x15` bytes with `unit_name_inline[9]` plus the currently safest trailing fields `source_stream_target_farptr`, `current_frame_payload_farptr`, and still-neutral `aux_farptr`.
|
||||
|
||||
The VM lane also advanced one more selective step without overpromoting inheritance: `Remorse::EntityVmContext::CreateFromSlotIndex` now has a caller-backed mixed parameter pack (`owner_source_farptr`, `pitemno_farptr`, `mode_flags`, `slot_index`, `value_add_offset`, `intra_chunk_offset`, `ucparam_farptr`, `ucparamsize`) and an explicit far return restored in `AX:DX`, even though the current live endpoint still textualizes that repaired signature conservatively as plain `dword __cdecl`.
|
||||
|
||||
## Bottom Line
|
||||
|
||||
The current prep work is now large enough that it should be treated as one coordinated lane rather than scattered notes.
|
||||
|
|
|
|||
221
docs/usecode-debugger-break-state-layout.md
Normal file
221
docs/usecode-debugger-break-state-layout.md
Normal file
|
|
@ -0,0 +1,221 @@
|
|||
# Usecode Debugger Break-State Layout
|
||||
|
||||
## Purpose
|
||||
|
||||
This note captures the current class-lift-relevant evidence for the dormant seg1408 debugger-state object.
|
||||
|
||||
The retail binary still appears to leave this family orphaned at runtime, but the object model itself is strong enough to justify explicit class authoring in Ghidra.
|
||||
|
||||
Current working family name:
|
||||
|
||||
- `UsecodeDebuggerBreakState`
|
||||
|
||||
## Current Best Class-Level Read
|
||||
|
||||
`UsecodeDebuggerBreakState` is a retained debugger object that owns:
|
||||
|
||||
- a small breakpoint table
|
||||
- current interpreted-line state
|
||||
- single-step / break-armed flags
|
||||
- a callstack entry stack
|
||||
- a small vtable-backed break/notify surface used by the interpreter callback lane
|
||||
|
||||
The compiled interpreter still calls into this object when the global debugger-state pointer is non-null, even though the retail binary no longer seems to instantiate it during normal play.
|
||||
|
||||
## Strongest Evidence Anchors
|
||||
|
||||
### Constructor
|
||||
|
||||
#### `1408:0000` `Create`
|
||||
|
||||
Current verified behavior:
|
||||
|
||||
- allocates `0x2f2` bytes when `this == null`
|
||||
- writes retail vtable offset `0x65ab` at object `+0x00`
|
||||
- fills the breakpoint-entry region starting at `+0x04` with `0xffff`
|
||||
- clears `+0x02`, `+0x75`, and `+0x7a`
|
||||
- returns the object far pointer in `DX:AX`
|
||||
|
||||
This is a real constructor-style path, not just a helper wrapper.
|
||||
|
||||
### Break gate
|
||||
|
||||
#### `1408:0053` `MaybeBreakOnCurrentLine`
|
||||
|
||||
Current verified behavior:
|
||||
|
||||
- stores the incoming interpreted line minus one at `+0x72`
|
||||
- resolves the current unit-name pointer through `CurrentEntryGetUnitName`
|
||||
- checks file+line breakpoints through `HasBreakpoint`
|
||||
- dispatches through the object vtable when a break condition is met
|
||||
|
||||
This is the strongest proof that the hidden debugger lane is object-based and that the interpreter-side callback still expects a live debugger object.
|
||||
|
||||
### Breakpoint table helpers
|
||||
|
||||
#### `1408:00dd` `BreakpointInsertSorted`
|
||||
|
||||
Current verified behavior:
|
||||
|
||||
- enforces a maximum of ten breakpoint entries
|
||||
- scans the `0x0b`-byte breakpoint-entry table rooted at `+0x04`
|
||||
- compares unit-name strings via the common string helper
|
||||
- inserts a new `(unit_name, line_number)` pair into the sorted table
|
||||
|
||||
#### `1408:01a5` `BreakpointRemove`
|
||||
|
||||
Current verified behavior:
|
||||
|
||||
- scans the same `0x0b`-byte breakpoint-entry table for an exact `(unit_name, line_number)` match
|
||||
- compares the stored inline name bytes first, then the stored line word at entry `+0x09`
|
||||
- compacts the remaining entries downward when a match is found
|
||||
- decrements `breakpoint_count`
|
||||
|
||||
#### `1408:029e` `HasBreakpoint`
|
||||
|
||||
Current verified behavior:
|
||||
|
||||
- scans the same breakpoint-entry table
|
||||
- compares the requested line number against entry `+0x09`
|
||||
- compares the requested unit-name pair against the stored name bytes
|
||||
- returns a boolean-style `uint` in `AX`
|
||||
|
||||
### Callstack helpers
|
||||
|
||||
#### `1408:02f5` `CallstackPushFrame`
|
||||
|
||||
Current verified behavior:
|
||||
|
||||
- computes the current callstack-entry base as `this + 0x7c + callstack_depth * 0x15`
|
||||
- copies an inline unit-name buffer into entry `+0x00`
|
||||
- enforces a maximum visible unit-name length of eight characters plus terminator
|
||||
- stores three trailing far-pointer/state dwords at `+0x09`, `+0x0d`, and `+0x11`
|
||||
- increments `callstack_depth`
|
||||
|
||||
#### `1408:03b0` `CallstackPushEntry`
|
||||
|
||||
Current verified behavior:
|
||||
|
||||
- uses `+0x7a` as the current callstack depth
|
||||
- acts as a thinner wrapper over `CallstackPushFrame` when only the inline unit-name payload matters
|
||||
- increments the depth and asserts when it reaches `0x1e`
|
||||
|
||||
#### `1408:03f7` `CallstackPopEntry`
|
||||
|
||||
Current verified behavior:
|
||||
|
||||
- decrements `+0x7a`
|
||||
- asserts if the depth underflows below zero
|
||||
|
||||
### Step-state helpers
|
||||
|
||||
#### `1408:0419` `EnableSingleStep`
|
||||
|
||||
- clears `+0x76/+0x78`
|
||||
- sets `+0x75 = 1`
|
||||
|
||||
#### `1408:0432` `ClearStepState`
|
||||
|
||||
- clears `+0x74`
|
||||
- clears `+0x75`
|
||||
|
||||
### Current-entry name accessor
|
||||
|
||||
#### `1408:0444` `CurrentEntryGetUnitName`
|
||||
|
||||
Current verified behavior:
|
||||
|
||||
- returns null when `callstack_depth <= 0`
|
||||
- otherwise returns a far pointer to the current callstack entry's inline unit-name buffer
|
||||
|
||||
## Recovered Entry Schemas
|
||||
|
||||
The live debugger-state model is now strong enough to split the old table blobs into concrete fixed-size entry records.
|
||||
|
||||
### `UsecodeDebuggerBreakpointEntry` (`0x0b` bytes)
|
||||
|
||||
| Offset | Current name | Confidence | Current meaning |
|
||||
|---|---|---|---|
|
||||
| `+0x00..+0x08` | `unit_name_inline[9]` | High | Inline unit-name buffer, consistent with eight visible characters plus terminator. |
|
||||
| `+0x09` | `line_number` | High | Breakpoint line number compared by `BreakpointInsertSorted`, `BreakpointRemove`, and `HasBreakpoint`. |
|
||||
|
||||
### `UsecodeDebuggerCallstackEntry` (`0x15` bytes)
|
||||
|
||||
| Offset | Current name | Confidence | Current meaning |
|
||||
|---|---|---|---|
|
||||
| `+0x00..+0x08` | `unit_name_inline[9]` | High | Inline unit-name buffer for the active frame. |
|
||||
| `+0x09` | `source_stream_target_farptr` | Medium | Far pointer derived from the interpreter source-stream lane plus one fetched word in the only verified caller. |
|
||||
| `+0x0d` | `current_frame_payload_farptr` | Medium | Far pointer to the current frame payload at `frame_base + 0x04` in the only verified caller. |
|
||||
| `+0x11` | `aux_farptr` | Low | Trailing auxiliary far pointer slot; still zero in the only verified caller. |
|
||||
|
||||
## Current Working Layout
|
||||
|
||||
The live datatype `/Remorse/UsecodeDebuggerBreakState` now exists in-session with the currently safest anchors:
|
||||
|
||||
| Offset | Current name | Confidence | Current meaning |
|
||||
|---|---|---|---|
|
||||
| `+0x00` | `vtable_offset` | High | Retail debugger-state vtable offset `0x65ab`. |
|
||||
| `+0x02` | `breakpoint_count` | High | Count of `0x0b`-byte breakpoint entries. |
|
||||
| `+0x04..+0x71` | `breakpoint_entries[10]` | High | Ten inline `UsecodeDebuggerBreakpointEntry` records. |
|
||||
| `+0x72` | `current_line` | High | Current interpreted line minus one. |
|
||||
| `+0x74` | `break_armed` | Medium | Break/armed flag cleared by `ClearStepState`. |
|
||||
| `+0x75` | `single_step_enabled` | High | Single-step flag set by `EnableSingleStep`. |
|
||||
| `+0x76/+0x78` | `step_state_lo/hi` | Medium | Step-state pair cleared by `EnableSingleStep`. |
|
||||
| `+0x7a` | `callstack_depth` | High | Current callstack depth. |
|
||||
| `+0x7c..+0x2f1` | `callstack_entries[30]` | High | Thirty inline `UsecodeDebuggerCallstackEntry` records. |
|
||||
|
||||
## Live Ghidra Authoring Status
|
||||
|
||||
Verified first live class batch landed on 2026-04-06.
|
||||
|
||||
- Created class owner `Remorse::UsecodeDebuggerBreakState`.
|
||||
- Created `/Remorse/UsecodeDebuggerBreakpointEntry` (`0x0b`) and `/Remorse/UsecodeDebuggerCallstackEntry` (`0x15`) in the live data-type manager.
|
||||
- Rewrote `/Remorse/UsecodeDebuggerBreakState` in-session so the old blob regions are now explicit `UsecodeDebuggerBreakpointEntry[10]` and `UsecodeDebuggerCallstackEntry[30]` arrays at the recovered offsets.
|
||||
- Moved the main seg1408 helpers under the class owner:
|
||||
- `1408:0000` -> `Create`
|
||||
- `1408:0053` -> `MaybeBreakOnCurrentLine`
|
||||
- `1408:00dd` -> `BreakpointInsertSorted`
|
||||
- `1408:01a5` -> `BreakpointRemove`
|
||||
- `1408:029e` -> `HasBreakpoint`
|
||||
- `1408:02f5` -> `CallstackPushFrame`
|
||||
- `1408:03b0` -> `CallstackPushEntry`
|
||||
- `1408:03f7` -> `CallstackPopEntry`
|
||||
- `1408:0419` -> `EnableSingleStep`
|
||||
- `1408:0432` -> `ClearStepState`
|
||||
- `1408:0444` -> `CurrentEntryGetUnitName`
|
||||
- Tightened the live method signatures to explicit object-style forms, including:
|
||||
- `UsecodeDebuggerBreakState * __cdecl16far Create(UsecodeDebuggerBreakState * this, dword init_spec)`
|
||||
- `void __cdecl16far MaybeBreakOnCurrentLine(UsecodeDebuggerBreakState * this, word current_line)`
|
||||
- `byte __cdecl16far BreakpointInsertSorted(UsecodeDebuggerBreakState * this, dword unit_name_farptr, word line_number)`
|
||||
- `void __cdecl16far BreakpointRemove(UsecodeDebuggerBreakState * this, dword unit_name_farptr, word line_number)`
|
||||
- `uint __cdecl16far HasBreakpoint(UsecodeDebuggerBreakState * this, dword unit_name_farptr, word line_number)`
|
||||
- `void __cdecl16far CallstackPushFrame(UsecodeDebuggerBreakState * this, dword unit_name_farptr, dword source_stream_target_farptr, dword current_frame_payload_farptr, dword aux_farptr)`
|
||||
- `byte __cdecl16far CallstackPushEntry(UsecodeDebuggerBreakState * this, dword unit_name_farptr)`
|
||||
- `void __cdecl16far CallstackPopEntry(UsecodeDebuggerBreakState * this)`
|
||||
- `void __cdecl16far EnableSingleStep(UsecodeDebuggerBreakState * this)`
|
||||
- `void __cdecl16far ClearStepState(UsecodeDebuggerBreakState * this)`
|
||||
- `dword __cdecl16far CurrentEntryGetUnitName(UsecodeDebuggerBreakState * this)`
|
||||
- Added decompiler comments on the breakpoint and callstack helpers so the recovered inline-record layout is visible in-session even before every field is formally typed.
|
||||
- Added decompiler comments on the only verified `Interpreter_NextUsecodeOp` caller of `CallstackPushFrame`, which confirms the current live read of the three trailing callstack dwords:
|
||||
- `source_stream_target_farptr` is source-stream-derived from the interpreter `+0xd6/+0xd8` lane plus one fetched word
|
||||
- `current_frame_payload_farptr` is current-frame-derived from the `frame_base + 0x04` lane
|
||||
- `aux_farptr` is still zero in the only verified caller
|
||||
|
||||
## Current Cautions
|
||||
|
||||
- The retail instantiation path still appears absent; no normal caller currently reaches `Create` in the unpatched retail binary.
|
||||
- The record boundaries inside both tables are now landed in the live datatype, and two of the three trailing callstack dwords now have caller-backed structural names. The exact gameplay role behind those two far pointers is still only partly recovered.
|
||||
- `init_spec` on `Create` and `unit_name_farptr` on the breakpoint/callstack helpers are intentionally neutral names; the live signatures are object-correct, but the payload semantics should stay conservative.
|
||||
|
||||
## Best Next Moves
|
||||
|
||||
1. Identify the real gameplay semantics of `source_stream_target_farptr` and `current_frame_payload_farptr` from the interpreter-side caller lanes before promoting subsystem-specific names.
|
||||
2. Identify the vtable callback slots used by `MaybeBreakOnCurrentLine` and decide whether one or two additional methods belong on the class owner.
|
||||
3. Cross-check the seg1408 class note against the interpreter callback site at `1418:04b5` so the dormant-orphan lifecycle remains explicit in the live notes.
|
||||
4. Decide whether `aux_farptr` should remain neutral or can be promoted after one more caller or consumer pass.
|
||||
|
||||
## Bottom Line
|
||||
|
||||
`UsecodeDebuggerBreakState` is now past the “interesting orphan subsystem” stage and into real class territory.
|
||||
|
||||
The live database now has a bounded debugger-state class with a constructor, breakpoint gate, breakpoint table helpers, callstack helpers, explicit recovered `0x0b` / `0x15` entry schemas, and step-state helpers, even though the retail game still appears to leave that object dormant.
|
||||
Loading…
Add table
Add a link
Reference in a new issue