Add script to apply NE far call fixups from TSV files

This commit introduces a new script, `apply_ne_far_call_fixups.py`, which processes internal and import far call fixups from TSV files. The script includes functions to parse addresses, load fixup data, patch internal far calls, and annotate import calls. It handles conflicts in fixup mappings and provides detailed output on the applied patches and any skipped instructions. The script is designed to enhance the handling of NE format far calls in Ghidra.
This commit is contained in:
MaddoScientisto 2026-03-21 13:25:21 +01:00
commit 55b3187469
9 changed files with 371 additions and 6 deletions

View file

@ -50,7 +50,15 @@ Functions at Ghidra `0003:XXXX` / `0004:XXXX` are **Phar Lap extender code** (fl
### `0000:ffff` — NE Fixup Placeholder (not a dispatcher)
`unresolved_far_thunk_dispatch` at `0000:ffff` is NOT a runtime function. Every `CALLF 0x0000:ffff` in the binary is a **different** external or inter-segment call patched by the NE loader at runtime. The decompiler body is garbled (it reads NE fixup-chain sentinel data). Decompiler comment added in Ghidra. See individual call sites for per-site behavioral annotations.
`unresolved_far_thunk_dispatch` at `0000:ffff` is NOT a runtime function. Every `CALLF 0x0000:ffff` in the original NE image is a **different** external or inter-segment call patched by the NE loader at runtime. The body at `0000:ffff` is just fixup placeholder data, so decompiling it as a function is meaningless.
Repair status in `CRUSADER-RAW.EXE`:
- A PyGhidra repair pass now applies the verified NE relocation table directly to the raw-program bytes for literal internal `CALLF 9A ptr16:16` sites, then re-disassembles each patched instruction.
- Current verified batch results:
- `8851` internal literal `CALLF` sites patched to their real segment:offset targets.
- `2841` far-pointer relocation entries skipped because they were not literal `CALLF` instructions (data or other non-call uses).
- `119` import callsites annotated as `NE IMPORT -> module.symbol`.
- Remaining xrefs to `0000:ffff` should now mostly be import callsites or non-literal far-pointer cases rather than unresolved intra-game far calls.
Known call-site classifications (by argument pattern):
- `PUSH DS; PUSH imm_ordinal; CALLF` — Phar Lap extender calling a runtime-imported procedure by ordinal
@ -149,9 +157,9 @@ Known call-site classifications (by argument pattern):
#### Current limitation in raw import
- The common thunk endpoint is still imported as `unresolved_far_thunk_dispatch` at `0000:ffff`.
- In this raw database, that body decompiles as overlapped/bad instruction data, so exact arithmetic internals of the final coordinate projection cannot yet be recovered from this symbol alone.
- Despite that, caller context + table shape + argument flow make the gameplay role of this helper clear enough for naming and control-flow analysis.
- The placeholder at `0000:ffff` still exists as a symbol, but the relevant internal literal `CALLF` sites are no longer the best source of truth: they have been patched in-place to their real NE targets.
- For `snap_entity_to_ground`, the formerly unresolved call at `0007:2261` now disassembles to `CALLF 0004:e7bd`, i.e. `world_to_screen_coords`.
- Remaining `0000:ffff` sightings in the raw import are now primarily import calls or non-literal far-pointer cases, not evidence that this gameplay helper still dispatches through a single shared runtime function.
#### Working pseudocode (behavioral)
@ -189,7 +197,7 @@ Decompiler comment added to `0000:ffff` in Ghidra documenting this.
#### Next RE targets for `snap_entity_to_ground`
- The `0007:224b` thunk call is an intra-NE inter-segment call (calling into a different game segment with ground-aligned coordinate math). Identifying it requires the NE relocation table or matching the disassembly in the standalone extracts.
- The repaired call at `0007:2261` now lands at `world_to_screen_coords` (`0004:e7bd`), so the next step is to reinterpret the helper with the real callee in view rather than through the old `0000:ffff` placeholder model.
### Raw 0007 Gameplay Helper Follow-up: AI sweep + checked spawn path
@ -213,6 +221,14 @@ Decompiler comment added to `0000:ffff` in Ghidra documenting this.
- Reads player entity FAR pointer from global `0x2de4`.
- Copies player world position fields (`+0x40`, `+0x42`) into globals `0x27e7` / `0x27e9` (AI focus position cache used by downstream logic).
- Iterates entity IDs from `2` through `255` and dispatches per-entity processing through two sequential thunked calls per entity.
- After the NE far-call repair pass, the first call at `0007:101c` now disassembles and decompiles directly as `entity_resolve_slot_ptr` (`0005:0466`) instead of `CALLF 0000:ffff`.
- The repaired call chain now exposes several concrete helpers used by the sweep:
- `0005:42c8` = `entity_projected_bbox_overlaps_viewport` — projects the entity slot via `world_to_screen_coords`, subtracts entity height from `0x7df5[slot]`, derives sprite/flag context, and returns `bbox_overlap_test` against the active viewport rectangle referenced from global `0x4014`.
- `0005:3cf5` = `entity_class_has_flag2000` — class-word flag test over `entity_get_class_word(slot) & 0x2000`.
- `0005:ff2d` = `entity_class_get_flag8` — returns bit `0x08` from entity-class detail byte `0x7e1e[type*0x79 + 0x59]`.
- `0006:1305` = `entity_class_get_word_02` — raw accessor for word `+0x02` in the `0x7e1e` class-detail record.
- `0006:0ca4` = `entity_class_get_word_0a` — raw accessor for word `+0x0a` in the same class-detail record.
- `0006:11a1` = `entity_class_clear_flag8_and_dispatch` — clears bit `0x08` in class-detail byte `+0x59`, then performs follow-up entity/type checks and callback dispatch. Name intentionally stays flag-centric until the downstream side effects are fully mapped.
- New disassembly comments added at both dispatch call sites:
- `0007:101c`: `entity_slot_fetch(SS:&entity_id)` — first call; resolves entity slot/pointer from loop ID
- `0007:1093`: `entity_tick_dispatch(SS:&entity_id, g_0x27c8)` — second call; per-entity AI tick with global `0x27c8` mode/context word
@ -710,6 +726,7 @@ All 35+ identified functions renamed and annotated in Ghidra.
### Entity Data Table at 0x7e1e
- Stride: `0x79` bytes (121 bytes per entry)
- Indexed by entity type (integer) or entity slot
- `+0x59` offset = class-detail flags byte (`entity_class_get_flag8` returns bit `0x08`; other callers also clear bit `0x10` here during at-target facing updates)
- `+0x5a` offset = flags byte (bit4 = special entity flag, bit3 = armor/shield flag)
- `+0x68` = targeting flag
@ -1000,6 +1017,7 @@ Small conservative rename batch from direct field-write behavior in the `0008:ba
| `0008:bbb6` | `entity_set_source_type` | Writes entry word field `+0x04` from incoming parameter, then dispatches through FAR thunk path |
| `0008:bc27` | `entity_set_event_type_checked` | Writes entry word field `+0x06`; when source field `+0x04` is non-zero, validates old/new event transition, including special checks for `0xF0-0xF7` and upper bound `<= 0x0FFF` |
| `0008:bca8` | `entity_set_group_id` | Validates group id range `1..31`, writes low 5-bit group in byte `+0x08`, decrements old per-group counter and increments new one via counter table pointed to by `0x39c5` |
| `0008:bd53` | `entity_dispatch_entry_unlink` | Clears bit `0x1000` in flags2 at `+0x18` and zeroes the four link/state words at `+0x0a..+0x10`; used as the common unlink/reset tail in the local dispatch-entry pruning path |
| `0008:be05` | `entity_increment_group_id` | Computes `((entry+0x08)&0x1F)+1`, validates against active-layer assumptions (`0x39c9`), then applies through `entity_set_group_id` |
### Verified call/xref notes
@ -1015,6 +1033,8 @@ Small conservative rename batch from direct field-write behavior in the `0008:ba
- This cluster appears to manage core dispatch-entry metadata (`source_type`, `event_type`, group/layer byte and counters) that feeds the seg021 scheduler/event system previously documented.
- The field offsets match the current seg021 entity/dispatch layout notes (`+0x04`, `+0x06`, `+0x08`), strengthening confidence in cross-function struct consistency.
- Follow-up on the same cluster:
- `0008:bd79` remains positional, but current decompiler evidence shows it compares an entry extent/position tuple against the player world position (`g_player_entity_farptr + 0x40/+0x42`), optionally fires the vtable callback at `+0x28` when flag `0x100` is armed, then calls `entity_dispatch_entry_unlink`.
## Raw 0008 Pair-Sync Helper Batch (new)
@ -1085,11 +1105,12 @@ Additional conservative renames from the `0008:d1a4-0008:d27d` cluster.
|---------|------|---------|
| `0008:d1a4` | `entity_set_flag100_in_flags2` | Gate-checked setter: ORs bit `0x100` into entry word at `+0x18` |
| `0008:d1dc` | `entity_clear_flag100_in_flags2` | Gate-checked clearer: ANDs entry word at `+0x18` with `0xFEFF` (clears bit `0x100`) |
| `0008:cefb` | `entity_dispatch_entry_ctor_vtbl_3ad2` | Constructor variant: allocates if null, reinitializes via `entity_dispatch_entry_init`, sets vtable `0x3ad2`, sets flag `0x100` at `+0x16`, and zeroes the extension words at `+0x32/+0x34` |
| `0008:d214` | `entity_dispatch_entry_ctor_vtbl_3aa6` | Constructor variant: allocates `0x40` bytes if null, reinitializes via `0008:cefb`, sets vtable to `0x3aa6`, sets flag `0x200` at `+0x16`, zeroes fields `+0x38..+0x3e` |
Notes:
- The `entity_set_flag100_in_flags2` / `entity_clear_flag100_in_flags2` pair is a verified complementary toggle with identical gate logic (`0x39a8/0x39f9/0x3991` check path).
- Constructor naming is intentionally vtable-centric (`0x3aa6`) until more direct gameplay semantics are recovered from its callback dispatch paths.
- Constructor naming is intentionally vtable-centric (`0x3ad2`, `0x3aa6`) until more direct gameplay semantics are recovered from their callback dispatch paths.
## Raw 0008 Periodic/Counter Helpers (new)
@ -1573,6 +1594,190 @@ Compares two 5-byte `map_position` structs: `{ x:word, y:word, layer:byte }`. Re
| 98 | `0003:e4d3` | `dos_file_open_wrapper` | 26 | Zeros output byte, delegates to file open impl at 0003:bb92 |
| 99 | `0005:033e` | `entity_resolve_base_parent` | 25 | Same hierarchy walk as `entity_resolve_base_type` but returns parent from [0x7ded] |
| 100 | `000a:87fd` | `render_clip_rect_to_viewport` | 25 | Clips 4 rect params to viewport bounds at [0x4014], sets dirty flag at 0x8a16, increments draw counter at 0x4716 |
| 101 | `0005:3cf5` | `entity_class_has_flag2000` | 25 | Returns `(entity_get_class_word(slot) & 0x2000) != 0` |
| 102 | `0009:80db` | `bbox_overlap_test` | 25 | Boolean rectangle overlap test for two 4-word bbox structs; unlike `bbox_intersect`, it does not write back an intersection |
| 103 | `000d:cc00` | `entity_compute_proximity_or_visibility_bucket` | 25 | Returns `0x40` if entity is null or projected bbox overlaps viewport; otherwise buckets world-distance from current reference entity (`0x7e22`) into `0x32/0x20/0x10/0x08` |
| 104 | `000d:d413` | `entity_refresh_recent_proximity_or_visibility_buckets` | 24 | Recomputes bucket values for the last four active entries in `0x69ac` and notifies backing handles via `000a:6343` when a bucket changes |
## Raw 000d Proximity/Visibility Bucket Cluster (new)
Small conservative rename batch from the `000d:cc00-d413` region after the far-call repair exposed the viewport helper in live decompilation.
| Address | Name | Evidence |
|---------|------|---------|
| `000d:cc00` | `entity_compute_proximity_or_visibility_bucket` | Returns bucket `0x40` for null or on-screen entities (`entity_projected_bbox_overlaps_viewport`), else computes a distance bucket from the current reference entity at `0x7e22` with thresholds `0x17d`, `0x281`, `0x3c1` mapping to `0x32`, `0x20`, `0x10`, `0x08` |
| `000d:d413` | `entity_refresh_recent_proximity_or_visibility_buckets` | Walks the last four active records in the `0x69ac` array, recomputes the same bucket, stores it back to each entry, and calls `000a:6343` when the bucket changes |
| `000d:cdd0` | `tracked_entity_bucket_prune_invalid_entries` | Walks the `0x69ac` array, validates backing handles through `000a:637a`, and clears entry handles to `0xffff` when the backing object is gone |
| `000d:cd62` | `tracked_entity_bucket_find_free_main_slot` | Finds the first free entry in the main portion of the `0x69ac` array (`0 .. count-4`) |
| `000d:cd9a` | `tracked_entity_bucket_find_free_aux_slot` | Finds the first free entry in the auxiliary tail portion of the `0x69ac` array (`count-4 .. count-1`) |
Supporting caller notes:
- `000d:ce1e` populates one `0x69ac` entry by reserving a free slot, computing the initial bucket through `entity_compute_proximity_or_visibility_bucket`, storing both current and previous bucket fields, then allocating/linking the backing handle through `000a:5f36`.
- `000d:d409` is a thin wrapper that only calls `entity_refresh_recent_proximity_or_visibility_buckets`.
- `000d:cfad` is an update-or-allocate helper for `(param_1,param_2)` pairs: it tries to update an existing tracked entry through `000a:606a`, clears dead entries, and falls back to `000d:ce1e` allocation when no live match remains.
- `000d:cec5` is the auxiliary-slot allocator: it prunes invalid entries, uses `tracked_entity_bucket_find_free_aux_slot`, tags the new entry with byte `+0x0a = 1`, and seeds its handle via `000a:5f36(..., flag=1)`.
- `000a:606a` = `tracked_entity_bucket_handle_update_or_alloc` — updates the backing handle for an existing tracked bucket entry when possible, or falls back to allocation via `000a:5f36` if the handle has gone stale.
- `000d:d350` = `tracked_entity_bucket_set_value` — finds a tracked `(entity_id, entity_ref)` entry in `0x69ac` and pushes a new bucket value into its backing handle through `000a:6343`.
- `000d:d10b` = `tracked_entity_bucket_clear_ref_field` — clears only the `+0x02` reference field for all matching entries.
- `000d:d151` = `tracked_entity_bucket_remove_by_ref` — marks matching entries' backing handles for removal and clears the local entry handle/reference fields.
- `000d:d1b1` = `tracked_entity_bucket_remove_tagged_by_ref` — same removal path, but only for entries whose byte `+0x0a` tag is set.
## Raw 000a Tracked-Handle Table (new)
The `0x4673` table now reads as the backing handle registry for the `0x69ac` tracked-entry bucket subsystem.
That client layer sits on top of a separate generic cache manager rooted at `0x4688..0x46b7`, so the current `tracked_entity_*` names should be read as client-side structure names rather than names for the cache internals themselves.
| Address | Name | Evidence |
|---------|------|---------|
| `000a:5f02` | `tracked_entity_handle_find_slot` | Linear scan over 12 entries in the `0x4673` table for a matching 32-bit handle id |
| `000a:602b` | `tracked_entity_handle_is_live` | Returns true only when a handle exists in `0x4673` and its flag word at `+0x0a` does not have bit `0x0002` set |
| `000a:60eb` | `tracked_entity_handle_mark_remove` | Sets bit `0x0002` in the handle-table flag word and dispatches through the unresolved cleanup path |
| `000a:612e` | `tracked_entity_handle_mark_remove_all` | Iterates all 12 handle-table entries and marks each live handle for removal |
| `000a:6167` | `tracked_entity_handle_alloc_slot` | Allocates a slot in one of two ranges (`0..7` or `8..11`) depending on the aux flag; when full, wraps in a ring and evicts via `tracked_entity_handle_mark_remove` before reusing the slot |
| `000a:6228` | `tracked_entity_handle_prune_removed` | Reaps entries previously marked with bit `0x0002`, clears dead slots, and refreshes high-index entries through `000a:6b2d` |
| `000a:63bc` | `tracked_entity_handle_find_by_entity` | Finds the first live handle-table entry whose key/entity word at `+0x04` matches the requested entity id |
Current structural read of one `0x4673` entry (stride `0x0c`):
- `+0x00` = 32-bit handle id
- `+0x04` = key/entity id
- `+0x06` = class/group/source-style selector passed in from tracked-entry allocation
- `+0x08` = current bucket/value
- `+0x0a` = flags (`bit0` set by aux-slot allocation, `bit1` = pending removal)
Thin public wrappers on top of the tracked-handle client layer:
- `000a:5276` = `entity_bucket_track_default_main` — gated by `0x45aa`; creates or refreshes a main-slot tracked handle with bucket `0x40` and selector `0xff`.
- `000a:5294` = `entity_bucket_track_main` — same path, but takes the bucket value as an argument for the main-slot range.
- `000a:52d0` = `entity_bucket_track_default_aux` — aux-slot variant with default bucket `0x40`.
- `000a:52ee` = `entity_bucket_track_aux` — aux-slot variant with explicit bucket argument.
## Raw 000a Generic Cache Manager (new)
Follow-up analysis of `000a:6b2d` and the `0x4688..0x46b7` globals shows that this region is a generic cache manager used by the tracked-handle layer, not part of the tracked-entity subsystem itself.
| Address | Name | Evidence |
|---------|------|---------|
| `000a:6b2d` | `cache_lookup_or_load_entry_by_id` | Fast-paths the last id via `0x46af/0x46b1`, otherwise searches `0x469d`, evicts older cache slots until there is room under byte budget `0x46a5`, allocates a block from the free-list, clears/initializes the payload, records the id, and dispatches through the loader interface at `0x468c` |
| `000a:6a95` | `cache_release_entry_by_slot` | Releases a cached slot by index, clears any client references through `000a:62d8`, frees its backing block through `cache_free_block_by_slot`, and marks the slot id in `0x469d` as unused (`0xffff`) |
| `000a:6d07` | `cache_alloc_block_for_slot` | Allocates or splits a block from the free-list anchored at `0x4688`, tags it with the owning cache slot index, and updates the in-use byte count at `0x46a9` |
| `000a:6f4d` | `cache_free_block_by_slot` | Finds the free-list node for a cache slot, marks it free, subtracts its size from `0x46a9`, and coalesces adjacent free blocks |
| `000a:67d9` | `cache_shutdown` | Tears down the generic cache manager: flushes/reset state, frees slot arrays at `0x4699/0x469d/0x46b3`, frees the free-list container at `0x4688`, and closes backing state at `0x4691` |
| `000a:6898` | `cache_set_loader_interface` | Installs the backend loader/callback interface pointer at `0x468c` |
Current structural read of the cache globals:
- `0x4688` = intrusive free-list / block-list head used by `cache_alloc_block_for_slot` and `cache_free_block_by_slot`
- `0x468c` = backend loader interface / callback table; `+0x34` returns the payload size for an id and `+0x0c` loads or binds a block after allocation
- `0x4695` = base pointer for the raw cache payload arena
- `0x4699` = per-slot payload-pointer table
- `0x469d` = per-slot cached id table (`0xffff` = unused)
- `0x46a5` = cache byte budget / arena capacity
- `0x46a9` = current bytes in use
- `0x46af` / `0x46b1` = one-entry fast-path cache of the last requested id and slot index
- `0x46b3` = per-slot block metadata pointer mirror used when releasing or refreshing slots
### Follow-up: caller resolution for the public bucket wrappers
- `0005:3b34` = `tracked_entity_bucket_alloc_main_if_enabled` and `0005:3b53` = `tracked_entity_bucket_alloc_aux_if_enabled`; these are the two thin gate wrappers that feed the `0x69ac` tracked-entry layer.
- `0005:3b72` = `tracked_entity_bucket_remove_by_entity_and_ref_if_enabled`, which forwards into `000d:d086 = tracked_entity_bucket_remove_by_entity_and_ref` when `0x45aa` is set.
- `0006:d404` is not a standalone function entry; it is the tail call inside the unnamed destructor block immediately before `0006:d414`. That block tears down a `0x2774` dispatch-entry object, pushes `(entity_id = 0xdb, entity_ref = *0x7e22)`, and removes the matching aux tracked bucket entry.
- `0006:d5ae` is the analogous tail call immediately before `0006:d5be`. It tears down the sibling `0x2750` dispatch-entry object, pushes `(entity_id = 0xa4, entity_ref = *0x7e22)`, and removes the matching aux tracked bucket entry.
- The matching constructor sides are `0006:d370` and `0006:d51a`: both allocate/init a dispatch entry, stamp source type `8`, seed a per-object field from the current reference entity at `0x7e22`, and then call `tracked_entity_bucket_alloc_aux_if_enabled`.
- `0007:cce8` is the tail call at the end of `scroll_camera_set_state_params`. After the camera scroll state is updated and the new screen-space origin is committed to `0x2bb7/0x2bb9`, it refreshes the recent proximity/visibility buckets through `000d:d409` when `0x45aa` is enabled.
### Follow-up: `0x45aa` gate and cache loader installation
- `0x45aa` now reads as `tracked_entity_bucket_system_enabled`, not a one-off debug/test flag. It gates all three public wrapper helpers above and the camera-side refresh in `scroll_camera_set_state_params`.
- The enable bit is set only by the unlabeled init block around `000a:51d0..5222`: that code stores the incoming backend/interface pointer into `0x45ab/0x45ad`, installs it into `0x468c` via `cache_set_loader_interface`, allocates the tracked-handle table (`000a:5e00`) and the 32-entry bucket array (`000d:cca3(0x20)`), then sets `0x45aa = 1`.
- The matching unlabeled shutdown block starts at `000a:5223`: it checks `0x45aa`, tears down the tracked-handle table through `000a:5e59`, frees the bucket array through `000d:ccec`, and only then clears/uses the backend pointer state at `0x45ab`.
- `0x468c` remains best named as a generic `cache_loader_interface`: the current verified evidence is a cache backend callback table (`+0x34` = size query, `+0x0c` = load/bind callback) shared by the tracked-entry service. The newly traced gameplay callers prove this service participates in camera/entity-interest updates, but they are not yet strong enough to justify an audio-specific or resource-specific subsystem rename.
### Follow-up: init/shutdown entry points around `000a:51xx`
- `000a:5186` = `tracked_entity_bucket_system_init`.
- `000a:538e` = `tracked_entity_bucket_system_init_if_configured`; it only calls the init routine when config/feature gate `0x89f4` is non-zero.
- `000a:5223` = `tracked_entity_bucket_system_shutdown`.
- `0x45ab/0x45ad` now read as `tracked_entity_bucket_backend_object`, a cached backend/interface object pointer used by init/shutdown in addition to the lower-level `cache_loader_interface` callback table at `0x468c`.
- `tracked_entity_bucket_system_init` first allocates a rotating buffer via `0009:3600`, lazily creates `tracked_entity_bucket_backend_object` through `0009:5600` when absent, installs that object into `cache_loader_interface`, allocates the tracked handle table via the missing function entry at `000a:5e00`, allocates the 32-entry `0x69ac` bucket array via `000d:cca3(0x20)`, then sets `tracked_entity_bucket_system_enabled`.
- `tracked_entity_bucket_system_shutdown` is called from the wider engine teardown routine at `0004:621b`; it tears down the tracked handle table, frees the `0x69ac` bucket array, calls backend-object vtable slot `+0x38` with `(3, backend_object)`, and clears `tracked_entity_bucket_backend_object`.
### Follow-up: backend object constructor at `0009:5600`
- The missing raw-import function entry at `0009:5600` has now been recovered in-place as `cache_backend_object_init` with body `0009:5600-0009:57b9`.
- Current verified behavior is still structural, but stronger than before:
- Allocates a `0x20`-byte object when the caller passes null.
- Initializes embedded DOS file-handle state via `dos_file_handle_init` (`0009:1c00`).
- Seeds internal method-table / state fields at object offsets `+0x08`, `+0x0c`, `+0x10`, `+0x14`, `+0x16`, `+0x18`, and `+0x1c`.
- Dispatches through the object method table during construction and returns the object pointer later cached at `0x45ab/0x45ad`.
- This is enough to justify the structural `cache_backend_object_init` name, but not yet enough to promote the backend object to a file-, audio-, or resource-specific subsystem name.
- Verified callback roles inside `cache_lookup_or_load_entry_by_id`:
- backend vtable `+0x34` = size query callback for a cache entry id (used before allocation/eviction).
- backend vtable `+0x0c` = load/bind callback that populates the newly allocated slot buffer for the requested id.
### Follow-up: missing function entry at `000a:5e00`
- The missing raw-import function entry at `000a:5e00` has now been recovered in-place as `tracked_entity_handle_table_init` with body `000a:5e00-000a:5e58`.
- Verified behavior: if `0x4672` is clear, it allocates `0x90` bytes at `0x4673/0x4675`, aborts through `runtime_init_or_abort` on allocation failure, calls `000a:577d` and local helper `000a:5e95`, then sets `0x4672 = 1`.
- Matching teardown helper promoted alongside it: `000a:5e59` = `tracked_entity_handle_table_shutdown`.
- Current structural names for that local state:
- `0x4672` = `tracked_entity_handle_table_active`
- `0x4673/0x4675` = `tracked_entity_handle_table`
- This matches the existing client-layer interpretation: `0x4673` holds 12 handle entries (`12 * 0x0c = 0x90` bytes), and `000a:5e00` is the table allocator/initializer used by the tracked bucket subsystem startup path.
### Follow-up: missing function entry at `000a:6600`
- The missing raw-import function entry at `000a:6600` has now been recovered in-place as `cache_init` with body `000a:6600-000a:67d8`.
- Verified behavior from the repaired body:
- Stores the requested slot count in `0x46ad`.
- Allocates the per-slot payload-pointer table at `0x4699` (`count * 4`) and aborts on failure.
- Seeds each slot with allocator-returned pointers / zero low words before running local pointer normalization helpers (`0009:c496`, `0009:c400`, `0009:c6ae`).
- Queries/derives the cache arena size, subtracts `0x1000`, and stores the byte budget in `0x46a5`.
- Allocates the arena backing object at `0x4691`, derives the payload base pointer `0x4695`, and aborts through `seg091_func_00fd` on failure.
- Allocates the per-slot block-metadata mirror at `0x46b3` (`count * 4`) and per-slot cached-id table at `0x469d` (`count * 2`).
- Allocates and initializes the free-list head object at `0x4688`, then calls local helper `000a:68aa` before returning.
### Follow-up: cache reset / handle-table helpers
- `000a:68aa` = `cache_reset_runtime_state`.
- Verified role: shared cache reset/bootstrap helper called from `cache_init`, `cache_shutdown`, and one wider external reset path. It allocates per-slot arena-header / metadata nodes, rebinds slot pointers to the arena base, clears the cached-id table, seeds the free-list head, and resets `0x46a9` (bytes in use) plus `0x46af` (last-id fast path).
- `000a:703e` = `cache_compact_arena_blocks`.
- Verified role: compacts live cache arena blocks into earlier free holes when `cache_alloc_block_for_slot` cannot find a large-enough free block, updates per-slot payload pointers, and merges adjacent free-list headers afterward.
- `000a:5e95` = `tracked_entity_handle_table_clear_and_dispatch`.
- Verified role: when `tracked_entity_handle_table_active` is set, it zeroes the full `0x90`-byte handle table at `0x4673`, resets adjacent local state at `0x4677/0x4679/0x467b`, then dispatches through the remaining thunked follow-up path.
- `000a:5339` = `tracked_entity_handle_mark_remove_all_if_enabled`.
- Verified role: thin gate wrapper that only forwards to `tracked_entity_handle_mark_remove_all` when `tracked_entity_bucket_system_enabled` is set.
### Follow-up: external reset paths using the cache/tracked-handle layer
- The unlabeled path around `0004:25a9` now has enough local evidence to classify as an external reset sequence: it calls `cache_reset_runtime_state`, then `tracked_entity_handle_table_clear_and_dispatch`, then continues through additional tracked-entry/cache-side refresh helpers (`000d:cd22`, `000d:44b3`, `0006:ae66`, `0006:ae00`, etc.).
- The unlabeled path around `0004:eb80` is a conditional tracked-bucket reset/update sequence: when `tracked_entity_bucket_system_enabled` is set, it calls `tracked_entity_handle_mark_remove_all_if_enabled`, then `tracked_entity_handle_table_clear_and_dispatch`, then `cache_compact_arena_blocks`, before resuming its outer flow.
- These caller sites strengthen the current interpretation that the `0x45aa` / `0x4673` / `0x4688..46b7` layer is a shared runtime cache service used by gameplay/system reset flows, but they still do not expose a resource-specific subsystem name by themselves.
### Follow-up: repaired seg004 reset-path function objects
- `0004:2592` had been mis-modeled as a one-instruction thunk body. It has now been repaired to the full body `0004:2592-25de` and renamed `runtime_cache_reset_sequence`.
- Current verified behavior for `runtime_cache_reset_sequence`:
- Calls `0008:7bfe`.
- Calls `game_mode_init(*(0x27c4))`.
- Executes one still-unresolved follow-up callsite at `0004:25a4`.
- Then resets cache runtime state, clears tracked handles, refreshes tracked-entry/cache helpers, and resumes the wider runtime reset flow.
- Known caller so far: `0004:262d` inside the tiny wrapper at `0004:2620`, which sets byte `+0x40` on the object at `0x6828` before invoking the reset sequence.
- `0004:eb1f` had also been truncated. It has now been repaired to the full body `0004:eb1f-eb9b` and renamed `entity_dispatch_entry_ctor_0f3a_with_cache_reset`.
- Verified behavior for `entity_dispatch_entry_ctor_0f3a_with_cache_reset`:
- Allocates/initializes an entity dispatch entry when needed.
- Stamps entry type `0x0f3a`.
- Stores its two word payload fields from the incoming args.
- Runs local setup through `0004:ebf4`.
- When `tracked_entity_bucket_system_enabled` is set, performs the tracked-handle removal / clear / cache-compaction sequence before finalizing through `0009:b1c3`.
- The sibling at `0004:eb9c` remains separate and valid; it builds the same `0x0f3a` entry type without the extra cache-reset tail, so the repaired `0004:eb1f` boundary stops cleanly at `0004:eb9b`.
### Repair note: overlapping bad function body
- Recovery of `cache_init` required a conservative boundary repair: a stray function object `FUN_000a_eee3` had incorrectly claimed body range `000a:6710-000a:fe79`, blocking creation of the real `cache_init` body.
- The bad overlap was removed, `cache_init` was created at `000a:6600-67d8`, and `FUN_000a_eee3` was recreated conservatively as the contiguous visible body `000a:eee3-f00b`.
Current interpretation of the `0x69ac` / `0x4673` client layer:
- It is an entity-linked consumer of the generic cache manager.
- Its bucket values (`0x40`, `0x32`, `0x20`, `0x10`, `0x08`) still look attenuation- or priority-like rather than purely visibility-like.
- That is enough to keep the structural names, but not enough yet to safely promote the subsystem to a concrete audio/effect name.
**Entity Table Pointers (DS-relative, discovered in tier 5):**

View file

@ -0,0 +1,160 @@
from __future__ import annotations
import csv
from pathlib import Path
from ghidra.app.cmd.disassemble import DisassembleCommand
from ghidra.util.task import ConsoleTaskMonitor
REPO_ROOT = Path(__file__).resolve().parents[1]
INTERNAL_FIXUPS_TSV = REPO_ROOT / "ne_reloc_far_calls.tsv"
IMPORT_FIXUPS_TSV = REPO_ROOT / "ne_reloc_far_imports.tsv"
def parse_address_text(address_text: str) -> int:
segment_text, offset_text = address_text.split(":", 1)
return (int(segment_text, 16) << 16) + int(offset_text, 16)
def to_address(address_text: str):
address_space = program.getAddressFactory().getDefaultAddressSpace()
return address_space.getAddress(parse_address_text(address_text))
def parse_seg_off(address_text: str) -> tuple[int, int]:
segment_text, offset_text = address_text.split(":", 1)
return int(segment_text, 16), int(offset_text, 16)
def signed_byte(value: int) -> int:
return value if value < 0x80 else value - 0x100
def load_internal_fixups() -> dict[str, tuple[str, str]]:
rows: dict[str, tuple[str, str]] = {}
with INTERNAL_FIXUPS_TSV.open("r", encoding="utf-8", newline="") as handle:
reader = csv.DictReader(handle, delimiter="\t")
for row in reader:
source = row["source_ghidra"].strip()
target = row["target_ghidra"].strip()
label = row["target_label"].strip()
previous = rows.get(source)
if previous is not None and previous != (target, label):
raise RuntimeError(
f"conflicting internal fixup mapping for {source}: {previous} vs {(target, label)}"
)
rows[source] = (target, label)
return rows
def load_import_fixups() -> dict[str, str]:
rows: dict[str, str] = {}
with IMPORT_FIXUPS_TSV.open("r", encoding="utf-8", newline="") as handle:
reader = csv.DictReader(handle, delimiter="\t")
for row in reader:
source = row["source_ghidra"].strip()
target = row["target"].strip()
previous = rows.get(source)
if previous is not None and previous != target:
raise RuntimeError(
f"conflicting import fixup mapping for {source}: {previous} vs {target}"
)
rows[source] = target
return rows
def patch_internal_far_call(operand_address_text: str, target_address_text: str, label: str) -> str | None:
memory = program.getMemory()
listing = program.getListing()
operand_address = to_address(operand_address_text)
instruction_address = operand_address.previous()
instruction_end = instruction_address.add(4)
opcode = int(memory.getByte(instruction_address)) & 0xFF
if opcode != 0x9A:
return None
listing.clearCodeUnits(instruction_address, instruction_end, False)
target_segment, target_offset = parse_seg_off(target_address_text)
operand_bytes = [
target_offset & 0xFF,
(target_offset >> 8) & 0xFF,
target_segment & 0xFF,
(target_segment >> 8) & 0xFF,
]
for index, byte_value in enumerate(operand_bytes):
memory.setByte(operand_address.add(index), signed_byte(byte_value))
command = DisassembleCommand(instruction_address, None, True)
if not command.applyTo(program, ConsoleTaskMonitor()):
raise RuntimeError(f"failed to re-disassemble patched CALLF at {instruction_address}")
comment = f"NE FIXUP APPLIED -> {target_address_text} ({label})"
helpers["set_comment"](program, str(instruction_address), comment, "eol")
return str(instruction_address)
def annotate_import_far_call(operand_address_text: str, import_target: str) -> str:
instruction_address = to_address(operand_address_text).previous()
helpers["set_comment"](program, str(instruction_address), f"NE IMPORT -> {import_target}", "eol")
return str(instruction_address)
def main() -> None:
internal_fixups = load_internal_fixups()
import_fixups = load_import_fixups()
applied_internal = 0
skipped_non_call_internal = 0
annotated_imports = 0
internal_examples: list[str] = []
skipped_examples: list[str] = []
import_examples: list[str] = []
transaction_id = program.startTransaction("Apply NE far-call fixups")
commit = False
try:
for source, (target, label) in sorted(internal_fixups.items()):
instruction_address = patch_internal_far_call(source, target, label)
if instruction_address is None:
skipped_non_call_internal += 1
if len(skipped_examples) < 10:
skipped_examples.append(f"{source} -> {target} ({label})")
continue
applied_internal += 1
if len(internal_examples) < 10:
internal_examples.append(f"{instruction_address} -> {target} ({label})")
for source, target in sorted(import_fixups.items()):
instruction_address = annotate_import_far_call(source, target)
annotated_imports += 1
if len(import_examples) < 10:
import_examples.append(f"{instruction_address} -> {target}")
helpers["set_comment"](
program,
"0000:ffff",
"NE fixup placeholder only. Internal far calls have been patched from the verified relocation table; remaining xrefs, if any, are import callsites or unresolved non-internal cases.",
"pre",
)
commit = True
finally:
program.endTransaction(transaction_id, commit)
print(f"Applied internal far-call patches: {applied_internal}")
for line in internal_examples:
print(f" {line}")
print(f"Skipped internal far-ptr fixups that were not literal CALLF instructions: {skipped_non_call_internal}")
for line in skipped_examples:
print(f" {line}")
print(f"Annotated import far calls: {annotated_imports}")
for line in import_examples:
print(f" {line}")
main()