diff --git a/.github/instructions/ghidra-mcp-server-updates.instructions.md b/.github/instructions/ghidra-mcp-server-updates.instructions.md index 9a26f37..87753b2 100644 --- a/.github/instructions/ghidra-mcp-server-updates.instructions.md +++ b/.github/instructions/ghidra-mcp-server-updates.instructions.md @@ -20,6 +20,7 @@ Use these instructions when making changes in the local Ghidra MCP source at K:/ - Java plugin endpoint in src/main/java/com/lauriewired/GhidraMCPPlugin.java. - Python MCP bridge wrapper in bridge_mcp_ghidra.py. - Keep backward compatibility for endpoint naming when practical by adding aliases instead of removing old routes. +- For POST endpoints, preserve existing form-urlencoded parameter handling and add `application/json` support when evolving request parsing; if a route still only accepts one format, return a structured unsupported-content-type error instead of a missing-parameter failure. - Keep xref and analysis outputs machine-friendly and evidence-rich (addresses, ref type, classification, function context). - **Terminal execution rule:** When running multi-line Python for testing or server-side scripts, do not paste multi-line code into interactive terminals. Always write the code to a temporary `.py` file and execute it with the Python interpreter. This prevents paste-related failures and preserves the intended execution environment. diff --git a/.github/instructions/ghidra.instructions.md b/.github/instructions/ghidra.instructions.md index d6ec023..bbfadf5 100644 --- a/.github/instructions/ghidra.instructions.md +++ b/.github/instructions/ghidra.instructions.md @@ -32,6 +32,7 @@ applyTo: "**" - Prefer a single decompile call first. - If the decompiler collapses to thunk-heavy output, use one disassembly lookup to confirm the wrapper or parameter setup. - **When `decompile_function` output is too large** (>~50KB), the result is written to a temp JSON file that `read_file` returns as empty `{}`. Use `disassemble_function` instead — it returns inline assembly directly and is fully navigable for large functions. +- For 16-bit NE decompiler failures such as `Low-level Error: Symbol $$undef... extends beyond the end of the address space`, do not assume the caller's frame is the only culprit. Inspect direct callees for parser-injected hidden `__return_storage_ptr__` parameters or bad pointer-return storage first, especially after prototype edits or function recreation. - Cross-reference new `CRUSADER.EXE` findings against the old raw notes before promoting a rename or behavioral claim. If the two differ, keep both addresses and explain the mismatch instead of silently preferring one. - Add a short decompiler comment when a rename is mapped from verified notes so the provenance stays visible in Ghidra. - Keep `crusader_decompilation_notes.md` updated after each verified batch. That file is now a short index — append new analysis to the appropriate file in `docs/` and add a row to the index table if a new file is created. @@ -58,6 +59,7 @@ applyTo: "**" # PyGhidra Fallback - Use the local PyGhidra toolkit in `tools/pyghidra_crusader` when MCP is missing an operation such as function creation, deletion, or batched scripted edits. +- If Ghidra was started with Python enabled, prefer live MCP `run_readonly_script(...)` for one-off inspection first; drop to the local PyGhidra CLI only when the work needs write access or MCP still lacks the required operation. - When PyGhidra is needed because MCP lacks a required operation, append a note to `ghidra_mcp_wishlist.md` in the same batch if the gap is not already documented. - The workspace-local Python environment for this toolkit is `.venv-pyghidra311`, created from `C:\Users\Maddo\.pyenv\pyenv-win\versions\3.11.6\python.exe` and installed from the bundled Ghidra 12.0.4 offline packages. - Default install dir for the toolkit is `I:\Apps\ghidra_12.0.4_PUBLIC`. diff --git a/.github/instructions/shell-operations.instructions.md b/.github/instructions/shell-operations.instructions.md new file mode 100644 index 0000000..20ad90f --- /dev/null +++ b/.github/instructions/shell-operations.instructions.md @@ -0,0 +1,46 @@ +--- +description: 'Shell-operation guardrails for shared PowerShell sessions, tool availability checks, and multiline command workarounds' +applyTo: '**' +--- + +# Shell Operations + +Use these rules when running shell commands from this workspace. + +## General Rules + +- Prefer built-in workspace tools for file search, text search, file reads, and edits before dropping to the terminal. +- Keep terminal commands short and single-line when possible. +- Treat the shared PowerShell session as fragile after any prompt-continuation or escape-sequence artifact appears. + +## Tool Availability + +- Do not assume `rg` is installed. +- Before using `rg`, check availability with `Get-Command rg -ErrorAction SilentlyContinue`. +- If `rg` is unavailable, use workspace search tools first. +- If shell-side fallback is still needed, use PowerShell-native alternatives: + - `Get-ChildItem -Recurse -File` for file discovery + - `Select-String` for text search + +## Multiline Command Safety + +- Do not paste multiline commands directly into the shared PowerShell terminal when a one-line command or script file will do. +- Do not send multiline PowerShell hash tables, arrays, or loops directly to the shared terminal if the same work can be expressed as: + - one single-line command, or + - a temporary `.ps1` script file executed with `pwsh -File`, or + - a temporary `.py` file executed with Python for Python-side work. +- Always write multiline Python to a temporary `.py` file and execute that file instead of pasting Python into the terminal. + +## Recovering From Terminal Weirdness + +- If the prompt shows continuation markers, partial prompt redraws, or escape-sequence artifacts, stop issuing more complex commands in that shell. +- Try a single `Esc` first to clear unfinished input. +- If the shell still looks unhealthy, ask the user to reset or fix the terminal with the ask-questions tool before continuing. +- After recovery, resume with single-line commands or temporary script files instead of retrying the same multiline paste. + +## Command Style + +- Prefer explicit PowerShell cmdlets over aliases in reusable commands. +- Avoid commands that rely on interactive input. +- Prefer one focused command per terminal invocation so failures are easier to attribute. +- When posting HTTP form data from PowerShell, prefer compact one-line request bodies over multiline literal blocks in the shared shell. \ No newline at end of file diff --git a/.github/skills/pyghidra-ghidra-ops/SKILL.md b/.github/skills/pyghidra-ghidra-ops/SKILL.md index 85d4886..ba807e5 100644 --- a/.github/skills/pyghidra-ghidra-ops/SKILL.md +++ b/.github/skills/pyghidra-ghidra-ops/SKILL.md @@ -30,9 +30,12 @@ Use this skill when Ghidra MCP is missing a needed operation and you need native - Stay conservative. Use the same rename and batch-size rules as the main Ghidra workflow. - Prefer one focused plan or 1-5 direct edits at a time. +- If a live MCP session was started with Python enabled, use live `run_readonly_script(...)` for quick inspection before falling back to the local CLI; reserve the local PyGhidra path for write-side work or still-missing MCP capabilities. - Write operations require the project to be openable for modification. If `Crusader.lock` is present because the GUI owns the project, close Ghidra first or work on a copy. - Keep `crusader_decompilation_notes.md` updated after verified repair batches. +For 16-bit NE decompiler failures after prototype edits or function recreation, inspect direct callees before assuming the caller frame is corrupt. In this repo a broken caller (`1420:1499`) was only fixed after repairing a shared callee (`1000:42e2`) whose pointer-return prototype had decompiled with a hidden `__return_storage_ptr__` and poisoned the caller stack model. + Refresh the local PyGhidra environment when the bundled Ghidra version changes: ```powershell diff --git a/Crusader.rep/idata/01/~00000015.db/change.data.gbf b/Crusader.rep/idata/01/~00000015.db/change.data.gbf index c362e08..72ec249 100644 Binary files a/Crusader.rep/idata/01/~00000015.db/change.data.gbf and b/Crusader.rep/idata/01/~00000015.db/change.data.gbf differ diff --git a/Crusader.rep/idata/01/~00000015.db/change.map.gbf b/Crusader.rep/idata/01/~00000015.db/change.map.gbf index cf58bf0..b8d0fbf 100644 Binary files a/Crusader.rep/idata/01/~00000015.db/change.map.gbf and b/Crusader.rep/idata/01/~00000015.db/change.map.gbf differ diff --git a/Crusader.rep/idata/01/~00000015.db/db.43.gbf b/Crusader.rep/idata/01/~00000015.db/db.67.gbf similarity index 98% rename from Crusader.rep/idata/01/~00000015.db/db.43.gbf rename to Crusader.rep/idata/01/~00000015.db/db.67.gbf index cb749fb..2670c98 100644 Binary files a/Crusader.rep/idata/01/~00000015.db/db.43.gbf and b/Crusader.rep/idata/01/~00000015.db/db.67.gbf differ diff --git a/Crusader.rep/idata/01/~00000015.db/db.42.gbf b/Crusader.rep/idata/01/~00000015.db/db.68.gbf similarity index 98% rename from Crusader.rep/idata/01/~00000015.db/db.42.gbf rename to Crusader.rep/idata/01/~00000015.db/db.68.gbf index 13bbb5f..ec12335 100644 Binary files a/Crusader.rep/idata/01/~00000015.db/db.42.gbf and b/Crusader.rep/idata/01/~00000015.db/db.68.gbf differ diff --git a/Crusader.rep/projectState b/Crusader.rep/projectState index 389bee4..1c9566e 100644 --- a/Crusader.rep/projectState +++ b/Crusader.rep/projectState @@ -11,122 +11,132 @@ - + - - - - - - - - - - - - - - - - - - + + + + + + - + + - - - - + + + + + + + + + + - - - - - - - - - - - + - + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - + + - - - - + + + + + + - + - + - + - + - + - + - + - + - + - + - + - + - + - + @@ -136,42 +146,42 @@ - + - + - + - + - + + + + + + - + - + - - - - - - + @@ -189,7 +199,7 @@ - + @@ -230,34 +240,35 @@ - + - + - - - + + - + + + + - - + + - - + - - + + - + @@ -276,48 +287,46 @@ - - + + - - - + + + - - + - - + + - + - + - + - - + + - - + - - + + - + @@ -333,20 +342,22 @@ - + - - + + - - - - + + + - + + + + diff --git a/Crusader.rep/user/00/~00000008.db/db.34.gbf b/Crusader.rep/user/00/~00000008.db/db.44.gbf similarity index 98% rename from Crusader.rep/user/00/~00000008.db/db.34.gbf rename to Crusader.rep/user/00/~00000008.db/db.44.gbf index b6efd2b..777b8c2 100644 Binary files a/Crusader.rep/user/00/~00000008.db/db.34.gbf and b/Crusader.rep/user/00/~00000008.db/db.44.gbf differ diff --git a/Crusader.rep/user/00/~00000008.db/db.35.gbf b/Crusader.rep/user/00/~00000008.db/db.45.gbf similarity index 98% rename from Crusader.rep/user/00/~00000008.db/db.35.gbf rename to Crusader.rep/user/00/~00000008.db/db.45.gbf index c050da9..ee47c03 100644 Binary files a/Crusader.rep/user/00/~00000008.db/db.35.gbf and b/Crusader.rep/user/00/~00000008.db/db.45.gbf differ diff --git a/_tmp_apply_vm_class_types.py b/_tmp_apply_vm_class_types.py new file mode 100644 index 0000000..addc58d --- /dev/null +++ b/_tmp_apply_vm_class_types.py @@ -0,0 +1,129 @@ +from java.util import ArrayList + +from ghidra.program.model.data import DWordDataType, VoidDataType, WordDataType +from ghidra.program.model.listing import Function, ParameterImpl, ReturnParameterImpl +from ghidra.program.model.symbol import SourceType + +from tools.pyghidra_crusader.common import transaction + + +def clone_type(data_type): + return data_type.clone(program.getDataTypeManager()) + + +def get_required_data_type(path): + data_type = program.getDataTypeManager().getDataType(path) + if data_type is None: + raise RuntimeError("Missing datatype: %s" % path) + return data_type + + +def pointer_to(data_type): + return program.getDataTypeManager().getPointer(data_type) + + +def build_params(specs): + params = ArrayList() + for spec in specs: + if spec.get("keep"): + params.add(ParameterImpl(spec["parameter"], program)) + else: + params.add(ParameterImpl(spec["name"], spec["datatype"], program, SourceType.USER_DEFINED)) + return params + + +def update_signature(function_address, return_type, param_specs): + function = helpers["get_function"](program, function_address) + if function is None: + raise RuntimeError("Function not found: %s" % function_address) + + params = build_params(param_specs) + return_param = ReturnParameterImpl(return_type, program) + function.updateFunction( + function.getCallingConventionName(), + return_param, + params, + Function.FunctionUpdateType.DYNAMIC_STORAGE_ALL_PARAMS, + True, + SourceType.USER_DEFINED, + ) + return function + + +runtime_struct = get_required_data_type("/Remorse/EntityVmRuntime") +slot_entry_struct = get_required_data_type("/Remorse/EntityVmSlotEntry") + +runtime_ptr = pointer_to(runtime_struct) +slot_entry_ptr = pointer_to(slot_entry_struct) +word_type = clone_type(WordDataType.dataType) +dword_type = clone_type(DWordDataType.dataType) +void_type = clone_type(VoidDataType.dataType) + +with transaction(program, "Apply VM class type fixes"): + create_or_clear = update_signature( + "1420:2040", + slot_entry_ptr, + [ + {"name": "slot_entry", "datatype": slot_entry_ptr}, + ], + ) + + acquire_slot = helpers["get_function"](program, "1420:167c") + if acquire_slot is None: + raise RuntimeError("Function not found: 1420:167c") + acquire_slot_params = acquire_slot.getParameters() + acquire_slot = update_signature( + "1420:167c", + slot_entry_ptr, + [ + {"name": "this", "datatype": runtime_ptr}, + {"name": "slot_index", "datatype": word_type}, + {"keep": True, "parameter": acquire_slot_params[2]}, + ], + ) + + init_slot_owner_buffers = update_signature( + "1420:1866", + void_type, + [ + {"name": "this", "datatype": runtime_ptr}, + {"name": "slot_index", "datatype": word_type}, + {"name": "slot_entry", "datatype": slot_entry_ptr}, + ], + ) + + create_runtime = update_signature( + "1420:1499", + dword_type, + [ + {"name": "this", "datatype": runtime_ptr}, + {"name": "owner_type", "datatype": word_type}, + {"name": "owner_id", "datatype": word_type}, + ], + ) + + init_slots = update_signature( + "1420:1536", + void_type, + [ + {"name": "this", "datatype": runtime_ptr}, + ], + ) + + release_slots = update_signature( + "1420:1575", + void_type, + [ + {"name": "this", "datatype": runtime_ptr}, + ], + ) + +for function in [ + create_or_clear, + acquire_slot, + init_slot_owner_buffers, + create_runtime, + init_slots, + release_slots, +]: + print("UPDATED", function.getEntryPoint(), function.getPrototypeString(True, True)) diff --git a/_tmp_create_entity_vm_slot_entry_datatype.py b/_tmp_create_entity_vm_slot_entry_datatype.py new file mode 100644 index 0000000..2d52e61 --- /dev/null +++ b/_tmp_create_entity_vm_slot_entry_datatype.py @@ -0,0 +1,24 @@ +from ghidra.program.model.data import CategoryPath, DataTypeConflictHandler, Structure, StructureDataType, WordDataType + + +data_type_manager = program.getDataTypeManager() +category_path = CategoryPath("/Remorse") +slot_entry = data_type_manager.getDataType("/Remorse/EntityVmSlotEntry") + +if slot_entry is None: + slot_entry = StructureDataType(category_path, "EntityVmSlotEntry", 0x26) + slot_entry = data_type_manager.addDataType(slot_entry, DataTypeConflictHandler.REPLACE_HANDLER) + +if not isinstance(slot_entry, Structure): + raise RuntimeError("/Remorse/EntityVmSlotEntry is not a structure: %s" % slot_entry.getClass().getName()) + +word_type = WordDataType.dataType.clone(data_type_manager) + +slot_entry.replaceAtOffset(0x1e, word_type, 2, "owner_buffer_offset", None) +slot_entry.replaceAtOffset(0x20, word_type, 2, "owner_buffer_segment", None) +slot_entry.replaceAtOffset(0x22, word_type, 2, "chunk_state_offset", None) +slot_entry.replaceAtOffset(0x24, word_type, 2, "chunk_state_segment", None) + +print("DATATYPE", slot_entry.getPathName(), slot_entry.getLength()) +for component in slot_entry.getDefinedComponents(): + print("FIELD", hex(component.getOffset()), component.getFieldName(), component.getDataType().getDisplayName()) \ No newline at end of file diff --git a/_tmp_entity_vm_context_this_type.py b/_tmp_entity_vm_context_this_type.py new file mode 100644 index 0000000..537201d --- /dev/null +++ b/_tmp_entity_vm_context_this_type.py @@ -0,0 +1,70 @@ +from java.util import ArrayList + +from ghidra.program.model.listing import Function, ParameterImpl +from ghidra.program.model.symbol import SourceType + + +TARGETS = [ + "1420:0eec", + "1420:10b6", + "1420:10da", + "1420:1162", + "1420:118f", + "1420:1278", +] + + +def main(): + dtm = program.getDataTypeManager() + this_base = dtm.getDataType("/Remorse/EntityVmContext") + if this_base is None: + print("failed\tdatatype\t/Remorse/EntityVmContext not found") + return + + this_type = dtm.getPointer(this_base) + ok_count = 0 + failed = [] + + for entry_text in TARGETS: + func = helpers["get_function"](program, entry_text) + if func is None: + failed.append((entry_text, "function not found")) + print("failed\t{}\tfunction not found".format(entry_text)) + continue + + before = func.getPrototypeString(True, True) + params = list(func.getParameters()) + replacements = ArrayList() + replacements.add(ParameterImpl("this", this_type, program, SourceType.USER_DEFINED)) + for param in params[1:]: + replacements.add(ParameterImpl(param, program)) + + try: + transaction_id = program.startTransaction("Type EntityVmContext this") + commit = False + try: + func.replaceParameters( + replacements, + Function.FunctionUpdateType.DYNAMIC_STORAGE_ALL_PARAMS, + True, + SourceType.USER_DEFINED, + ) + commit = True + finally: + program.endTransaction(transaction_id, commit) + ok_count += 1 + print( + "ok\t{}\tbefore={}\tafter={}".format( + entry_text, + before, + func.getPrototypeString(True, True), + ) + ) + except Exception as exc: + failed.append((entry_text, str(exc))) + print("failed\t{}\t{}".format(entry_text, exc)) + + print("summary\tok={}\tfailed={}".format(ok_count, len(failed))) + + +main() \ No newline at end of file diff --git a/_tmp_fix_entity_vm_runtime_create.py b/_tmp_fix_entity_vm_runtime_create.py new file mode 100644 index 0000000..4e5bee7 --- /dev/null +++ b/_tmp_fix_entity_vm_runtime_create.py @@ -0,0 +1,48 @@ +from java.util import ArrayList + +from ghidra.program.model.data import DWordDataType, WordDataType +from ghidra.program.model.listing import Function, ParameterImpl, ReturnParameterImpl, VariableStorage +from ghidra.program.model.symbol import SourceType + + +def _clone(data_type): + return data_type.clone(program.getDataTypeManager()) + + +function = helpers["get_function"](program, "1420:1499") +if function is None: + raise RuntimeError("Function 1420:1499 not found") + +dword_type = _clone(DWordDataType.dataType) +word_type = _clone(WordDataType.dataType) +runtime_offset_param = ParameterImpl("this", word_type, VariableStorage(program, 4, 2), program, SourceType.USER_DEFINED) +runtime_segment_param = ParameterImpl("runtime_segment", word_type, VariableStorage(program, 6, 2), program, SourceType.USER_DEFINED) +owner_type_param = ParameterImpl("owner_type", word_type, VariableStorage(program, 8, 2), program, SourceType.USER_DEFINED) +owner_id_param = ParameterImpl("owner_id", word_type, VariableStorage(program, 10, 2), program, SourceType.USER_DEFINED) + +ax_reg = program.getRegister("AX") +dx_reg = program.getRegister("DX") +return_param = ReturnParameterImpl(dword_type, VariableStorage(program, ax_reg, dx_reg), program) + +params = ArrayList() +params.add(runtime_offset_param) +params.add(runtime_segment_param) +params.add(owner_type_param) +params.add(owner_id_param) + +function.updateFunction( + function.getCallingConventionName(), + return_param, + params, + Function.FunctionUpdateType.CUSTOM_STORAGE, + True, + SourceType.USER_DEFINED, +) + +function.setName("Create", SourceType.USER_DEFINED) +function.setComment("Factory-style runtime creator. Uses split 16-bit this/segment parameters so Ghidra can represent the incoming far runtime pointer without corrupting decompilation.") + +print("updated", function.getEntryPoint(), function.getSignature()) +for param in function.getParameters(): + print("param", param.getName(), param.getDataType().getDisplayName(), param.getVariableStorage()) +print("return", function.getReturnType().getDisplayName(), function.getReturn().getVariableStorage()) \ No newline at end of file diff --git a/_tmp_fix_entity_vm_runtime_create_typed.py b/_tmp_fix_entity_vm_runtime_create_typed.py new file mode 100644 index 0000000..4aa98e3 --- /dev/null +++ b/_tmp_fix_entity_vm_runtime_create_typed.py @@ -0,0 +1,50 @@ +from java.util import ArrayList + +from ghidra.program.model.data import DWordDataType +from ghidra.program.model.listing import Function, ParameterImpl, ReturnParameterImpl, VariableStorage +from ghidra.program.model.symbol import SourceType + +from tools.pyghidra_crusader.common import transaction + + +function = helpers["get_function"](program, "1420:1499") +if function is None: + raise RuntimeError("Function 1420:1499 not found") + +runtime_type = program.getDataTypeManager().getDataType("/Remorse/EntityVmRuntime") +if runtime_type is None: + raise RuntimeError("Missing datatype /Remorse/EntityVmRuntime") + +runtime_ptr = program.getDataTypeManager().getPointer(runtime_type) +dword_type = DWordDataType.dataType.clone(program.getDataTypeManager()) +word_type = program.getDataTypeManager().getDataType("/undefined2") +if word_type is None: + raise RuntimeError("Missing datatype /undefined2") + +this_param = ParameterImpl("this", runtime_ptr, VariableStorage(program, 4, 4), program, SourceType.USER_DEFINED) +owner_type_param = ParameterImpl("owner_type", word_type, VariableStorage(program, 8, 2), program, SourceType.USER_DEFINED) +owner_id_param = ParameterImpl("owner_id", word_type, VariableStorage(program, 10, 2), program, SourceType.USER_DEFINED) + +ax_reg = program.getRegister("AX") +dx_reg = program.getRegister("DX") +return_param = ReturnParameterImpl(dword_type, VariableStorage(program, ax_reg, dx_reg), program) + +params = ArrayList() +params.add(this_param) +params.add(owner_type_param) +params.add(owner_id_param) + +with transaction(program, "Repair EntityVmRuntime Create custom storage"): + function.updateFunction( + function.getCallingConventionName(), + return_param, + params, + Function.FunctionUpdateType.CUSTOM_STORAGE, + True, + SourceType.USER_DEFINED, + ) + +print("UPDATED", function.getEntryPoint(), function.getSignature()) +for param in function.getParameters(): + print("PARAM", param.getName(), param.getDataType().getDisplayName(), param.getVariableStorage()) +print("RETURN", function.getReturnType().getDisplayName(), function.getReturn().getVariableStorage()) diff --git a/_tmp_fix_probably_some_alloc_1000_42e2.py b/_tmp_fix_probably_some_alloc_1000_42e2.py new file mode 100644 index 0000000..0bec3eb --- /dev/null +++ b/_tmp_fix_probably_some_alloc_1000_42e2.py @@ -0,0 +1,50 @@ +from java.util import ArrayList + +from ghidra.program.model.data import DWordDataType, WordDataType +from ghidra.program.model.listing import Function, ParameterImpl, ReturnParameterImpl, VariableStorage +from ghidra.program.model.symbol import SourceType + + +def _clone(data_type): + return data_type.clone(program.getDataTypeManager()) + + +function = helpers["get_function"](program, "1000:42e2") +if function is None: + raise RuntimeError("Function 1000:42e2 not found") + +dword_type = _clone(DWordDataType.dataType) +word_type = _clone(WordDataType.dataType) + +param_1 = ParameterImpl("base_ptr", dword_type, VariableStorage(program, 4, 4), program, SourceType.USER_DEFINED) +blocksize = ParameterImpl("blocksize", word_type, VariableStorage(program, 8, 2), program, SourceType.USER_DEFINED) +nblocks = ParameterImpl("nblocks", word_type, VariableStorage(program, 10, 2), program, SourceType.USER_DEFINED) +param_4 = ParameterImpl("param_4", word_type, VariableStorage(program, 12, 2), program, SourceType.USER_DEFINED) +param_5 = ParameterImpl("param_5", word_type, VariableStorage(program, 14, 2), program, SourceType.USER_DEFINED) +transform_func = ParameterImpl("transform_func", dword_type, VariableStorage(program, 16, 4), program, SourceType.USER_DEFINED) + +ax_reg = program.getRegister("AX") +dx_reg = program.getRegister("DX") +return_param = ReturnParameterImpl(dword_type, VariableStorage(program, ax_reg, dx_reg), program) + +params = ArrayList() +params.add(param_1) +params.add(blocksize) +params.add(nblocks) +params.add(param_4) +params.add(param_5) +params.add(transform_func) + +function.updateFunction( + function.getCallingConventionName(), + return_param, + params, + Function.FunctionUpdateType.CUSTOM_STORAGE, + True, + SourceType.USER_DEFINED, +) + +print("updated", function.getEntryPoint(), function.getSignature()) +for parameter in function.getParameters(): + print("param", parameter.getName(), parameter.getDataType().getDisplayName(), parameter.getVariableStorage()) +print("return", function.getReturnType().getDisplayName(), function.getReturn().getVariableStorage()) \ No newline at end of file diff --git a/_tmp_inspect_entity_vm_runtime_create_frame.py b/_tmp_inspect_entity_vm_runtime_create_frame.py new file mode 100644 index 0000000..6f0ae8e --- /dev/null +++ b/_tmp_inspect_entity_vm_runtime_create_frame.py @@ -0,0 +1,55 @@ +def dump_function(address): + function = helpers["get_function"](program, address) + if function is None: + raise RuntimeError("Function %s not found" % address) + + frame = function.getStackFrame() + symbol_table = program.getSymbolTable() + print("FUNCTION", address) + print("SIGNATURE", function.getSignature()) + print("CALLING_CONVENTION", function.getCallingConventionName()) + print("CUSTOM_STORAGE", function.hasCustomVariableStorage()) + print("STACK_PURGE_SIZE", function.getStackPurgeSize()) + print("STACK_PURGE_VALID", function.isStackPurgeSizeValid()) + print("HAS_NO_RETURN", function.hasNoReturn()) + print("ENTRY", function.getEntryPoint()) + print("BODY", function.getBody().getMinAddress(), function.getBody().getMaxAddress()) + print("PARENT_NAMESPACE", function.getParentNamespace()) + print("STACK_GROWS_NEGATIVE", frame.growsNegative()) + print("LOCAL_SIZE", frame.getLocalSize()) + print("PARAM_OFFSET", frame.getParameterOffset()) + print("RETURN_ADDR_OFFSET", frame.getReturnAddressOffset()) + print("FRAME_SIZE", frame.getFrameSize()) + print("PARAM_SIZE", frame.getParameterSize()) + print("SYMBOLS_AT_ENTRY") + for symbol in symbol_table.getSymbols(function.getEntryPoint()): + print(" ", symbol.getName(True), symbol.getSymbolType(), symbol.getSource()) + + print("PARAMETERS") + for parameter in function.getParameters(): + print(" ", parameter.getName(), parameter.getDataType().getDisplayName(), parameter.getVariableStorage()) + + print("LOCALS") + for variable in function.getLocalVariables(): + print(" ", variable.getName(), variable.getDataType().getDisplayName(), variable.getVariableStorage(), type(variable).__name__) + + print("ALL_VARS") + for variable in function.getAllVariables(): + print(" ", variable.getName(), variable.getDataType().getDisplayName(), variable.getVariableStorage(), type(variable).__name__) + + print("STACK_REFERENCES") + listing = program.getListing() + instruction = listing.getInstructionAt(function.getEntryPoint()) + while instruction is not None and function.getBody().contains(instruction.getAddress()): + operands = [] + for operand_index in range(instruction.getNumOperands()): + references = instruction.getOperandReferences(operand_index) + if references: + operands.append((operand_index, [str(reference) for reference in references])) + print(instruction.getAddress(), instruction, operands) + instruction = instruction.getNext() + print() + + +dump_function("1420:1499") +dump_function("1430:0000") \ No newline at end of file diff --git a/_tmp_rebind_entity_vm_runtime_create.py b/_tmp_rebind_entity_vm_runtime_create.py new file mode 100644 index 0000000..8d5f2a0 --- /dev/null +++ b/_tmp_rebind_entity_vm_runtime_create.py @@ -0,0 +1,27 @@ +from ghidra.program.model.symbol import SourceType, SymbolType + + +function = helpers["get_function"](program, "1420:1499") +if function is None: + raise RuntimeError("Function 1420:1499 not found") + +symbol_table = program.getSymbolTable() +global_namespace = program.getGlobalNamespace() +remorse_namespace = symbol_table.getNamespace("Remorse", global_namespace) +if remorse_namespace is None: + raise RuntimeError("Namespace Remorse not found") + +runtime_namespace = symbol_table.getNamespace("EntityVmRuntime", remorse_namespace) +if runtime_namespace is None: + raise RuntimeError("Namespace Remorse::EntityVmRuntime not found") + +for symbol in list(symbol_table.getSymbols(function.getEntryPoint())): + if symbol.getSymbolType() == SymbolType.LABEL and symbol.getName() == "Create" and symbol.getParentNamespace() == runtime_namespace: + symbol.delete() + +function.setParentNamespace(runtime_namespace) +function.setName("Create", SourceType.USER_DEFINED) + +print("PARENT", function.getParentNamespace()) +for symbol in symbol_table.getSymbols(function.getEntryPoint()): + print("SYMBOL", symbol.getName(True), symbol.getSymbolType(), symbol.getSource()) \ No newline at end of file diff --git a/docs/entity-vm-runtime-owner-resource-layout.md b/docs/entity-vm-runtime-owner-resource-layout.md index 5c1678a..b38f681 100644 --- a/docs/entity-vm-runtime-owner-resource-layout.md +++ b/docs/entity-vm-runtime-owner-resource-layout.md @@ -41,6 +41,24 @@ Current strongest structural claims: - tail region around `+0x1300..+0x1318` holds runtime budget/default metadata plus the owner-resource helper pointer - helper attachment lives at `+0x1315/+0x1317` +Live runtime-helper classification added during the 2026-04-05 MCP-upgrade pass: + +- `1420:167c Remorse::EntityVmRuntime::AcquireSlotForEntity` + - scans the `0x80`-entry slot table for an existing entity match or the first free slot + - can evict the currently selected slot via the owner-row iterator when the runtime needs to recycle one +- `1420:1866 Remorse::EntityVmRuntime::InitSlotOwnerBuffers` + - reads owner-resource metadata for one slot id + - allocates the two slot-local working buffers later stored at `+0x1e/+0x20` and `+0x22/+0x24` + - seeds the sentinel-filled chunk-state array used by later lazy chunk loads +- `1420:19fd Remorse::EntityVmRuntime::EnsureSlotChunkLoaded` + - lazily materializes one indexed owner chunk for a slot into runtime memory + - marks the chunk present through the slot-local state arrays and updates the runtime-wide budget counters +- `1420:1f24 entity_vm_runtime_apply_to_matching_owner_rows` + - runtime-owned iterator over the owner list, used both for broad cleanup and for filtered slot/category work +- `1420:2040 entity_vm_slot_entry_create_or_clear` + - allocates or clears one `0x26`-byte slot entry record + - gives the current strongest live evidence for the stable slot-entry size and the buffer/cache lane at `+0x1e..+0x24` + Current safe class role: - long-lived VM root object that owns slot state, owner resource, category-base words, and runtime-wide value budgets @@ -148,6 +166,103 @@ Why this order: - runtime has clear ownership over the helper and slot table - context has the richest semantics but also the most unresolved dispatcher behavior +## Live Ghidra Authoring Status + +Verified first batch landed in the live `CRUSADER.EXE` session on 2026-04-05. + +- Created namespace `Remorse` and class owners `Remorse::EntityVmOwnerResource`, `Remorse::EntityVmRuntime`, and `Remorse::EntityVmContext`. +- Created provisional datatype `/Remorse/EntityVmOwnerResource` with current stable anchors: + - `+0x10 owner_row_table` + - `+0x14 entry_count` + - `+0x18 entry_word_table` +- Created provisional datatype `/Remorse/EntityVmRuntime` with size `0x1319` and only the currently stable tail anchors around the owner-resource attachment lane. +- Created provisional datatype `/Remorse/EntityVmSlotEntry` with size `0x26` and only the currently stable tail buffer fields named: + - `+0x1e owner_buffer_offset` + - `+0x20 owner_buffer_segment` + - `+0x22 chunk_state_offset` + - `+0x24 chunk_state_segment` +- Moved `1430:0000` under `Remorse::EntityVmOwnerResource::Create`. +- Moved `1430:00fd` under `Remorse::EntityVmOwnerResource::Destroy`. +- Moved `1420:1499` under `Remorse::EntityVmRuntime::Create`. +- Moved `1420:1536` under `Remorse::EntityVmRuntime::InitSlots`. +- Moved `1420:1575` under `Remorse::EntityVmRuntime::ReleaseSlots`. +- Moved `1420:1601` under `Remorse::EntityVmRuntime::Destroy`. +- Moved `1420:167c` under `Remorse::EntityVmRuntime::AcquireSlotForEntity` after live decompilation showed a `0x80`-entry scan over the runtime slot table with free-slot fallback and eviction of the currently selected slot. +- Moved `1420:1866` under `Remorse::EntityVmRuntime::InitSlotOwnerBuffers` after live decompilation showed owner-resource reads plus the two slot-local buffer allocations and initial sentinel fill. +- Moved `1420:19fd` under `Remorse::EntityVmRuntime::EnsureSlotChunkLoaded` after live decompilation showed per-slot chunk materialization and cache-presence marking. +- Renamed `1420:1cca` to `entity_vm_runtime_debug_dump_slot_memory` after live decompilation showed a debug-gated walk of the runtime slot list with slot memory usage output. +- Renamed `1420:1f24` to `entity_vm_runtime_apply_to_matching_owner_rows` after live decompilation showed filtered iteration over the runtime owner-row list. +- Renamed `1420:2040` to `entity_vm_slot_entry_create_or_clear` after live decompilation showed allocation and zeroing of one `0x26`-byte slot record. +- Added short decompiler comments at `1430:0000` and `1430:00fd` to preserve the raw `000d:7000` / `000d:70fd` provenance. +- Added short decompiler comments at `1420:1499`, `1420:1536`, `1420:1575`, and `1420:1601` to preserve the runtime-lifecycle provenance and current layout claims. +- Added short decompiler comments at `1420:167c`, `1420:1866`, `1420:19fd`, `1420:1cca`, `1420:1f24`, and `1420:2040` so the slot-table evidence stays visible in the live database. +- Repaired the decompiler health of `1420:1499 Remorse::EntityVmRuntime::Create` after the delete/recreate cycle left it throwing `Low-level Error: Symbol $$undef00000006 extends beyond the end of the address space`; the root cause was the shared allocator helper at `1000:42e2`, whose pointer-return signature decompiled with a hidden `__return_storage_ptr__` and poisoned the caller stack model until it was normalized to an explicit `dword` return plus explicit stack storage. +- Verified second batch landed in the live `CRUSADER.EXE` session on 2026-04-06. +- Moved `1420:0eec` under `Remorse::EntityVmContext::CreateFromSlotIndex`. +- Moved `1420:10b6` under `Remorse::EntityVmContext::FreeBuffer`. +- Moved `1420:10da` under `Remorse::EntityVmContext::SyncGlobalValueAndDispatch`. +- Moved `1420:1162` under `Remorse::EntityVmContext::Destroy`. +- Moved `1420:118f` under `Remorse::EntityVmContext::Save`. +- Moved `1420:1278` under `Remorse::EntityVmContext::Load`. +- Added short decompiler comments at `1420:0eec`, `1420:10b6`, `1420:10da`, `1420:1162`, `1420:118f`, and `1420:1278` to preserve the raw `000d:46ec`, `000d:48b6`, `000d:48da`, `000d:4962`, `000d:498f`, and `000d:4a78` provenance after the class-owner move. +- Verified third batch landed through local PyGhidra on 2026-04-06 after the live `run_write_script(...)` route still returned `404 No context found for request` against the active GUI session. +- Created provisional datatype `/Remorse/EntityVmContext` with size `0x124` and the currently safest stable anchors: + - `+0x32 slot_index` + - `+0x34 value_add_offset` + - `+0xd6/+0xd8 source_stream_offset/source_stream_segment` + - `+0x10c/+0x10e derived_pair_lo/derived_pair_hi` + - `+0x117/+0x119 owner_source_offset/owner_source_segment` + - `+0x123 busy_flag` +- Updated `1420:2040 entity_vm_slot_entry_create_or_clear` to `EntityVmSlotEntry * __cdecl16far entity_vm_slot_entry_create_or_clear(EntityVmSlotEntry * slot_entry)` so the slot-record helper no longer falls back to anonymous `byte *` parameters. +- Updated `1420:167c Remorse::EntityVmRuntime::AcquireSlotForEntity` to return `EntityVmSlotEntry *`, while leaving the third `param_3` entity-like pointer conservative until caller-side role recovery is tighter. +- Updated `1420:1866 Remorse::EntityVmRuntime::InitSlotOwnerBuffers` so the third parameter is now `EntityVmSlotEntry * slot_entry`. +- Updated `1420:1536 Remorse::EntityVmRuntime::InitSlots` to `void __cdecl16far InitSlots(EntityVmRuntime * this)`. +- Updated `1420:1575 Remorse::EntityVmRuntime::ReleaseSlots` to `void __cdecl16far ReleaseSlots(EntityVmRuntime * this)`. +- Tried the same typed-`this` collapse on `1420:1499 Remorse::EntityVmRuntime::Create`, but the pointer-sized `this` variant reintroduced a hidden `__return_storage_ptr__`; the function was restored immediately to the known-good split-word custom-storage signature `dword __cdecl16far Create(word this, word runtime_segment, word owner_type, word owner_id)`. +- Verified fourth live batch landed on 2026-04-06. +- Updated the `local_a` decompiler local in `1420:19fd Remorse::EntityVmRuntime::EnsureSlotChunkLoaded` to `EntityVmSlotEntry *`, so the slot-entry cache path now renders the stable `owner_buffer_offset` and `chunk_state_offset` fields directly instead of anonymous `undefined4` pairs. +- Retried the `EntityVmContext` lifecycle typing pass through live MCP. `apply_class_layout` dry-run for `/Remorse/EntityVmContext` now returns a normal structured preview instead of the earlier null failure, but the real apply path still fails with `Failed to apply this type: Storage size does not match data type size: 2`, and direct live `set_function_this_type` calls on `FreeBuffer`, `SyncGlobalValueAndDispatch`, `Destroy`, `Save`, and `Load` hit the same storage-size mismatch. +- Retried live `run_write_script(...)` with and without explicit target selectors on `CRUSADER.EXE`, but the route still returned `404 No context found for request`, so there is still no live in-session fallback for forcing the dynamic-storage rewrite on the context methods. +- Verified fifth batch landed through local PyGhidra on 2026-04-06 after the live write-side routes still blocked the context pass. +- Updated `1420:0eec Remorse::EntityVmContext::CreateFromSlotIndex` so the first parameter is now `EntityVmContext * this` while preserving the existing `UsecodeProcess *` return type until the constructor/factory semantics are tighter. +- Updated `1420:10b6 FreeBuffer`, `1420:10da SyncGlobalValueAndDispatch`, `1420:1162 Destroy`, `1420:118f Save`, and `1420:1278 Load` so the first parameter is now `EntityVmContext * this` instead of `UsecodeProcess *`. +- That local fallback confirms the newer dynamic-storage rewrite is sufficient for the context lifecycle cluster when applied outside the live GUI session. The remaining MCP issue is deployment/session parity, not whether the typing model itself works. +- Verified sixth analysis-only live batch on 2026-04-06. +- Exercised the new storage-aware prototype route against the two known 16-bit repair cases (`1000:42e2` and `1420:1499`) through the active MCP session. The checked-in source has the new route wiring, but the live GUI plugin still answered with legacy behavior: `/set_function_prototype_storage` returned the old `set_function_prototype` failure body, and `/set_storage_aware_prototype` returned `404 No context found for request`. That confirms the remaining issue is live deployment parity, not endpoint design. +- Rechecked the direct callers of `CreateFromSlotIndex`: `Usecode_ItemCallEvent` plus two `Interpreter_NextUsecodeOp` call sites. The `Usecode_ItemCallEvent` path explicitly calls `CreateFromSlotIndex((EntityVmContext *)0x0,0,...)` as an allocate-and-return factory, and the current caller-side uses immediately consume only base `Process`-style fields such as `procid` and termination flags. The two interpreter call sites likewise just store the returned far pointer in `DX:AX` scratch pairs before later base-process handling. +- That caller evidence is enough to keep the current conservative return type for now: `CreateFromSlotIndex` is clearly manufacturing an `EntityVmContext`, but promoting the return to `EntityVmContext *` before the inheritance/base-process datatype story is explicit would probably make current caller decompilation less clear rather than more clear. + +Current live datatype state: + +- `/Remorse/EntityVmOwnerResource` is the cleanest landed class in this lane so far. +- `/Remorse/EntityVmRuntime` currently only freezes the stable tail fields and helper pointer, not the full slot-entry schema. +- `/Remorse/EntityVmSlotEntry` now exists as a bounded helper datatype, but only the stable tail buffer-pair fields are named so far. +- `/Remorse/EntityVmContext` now exists and matches the current owned lifecycle cluster, but it still only records the safest field anchors rather than the full embedded mini-VM layout. +- `apply_class_layout` succeeded for `Remorse::EntityVmOwnerResource` but failed for `Remorse::EntityVmRuntime` when the binder tried to apply a `this` type, even though plain ownership moves worked. +- The old `apply_class_layout` dry-run null failure for `Remorse::EntityVmContext` no longer reproduces on the current live server, but the actual write-side `this` typing path is still effectively old-build behavior: the real apply and direct `set_function_this_type` calls still fail on the existing `UsecodeProcess *` lifecycle signatures with `Storage size does not match data type size: 2`. +- The `EntityVmContext` lifecycle signatures are now locally repaired through PyGhidra: `CreateFromSlotIndex` plus `FreeBuffer` / `SyncGlobalValueAndDispatch` / `Destroy` / `Save` / `Load` all carry `EntityVmContext * this` as their first parameter. +- `CreateFromSlotIndex` should still keep its conservative `UsecodeProcess *` return type for the moment. The allocate-and-return behavior is clear, but the known callers currently consume it through base-process fields, and the repo does not yet have an inheritance-aware `EntityVmContext : UsecodeProcess` datatype model that would make a promoted return cleaner across the call sites. +- The runtime lane is now split more accurately: `InitSlots` and `ReleaseSlots` can carry a direct `EntityVmRuntime * this`, while `Create` still needs the split-word custom-storage form to avoid hidden return-storage breakage. +- The first slot-entry prototype batch is tighter now that `EnsureSlotChunkLoaded` carries a real `EntityVmSlotEntry *` local on the acquired-slot path, but the wider slot-entry model is still improved rather than finished. + +Current scope of that batch stayed intentionally conservative: + +- no final source-format schema naming for the owner rows +- no speculative promotion of additional seg069/070 helper callbacks into owned methods yet +- no speculative promotion of the masked-create wrapper ladder into `EntityVmContext` methods +- no speculative typing yet for the entity-like pointer parameter on `AcquireSlotForEntity` +- no attempt yet to force slot-entry field names beyond the stable `+0x1e..+0x24` tail region and the current conservative helper prototypes + +Best immediate next moves after this landed: + +- inspect `EnsureSlotChunkLoaded` and adjacent `1420:` helpers again now that `AcquireSlotForEntity` returns `EntityVmSlotEntry *`, and push the slot-entry type one step deeper only where the resulting local/object read is genuinely clearer +- decide whether `CreateFromSlotIndex` can safely promote its return type from `UsecodeProcess *` to `EntityVmContext *`, or whether it should stay a factory-style bridge that only types `this` +- if the context/base-process inheritance story becomes explicit in datatypes, revisit `CreateFromSlotIndex` return typing then; until that point, keep the current `UsecodeProcess *` return even though the body itself clearly builds an `EntityVmContext` +- recover a storage-aware `this`-typing path for `Create` specifically; `InitSlots` and `ReleaseSlots` no longer need to stay in the unresolved set +- redeploy or otherwise verify the live storage-fallback `set_function_this_type` / `apply_class_layout` build, then retry the `EntityVmContext` lifecycle typing pass in-session before dropping back to local PyGhidra +- identify one or two additional strongly owned runtime or owner-resource helpers if the live session exposes them cleanly +- keep the masked-create hub and offset-specialized wrapper ladder outside the class until caller-side role recovery is tighter + ## Source-Emission Guidance If emitted as provisional C++ later, safest early skeleton is: diff --git a/docs/movable-wall-trigger-investigation.md b/docs/movable-wall-trigger-investigation.md new file mode 100644 index 0000000..e01cf52 --- /dev/null +++ b/docs/movable-wall-trigger-investigation.md @@ -0,0 +1,306 @@ +# Movable Wall Trigger Investigation + +This note records the current evidence for movable walls that do not advertise their opener through the viewer's older helper-arrow rules. + +## Result + +The checked cases fit one local cluster pattern: + +- a family-4 usecode trigger egg sits beside the wall +- a nearby `0x04B1` CMD_LINK helper uses the egg id from `mapNum` as its `QLo` link byte +- that helper is the practical local opener for the wall face + +The viewer now draws that cluster as two links when the pattern is present: + +- egg -> CMD_LINK +- CMD_LINK -> movable wall + +The rule is intentionally narrow. It only applies to checked `TRIGEGG` / `ONCEEGG` style eggs when a nearby CMD helper and one of the verified movable-wall target shapes are both present. + +## Checked Cases + +### Map 13 + +- wall: `fixed:3964` +- visible wall face near `49790, 50206, 0` +- nearby trigger egg: `fixed:3970`, shape `0x0011`, `mapNum = 5`, near `49794, 50598, 0` +- second nearby trigger egg with the same egg id: `fixed:3955`, shape `0x0011`, `mapNum = 5`, near `49604, 50516, 0` +- stacked CMD helpers in the same local lane: + - `fixed:3972`, shape `0x04B1`, near `49790, 50558, 0`, `mapNum = 38`, `npcNum = 155`, `quality = 0x0105` + - `fixed:3956`, shape `0x04B1`, near `49790, 50558, 4`, `mapNum = 46`, `npcNum = 155`, `quality = 0x0005` + - `fixed:3976`, shape `0x04B1`, near `49790, 50558, 8`, `mapNum = 134`, `npcNum = 208`, `quality = 0x0005` + +This is still the cleanest same-column movable-wall cluster, but the deeper map-13 pass shows that it is not just one egg wired straight to one wall. + +#### Map 13: Proven Trigger Chain + +The currently proven chain is: + +- avatar crosses the hidden family-4 `TRIGEGG` footprint for egg id `5` +- leaving that footprint runs `TRIGEGG::unhatch`, not `hatch` +- `TRIGEGG::unhatch` spawns `TRIGGER.slot_20(..., 0x81, ...)` +- `TRIGGER.slot_20` scans nearby `0x04B1` CMD helpers and keeps only the phase-1 lane (`mapNum & 8`) +- in this local stack, that selects `fixed:3956` +- the selected CMD then routes into the item-targeting trigger worker path + +So the wall-opening logic for `fixed:3964` is definitely tied to the `0x81` unhatch path, meaning the authored event fires when the player leaves the trigger region rather than when entering it. + +#### Map 13: Why This Still Looks Like A Set Piece + +Several nearby placements suggest a more specific authored sequence than a simple hidden strip trigger: + +- there are two separate `TRIGEGG` items using the same egg id `5` in the same local area +- there are three stacked `0x04B1` helpers at the same `x/y` with different `z` values and different encoded payloads +- nearby helper/door-like pieces also sit in the same cluster, including shape `1278` and several `353` / `354` objects + +That combination looks more like a local puzzle or staged control cluster than a generic walk-near-door opener. + +#### Map 13: Current Open Gap + +The last mutation step is still unresolved, but the current gap is narrower than before. + +The relevant phase-1 CMD path does reach `TRIGGER.slot_23`, and the selected helper in the checked stack (`fixed:3956`, `mapNum = 46`, `quality = 0x0005`) now decompiles cleanly enough to read its top-level lane. Its `mapNum & 3` mode selects `slot_23` case `2`, and that case is not a hidden mutation body. It is currently just a scan over nearby decoded target-shape items with two filters: + +- `Item.getQLo(item) == baseLink` +- `ref != item` + +and no direct action on the matches. + +That point is now supported by both the repaired high-level pseudocode and the lower semantic-layer output for `TRIGGER.slot_23`, `slot_24`, and `slot_26`: all three families still reduce their case-`2` lane to a bodyless match scan. + +For the checked map-13 cluster, that scan target is now known more precisely: + +- `fixed:3956` and its phase-0 sibling `fixed:3972` both decode to target shape `0x019B` (`shape:411`) +- the only nearby non-CMD object with low-byte link `5` is `fixed:3968`, a `shape:411` helper-geometry record at `49918, 50238, 48` +- the upper stacked helper `fixed:3976` decodes to target shape `0x04D0` (`shape:1232`), but there are no nearby `shape:1232` targets in this local cluster at all + +So the local `QLo = 5` objects around the wall reduce to four items: + +- `fixed:3972` phase-0 CMD helper +- `fixed:3956` phase-1 CMD helper +- `fixed:3976` upper CMD helper with no local target +- `fixed:3968` helper-geometry target + +Current best interpretation: + +- the local trigger source is known +- the relevant phase-1 helper is known +- the selected phase-1 helper now looks like an intentional scan-only selector aimed at helper target `fixed:3968`, not a direct wall mutator +- the upper stacked helper currently looks inactive for this cluster because it has no local `shape:1232` target to scan +- that makes the final wall mutation for `fixed:3964` more likely to live in a sibling helper, another usecode family that reacts to the scanned helper-geometry target, or a different edge/phase of the local trigger cluster, not inside this specific case-2 body + +So map 13 should still be documented as a verified trigger cluster with an unresolved final mutation, but the strongest current read is no longer “empty loop probably means bad decompilation.” The stronger read is “the chosen phase-1 CMD helper appears to be a deliberate scan-only lane.” + +#### Map 13: Current Best Gameplay Trigger Path + +Even without the final helper-consumer body, the local egg geometry now narrows the likely player action down substantially. + +The hidden family-4 trigger eggs beside the wall form an ordered west-to-east run by egg id: + +- `fixed:3910` = egg id `1` +- `fixed:3945` = egg id `2` +- `fixed:3944` = egg id `3` +- `fixed:3959` = egg id `4` +- `fixed:3955` = egg id `5` +- `fixed:3970` = egg id `5` + +The last two id-`5` eggs are narrow strips that sit directly in the northbound approach lane for wall `fixed:3964`. + +- `fixed:3955` covers `x = 49540..49668`, `y = 50196..50836` +- `fixed:3970` covers `x = 49730..49858`, `y = 50278..50918` +- wall `fixed:3964` sits at `49790, 50206`, which lines up with the eastern strip and lies just beyond its north boundary + +That means the most likely gameplay action is: + +- approach the wall from the south side, not from the north +- enter the eastern id-`5` strip aligned with the wall face +- keep moving north toward the wall until the avatar leaves that strip across its north edge + +That northbound exit is the exact movement that matches the proven `TRIGEGG::unhatch -> TRIGGER.slot_20(..., 0x81, ...) -> fixed:3956` chain. + +So the current best practical trigger instruction is not just “walk through an egg.” It is “come up from the south, get into the narrow id-`5` lane in front of the wall, and cross out of that lane northward toward the wall.” + +What is still not closed is whether the earlier id-`1..4` strips are mandatory setup for the wall or just sibling lanes in the same authored corridor. The current code evidence proves the final northbound id-`5` exit, but does not yet prove that ids `1..4` must be traversed first. + +For gameplay testing, the screen-space direction should be stated more clearly: + +- decreasing world `y` is not plain screen-up; in this isometric projection it reads as up-right on the screen +- increasing world `x` with roughly stable `y` reads as rightward on the screen + +So the current best practical route on screen is: + +- sweep left-to-right across the lower hidden trigger corridor for ids `1 -> 2 -> 3 -> 4` +- continue a little farther right into the first narrow id-`5` strip (`fixed:3955`) +- drift slightly down-right into the second narrow id-`5` strip (`fixed:3970`) +- then cut up-right out of that second strip toward the wall face at `fixed:3964` + +This wording matches the actual screen anchors in the authored scene cache: + +- id `4` strip anchor: `(15555, 11793)` +- first id `5` strip anchor: `(15645, 11796)` +- second id `5` strip anchor: `(15672, 11830)` +- wall face anchor: `(15741, 11781)` + +That makes the final move leg visually specific: from the lower-left side of the wall, move up-right toward the wall after entering the second id-`5` strip. + +#### Map 13: Practical Clarifications + +##### Do the wall shooters matter? + +Current best read: probably not for the wall opener itself. + +The visible wall launcher near this setup is `fixed:3958` at `49246, 50942, 48`. Its low quality byte is `7`, while the checked movable-wall trigger lane uses local link ids `1..5` and the final wall-facing lane uses id `5`. + +Nearby authored objects support that split: + +- wall lane helpers: `fixed:3972`, `fixed:3956`, `fixed:3976` with local ids centered on the id-`5` corridor +- wall-facing helper target: `fixed:3968` with low byte `5` +- launcher object: `fixed:3958` with low byte `7` + +No nearby id-`7` trigger egg or id-`7` `0x04B1` controller helper has been found in the same local corridor. So the current best interpretation is that the launcher is a sibling trap lane, not the wall-opening lane. + +That means destroying the launcher should not be expected to disable the wall trigger path. This is still an evidence-backed inference rather than a fully closed runtime proof, but the current local link data points strongly that way. + +##### Can the wall-trigger sequence be retried? + +On the trigger side, yes. + +The last two wall-facing id-`5` eggs are plain `TRIGEGG` subtype eggs (`quality low byte = 0`), not `ONCEEGG`. The live egg-hatcher runner sets the internal hatched flag when the avatar enters the trigger footprint and clears it again on exit, calling `hatch` and `unhatch` on those boundary crossings. + +So the trigger geometry is inherently reusable: + +- enter the strip -> `hatch` +- leave the strip -> `unhatch` +- walk back in and out again -> the same boundary logic can fire again + +Also, the selected phase-1 helper `fixed:3956` uses `mapNum = 46`, which does not set the `0x10` low-priority self-clear bit in `TRIGGER.slot_20`. So the helper itself does not look one-shot from the currently recovered routing path. + +The remaining caveat is only the still-unresolved final consumer after the scan lane. The trigger side is repeatable; the last unclosed world-change side might still have an authored one-shot elsewhere, but there is no current evidence for that in the id-`5` egg path itself. + +##### How do the last two id-`5` eggs work if they are lines? + +They are narrow trigger strips, not square areas. + +For both wall-facing id-`5` eggs, `npcNum = 4`, which decodes to: + +- `xRange = 0` +- `yRange = 4` +- `worldXRange = 0` +- `worldYRange = 256` +- `zWindow ~= +/- 48` + +So in world-space they are centered on a single `x` line and extend along `y`. In the isometric screen projection that reads as a narrow diagonal strip running roughly down-left / up-right on screen. + +The runner compares the avatar footprint against that trigger window, not just a single point. So a zero-width `xRange` does not mean the player has to hit one impossible exact pixel. It behaves like a thin line trigger that the avatar body can cross. + +Practical movement consequence: + +- if you move along the strip, you stay inside it and nothing new happens after the first entry +- if you cross into it, `hatch` fires +- if you cross back out of it, `unhatch` fires + +For the wall case, the proven useful event is the `unhatch` on the second id-`5` strip. So the movement that matters is not "stand in the line" but "pass through the line and leave it on the wall-facing side." + +##### Are ids `1..4` actually required? + +Current best read: no, they do not look required. + +This is now supported by the recovered trigger code and the authored local lane layout. + +Locally, ids `1..4` are four copies of the same controller pattern: + +- each egg is a plain `TRIGEGG` subtype egg +- each one routes into a pair of nearby `0x04B1` helpers with the same `mapNum` values as the id-`5` lane (`38` at `z=0`, `46` at `z=4`) +- each helper pair targets exactly one nearby `shape:411` helper-geometry object whose `QLo` matches the egg id + +The code path for those helpers does not show any accumulating setup behavior. + +- `TRIGEGG::hatch` and `unhatch` temporarily rewrite the egg item's local `QLo` to the egg id and call `TRIGGER.slot_20` +- `TRIGGER.slot_20` carries that id as the current `baseLink` +- the selected `slot_21 -> slot_23 case 2` path for ids `1..5` is a scan-only lane over nearby matching targets +- that lane returns `process_result = baseLink` unchanged +- none of the id-`1..4` helpers use the add/subtract cases that would roll the link id forward to the next lane +- none of the id-`1..4` helpers set the `mapNum & 16` self-clear bit either + +So crossing ids `1..4` does not currently show any recovered mechanism that would arm id `5`, advance a shared link counter, or mutate a helper object. + +The strongest current interpretation is: + +- ids `1..4` are sibling scan lanes in the same authored corridor +- they are not the required setup for wall `fixed:3964` +- the only lane that still lines up with the wall is the paired id-`5` trigger near the wall face + +So for practical gameplay testing, ids `1..4` can be dropped from the step list. If the wall opens at all through this local trigger cluster, the critical movement should be the paired id-`5` strip crossing beside the wall, not a full `1 -> 2 -> 3 -> 4 -> 5` sweep. + +#### Map 13: Door-Family Closure And Remaining Gap + +The wall face itself is now better closed than the opener. + +- `fixed:3964` is shape `0x01AB` +- recovered `DOOR.slot_21` handles shape `0x01AB` by dispatching `DOOR2.slot_2C` +- `DOOR2.slot_2C` is the actual open animation for this wall family: it advances frames `1..11`, then changes the item to shape `0x0215` +- shape `0x0215` is a non-solid, non-drawn terrain state in the exported reference data, so this is a real hidden-wall-open transition rather than just a cosmetic frame swap + +So the unresolved part is no longer what the wall does when opened. The unresolved part is still the caller that reaches `DOOR.slot_20 -> DOOR.slot_21 -> DOOR2.slot_2C` for this specific wall. + +The most important newly ruled-out direct-caller families on map 13 are: + +- no controller-family object near this wall room with local `QLo = 202` +- no family-4 egg on map 13 whose encoded egg id is `202` +- no nearby `SECRTEGG` or `DOOREGG` close enough to target this wall through the recovered secret-door scan radius +- no nearby watcher / secret-door-post family object in the wall room + +That leaves the current state as: + +- the local id-`5` trigger strip is proven to fire a real local trigger lane +- the wall face is proven to have a real hidden-door open path +- the exact authored bridge between those two facts is still not closed from the current recovered data + +So the exact player steps for opening `fixed:3964` are still unresolved. The current local egg/CMD corridor remains evidence-bearing, but it can no longer be promoted to a verified opener sequence. + +### Map 1 + +- wall: `fixed:6850` +- visible wall face near `53278, 61054, 96` +- nearby trigger egg: shape `0x0011`, `mapNum = 17`, near `53662, 61086, 104` +- nearby CMD helper: shape `0x04B1`, `quality = 0x0111`, near `53566, 61054, 96` + +This case gives the strongest byte-level confirmation because the CMD helper `QLo` is `17`, which matches the egg id `mapNum = 17` exactly. + +### Map 27 + +- wall: `fixed:6510` +- visible wall face near `20938, 23870, 3` +- nearby trigger egg: shape `0x0011`, `mapNum = 15`, near `19966, 23582, 0` +- nearby CMD helper: shape `0x04B1`, `quality low byte = 15`, near `19934, 23614, 0` + +This is the loosest of the three examples, but it still fits the same egg-id -> CMD `QLo` neighborhood pattern and falls inside the wider movable-wall cluster distance used by the overlay. + +## Viewer Rule + +The current implementation does not try to solve every movable wall in the game. It only draws this verified cluster: + +- source egg class: `TRIGEGG` or `ONCEEGG` +- match byte: egg `mapNum` == nearby CMD helper `QLo` +- CMD helper must be item-targeting and locally near the egg +- CMD helper must also be locally near one of the checked movable-wall face shapes + +Current movable-wall target shape whitelist: + +- `0x01AB` +- `0x0393` +- `0x03E8` + +Those shapes came from the inspected examples above. If future investigations recover more walls using the same cluster, extend the whitelist only with checked placements. + +## Why Not Reuse The Existing Door Rule + +The older door overlay path expects a direct same-egg-id or same-`QLo` match against door-family targets. + +These walls did not fit that assumption cleanly: + +- the opener signal is carried by a nearby usecode egg id +- the local `0x04B1` helper is the observable routing object beside the wall +- at least one checked wall face does not expose the same `QLo` as the helper + +That is why the viewer now treats this as a separate trigger-cluster overlay instead of quietly broadening the generic door whitelist. \ No newline at end of file diff --git a/ghidra_mcp_wishlist.md b/ghidra_mcp_wishlist.md index a3719de..fe2d9bc 100644 --- a/ghidra_mcp_wishlist.md +++ b/ghidra_mcp_wishlist.md @@ -11,6 +11,50 @@ For each new entry, keep the format short: ## Current Wishlist +### POST Body Contract Gap Hit During Runtime Prototype Repair (2026-04-05) + +- Missing capability: POST endpoints only accept form-urlencoded key/value parameters; direct JSON bodies fail as if required parameters were omitted. +- Current fallback: use bridge helpers or manual form-encoded POSTs when testing endpoints such as `set_function_prototype(...)` directly. +- Why it matters: MCP clients, ad hoc terminal tests, and future automation naturally try JSON first for structured payloads, especially on newer class-lift and prototype endpoints. +- Proposed MCP behavior: accept both `application/x-www-form-urlencoded` and `application/json` on POST endpoints, or return a structured unsupported-content-type error that explicitly says the route only accepts form parameters. +- Status update (2026-04-05): local plugin `parsePostParams(...)` still only splits `key=value&...` bodies and ignores JSON payloads entirely, which is why direct JSON POSTs looked like missing-parameter failures during the `EntityVmRuntime::Create` repair. +- Status update (2026-04-05, local fork): plugin `parsePostParams(...)` now accepts both form-urlencoded bodies and JSON object bodies across POST routes. Unsupported POST bodies now fail early with an explicit `unsupported-content-type` parser error instead of silently degrading into missing-parameter behavior. + +### Live PyGhidra Write Gap Hit During Runtime Repair Pass (2026-04-05) + +- Missing capability: constrained live PyGhidra write execution through MCP when Ghidra was started with Python enabled. +- Current fallback: keep read-only inspection in live MCP via `run_readonly_script(...)`, but close the GUI and drop back to local project-open PyGhidra for write-side repairs such as custom-storage prototype fixes and datatype edits. +- Why it matters: the runtime class-lift batch had to leave the live session and reopen the project locally just to repair one 16-bit function signature and one allocator-helper callee, even though the live Ghidra instance could already host Python scripts. +- Proposed MCP behavior: add a narrowly scoped live write-script or transaction endpoint family that runs against the active writable program with explicit safety limits, dry-run support where possible, and machine-friendly transaction results. +- Status update (2026-04-05): the local fork can already probe and run live read-only Python when Ghidra starts with PyGhidra enabled, so the remaining gap is write-side exposure and safety policy rather than Python availability itself. +- Status update (2026-04-05, local fork): local plugin and bridge now expose `run_write_script(script_path|script_text, dry_run?)` plus the alias route `run_transaction_script`. The implementation reuses explicit write-target selectors, validates inline or file-backed scripts against a write-policy denylist, wraps execution in a single MCP-managed transaction, reports machine-friendly status/output, and surfaces `write_script_*` capability fields from `get_runtime_capabilities()`. The remaining gap is finer-grained safety policy and live workflow verification, not basic write-side exposure. +- Status update (2026-04-06, VM class-lift pass): direct bridge `run_write_script(...)` still returned `404 No context found for request` against the active `CRUSADER.EXE` GUI session even with explicit target selectors, so the `EntityVmContext` datatype plus the slot-entry/runtime prototype batch still had to fall back to closed-project local PyGhidra. The remaining gap is now active-session context binding for the write-script route, not route availability alone. +- Status update (2026-04-06, local fork hardening): plugin explicit-target binding now normalizes Windows `project_dir` casing/separators, infers missing `project_dir` / `project_name` from the active program when possible, and fills the matching `folder_path` from the active domain file before trying to reopen a target. Bridge `run_write_script(...)` now retries the `run_transaction_script` alias on `404` or `No context found for request`, reducing mixed-build false negatives while live-session verification continues. +- Status update (2026-04-06, live context-typing retry): the trivial dry-run probe for `run_write_script(...)` still returned `404 No context found for request` against the active `CRUSADER.EXE` session both with implicit active-program targeting and with explicit `project_dir` / `project_name` / `folder_path` / `program_name` selectors. The route is still not usable as an in-session fallback for the `EntityVmContext` typing pass. + +### Class-Lift Typing Gap Hit During VM Runtime Pass (2026-04-05) + +- Missing capability: a storage-aware class-layout or `this`-typing path for 16-bit NE methods whose current function storage does not match the default pointer storage the binder tries to apply. +- Current fallback: create/update the class namespace and datatype, then move methods individually with `set_function_class(...)` and leave `this` typing/manual prototype cleanup for later. +- Why it matters: the current Remorse class-lift workflow can land ownership cleanly for `EntityVmRuntime`, but `apply_class_layout(...)` failed on the runtime lifecycle cluster with `Failed to apply this type: Storage size does not match data type size: 2` even though the same binder succeeded for `EntityVmOwnerResource`. +- Proposed MCP behavior: let `apply_class_layout(...)` either skip/soft-fail `this` typing per method with structured results, or accept an explicit storage/calling-convention override for `this` so 16-bit segmented/custom-storage methods can still be class-bound and partially typed in one pass. +- Status update (2026-04-05, later MCP-upgrade pass): the upgraded tool surface now allows direct `set_function_class(...)` moves for additional `EntityVmRuntime` helpers and `set_function_this_type(...)` succeeded on `1420:1601 Destroy` when forced to `this_storage=farptr`, but `1420:1499 Create`, `1420:1536 InitSlots`, and `1420:1575 ReleaseSlots` still fail with the same storage-size mismatch, so the gap is narrower but not resolved. +- Status update (2026-04-05, local fork): `set_function_this_type(...)` now treats `this_storage` as a real storage strategy hint instead of always reusing the old first-parameter storage. For existing parameters it tries preserved custom storage first only when the caller asked to preserve/current storage, then falls back to `DYNAMIC_STORAGE_ALL_PARAMS` when the preserved storage is incompatible with the requested `this` type. `apply_class_layout(...)` now records per-method typing failures as structured warnings instead of aborting the entire batch, and bridge method payloads can carry per-method `this_storage` and `calling_convention` overrides. +- Status update (2026-04-06, VM class-lift pass): after landing `/Remorse/EntityVmContext` and the first slot-entry prototype batch, local PyGhidra could collapse `1420:1536 InitSlots` and `1420:1575 ReleaseSlots` to direct `EntityVmRuntime * this`, but `1420:1499 Create` still reintroduced hidden `__return_storage_ptr__` corruption whenever the split-word far runtime pointer was collapsed to a typed `this`. The open gap is now mostly `Create` plus any future 16-bit constructors/factories with the same far-pointer/custom-storage shape. +- Status update (2026-04-06, live context-typing retry): the old `apply_class_layout(...)` dry-run null failure for `/Remorse/EntityVmContext` no longer reproduces, but the real live write path still behaves like the older storage-preserving build. Actual `apply_class_layout(...)` and direct `set_function_this_type(...)` calls on `1420:10b6`, `1420:10da`, `1420:1162`, `1420:118f`, and `1420:1278` all still fail with `Storage size does not match data type size: 2`, so the open gap is now specifically live deployment parity for the dynamic-storage fallback rather than dry-run binder coverage. +- Status update (2026-04-06, local PyGhidra confirmation): after closing the GUI and running the local `tools.pyghidra_crusader` script path, the same context lifecycle entries (`1420:0eec`, `1420:10b6`, `1420:10da`, `1420:1162`, `1420:118f`, `1420:1278`) all accepted `EntityVmContext * this` cleanly via `DYNAMIC_STORAGE_ALL_PARAMS`. That confirms the typing model is valid and the remaining gap is live-session deployment parity, not the class layout itself. + +### 16-bit Prototype And Hidden Return-Storage Gap Hit During Runtime Repair (2026-04-05) + +- Missing capability: a semantics-preserving prototype/storage endpoint for 16-bit NE functions that can set explicit parameter storage, explicit return storage, and avoid parser-induced hidden `__return_storage_ptr__` rewrites. +- Current fallback: inspect the broken caller plus its direct callees, then use local PyGhidra to normalize callee prototypes and apply custom storage manually. +- Why it matters: `1420:1499 Remorse::EntityVmRuntime::Create` kept throwing `Low-level Error: Symbol $$undef00000006 extends beyond the end of the address space` until the shared allocator helper at `1000:42e2` was repaired from a pointer-return signature that decompiled with a hidden return-storage parameter. +- Proposed MCP behavior: expose a storage-aware prototype/update endpoint that accepts explicit parameter and return storage, plus optionally a decompiler-health check or warning when a candidate prototype would inject hidden return storage into a 16-bit caller chain. +- Status update (2026-04-05): parser-string prototype updates alone were not sufficient here; the stable repair required explicit `AX:DX` return storage on `1000:42e2` and split-stack-word modeling for the runtime far pointer on `1420:1499`. +- Status update (2026-04-05, later MCP-upgrade pass): the new live `run_write_script(...)` path gives MCP a constrained way to perform these repairs inside the active writable session, but there is still no first-class storage-aware prototype endpoint that models explicit return/parameter storage declaratively. This wishlist item remains open. +- Status update (2026-04-06, local fork): local plugin and bridge now expose `set_function_prototype_storage(...)` plus the alias `set_storage_aware_prototype(...)`. The endpoint accepts declarative `return_type`, `return_storage`, and ordered `parameters` lines (`name|type|storage`), supports explicit target selectors, applies custom return/parameter storage in one transaction, and reports a warning when the resulting signature still contains hidden `__return_storage_ptr__` state. +- Status update (2026-04-06, live in-session verification): the checked-in Java source now wires both `/set_function_prototype_storage` and `/set_storage_aware_prototype` to the storage-aware implementation, but the active GUI session still does not match that build. Direct live POSTs to `/set_function_prototype_storage` returned HTTP 200 with the old legacy body `failed: set_function_prototype ... Function prototype is required`, while the alias route `/set_storage_aware_prototype` still returned `404 No context found for request`. So the live session still cannot exercise the new explicit-storage modeling in-session, and this remains a deployment/runtime parity gap rather than a source-level endpoint absence. + ### Live MCP Issues Hit During Spanish Cheat Pass (2026-03-26) - Missing capability: working `search_bytes(...)` requests against the currently opened program. @@ -28,6 +72,9 @@ For each new entry, keep the format short: - Why it matters: these are the exact helper endpoints needed to validate which program is active, enumerate comparison targets, and reason about whether a failure is a real analysis result or an MCP/session problem. - Proposed MCP behavior: metadata helpers should either work whenever an active program exists or return structured unsupported-state details, not raw 404 context failures. - Status update (2026-03-26, later Spanish pass): the refreshed live server still returned `404 No context found for request` for `get_runtime_capabilities(...)` and `get_callers(...)` during an active `/es/CRUSADER.EXE` session, so this is still a live deployment or routing problem, not just an earlier-session artifact. +- Status update (2026-04-05, class-lift pass): after reloading the updated plugin, `get_project_access_info(...)` and the new class-lift write routes were reachable in the active `CRUSADER.EXE` session, but `list_project_programs(...)` still returned `404 No context found for request`, so the metadata-helper context issue is not fully resolved. +- Status update (2026-04-05, local bridge hardening): bridge `list_project_programs(...)` now retries the legacy `/project_programs` alias whenever the live server answers with `404` or `No context found for request`, which should smooth mixed-build sessions while the remaining live metadata routing issue is verified after redeploy. +- Status update (2026-04-06, local fork hardening): bridge `get_runtime_capabilities(...)` now retries the `/runtime_capabilities` alias on `404` or `No context found for request`, and plugin explicit-target matching no longer depends on exact Windows path casing or slash style when deciding whether an already-open program satisfies the request. This should reduce false context failures in mixed-build live sessions, though full deployment verification is still pending. ### Open Gaps Found During Hidden Usecode Debugger Patch Batch (2026-03-24) @@ -279,6 +326,7 @@ Short, concrete gaps hit during live Crusader work. Each entry records what MCP - `move_symbol_to_namespace(symbol_address_or_name, namespace_path, new_name?)` - `set_function_class(function_address, class_path, method_name?, this_param_name?, calling_convention?)` - machine-friendly responses that include the final symbol path and any rename collisions. +- Status update (2026-04-05): local fork now exposes `create_namespace(...)`, `list_namespace_members(...)`, `move_symbol_to_namespace(...)`, and `set_function_class(...)` in both the Java plugin and Python bridge. The implementation supports explicit target selectors, dry-run moves, collision policies (`fail|keep_existing|rename_incoming`), and compatibility aliases (`create_class`, `move_function_to_class`). ### Vtable / OO recovery helpers for class reconstruction @@ -290,4 +338,5 @@ Short, concrete gaps hit during live Crusader work. Each entry records what MCP - `create_or_update_struct(name, size?, fields)` - `set_function_this_type(function_address, struct_name, this_storage=stack|register|farptr)` - `apply_class_layout(class_path, instance_struct, vtable_struct?, methods)` - - optional dry-run output showing inferred slots, unresolved targets, and conflicting field/size evidence. \ No newline at end of file + - optional dry-run output showing inferred slots, unresolved targets, and conflicting field/size evidence. +- Status update (2026-04-05): local fork now exposes `analyze_vtable(...)`, `create_or_update_struct(...)`, `create_or_update_vtable(...)`, `set_function_this_type(...)`, and `apply_class_layout(...)` in both layers. Struct and vtable authoring accept line-encoded field/slot batches from the bridge, `set_function_this_type(...)` updates the first parameter to a typed `this` pointer while preserving storage when possible, and `apply_class_layout(...)` batches namespace moves plus `this` typing with dry-run support. Compatibility aliases now also cover `set_this_type` and `build_vtable`. \ No newline at end of file diff --git a/plan-mid.md b/plan-mid.md index 869d546..6f86fb0 100644 --- a/plan-mid.md +++ b/plan-mid.md @@ -59,6 +59,15 @@ Latest verified batch: [docs/combat-dat.md](docs/combat-dat.md) now closes the s - The PSX lane is no longer just side inventory. Retail/pre-alpha bundle loading, mission-briefing/passcode structure, and the reduced-content pre-alpha disc now have dedicated notes and enough stable naming to support future targeted passes. - The Remorse class-lift preparation lane now has a usable document cluster: overall plan, candidate inventory, endpoint spec, ABI constraints, family notes for `EntityDispatchEntry` and `SpriteNode`, a conservative `Entity` family split, a VM runtime/owner-resource layout note, a compatibility-header draft, and one grouped resume index. - The same class-lift prep lane is now more execution-ready: the `0x4588` broker family has its own focused object note, the toolchain story has a dedicated fingerprint-evidence note, and there is now a concrete first-batch class-authoring checklist ready for the first MCP-backed namespace/struct/vtable pass. +- The live Remorse VM class-lift lane also recovered from a decompiler breakage in `Remorse::EntityVmRuntime::Create`: the root cause was a hidden-return-storage allocator helper signature at `1000:42e2`, `Create` now decompiles again, and the provisional `/Remorse/EntityVmSlotEntry` datatype now exists with the stable `+0x1e..+0x24` buffer-pair fields named. +- The live Remorse VM class-lift lane is tighter again: the old `UsecodeProcess_*` context lifecycle bodies at `1420:0eec`, `1420:10b6`, `1420:10da`, `1420:1162`, `1420:118f`, and `1420:1278` now live under `Remorse::EntityVmContext::{CreateFromSlotIndex, FreeBuffer, SyncGlobalValueAndDispatch, Destroy, Save, Load}`, with short raw `000d:` provenance comments preserved on each entry. +- The same VM class-lift lane tightened one step further through local PyGhidra fallback once the live `run_write_script(...)` route still returned `404 No context found for request`: `/Remorse/EntityVmContext` is now a real datatype, `entity_vm_slot_entry_create_or_clear` and `InitSlotOwnerBuffers` now carry `EntityVmSlotEntry *`, `AcquireSlotForEntity` now returns `EntityVmSlotEntry *`, and `InitSlots` / `ReleaseSlots` now take direct `EntityVmRuntime * this`. +- That pass also made the remaining blocker more precise: `Create` still cannot hold a fully typed far `this` without reintroducing hidden `__return_storage_ptr__` corruption, so it was restored to the verified split-word custom-storage signature instead of forcing a broken prettier form. +- Tooling follow-up from that same batch is now clearer too: live MCP read-only Python is usable when Ghidra starts with PyGhidra enabled, but write-side repairs still had to fall back to closed-project local PyGhidra because MCP does not yet expose a constrained live write-script or equivalent custom-storage edit path. +- The live VM class-lift lane tightened slightly again in-session: `1420:19fd Remorse::EntityVmRuntime::EnsureSlotChunkLoaded` now carries a real `EntityVmSlotEntry *` local for the acquired slot path, so the slot-entry cache tail fields decompile directly instead of through anonymous `undefined4` pairs. +- The matching MCP gap is also clearer now: the old `apply_class_layout` dry-run null failure no longer reproduces for `/Remorse/EntityVmContext`, but the real write path still behaves like the older storage-preserving build. Actual `apply_class_layout` and direct `set_function_this_type` calls on the context lifecycle methods still fail with `Storage size does not match data type size: 2`, and live `run_write_script(...)` still returns `404 No context found for request` even with explicit target selectors. +- Closing the GUI and dropping to the local PyGhidra fallback then landed the blocked context typing work cleanly: `CreateFromSlotIndex`, `FreeBuffer`, `SyncGlobalValueAndDispatch`, `Destroy`, `Save`, and `Load` now all carry `EntityVmContext * this` as their first parameter in `CRUSADER.EXE`, which confirms the newer dynamic-storage rewrite is sound even though the live MCP session still is not taking it. +- The next live verification pass tightened two details. First, the new checked-in storage-aware prototype endpoint still is not the build currently serving the active GUI session: direct live POSTs to `/set_function_prototype_storage` still answered with the legacy `set_function_prototype` failure body, and the alias route still returned `404 No context found for request`. Second, the direct callers of `CreateFromSlotIndex` still mostly consume the result as a base process object, so the current conservative `UsecodeProcess *` return should stay in place until the inheritance-aware datatype story is explicit. ### Areas That Are No Longer Live Priorities @@ -92,10 +101,12 @@ Latest verified batch: [docs/combat-dat.md](docs/combat-dat.md) now closes the s 4. Tighten the higher-slot wrapper ladder around `0005:3115..31da` so future event-label promotion depends on compiled caller behavior instead of external tables. 5. Tighten the seg006 masked-helper caller chains so the local state-selector/value family can be tied to concrete gameplay subsystems. 6. Classify the paired seg070 loops behind `entity_vm_runtime_owner_resource_create`, especially which temporary buffers and record schemas each family populates. -7. Promote additional ledger rows directly from already-verified docs and live comments, especially where segments already deserve `Foothold`, `Partial`, or `Deep`; the new seg029 step-aware sweep batch, seg031 queue-release batch, and seg090 movement-helper batch should be the immediate template. -8. If the VM lane stalls, revisit `000e:ffb0` from the now-better-constrained video/audio caller windows and try to recover an adjacent non-overlapped helper before attempting broad boundary repair. -9. Continue the map-renderer cross-check lane by building one conservative shape-id/map-placement crosswalk from `shapedata_more_complete.txt`, extracted corpora, and authored scene evidence before promoting more trigger-heavy classes in NE. -10. Keep the PSX pre-alpha lane alive as a secondary target: classify the `LoadExec` callers, test whether the stale `TALK1.XA` path is still reachable, and compare the shipped `LSET1` bundles against the retail extractor outputs. +7. Stay on the Remorse VM class-lift batch while the repaired runtime lane is warm: redeploy or otherwise verify the live storage-aware prototype and storage-fallback class-layout builds so future context and slot-entry typing can stay in-session, then push `/Remorse/EntityVmSlotEntry` one step deeper through `EnsureSlotChunkLoaded` and adjacent slot helpers, keep `CreateFromSlotIndex` on the conservative `UsecodeProcess *` return until the base-process inheritance model is explicit, and keep the storage-aware `this` investigation focused on `Create` specifically now that `InitSlots` / `ReleaseSlots` and the broader context lifecycle are already typed. +8. In the local GhidraMCP upgrade lane, add support for dual POST body decoding (`application/json` plus form-urlencoded) and a constrained live write-side PyGhidra endpoint family so future custom-storage/type repairs can stay inside the active MCP session when Python is enabled. +9. Promote additional ledger rows directly from already-verified docs and live comments, especially where segments already deserve `Foothold`, `Partial`, or `Deep`; the new seg029 step-aware sweep batch, seg031 queue-release batch, and seg090 movement-helper batch should be the immediate template. +10. If the VM lane stalls, revisit `000e:ffb0` from the now-better-constrained video/audio caller windows and try to recover an adjacent non-overlapped helper before attempting broad boundary repair. +11. Continue the map-renderer cross-check lane by building one conservative shape-id/map-placement crosswalk from `shapedata_more_complete.txt`, extracted corpora, and authored scene evidence before promoting more trigger-heavy classes in NE. +12. Keep the PSX pre-alpha lane alive as a secondary target: classify the `LoadExec` callers, test whether the stale `TALK1.XA` path is still reachable, and compare the shipped `LSET1` bundles against the retail extractor outputs. ## Remaining Work To Reach A Reasonably Complete Decompilation State