Translucency doc

Crusader_Decomp.code-workspace

@@ -15,9 +15,6 @@
 		{
 			"path": "../crusader-disasm"
 		},
-		{
-			"path": "../Crusader_Decomp_Public"
-		},
 		{
 			"path": "E:/emu/psx/Crusader - No Remorse"
 		},

docs/map_renderer/translucency-xformpal.md

@@ -0,0 +1,146 @@
+# Translucency and XFORMPAL Work Log
+
+This note tracks the renderer-side work on Crusader translucent shapes, especially the recurring `wrong yellow/white` failure on editor walls and related helper surfaces.
+
+## Current Goal
+
+Make translucent shapes in the public map renderer use colors closer to the original Crusader appearance instead of the old bright yellow/white fallback.
+
+## Verified Test Anchors
+
+Work in this batch used the local renderer project at `K:/ghidra/crusader_map_viewer/map_renderer` with the retail executables now present in `STATIC/CRUSADER.EXE` and `STATIC_REGRET/REGRET.EXE`.
+
+Representative translucent shapes used as direct probes:
+
+- `0x005A`, `0x005B`, `0x005C`, `0x005D`, `0x0067`, `0x0068`, `0x0069`: invisible/editor wall family
+- `0x00E9`: additional invisible wall helper
+- `0x02F8`: unnamed translucent base shape present in Remorse map `10`
+- `0x044D`, `0x044E`: `Zappy_Surface_SW_044D` / `Zappy_Surface_SE_044E`
+
+Verified scene/cache anchors from this batch:
+
+- old remap-only build: fingerprint `5795f5362a7450fa` (`v11-atlas-scene-xformpal-opaque-remap`)
+- ScummVM-slot-only build: fingerprint `34b9a03b15bdf9da` (`v12-atlas-scene-crusader-xform-blend`)
+- current hybrid build: fingerprint `9bc2bb02d545279d` (`v13-atlas-scene-crusader-xform-remap-rgb`)
+
+Verified fresh scene file for the current implementation:
+
+- `K:/ghidra/crusader_map_viewer/map_renderer/.cache/scene-cache/remorse/map-10/9bc2bb02d545279d/scene.json`
+
+## What The Pixel Data Proved
+
+Direct frame inspection for the bad editor-wall and helper shapes showed that they are dominated by palette indices `8`, `9`, `10`, `11`, `13`, and sometimes `14`.
+
+That matters because ScummVM and Pentagram treat those low slot values as special Crusader xform slots rather than ordinary color indices.
+
+Example direct probe results from this batch:
+
+- shape `0x005A` / `90`: top source indices `11`, `10`, `13`, `9`
+- shape `0x00E9` / `233`: top source indices `11`, `9`, `10`, `13`
+- shape `0x044D` / `1101`: almost entirely slot `8`, with a small amount of `10`
+- shape `0x044E` / `1102`: almost entirely slot `10`, with a small amount of `8`
+
+This falsified the earlier local assumption that `XFORMPAL.DAT` should be consumed as a simple `256 -> 256` color remap for translucent world shapes.
+
+## Attempt 1: Simple XFORMPAL Remap
+
+Implementation shape:
+
+- parse the first `256` bytes of `XFORMPAL.DAT`
+- remap every translucent source pixel to a different palette index
+- bake the result into the atlas PNG
+
+Result:
+
+- failed visually
+- translucent shapes still landed in the same obviously wrong bright yellow/white family
+- this also did not model the destination-aware blend behavior used by ScummVM/Pentagram
+
+Useful negative result:
+
+- `XFORMPAL.DAT` object `0` is not a single `256`-byte table; it is `2304` bytes
+- the current files in both Remorse and Regret are `2480` bytes total with:
+  - entry `0`: offset `144`, length `2304`
+  - entry `1`: offset `2448`, length `32`
+
+## Attempt 2: ScummVM Crusader Slot Blend
+
+Implementation shape:
+
+- use the Crusader slot alpha/color behavior modeled in ScummVM `CruXFormPal`
+- treat slots `8..14` as special translucent blend slots
+- keep those pixels per-pixel translucent instead of globally fading the entire item
+
+What this fixed:
+
+- removed the earlier `double fade` behavior where translucent items were being globally faded on top of a pre-faded atlas
+- direct probe on shape `0x005A` produced the expected slot alpha set `64`, `80`, `128`, `140`
+
+What still failed:
+
+- editor walls still looked like bright white/yellow slabs
+- this was better structurally than attempt 1, but still wrong in hue
+
+Status:
+
+- partial success only
+
+## Attempt 3: Compare All On-Disk XFORMPAL Tables
+
+The `2304`-byte payload was also split into `9` separate `256`-byte remap tables and compared visually against the bad translucent shapes.
+
+Observed result:
+
+- tables are not equivalent
+- several tables still produced extreme orange/red/black results
+- some later tables produced substantially more muted grey/blue/red translucent helper colors than the ScummVM hardcoded white/yellow slot colors
+
+Most useful empirical result from this comparison:
+
+- remap table index `5` (zero-based) was the most consistently plausible across the tested shape families in this batch
+- it improved the editor wall family and also gave better-looking muted tones for the tested `Zappy_Surface_*` probes
+
+This table choice is still empirical, not format-closed.
+
+## Current Implementation: Hybrid Alpha + XFORMPAL RGB
+
+Current code path in the renderer:
+
+- keep the Crusader slot alpha behavior from the ScummVM-style `8..14` xform slots
+- source the translucent RGB from `XFORMPAL.DAT` remap table `5` instead of the hardcoded white/yellow slot RGB
+- do not apply the old item-wide `0.7` opacity to translucent items anymore
+- invalidate scene caches with renderer cache version `v13-atlas-scene-crusader-xform-remap-rgb`
+
+Files touched in the renderer repo for this batch:
+
+- `src/lib/formats.js`
+- `src/lib/png.js`
+- `src/lib/build-manager.js`
+
+Verified result:
+
+- fresh isolated build probe succeeded with scene fingerprint `9bc2bb02d545279d`
+- fresh dark-background debug render no longer showed the old bright yellow/white slabs for the tested editor-wall and helper-family probes
+- the current debug artifact from the live code path is:
+  - `K:/ghidra/crusader_map_viewer/map_renderer/.cache/translucency-debug-sheet-v13.png`
+
+Best short summary of the current state:
+
+- `alpha behavior`: materially improved and now matches Crusader xform-slot expectations much better than the old remap-only path
+- `hue behavior`: materially improved for the tested shapes, but still not proven closed for every translucent family
+
+## What Is Still Unresolved
+
+- the exact semantic structure of `XFORMPAL.DAT` is still not closed
+- entry `1` (`32` bytes) remains unexplained in this batch
+- the choice of remap table `5` is evidence-backed for the tested shapes, but still empirical rather than formally decoded
+- other translucent families may need a different XFORMPAL table than the current default
+- this batch validated via direct shape probes and a fresh scene build, but not yet via a broad multi-map manual browser pass
+
+## Recommended Next Steps
+
+1. Run a manual browser-side check on a few representative maps and note which translucent families now look correct versus still wrong.
+2. Identify at least one real `window` family shape from live cached scenes and add it to the comparison set, instead of relying mostly on editor-wall and helper probes.
+3. Compare remap table `5` versus nearby candidates `6` and `7` for the remaining suspect families if any still look off.
+4. Reverse-engineer the meaning of the `32`-byte second XFORMPAL entry and look for any runtime selector or table-id field that explains why multiple remap tables exist.
+5. If one-family-per-table evidence appears, move from a single global translucent RGB table to a family- or shape-driven selector.