# Translucency and XFORMPAL Work Log This note tracks the renderer-side work on Crusader translucent shapes, especially the recurring `wrong yellow/white` failure on editor walls and related helper surfaces. ## Current Goal Make translucent shapes in the public map renderer use colors closer to the original Crusader appearance instead of the old bright yellow/white fallback. ## Verified Test Anchors Work in this batch used the local renderer project at `K:/ghidra/crusader_map_viewer/map_renderer` with the retail executables now present in `STATIC/CRUSADER.EXE` and `STATIC_REGRET/REGRET.EXE`. Representative translucent shapes used as direct probes: - `0x005A`, `0x005B`, `0x005C`, `0x005D`, `0x0067`, `0x0068`, `0x0069`: invisible/editor wall family - `0x00E9`: additional invisible wall helper - `0x02F8`: unnamed translucent base shape present in Remorse map `10` - `0x044D`, `0x044E`: `Zappy_Surface_SW_044D` / `Zappy_Surface_SE_044E` Verified scene/cache anchors from this batch: - old remap-only build: fingerprint `5795f5362a7450fa` (`v11-atlas-scene-xformpal-opaque-remap`) - ScummVM-slot-only build: fingerprint `34b9a03b15bdf9da` (`v12-atlas-scene-crusader-xform-blend`) - current hybrid build: fingerprint `9bc2bb02d545279d` (`v13-atlas-scene-crusader-xform-remap-rgb`) Verified fresh scene file for the current implementation: - `K:/ghidra/crusader_map_viewer/map_renderer/.cache/scene-cache/remorse/map-10/9bc2bb02d545279d/scene.json` ## What The Pixel Data Proved Direct frame inspection for the bad editor-wall and helper shapes showed that they are dominated by palette indices `8`, `9`, `10`, `11`, `13`, and sometimes `14`. That matters because ScummVM and Pentagram treat those low slot values as special Crusader xform slots rather than ordinary color indices. Example direct probe results from this batch: - shape `0x005A` / `90`: top source indices `11`, `10`, `13`, `9` - shape `0x00E9` / `233`: top source indices `11`, `9`, `10`, `13` - shape `0x044D` / `1101`: almost entirely slot `8`, with a small amount of `10` - shape `0x044E` / `1102`: almost entirely slot `10`, with a small amount of `8` This falsified the earlier local assumption that `XFORMPAL.DAT` should be consumed as a simple `256 -> 256` color remap for translucent world shapes. ## Attempt 1: Simple XFORMPAL Remap Implementation shape: - parse the first `256` bytes of `XFORMPAL.DAT` - remap every translucent source pixel to a different palette index - bake the result into the atlas PNG Result: - failed visually - translucent shapes still landed in the same obviously wrong bright yellow/white family - this also did not model the destination-aware blend behavior used by ScummVM/Pentagram Useful negative result: - `XFORMPAL.DAT` object `0` is not a single `256`-byte table; it is `2304` bytes - the current files in both Remorse and Regret are `2480` bytes total with: - entry `0`: offset `144`, length `2304` - entry `1`: offset `2448`, length `32` ## Attempt 2: ScummVM Crusader Slot Blend Implementation shape: - use the Crusader slot alpha/color behavior modeled in ScummVM `CruXFormPal` - treat slots `8..14` as special translucent blend slots - keep those pixels per-pixel translucent instead of globally fading the entire item What this fixed: - removed the earlier `double fade` behavior where translucent items were being globally faded on top of a pre-faded atlas - direct probe on shape `0x005A` produced the expected slot alpha set `64`, `80`, `128`, `140` What still failed: - editor walls still looked like bright white/yellow slabs - this was better structurally than attempt 1, but still wrong in hue Status: - partial success only ## Attempt 3: Compare All On-Disk XFORMPAL Tables The `2304`-byte payload was also split into `9` separate `256`-byte remap tables and compared visually against the bad translucent shapes. Observed result: - tables are not equivalent - several tables still produced extreme orange/red/black results - some later tables produced substantially more muted grey/blue/red translucent helper colors than the ScummVM hardcoded white/yellow slot colors Most useful empirical result from this comparison: - remap table index `5` (zero-based) was the most consistently plausible across the tested shape families in this batch - it improved the editor wall family and also gave better-looking muted tones for the tested `Zappy_Surface_*` probes This table choice is still empirical, not format-closed. ## Current Implementation: Hybrid Alpha + XFORMPAL RGB Current code path in the renderer: - keep the Crusader slot alpha behavior from the ScummVM-style `8..14` xform slots - source the translucent RGB from `XFORMPAL.DAT` remap table `5` instead of the hardcoded white/yellow slot RGB - do not apply the old item-wide `0.7` opacity to translucent items anymore - invalidate scene caches with renderer cache version `v13-atlas-scene-crusader-xform-remap-rgb` Files touched in the renderer repo for this batch: - `src/lib/formats.js` - `src/lib/png.js` - `src/lib/build-manager.js` Verified result: - fresh isolated build probe succeeded with scene fingerprint `9bc2bb02d545279d` - fresh dark-background debug render no longer showed the old bright yellow/white slabs for the tested editor-wall and helper-family probes - the current debug artifact from the live code path is: - `K:/ghidra/crusader_map_viewer/map_renderer/.cache/translucency-debug-sheet-v13.png` Best short summary of the current state: - `alpha behavior`: materially improved and now matches Crusader xform-slot expectations much better than the old remap-only path - `hue behavior`: materially improved for the tested shapes, but still not proven closed for every translucent family ## What Is Still Unresolved - the exact semantic structure of `XFORMPAL.DAT` is still not closed - entry `1` (`32` bytes) remains unexplained in this batch - the choice of remap table `5` is evidence-backed for the tested shapes, but still empirical rather than formally decoded - other translucent families may need a different XFORMPAL table than the current default - this batch validated via direct shape probes and a fresh scene build, but not yet via a broad multi-map manual browser pass ## Recommended Next Steps 1. Run a manual browser-side check on a few representative maps and note which translucent families now look correct versus still wrong. 2. Identify at least one real `window` family shape from live cached scenes and add it to the comparison set, instead of relying mostly on editor-wall and helper probes. 3. Compare remap table `5` versus nearby candidates `6` and `7` for the remaining suspect families if any still look off. 4. Reverse-engineer the meaning of the `32`-byte second XFORMPAL entry and look for any runtime selector or table-id field that explains why multiple remap tables exist. 5. If one-family-per-table evidence appears, move from a single global translucent RGB table to a family- or shape-driven selector.