Crusader_Decomp/docs/map-rendering.md

# Crusader Map Rendering Workbench

## Purpose

This note starts a dedicated lane for offline Crusader map extraction and PNG rendering from the shipped data files in this workspace.

Current implementation entry point:

- `tools/render_crusader_map.py`

Current supported data roots:

- `STATIC` for No Remorse
- `STATIC_REGRET` for No Regret

Current asset note:

- `STATIC_REGRET` in this workspace now includes `FIXED.DAT`
- the renderer still accepts `--fixed-dat` so alternate map copies can be tested without changing the rest of the static asset path

The immediate goal is practical and narrow: load a fixed map, expand glob placements, decode the required shapes from `SHAPES.FLX`, apply `GAMEPAL.PAL`, and render a deterministic PNG.

## Source Cross-Checks Used

The first renderer is grounded in the overlapping parts of three sources rather than in ad hoc guesses.

1. Pentagram Crusader shape/map loaders
   - `convert/crusader/ConvertShapeCrusader.cpp`
   - `graphics/Shape.cpp`
   - `graphics/ShapeFrame.cpp`
   - `world/Map.cpp`
   - `world/MapGlob.cpp`
   - `graphics/Palette.cpp`
   - `graphics/TypeFlags.cpp`

2. ScummVM Ultima8 Crusader paths
   - `gfx/shape_archive.cpp`
   - `gfx/type_flags.cpp`
   - `world/map.cpp`
   - `world/glob_egg.cpp`
   - `world/coord_utils.h`
   - `world/item_sorter.cpp`
   - `world/sort_item.cpp`

3. Local workspace evidence
   - `docs/scummvm-crusader-reference.md`
   - `docs/pentagram-crusader-reference.md`
   - `docs/raw-0007-rendering.md`
   - `crusader-disasm/shapedata.txt`
   - `crusader-disasm/mapdump/mapdump.py`

## File Formats Used By The First Tool

### `FIXED.DAT`

The map container is treated as a header plus a map table:

- map count at file offset `0x54`
- map table at file offset `0x80`
- each table row is `<u32 offset, u32 size>`

Each map payload is read as packed 16-byte item records:

- `x: u16`
- `y: u16`
- `z: u8`
- `shape: u16`
- `frame: u8`
- `flags: u16`
- `quality: u16`
- `npc_num: u8`
- `map_num: u8`
- `next: u16`

Crusader-specific coordinate adjustment matches the Pentagram and ScummVM runtime loaders:

- world `x = disk_x * 2`
- world `y = disk_y * 2`

### `GLOB.FLX`

`GLOB.FLX` is handled as a normal FLEX archive, not as a one-off format.

Each non-empty glob object contains:

- object count: `u16`
- repeated entries of `x:u8 y:u8 z:u8 shape:u16 frame:u8`

Glob expansion matches the Crusader `GlobEgg::enterFastArea()` rule in ScummVM/Pentagram:

- `coordmask = ~0x3ff`
- `coordshift = 2`
- `offset = 2`
- `itemx = (parent_x & coordmask) + (glob_x << 2) + 2`
- `itemy = (parent_y & coordmask) + (glob_y << 2) + 2`
- `itemz = parent_z + glob_z`

The first renderer expands glob contents and skips drawing the source glob egg itself.

### `SHAPES.FLX`

World shapes use the Crusader shape layout documented by Pentagram/ScummVM:

- shape header: 6 bytes
  - 4 bytes unknown
  - 2-byte frame count
- frame header table: 8 bytes per frame
  - 3-byte frame offset
  - 1 unknown byte
  - 4-byte frame length
- frame body header: 28 bytes
  - 8 unknown bytes
  - 4-byte compression flag
  - 4-byte width
  - 4-byte height
  - 4-byte x offset
  - 4-byte y offset
- then `height` 4-byte line offsets
- then per-line RLE data

The current decoder follows the runtime line walker used in `ShapeFrame::getPixelAtPoint()`:

- each line is a series of skip/run pairs
- compressed runs use the low bit to choose literal versus repeated-color mode
- pixels absent from the RLE stream are treated as transparent

### `GAMEPAL.PAL`

`GAMEPAL.PAL` is read as 768 bytes of VGA-style palette data.

Each component is promoted from `0..63` to `0..255` using the same scaling used by Pentagram:

- `rgb8 = (rgb6 * 255) / 63`

### `TYPEFLAG.DAT`

The renderer currently uses Crusader's 9-byte records to extract:

- family id
- shape footpad dimensions (`x`, `y`, `z`)
- editor flag

This is enough for:

- skipping known egg families in the first pass
- expanding `SF_GLOBEGG`
- documenting future work toward a better sorter

The current tool does not yet use the footpad values for full ItemSorter-equivalent overlap resolution.

## Current Projection And Painting Rules

The renderer anchors each shape at the same world-to-screen bottom point used by the runtime shape painter:

$$
screen_x = \frac{x - y}{4}
$$

$$
screen_y = \frac{x + y}{8} - z
$$

Frame placement then follows the shape-frame offsets used by the runtime sorter:

- unflipped: `left = screen_x - xoff`
- flipped: `left = screen_x + xoff - width`
- `top = screen_y - yoff`

The renderer now uses a ScummVM/Pentagram-style dependency graph sorter rather than a plain scalar key.

The current implementation ports the crucial parts of `SortItem` and `ItemSorter`:

- footpad-derived world boxes from `TYPEFLAG.DAT`
- screen-diamond overlap and containment checks
- `below()` ordering rules for flat pieces, tall pieces, roofs, translucent items, and Crusader inventory-item families
- dependency expansion so overlapping items are painted only after everything behind them

This is materially better than the initial `z / x+y` heuristic and is the main path for reducing wall and prop overdraw artifacts, though it still omits some of the engine's more specialized runtime-only cases.

## Command Examples

Render No Remorse map `0`:

```powershell
c:/Users/Maddo/.PYENV/PYENV-WIN/versions/3.14.3/python.exe tools/render_crusader_map.py --game remorse --map 0 --output out/map0-remorse.png
```

Render No Regret map `0` and emit metadata:

```powershell
c:/Users/Maddo/.PYENV/PYENV-WIN/versions/3.14.3/python.exe tools/render_crusader_map.py --game regret --fixed-dat K:/path/to/REGRET/FIXED.DAT --map 0 --output out/map0-regret.png --metadata out/map0-regret.json
```

Render a bounded world-space region only:

```powershell
c:/Users/Maddo/.PYENV/PYENV-WIN/versions/3.14.3/python.exe tools/render_crusader_map.py --game remorse --map 0 --world-rect 0 0 4096 4096 --output out/map0-quarter.png
```

## Current Deliberate Limits

This tool is a start, not a complete engine clone.

Current gaps:

1. It renders `FIXED.DAT` only. It does not yet merge save-state or `NONFIXED.DAT` style movable items.
2. It expands globs, but it does not yet emulate broader fast-area/runtime-driven materialization behavior.
3. It skips several egg-family placements instead of trying to visualize their hidden runtime helpers.
4. It now implements the core dependency graph sorter, but it still omits experimental occlusion grouping and some runtime-only sprite/highlight cases.
5. It does not yet consume `ANIM.DAT`, `DAMAGE.FLX`, `DTABLE.FLX`, `WPNOVLAY.DAT`, or palette transforms such as `XFORMPAL.DAT`.
6. It uses `GAMEPAL.PAL` directly and does not yet model alternate or transformed palettes.
7. It writes a plain RGBA PNG using only the standard library; there is no zoomed viewer, tile atlas exporter, or sprite manifest yet.

## Immediate Follow-Ups

1. Validate and tune the dependency sorter against representative Remorse and Regret rooms, especially tall wall seams and dense prop clusters.
2. Add optional atlas export for all shapes touched by a chosen map.
3. Add a second path for movable/dynamic content once the relevant Crusader save/runtime files are pinned down for both games.
4. Compare a few rendered regions against known in-game screenshots to tighten projection and ordering errors.
5. Add optional per-item manifest output with `(shape, frame, x, y, z, source)` rows for debugging bad composites.
6. Revisit raw `0007` rendering notes and the live executable only if the current Pentagram/ScummVM overlap model proves insufficient for specific remaining errors.