Full-Precision Textures: Undoing the 4444 Cut Inside the Game's Own Driver (WIP)

Range: v0.0.212 to v0.0.213 (issue #37)

Run pumpitea and the faces on the song select screen and the folds in the BGA look blotchy. It looks like an emulator defect, but the cause is inside the game's executable. PIU draws through OpenGL and links a whole Mesa 3.x driver into the executable to turn that OpenGL into Glide; that driver cuts every color texture to 4 bits (ARGB4444) or 5/6 bits (RGB565) per channel before the hardware sees it. The original 8-bit image stays in the game's memory afterwards. This post records finding that original and adding a quality option that uploads it instead, only when it is verified to be exactly the same texture.

Major Changes

1. What it looks like

pumpitea was run twice with the same input script and captured at the same moments, once with the option off (=0, what the arcade hardware showed) and once on (=1, the default), on the Win32 x86 Release build, an RTX 4090 and the 2x window (1280×960). Every comparison below is off on the left, on on the right. Snowflakes and background animation land differently in each run; ignore them.

Song select — the jacket of the first song, "Jachaek" (U TWO)

At 3x (nearest), 4444 breaks the skin into 16-step bands and blotches, while 8 bits keeps it smooth.

The first song's BGA

The picture slides sideways in this scene and the two runs differ by 30 pixels, so the crops are aligned.

At 3x, the shading of the white turtleneck and the folds of the red coat show the difference best.

Off (4444/565)	On (8-bit original)

2. Where the quality went — Mesa inside the game

repiu_aot_probe gained --dump-image to write the relocated executable image to files, which were read with capstone. Mesa 3.x's 3dfx driver (fxMesa) is there as is; fxTMInit, met while chasing a Glide crash (Task 235), was this driver's texture memory manager.

flowchart LR
    G["game<br/>glTexImage2D(GL_RGBA, 8-bit)"] --> F["fxTexGetFormat<br/>RGBA → ARGB_4444<br/>RGB → RGB_565"]
    F --> B["fxTexBuildImageMap<br/>(A&0xF0)#lt;#lt;8 | (R&0xF0)#lt;#lt;4<br/>| (G&0xF0) | (B&0xF0)#gt;#gt;4"]
    B --> D["grTexDownloadMipMapLevel<br/>(only 4444/565 passes)"]
    D --> V["Voodoo / rePIU Glide HLE"]
    G -.the original stays.-> I["gl_texture_image +0x34<br/>(8-bit RGBA)"]

• It is cut whatever the game asks for. fxTexGetFormat maps everything from GL_RGBA to GL_RGBA16 to ARGB4444 and the RGB formats to RGB565, dropping the low bits with no rounding or dithering.
• The arcade showed the same. Real Voodoo hardware got this 4444 data too, so the option is not a fix but better than the original, and turning it off gives the arcade picture.
• The same build in all 19 ROM sets. Three byte signatures — the format function's entry, the 4444 packing and the upload call tail — occur once each in every set at the same relative distance; only the addresses differ.

3. Finding the original — from registers and the stack

The Glide gate (grTexDownloadMipMapLevel) never changes game code; it only reads the guest registers and memory at that moment. Just before the call, Mesa's upload function holds the fx texture info (ti) in ESI and the Mesa level in EBP, and has saved its caller's EBP — the texture object tObj — on the stack, at [ESP+0x34] as seen from the gate.

flowchart TD
    G["gate: format 12 (4444) or 10 (565)"] --> R["ti = ESI, level = EBP,<br/>tObj = [ESP+0x34]"]
    R --> V1{"tObj+0x484 == ti ?"}
    V1 -->|no| F["the game's data as is"]
    V1 -->|yes| V2{"ti's level data ==<br/>the gate's data ?"}
    V2 -->|no| F
    V2 -->|yes| I["tObj.Image[level]<br/>width, height, original pixels"]
    I --> B["RGBA8 at the Glide size<br/>(Mesa's whole-factor nearest)"]
    B --> V3{"cut to 4444/565 again,<br/>equal to the game's data byte for byte?"}
    V3 -->|no| F
    V3 -->|yes| U["use the 8-bit original"]

The third check is the key: the original is used only if cutting it again exactly as Mesa does yields not one byte different from what the game passed. Should another call path or build lead to the wrong memory, that texture simply stays 4444; a wrong picture cannot appear. The stack offset was first inferred by static analysis and confirmed when every upload in real runs passed all three checks.

4. Switching it in the OSD

The guest uploads a texture once and never sends it again. So the backend keeps both the game's data (expanded to RGBA8) and the 8-bit original for each verified texture, and when the option changes the host thread re-uploads them on its next event pump.

sequenceDiagram
    participant O as OSD checkbox
    participant A as atomic option value
    participant P as PumpEvents (host thread)
    participant T as texture entries
    participant GL as OpenGL
    O->>A: store(on/off)
    P->>A: load — returns at once if equal to the applied value
    P->>T: only entries holding both images
    T->>GL: glTexSubImage2D(same GL name, original or game data)
    P->>GL: rebind the texture the game selected
    P->>P: log "full-precision textures on: N re-uploaded"

Only the contents of the same GL texture are replaced, so the texture addresses and TMU state the game holds do not change, and neither does GPU memory (the data was already expanded to RGBA8 before).

5. Three places to choose

It is on by default.

6. Execution log

The final count lines of two user runs: every upload was replaced by its original, and every failure reason counts zero.

pumpitea
[loader] Glide texture full precision used/not-applicable/unreadable/link-mismatch/data-mismatch/verify-mismatch: 61/0/0/0/0/0
[repiu-glide] full-precision textures off: 18 re-uploaded
[repiu-glide] full-precision textures on: 18 re-uploaded

pumpit8 (about 65 s, 41 toggles)
[loader] Glide texture census uploads/distinct/identical-repeats/changed-repeats: 130/39/2/89
[loader] Glide texture full precision used/not-applicable/unreadable/link-mismatch/data-mismatch/verify-mismatch: 130/0/0/0/0/0
[loader] Glide texture census format 10: 42
[loader] Glide texture census format 12: 88

There is no blocker.

7. Sample test results

xychart-beta
    title "Texture uploads replaced by the 8-bit original"
    x-axis ["pumpitea 4444", "pumpitea 565", "pumpit8 4444", "pumpit8 565"]
    y-axis "uploads" 0 --> 100
    bar [48, 13, 88, 42]

Known

• Memory. Each verified texture keeps two extra RGBA8 copies in host memory (about 32 MB at most). The game-data copy can be rebuilt by cutting the original again, halving that, but it stays for now and is recorded as a follow-up.
• ROM sets other than pumpitea and pumpit8 have not been run (their signatures match).
• Whether the path that enlarges an original smaller than the Glide size occurs in practice was not counted; a failed check falls back to 4444.

Technology Stack Used

Mesa 3.x and the 3dfx Glide driver

In the late 1990s a common way to use OpenGL on a Voodoo card was Mesa's fx driver, a layer turning OpenGL calls into Glide calls. PIU links that driver statically, so what we see at the Glide boundary is not the game's request but the driver's already converted result. To save Voodoo texture memory, the driver chose 16-bit-per-texel formats for color textures.

Glide 16-bit texture formats

A smooth gradient reduced to 16 steps becomes visible bands, most of all on bright, gently varying surfaces such as skin.

Static analysis: the relocated image and capstone

Addresses in an LE/PE executable mean something only once it is relocated to its load address. repiu_aot_probe --dump-image writes each object, relocated as the loader does, as obj<N>_<base>.bin, which capstone disassembled at those very addresses. Constant comparisons such as GL_RGBA (0x1908) and the and 0xF0 packing were the handles for finding the functions.

Finding an object through a register the caller saved

An x86 function saves the registers it uses on the stack at entry and restores them on exit. Mesa's upload function pushes ecx, esi, edi, ebp, reserves 0x10 bytes of locals, then pushes the eight arguments of grTexDownloadMipMapLevel (0x20 bytes) and calls. From the gate, then, past the return address (4), the arguments (0x20) and the locals (0x10) — 0x34 bytes above ESP — lies the texture object the caller held in EBP. No pointer leads back from ti to tObj, hence this route, and the tObj->DriverData == ti check confirms it is the right object.