Preserving DOS4GW Execution: Work in Progress 1

Range: 40fc5a6 through e06f13c

Major Changes

The core of this progress is a step-by-step foundation for running the original 32-bit x86 DOS/4G code inside a Win32 process without rewriting the game logic in C++. The work started with a non-executing analyzer for PIU.EXE, then grew into LE object mapping, fixup decoding, internal relocation dry-runs, relocated image buffers, Win32 process memory placement, guest stack switching, and observation-driven HLE trap handling.

flowchart TD
    A[TargetProfile: piu_1st] --> B[DOS/4GW MZ + LE parser]
    B --> C[LE object/page/fixup analysis]
    C --> D[Relocatable runtime image plan]
    D --> E[Relocated image buffers]
    E --> F[Win32 VirtualAlloc placement]
    F --> G[Guest stack trampoline]
    G --> H{Original x86 code}
    H -->|STI/CLI| I[Privileged trap HLE]
    H -->|INT 21h| J[DOS HLE dispatcher]
    H -->|segment access| K[Segment shadow HLE]
    J --> L[DOS virtual filesystem]
    J --> M[Resize/IOCTL/write observations]
    M --> N[Current blocker: 0x020F7340 C7 memory write]

The early commits established the project rules and architecture documents first, then registered the piu_1st target profile and HLE profile statically. After that, Dos4gwExecutableLoader learned to read the MZ header, LE header, object table, page table, fixup page table, and fixup record table, then classify relocation source types. Representative commits include 364ddbc, d872740, 6904572, and 3b7bc41.

In the middle phase, the Win32 x86 host tried to reserve the original low address range expected by the original code, but that range is often already occupied. Instead of relying only on fixed low-address execution, the loader moved toward placing original LE objects at a safe relocated base and reapplying fixups for that base. This direction spans 3e59c49, afcbc71, 21948d5, 831b8ae, and 38be1eb.

In the execution phase, repiu_loader_win32 places the relocated image into Win32 process memory, switches to the guest stack, and enters the original entry point. Exceptions and traps are treated as observation points rather than dead ends. The loader handles STI as the first HLE trap, then gradually connects INT 21h AH=0x30, the observed AH=0xFF, segment register load/store, segment override memory loads, and DOS filesystem HLE. Representative commits include 58db6f2, baa89f4, 72ab28b, 0e7a80b, ed3ccc2, a8cf592, and 5270ee9.

As of the latest commit, piu_1st reaches a stage.cfg open attempt after passing DOS current-directory change, file open, IOCTL, console write, and resize observations. The file currently does not exist at the asset path, so the DOS open is reported as error 0x0002; after that, the next blocker is a normal memory write.

Win32 minimal execution exception caught: true
Win32 minimal execution exception code: 0xC0000005
Win32 minimal execution exception address: 0x020F7340
Win32 handled DOS interrupt count: 88
Win32 last handled DOS interrupt AH: 0x4A
Win32 handled DOS chdir count: 1
Win32 last DOS chdir guest path: \datas\bga
Win32 last DOS chdir virtual path: \DATAS\BGA
Win32 handled DOS open count: 2
Win32 last DOS open guest path: stage.cfg
Win32 last DOS open virtual path: \DATAS\BGA\STAGE.CFG
Win32 last DOS open result: failure
Win32 last DOS open error: 0x0002
Win32 handled DOS IOCTL count: 2
Win32 handled DOS resize count: 40
Win32 last DOS resize selector: 0x0024
Win32 last DOS resize paragraphs: 0x4AE1
Win32 last DOS resize result: success
Relocated exception bytes: ... [C7] 01 FF FF FF FF ...
Current execution blocker: unhandled or unclassified instruction/memory access at exception point

The OpenWatcom sample test was added as a regression metric for DOS/4GW console sample compatibility. The latest baseline covers 819 samples: 793 build passes, 26 explicit build skips, 793 run-eligible samples, and 473 run passes. The overall pass rate is 57.8%, the build pass rate is 96.8%, and the run pass rate is 59.6%.

xychart-beta
    title "OpenWatcom Sample Cumulative Results"
    x-axis ["0.0.1", "0.0.4", "0.0.5-a", "0.0.5-b"]
    y-axis "Samples" 0 --> 850
    line "Total" [819, 819, 819, 819]
    line "Build Passed" [788, 788, 793, 793]
    line "Run Passed" [419, 470, 473, 473]
    line "Build Skipped" [0, 0, 26, 26]

Current verification used scripts/test_all.ps1. The normal sandbox run failed because CMake could not restore a timestamp under build/win32_x86_debug/_deps/spdlog-subbuild, but rerunning the same command with elevated permissions passed the Win32 x86 host build, dos4gw_hello execution, and the piu_1st observation check.

Technology Stack Used

The first axis is DOS/4GW and Linear Executable analysis. LE is a protected-mode executable format following an MZ header, used by DOS extenders to run 32-bit protected-mode code. This project reads the LE object and page tables to reconstruct original code/data objects, then decodes fixup records and writes relocated internal pointer values for the selected relocated base. Reference: Linear Executable overview.

The second axis is direct Win32 x86 execution. Instead of reimplementing original code inside an emulator, the loader creates executable memory inside a 32-bit host process and jumps to the original entry. VirtualAlloc can reserve and commit process virtual address space; this project probes relocated base candidates when the fixed low address range is blocked, then places object buffers there. Reference: Microsoft VirtualAlloc.

sequenceDiagram
    participant Host as Win32 Loader
    participant Image as Relocated Image
    participant CPU as Original x86 Entry
    participant HLE as HLE Dispatcher
    Host->>Image: Reserve/commit relocated arena
    Host->>Image: Copy LE objects and apply fixups
    Host->>CPU: Switch to guest stack and call entry
    CPU-->>HLE: Exception/trap/INT observation
    HLE-->>CPU: Update context, flags, registers, EIP
    CPU-->>Host: Return or next blocker

The third axis is DOS INT 21h HLE. The DOS API uses INT 21h plus an AH subfunction for file, directory, console, and memory services. Important observed services in this range include AH=0x30 DOS version query, AH=0x3B chdir, AH=0x3D open, AH=0x40 write, AH=0x44 IOCTL, and AH=0x4A memory resize. The implementation does not emulate all of DOS at once; it connects only the services actually reached by piu_1st and the samples, with minimal semantics. Reference: DOS API INT 21h list.

The fourth axis is segment-register shadowing and privileged-instruction traps. Win32 user mode cannot execute privileged instructions such as STI directly, and guest DS/ES/FS semantics do not map one-to-one to host segment registers. The loader observes exception contexts and, when needed, keeps guest segment selectors in separate shadow state while correcting memory access behavior through HLE. This preserves the original execution flow while replacing only the OS/CPU privilege boundary.

The fifth axis is OpenWatcom sample-based regression testing. OpenWatcom provides C/C++ samples that are useful for DOS/4GW console runtime coverage, but sample sources and EXEs are not vendored into the repository because of license conditions. Instead, local installed samples are built outside Git, while test scripts, baselines, and history JSON are tracked. References: OpenWatcom v2 repository, OpenWatcom license.