I suppose that there’s some irony that it’s taken me over 4 years to write the next entry of the ZX Spectrum emulator blog which was going to be centered around the machine’s bizarre display memory layout.

After writing that initial blog post, I did immediately start writing the next entry describing the Spectrum’s display memory and how I had handled it within the work in progress emulator, and even started drafting a third post describing the process of starting to support the Lords of Midnight game snapshot. I also progressed the emulator codebase to the point of being able to boot into the ZX Spectrum 48k boot screen, and to the Lords of Midnight loading screen using the game data. And then life and contract work got in the way, and it all paused.

Earlier this year, whilst doing a bit of an audit of the Gentlemen Of Science code repositories during some down time between contracts, I moved the archived emulator codebase out of our previous source control solution and into GitHub. At the same time, I imported it into a more up-to-date version of Unity and ‘had a play’. The state it had been left in was that keyboard input was not functioning on the boot screen when starting up normally, and in Lords of Midnight you could start the game from the loading screen by pressing any key, and it would attempt to start drawing the first screen in game, but then the display would quickly corrupt and the game would crash back to the boot screen. I had a little bit of time on my hands, so thought I’d debug The Lords of Midnight problem and see if I could fix it. It took a bit of time to diagnose, but fix it I did, and then I carried on for a bit to see how playable I could actually make the whole game in the emulator, and with a few more fixes and updates, I got to the point where - as far as I could tell - Lords of Midnight became fully playable! At this juncture, I was pretty happy to put the whole project down again for now. The original intention was to disassemble and recreate the source code for the game in a readable format, and I may well return to do this again in another four years. But if that’s the case, in the meantime, I’m going to document everything that I intended to document previously!

I’m not going to put chunks of code into the update this time, now that it’s progressed beyond foundational stuff, so if anyone is reading this and is interested in the complete source code, let me know and I am happy to share.

Without further ado, let’s (finally) start with that display memory …

In the last episode, we’d just got to the point where we were processing the first few bytes of contiguous memory from our 48k Spectrum memory map, and starting to emulate the boot process. If we now fast forward over a couple of weeks worth of churning through our emulator hitting byte instructions that it doesn’t yet recognise, looking them up in our Z80 reference guide, and implementing them, we eventually get to the point where the memory area reserved for the display is starting to get written to.

Displaying stuff on the screen

When we talk about the “memory at address zero”, we’re talking about the first byte. When we talk about the memory address 16384, we’re talking about the 16384th sequential byte. And it just so happens that this is the memory address from which we can start poking pixels onto the screen.

Reset your trusty Spectrum, or emulator, and type the following, then press Enter.

POKE 16384, 255

Congratulations, you drew a really fast, really small black line in the top left of your screen!

The layout of the Spectrum’s display memory is weird. When you figure out the formula, it’s logical, but it’s a conundrum for the game programmer.

The main screen of the Spectrum is 256 pixels wide by 192 pixels high. This equates to 32 8x8 pixel characters wide, by 24 pixel characters high. This part is important to note, because clearly the designers of the Spectrum expected most folks would find it more practical to work in character space rather than pixel space.

As we’ve just established, the memory address for drawing pixels onto the screen starts at 16384, and if you fill that byte with ones (i.e. 255), you’ll fill-out a line of 8 pixels at the top-left of the screen. If you want to draw just the first pixel in the line, set the first bit only (i.e. 128). So now, if you poke 16385, the next memory address, with 255, you’ll draw the next 8 pixels. So far so good. In fact you can repeat this process 31 times and draw along the whole top of the screen. Now things start getting a bit weird. A 256 increment from the first byte address (16384+256=16640) will take you onto the next pixel line at the start of the next row. After 8 lines have been drawn however, we go back to the first address and increment by 32 to move onto the 9th line (the second 8x8 character). We can carry on happily and confusedly drawing from left-to-right, top-to-bottom like this. Until we hit the bottom third of the display, and then, if they weren’t already, things get really odd.

I could carry on at this point and go into detail about how the rest of the pixel memory is laid out, and then onto how the separate background and foreground colour and attribute memory is stored in character space rather than per-pixel, which explains the Spectrum’s infamous colour/attribute clash, but there are plenty of books, documents and websites out there which explain the precise layout in much greater clarity than I can, so suffice to say I read some of these, parsed the appropriate bytes and bits in the emulator code, and coloured them in on a custom 256x192 Render Texture in Unity.

The Lords of Midnight

In the first part of this ongoing emulator saga, I mentioned that I’d set out on this journey originally hoping to disassemble the TAP file of The Lords of Midnight, publicly available on the World of Spectrum https://worldofspectrum.org/archive/software/games/the-lords-of-midnight-beyond-software site, to get a look at the inner workings of the game.
Rather than figuring out how to implement the tape loading for this file into my emulator at this stage, and taking another obvious diversion, I decided it might be quicker just to use the Spectaculator https://www.spectaculator.com emulator memory dump feature to load the game in, and get the complete 64k memory map out instead. Then I could load these bytes straight into my current emulator code, and see where I could go from there.

Loading the TAP file, and immediately opening the debugger, shows that the program counter is captured at 0x02AE, and the display still shows the game loading screen/splash. We also know from running the game properly, that any keyboard input at this point will start the game, so we can assume that from or around 0x02AE where the program counter is sat is likely to be within some kind of input update loop.

If we now use Spectaculator’s handy memory export feature to dump the entire contents of the 65536 bytes of memory, we can use this binary file to populate our emulator’s memory wholesale, and start executing from 0x02AE, see what happens.

And we have a splash (screen)!

Bugs, keyboards and interrupts

Keyboard input on the Spectrum operates via instructions for reading values at a specified port address. To emulate this in the code, I added another byte array in an identical manner to the byte array used for the memory map, and used the Unity Input API to parse the keys on the PC keyboard and write to the corresponding bits in the port array. Then it’s just a case of adding support to our emulated IN [register], (C) instruction to read back these bits in the new port array, and we can now parse the keyboard in our emulated Z80!

From the point of getting the splash screen displaying, and the keyboard support implemented. Pressing any key “just worked”, to progress into the game!
At address 5B1A in the game code memory snapshot, there’s an instruction CALL 028E. This is a routine in the ROM, documented in the ROM disassembler book as KEY-SCAN, and subsequent register and flags reading can then be made to determine if a key has been pressed, and which specific one it was.

The first time I saw the first game screen start to appear after the loading screen had progressed, though, resulted in some mixed emotions. We got the location description text, and the background and border, but then as it tried to draw the actual environment (trees and mountains and so forth), the drawing would corrupt and after a few seconds it would crash back to the ZX Spectrum OS screen.
None of my usual debug error output was showing anything untoward (- for example, if I’d missed implementing a whole instruction, I would log an error saying that the program counter had reached some bytes in memory that it didn’t know what to do with). It was looking like it was going to be a case of stepping through the code line by line until I found the problem, and so that is what I did. Fortunately I had the Spectaculator debugger that I was able to run side-by-side with my emulator debugger, which felt like a bit of a cheat, but knowing where I could breakpoint the code - from the splash keyboard input address above - meant that I could step and compare memory and register state between what my emulator thought everything should be, and what Spectaculator reliably said it should be.
After a fairly laborious step by step walk through the code in both applications, I found a bug! The instruction LD IX, ([address]) had the upper and lower bytes of the address the wrong way around when comparing between my emulated instruction, and Spectaculator. I’d got my endian-ness wrong here! I did a quick pass through all of the other instructions I’d implemented support for which might have a similar issue, but as it turned out, this was the only one where I’d got it wrong. After fixing this, I eagerly booted into The Lords of Midnight in the emulator again, pressed a key to start … and the display was still corrupted. But at least it didn’t crash back to the OS this time, so that fix had achieved something. Obviously there was at least another bug to find, though, so back to it. Although this time I could restart my debugging from the address of the bugged instruction, though, saving a bit of time.

The next bug, found a little quicker, was an interesting one, in that I came across an instruction which my emulator code had wrongly parsed as something else entirely. Spectaculator interpreted DEC IX and DEC IY instructions, which I’d omitted to add support for in my code, but rather than just failing to interpret them and report back the error, my parser code had instead picked another instruction with the same expected bits and bytes and wrongly interpreted that instead. So I quickly added support for these simple instructions in my code, fired up the emulator again with bated breath and … we have correctly drawing environments!

But once again, a brief period of elation turned into confusion, as no keyboard input was working in the game. I had a hunch that I knew exactly what the problem was this time though, as I hadn’t implemented any support in my emulator code yet for Interrupt Modes.

The Spectrum 48k supports three different interrupt modes (0, 1 and 2), and I’m not going to go into detail about how they work and what they mean either, because again there’s plenty of good documentation out there which can explain the technical details better than I - but suffice to say, the Speccy defaults to IM1 (Interrupt Mode 1), which basically interrupts the executing code every so often to make a call to the ROM keyboard input handler routine at address 0x0038, before resuming again from the address of interruption when the ROM subroutine returns (finishes). The executing game code was relying on this happening and not explicitly calling anything itself to read the keyboard, unlike how it did exactly that to progress from the splash screen. In game, it is waiting for those flag and register changes to occur on a keypress, which were never allowed to happen in my emulator. So to test the theory, I quickly bodged in some code to simulate this interrupt mode occurring into the emulator’s main update loop, added support for enabling and suppressing this through calling the instructions EI and DI (Enable Interrupts and Disable Interrupts, respectively) and suddenly we really did have the whole game playing!

And this is where the emulator has gone back into hibernation again for now.

I am quite keen to try out an arcade game in it at some point, something like Manic Miner maybe, and add support for audio as well, but the main thrust of this whole process was still to document the source code for Lords of Midnight, and so perhaps I shall revisit that goal again first.

Maybe in another four years.