To mark the culmination of a lot of work on the BusRaider (retro computer display and debug board), I’ve started a series of short(ish) posts to answer the question: “What can you do with an RC2014” (a question asked on the RC2014 forum). The twist though is that my answers make the assumption that you have a BusRaider too 🙂
This is the first post and concerns the use of BusRaider as a tool for assisting in the development of other RC2014 hardware. It makes use of the BusRaider’s ability to drive the RC2014 bus in “Bus Master” mode. This is generally used for grabbing the display contents (when emulating a retro computer that had a memory-mapped display) and for grabbing the RAM contents during debugging. But the capability also allows the BusRaider to access the bus for other purposes and I’m going to document this capability for use in developing add-on hardware for the RC2014.
Developing RC2014 Hardware
When developing a new piece of hardware for a target computer it is desirable to put the board through its paces using some test programs. Normally these would execute on the target processor and would exercise the logic on the board. BusRaider provides an alternative for some simple tests by allowing a script in a high-level language on a separate computer to exercise the board’s functions.
To demonstrate how this is done I’m using a variant of Spencer’s 512KRAM+512KROM board that I’m working on which has additional support for paging using the PAGE (RST2) line on the RC2014 PRO bus. This board isn’t yet released as the current version has known issues – which we are about to resolve using the BusRaider’s super-powers! I will publish all the details on the board when I have a working version. In normal use (if the PAGE line is inactive/pulled-up) the board should be a drop-in replacement for Spencer’s board – BUT it isn’t working so here goes fixing it.
Firstly we need to establish how to communicate with the BusRaider. This is most simply done over WiFi and using the BusRaider REST API which is the same API that the BusRaider’s Web UI uses. Using this API can be as simple as typing the API command into a browser window. For instance, to disable the regular BusRaider bus control you could use the command directly in your browser’s address bar:
http://busraideripaddr/targetcmd/rawBusControlOn
Where: busraiderapiaddr is the IP Address of your BusRaider (see the manual for details of getting the BusRaider onto WiFi and finding the IP Address).
This should elicit a response like this:
{"cmdName":"rawBusControlOnResp","err":"ok","raw":"3fe6d033","pib":"fe","ctrl":"MIRW.","msgIdx":0,"dataLen":0}
The important part here is the “err”:”ok” which indicates the command completed successfully. The additional information isn’t relevant right now but is explained later on.
Using the API from Python
Instead of typing each API command into a browser’s address bar it would be good to setup a script to do this for us. Python is a suitably simple scripting language and we can use the Requests module to allow Python to make http requests. To install requests just use:
pip install requests
It is best to use Python 3 and on some computers with both Python 2 and Python3 installed you can use pip3 in place of pip to ensure you use the right package manager.
I have added a number of useful API commands which we can use for this kind of hardware debugging:
- rawBusControlOn / rawBusControlOff
- rawBusWaitDisable / rawBusWaitClear
- rawBusTake / rawBusRelease
- rawBusClockEnable / rawBusClockDisable
- rawBusSetAddress / rawBusSetData
- rawBusSetLine
- rawBusGetData
Using these commands allows reasonably complete control over the Z80 bus and allows exercising of hardware. An example is this:
# Import the requests library
import requests
url = "http://busraideripaddr/targetcmd/"
# Disable BusRaider's managed control of the bus
req = requests.request("get", url + "rawBusControlOn")
print(req.json())
# Disable and clear WAIT state generation
req = requests.request("get", url + "rawBusWaitDisable")
req = requests.request("get", url + "rawBusWaitClear")
# Take control of the bus
req = requests.request("get", url + "rawBusTake")
# Set the address bus to 1234 hexadecimal
req = requests.request("get", url + "rawBusSetAddress/1234")
# Set the data bus to 55 hexadecimal
req = requests.request("get", url + "rawBusSetData/55")
# Write to memory at this address
req = requests.request("get", url + "rawBusSetLine/MREQ/0")
req = requests.request("get", url + "rawBusSetLine/WR/0")
req = requests.request("get", url + "rawBusSetLine/WR/1")
req = requests.request("get", url + "rawBusSetLine/MREQ/1")
# Note that BUSRQ isn't released or control returned to BusRaider
# This is to ensure the test condition can be viewed as required
# To restore control either use rawBusRelease and rawBusControlOff
# Or just reset the system
Note that all the values used for address and data are hexadecimal values (without any prefix) and make sure you set your IP address for these examples to work.
What this does is to walk the bus through the following steps:
- Disable BusRaider’s managed control of the bus
- Disable wait states
- Take control of the bus (using BUSRQ)
- Set the address bus
- Set the data bus
- Sequence the MREQ and WR lines to write the value to memory
In addition information from the bus is returned too, the “pib” value in the JSON is the data read from the multiplexed bus on the BusRaider and the “raw” value is what the Pi read from all of its GPIO block 0 pins (which includes all of the external pins that we use – for details of which bit means what you would need to view the schematic).
A More Complex Example
In order to debug the 512K RAM / ROM with Paging card I set up the following script initially:
import requests
url = "http://busraideripaddr/targetcmd/"
req = requests.request("get", url + "rawBusControlOn")
print(req.json())
req = requests.request("get", url + "rawBusWaitDisable")
req = requests.request("get", url + "rawBusWaitClear")
req = requests.request("get", url + "rawBusTake")
ramRomPageWrBase = '70'
ramRomPgenWrBase = '74'
req = requests.request("get", url + "rawBusSetAddress/" + ramRomPageWrBase)
print(req.json())
for i in range(8):
if i % 2 == 0:
req = requests.request("get", url + "rawBusSetData/01")
else:
req = requests.request("get", url + "rawBusSetData/00")
req = requests.request("get", url + "rawBusSetLine/IORQ/0")
req = requests.request("get", url + "rawBusSetLine/WR/0")
req = requests.request("get", url + "rawBusSetLine/WR/1")
req = requests.request("get", url + "rawBusSetLine/IORQ/1")
This script is similar to the previous example but instead writes the values 0x01 & 0x00 alternately to the IO address 0x70 (which sets and clears the PAGE_EN value in the 74HCT74 flip-flop chip). See the schematic for Spencer’s 512K RAM/ROM card here.
Looking on a Logic Analyzer while this script runs you can see the following:
Clearly we’re not breaking any speed records here! But there is a lot of debug information going on in the background so it will be faster than this when that’s disabled. But the important thing is that the PAGE_EN line (bottom trace) toggles as we expect and the timing on the IORQ and WR lines can also be seen.
There isn’t a problem with that part of the circuit so let’s keep looking.
Exercising More of the Circuit
Let’s now jump into a more extensive test where we set up a few pages in the RAM, write some data into each page, then go back and check the right stuff is where we put it:
import requests
url = "http://192.168.86.192/targetcmd/"
ramRomPageWrBase = 0x70
ramRomPgenWrBase = 0x74
def takeControl():
req = requests.request("get", url + "rawBusControlOn")
req = requests.request("get", url + "rawBusWaitDisable")
req = requests.request("get", url + "rawBusWaitClear")
req = requests.request("get", url + "rawBusTake")
def enablePaging():
req = requests.request("get", url + "rawBusSetAddress/" + f"{ramRomPageWrBase:04x}")
req = requests.request("get", url + "rawBusSetData/01")
req = requests.request("get", url + "rawBusSetLine/IORQ/0")
req = requests.request("get", url + "rawBusSetLine/WR/0")
req = requests.request("get", url + "rawBusSetLine/WR/1")
req = requests.request("get", url + "rawBusSetLine/IORQ/1")
def writeRegister(regIdx, val):
req = requests.request("get", url + "rawBusSetAddress/" + f"{(ramRomPgenWrBase + regIdx):04x}")
req = requests.request("get", url + "rawBusSetData/" + f"{val:02x}")
req = requests.request("get", url + "rawBusSetLine/IORQ/0")
req = requests.request("get", url + "rawBusSetLine/WR/0")
req = requests.request("get", url + "rawBusSetLine/WR/1")
req = requests.request("get", url + "rawBusSetLine/IORQ/1")
def writeData(addr, data):
print(f"Writing addr {addr:02x} data {data:02x}")
req = requests.request("get", url + "rawBusSetAddress/" + f"{addr:04x}")
req = requests.request("get", url + "rawBusSetData/" + f"{data:02x}")
req = requests.request("get", url + "rawBusSetLine/MREQ/0")
req = requests.request("get", url + "rawBusSetLine/WR/0")
req = requests.request("get", url + "rawBusSetLine/WR/1")
req = requests.request("get", url + "rawBusSetLine/MREQ/1")
def readData(addr):
req = requests.request("get", url + "rawBusSetAddress/" + f"{addr:04x}")
req = requests.request("get", url + "rawBusGetData")
req = requests.request("get", url + "rawBusSetLine/MREQ/0")
req = requests.request("get", url + "rawBusSetLine/RD/0")
dataVal = req.json()["pib"]
req = requests.request("get", url + "rawBusSetLine/RD/1")
req = requests.request("get", url + "rawBusSetLine/MREQ/1")
return dataVal
takeControl()
enablePaging()
for i in range(4):
writeRegister(i, i + 1 + 0x20)
dataToWrite = (0x44 + i * 57) % 0xff
writeData(0x0000 + i * 0x4000, dataToWrite)
for i in range(4):
print("Read", readData(0x0000 + i * 0x4000))
The script is a little more structured but basically sets up values 0x21, 0x22, x023, x024 in the four 8 bit registers provided by the 74HC670 chips. It also writes data into memory in each of these banks and then reads back what should be the same data.
This generates the following output:
Writing addr 00 data 44 Writing addr 4000 data 7d Writing addr 8000 data b6 Writing addr c000 data ef Read 01 Read 01 Read 01 Read 01
So not what I was expecting!
I didn’t work out what to look at with the logic analyzer immediately and always rue the paucity of lines which can be monitored simultaneously (even with a 16 channel analyzer) but eventually I looked at the A14 and MA14 lines and noticed that they are the same. They shouldn’t be! So I traced this back to an error in my schematic where I had incorrectly applied the A14 (and A15) lines directly to the RAM and ROM chips instead of their MA14 (and MA15) counterparts.
A bit of hacking with an Olfa cutter and some wiring fixed that on the prototype board, so then I tried again…
Writing addr 00 data 44 Writing addr 4000 data 7d Writing addr 8000 data b6 Writing addr c000 data ef Read 44 Read 7d Read b6 Read ef
Hurrah!
At this point I assumed the job was done so I plugged the RomWBW ROM and RAM back in and turned-on, fully expecting to see everything working fine, but no joy! Blank terminal output.
Bringing out the Big Guns
At this point I was a little stumped. I’d checked out the hardware to a reasonable degree and found that it operated as I expected. So failure to work with actual firmware must be down to something I hadn’t considered.
I thought about this for a bit and then decided to do a little more work on a feature of the BusRaider that I’ve mainly only used for testing of the BusRaider itself – so far. That is the ability to monitor every bus access the Z80 makes (and optionally compare it to an emulated Z80 running on the Pi to check it is doing everything correctly). Alan Cox (who wrote the aforementioned Z80 emulator and is active on the RC2014 forums) suggested some time ago that this functionality would be more useful to him than single-step debugging and I think I can now see why.
So I set about improving the functionality to the point that it would be useful for comparing the operation of the original board with the new one. The main challenge here wan’t getting the bus monitoring functionality working – that part was already done and quite extensively tested – but in getting a large amount of data out of the BusRaider quickly enough to be useful. The issue here is that the data grabbed from the Z80 bus is in the Pi Zero, and the Pi Zero is connected to the ESP32 via a 920Kbps serial link, then the ESP32 is connected over WiFi to a fourth processor on a desktop or similar.
I made a few attempts at this (which failed as the serial buffers became swamped and it was tough to get end-to-end flow control working over so many interfaces) before alighting on a mechanism which buffers up captured bus activity and then holds the Z80 in WAIT until some of the data has been removed from the buffers. With sufficiently big buffers (2000 Z80 instructions, 32K of serial transmit buffer on the Pi, 32K receive buffer on the ESP32) I have managed to get around 1800 bus accesses per second traced in this way. Certainly no record breaker and equivalent, perhaps, to clocking the Z80 at about 4KHz (so 1000 times slower than a ZX Spectrum!) – but there’s lots of room for improvement 🙂
To create a dump using the tracing API is a little more involved than the previous examples as the REST API isn’t really suitable for grabbing data about execution (although I guess it might be possible). Anyhow, the start of the code to do this is as follows (the full file is on GitHub in the examples/hardwareDebug folder):
# Script to collect Z80 instruction execution data using a BusRaider
import time, datetime, json, logging, os
from SimpleHDLC import HDLC
from SimpleTCP import SimpleTCP
class Tracer:
# Start trace
def start(self):
# Setup
self.setup("192.168.86.192", "./examples/logs", "TraceLong.txt")
# Stop any previous tracer
self.sendFrame("tracerStop", b"{\"cmdName\":\"tracerStop\"}\0")
# Set serial terminal machine (with emulated 6850) - to avoid conflicts with display updates, etc
mc = "Serial Terminal ANSI"
self.sendFrame("SetMachine", b"{\"cmdName\":\"setMcJson\"}\0"+b"{\"name\":\"Serial Terminal ANSI\",\"hw\":[],\"emulate6850\":{}}\0")
time.sleep(2)
# Bus reset
self.sendFrame("busInit", b"{\"cmdName\":\"busInit\"}\0")
time.sleep(1)
# Start tracer with recording (dense binary records) turned on (logging should be off as this creates verbose text log)
self.sendFrame("tracerStart", b"{\"cmdName\":\"tracerStart\",\"logging\":0,\"record\":1}\0")
# Start message
timeStart = datetime.datetime.now()
self.logger.info(f"Execution trace started at {str(timeStart)}")
# Run the trace
self.awaitingTraceResponse = False
for i in range(10):
j = 0
while self.awaitingTraceResponse and j < 2000:
time.sleep(0.001)
if j >= 2000:
break
self.sendFrame("tracerGetLong", b"{\"cmdName\":\"tracerGetBin\"}\0")
self.awaitingTraceResponse = True
time.sleep(0.01)
# Calculate rate
timeEnd = datetime.datetime.now()
elapsed = (timeEnd - timeStart).total_seconds()
if elapsed > 0 and not self.instrCountAtStart is None:
instrCount = self.curInstructionCount - self.instrCountAtStart
self.logger.info(f"{instrCount} instructions in {elapsed:.2f} secs => {instrCount/elapsed:.1f} per second {self.instructionsSkipped} lost")
# Stop tracing
self.cleardown()
def cleardown(self):
# Clear down
try:
self.rdpTCP.stopReader()
except Exception as excp:
self.logger.error(f"{excp}")
if self.dumpTraceFile:
self.dumpTraceFile.close()
# Handle received messages
def frameCallback(self, msgContent, binContent, logger):
if (msgContent['cmdName'] == "tracerGetBinData") and (not self.dumpTraceFile is None):
# Traced bus access contents
dataCount = msgContent['dataLen'] // self.sizeOfTraceBinElem
traceCount = msgContent['traceCount']
# Check if any bus accesses lost in communication
if self.curInstructionCount != traceCount:
self.instructionsSkipped += traceCount - self.curInstructionCount
# For calculating rate
if self.instrCountAtStart is None:
self.instrCountAtStart = traceCount
self.curInstructionCount = traceCount + dataCount
elemPos = 0
# Extract the data, format and write to file
for i in range(dataCount):
dataStr = self.formatTraceBinElem(binContent, elemPos, traceCount)
elemPos += self.sizeOfTraceBinElem
traceCount += 1
self.dumpTraceFile.write(dataStr + "\n")
self.awaitingTraceResponse = False
# Format for trace dumping
def formatTraceBinElem(self, binContent, contentPos, traceCount):
addr = binContent[contentPos] + (binContent[contentPos+1]*256)
busData = binContent[contentPos+2]
retData = binContent[contentPos+3]
flags = binContent[contentPos+4]
retData = f"{traceCount:08d} {addr:04x} {busData:02x} {self.formatFlags1(flags)}{self.formatFlags2(flags)}"
if (flags & 0x80) != 0:
retData += f"{retData:02x}"
return retData
def formatFlags1(self, busFlags):
return self.flags1[(busFlags >> 3) & 0x07]
def formatFlags2(self, busFlags):
return self.flags2[busFlags & 0x07]
# Send frame to BusRaider
def sendFrame(self, comment, content):
frame = bytearray(content)
self.hdlcHandler.sendFrame(frame)
# HDLC Frame handler
def onHDLCBinFrame(self, fr):
msgContent = {'cmdName':''}
try:
# Split string - the format is a JSON string terminated with NULL then
# a binary section to the end of the buffer
nullPos = fr.find(b"\0")
binContent = b""
if nullPos >= 0:
jsonStr = fr[:nullPos].decode('utf-8').rstrip('\0')
msgContent = json.loads(jsonStr)
binContent = fr[nullPos+1:]
else:
jsonStr = fr.decode('utf-8').rstrip('\0')
msgContent = json.loads(jsonStr)
except Exception as excp:
self.logger.error(f"Failed to parse Json from {fr}, {excp}")
try:
# All messages should contain cmdName to identify message type
if 'cmdName' in msgContent:
if msgContent['cmdName'] == "log":
# Check for a log message from Pi
try:
self.logger.info(f"{msgContent['lev']} : {msgContent['src']} {msgContent['msg']}")
except Exception as excp:
self.logger.error(f"LOG CONTENT NOT FOUND IN FRAME {fr}, {excp}")
else:
self.frameCallback(msgContent, binContent, self.logger)
except Exception as excp:
self.logger.error(f"Failed to extract cmdName {fr}, {excp}")
# Check for using IP address
def setup(self, ipAddrOrHostName, fileBase, dumpTraceFileName):
# Format of trace binary element
self.sizeOfTraceBinElem = 5
self.flags1 = ("...","..I",".1.",".1I","W..","W.I","W1.","W1I")
self.flags2 = ("...","..R",".W.",".WR","M..","M.R","MW.","WWR")
# Instruction timing
self.instrCountAtStart = None
self.curInstructionCount = 0
self.instructionsSkipped = 0
# Logging
self.logger = logging.getLogger(__name__)
self.logger.setLevel(logging.DEBUG)
consoleLogger = logging.StreamHandler()
consoleLogger.setLevel(logging.DEBUG)
consoleLogger.setFormatter(logging.Formatter('%(asctime)s: %(message)s'))
self.logger.addHandler(consoleLogger)
# Open trace file
self.dumpTraceFile = None
try:
if dumpTraceFileName is not None and len(dumpTraceFileName) > 0:
self.dumpTraceFile = open(os.path.join(fileBase, dumpTraceFileName), "w")
except Exception(excp):
self.logger.warning("Can't open dump trace file " + os.path.join(fileBase, dumpTraceFile))
# HDLC port
self.tcpHdlcPort = 10001
# Callback to send to TCP
def sendDataToTCP(dataToSend):
self.rdpTCP.sendFrame(dataToSend)
# Frame handler
def onTCPFrame(fr):
# Send to HDLC
self.hdlcHandler.processBytes(fr)
# TCP Reader
self.rdpTCP = SimpleTCP(ipAddrOrHostName, self.tcpHdlcPort)
self.rdpTCP.startReader(onTCPFrame)
# Setup HDLC
self.hdlcHandler = HDLC(None, sendDataToTCP, None)
self.hdlcHandler.setCallbacks(None, self.onHDLCBinFrame)
# Welcome
self.logger.info(f"UnitTest BusRaider IP {ipAddrOrHostName} port {self.tcpHdlcPort}")
# Create and start the tracer
tracer = Tracer()
tracer.start()
And the output from running this on my 512K RAM/ROM with Paging board is as follows:
00000000 0000 c3 1.M.R 00000001 0001 00 ..M.R 00000002 0002 05 ..M.R 00000003 0500 f3 1.M.R 00000004 0501 ed 1.M.R 00000005 0502 56 1.M.R 00000006 0503 31 1.M.R 00000007 0504 00 ..M.R 00000008 0505 fe ..M.R 00000009 0506 af 1.M.R 00000010 0507 d3 1.M.R 00000011 0508 78 ..M.R 00000012 0078 00 .I.W. 00000013 0509 3c 1.M.R 00000014 050a d3 1.M.R 00000015 050b 79 ..M.R 00000016 0179 01 .I.W. 00000017 050c 00 1.M.R 00000018 050d 00 1.M.R 00000019 050e 00 1.M.R 00000020 050f 00 1.M.R 00000021 0510 00 1.M.R 00000022 0511 00 1.M.R 00000023 0512 00 1.M.R 00000024 0513 00 1.M.R 00000025 0514 00 1.M.R 00000026 0515 00 1.M.R 00000027 0516 00 1.M.R 00000028 0517 00 1.M.R 00000029 0518 00 1.M.R 00000030 0519 00 1.M.R
I collected a few hundred thousand lines in a log file only to find that the problem is visible after record 16 in the above. The instruction at record 14 is d3 79 which is OUT (79), A and after that every instruction fetch returns 00 NOP – so that doesn’t look too good! By comparison this is the trace from the working board:
00000000 0000 c3 1.M.R 00000001 0001 00 ..M.R 00000002 0002 05 ..M.R 00000003 0500 f3 1.M.R 00000004 0501 ed 1.M.R 00000005 0502 56 1.M.R 00000006 0503 31 1.M.R 00000007 0504 00 ..M.R 00000008 0505 fe ..M.R 00000009 0506 af 1.M.R 00000010 0507 d3 1.M.R 00000011 0508 78 ..M.R 00000012 0078 00 .I.W. 00000013 0509 3c 1.M.R 00000014 050a d3 1.M.R 00000015 050b 79 ..M.R 00000016 0179 01 .I.W. 00000017 050c 3e 1.M.R 00000018 050d 3e ..M.R 00000019 050e d3 1.M.R 00000020 050f 7a ..M.R 00000021 3e7a 3e .I.W. 00000022 0510 3c 1.M.R 00000023 0511 d3 1.M.R 00000024 0512 7b ..M.R 00000025 3f7b 3f .I.W. 00000026 0513 3e 1.M.R 00000027 0514 01 ..M.R 00000028 0515 d3 1.M.R 00000029 0516 7c ..M.R 00000030 017c 01 .I.W.
No such problem here!
The Final Step
Having now a clear idea that the (or at least another) problem lies with the paging mechanism I took a closer look at the circuitry around the paging selection and realised that I’d juxtaposed the PAGE_WR and PGEN_WR lines. So in-fact I would have found this problem earlier if I’d done a comparison of the output from the first test program on the original board. It wouldn’t have worked! And I’d have realized (perhaps) that the register values were actually the wrong way around in that program.
ramRomPageWrBase = '74' ramRomPgenWrBase = '70'
If you are interested in extending any of this the bulk of the code used in the first part of the post is in the BusController.cpp file in the PiSw/BusController folder. The code for the tracer is in the file StepTracer.cpp in the PiSw/StepTracer folder. And, finally the example code used here is in the PiSw/examples/hardwareDebug folder.
All of this is on GitHub.