Print

[stepleton]

The upper parallel port boot bug is now fixed!

I was halfway through writing my shared memory prober when a friend who is much better at mental boolean arithmetic pinged me for a video call. I told him the story of my woes and showed him some traces, and he cleverly determined that my code was too picky!

There's a part of the Cameo/Aphid state machine that was waiting for the ~PCMD signal line to be high and the PR/~W signal line to be low. It turns out that valid ProFile transactions usually lower PR/~W before raising ~PCMD, so this change here:

https://github.com/stepleton/cameo/commit/b6bed36619bc3c256e727477c11d96acbedbc3a7

abandons the transaction if it sees ~PCMD go high when PR/~W is already high. Previously the software was just waiting for PR/~W to go low eventually, and in those random wiggles observed during the parallel card's initialisation routine, that can't be relied on to happen, so it was a long wait ending in a timeout (during which the emulator was unusable). The fact that the bug didn't affect the lower port is probably down to minor differences in signal propagation --- in other words, we got lucky there.

There's a chance that some Lisa OSs are sloppy about the order in which they change ~PCMD and PR/~W, which would mean this change makes Cameo/Aphid newly incompatible. Fortunately, the workaround is pretty simple: rather than give up immediately if PR/~W is high, just delay for a little bit to give it a chance to come down. I'll try some more testing tomorrow, but I have a good feeling. If things look good, I'll prepare a new microSD card image to replace what's found on the current release.

Ultimately, it was a software bug all along. I'm very glad that no revisions to the hardware are necessary!

[rayarachelian]

The upper parallel port boot bug is now fixed!

Congrats and well done!

[stepleton]

Hi again. I'm happy to find that the latest system software for Cameo/Aphid (available now) has shortened boot time to under 27 seconds. This is a long time for a hard drive emulator (IDEfile and X/ProFile are both ready in only a couple seconds), but the critical thing is that it's short enough that if the Lisa and the emulator turn on at the same time, Cameo/Aphid is ready soon enough that the boot ROM can boot from it instead of timing out.

What this means is that if you make a little adaptor that converts from the Widget power supply connector to an ordinary USB Micro-B plug, you can throw a Cameo/Aphid into a Lisa 2/10 drive bay and it should just work.

Now, I still have to make this adaptor for myself (it's easy if you have the connector for the Widget power plug---just hack up an ordinary USB "charging cable"), but I have simulated the setup by turning the Lisa and the emulator on together. To keep myself honest, I gave the Lisa a 3-second head start, and it still worked. I'll probably try to make the cable tomorrow, and if it makes a monkey out of me, I'll let you know. Otherwise, please assume it worked


"Fast-enough" startup was the last of the big features I wanted to add in order to make Cameo/Aphid useful to me. There are a few more items to add before I actually stop this round of developing Cameo/Aphid and just start using it:

• ProFile-10 emulation. This should be pretty easy to do and only requires changes on the Python side. Now, I don't think I'm likely to try emulating the Widget anytime soon: the much larger Widget command set would take a lot of time to implement, and the multi-block Widget reads and writes in particular would require expanding the state machine in my low-level firmware. I hope that the Lisa OSs don't care that I've somehow managed to fit a ProFile-10 into the drive cage of my 2/10.
  
  
• Slow mode. I'd like to make the emulator add a delay to reads and writes whenever it detects that it's working with a drive image file whose name ends in ".slow.image". This is for Xenix (and maybe also UniPlus?), which expects reads and writes to take at least a certain amount of time. Patches exist to make these OSs more tolerant of fast drives, but you know authenticity is very important for me when I'm using a hard drive emulator with 500x the memory, 200x the clock speed, and 6400x the durable storage of the Lisa. Anyway, this is an easy Python change, too.

Does anyone know how slow a hard drive has to be in order for Xenix to work?

[rayarachelian]

Does anyone know how slow a hard drive has to be in order for Xenix to work?

I don't, but there's two ways you could find out:

1. look at what change the patch for Xenix does and what the previous code expects.

from: http://sigmasevensystems.com/xpf_xenix.html

Code: [Select]

c) To assert CMD after BSY interrupt is enabled and accept fast response
 
Sectors:  $22 (#34), $CF (#207), $183 (#387)
Search:   0200 0002 0C00 0002 6680
Replace:  4E71 08EB 0004 0001 6080 (change 9 bytes)

2. time an actual ProFile or Widget drive by reading from it. You could attach a scope to the ProFile while you use BLU to do a dump, or boot an OS and run tar to /dev/null or dd to /dev/null, and see how fast the data streams through between sectors.

3. estimate. Likely the rotational speed between two consecutive sectors is what they're after, and not track seek time. I think they're 3600 or so RPM, likely the Widget ERD has that info also.

a. Maybe divide the RPMs (3600) by number of sectors per track, this gives 57, so you're looking at reading 57 sectors/minute, so somewhat less than 1 sector/second.
I don't know if this applies to the ProFile, but the ST506 when MFM formatted used 63 sectors per track.
Now it's possible to not read sequentially, but instead read in parallel across the heads (I think the ST506 might have had 8 heads, but not sure), so perhaps start with a multiple of 1sector/sec upto maybe 8 or 16 and then slow down from that until it starts to work.

b. Another way to measure: off the top of my head it takes about 30mins or so to transfer a whole ProFile over BLU, I may be off here, but that could be used to divide the number of sectors by that time, however, the question is, is 57.6kbps the limit or the parallel interface? Likely the serial port is the bottle neck, so that's not going to be a great estimate, but it could be a starting point, and then ratchet up from there.

This would be 9728 sectors / ( 30mins*60s)=5.4 sectors/sec?

So that's quite the range from ~1sector/second to 5.4, so the math here is off here somewhere, but yeah, it's a start.

One thought, perhaps it might be best to not have to force the user to rename the hard drive ahead of time - you could detect their presence of UniPlus or Xenix it by reading the boot block, and then, if it's Xenix, figure out where the speed patch gets carried to onto the ProFile hard drive, and detect if it's been patched or not, and adjust the speed accordingly - likely it's in a fixed place after it's installed (possibly the boot loader, and certainly the "xenix" kernel file) and use that value to decide whether or not you should slow down the interface.

X/ProFile doesn't requires patching UniPlus for speed, only Xenix, so likely that's the more sensitive of the two, so you could go at least as fast as the X/ProFile for UniPlus and only slow down if unpatched needed for Xenix, unless your device is significantly faster than the X/ProFile, if in which case: Woo Hoo!

[compu_85]

This is just anecdotal, but both Xenix and UniPlus work fine from a 200mb conner IDE drive I use with an IDEFile. I don't know how much faster this is than a Widget - but it is a noticeable improvement.

[stepleton]

Well... Xenix 3.0 works with the emulator with the modifications Ray mentioned in place --- I have to admit that I didn't take a very close look at those mods before, and I see now that there's more to it than just speed. And since there's a fine workaround in the modified disk, that's probably too much to try to accommodate for the time being.

And that's because it's time to forget about Xenix for now, since I have MacWorks problems! I seldom run MacWorks, and an installation of XL 3.0 proceeds normally. MacWorks also boots without trouble. Unfortunately, it's a bit flaky once it does, with occasional crashes. And it gets even worse... MacWorks XL 4.5 (the HFS-modded version of XL that you can find on Macintosh Garden) will give a bomb error when you try to install it onto the drive, and while MacWorks Plus 1.1 will install, it'll start raising errors (and crash eventually) if you try to copy the System Folder onto the simulated hard drive.

So, it looks like it's back to Logic Analyser Land for me. I suspect it's another handshaking hiccup, similar to what the problem was with the upper parallel port.

[rayarachelian]

And that's because it's time to forget about Xenix for now, since I have MacWorks problems! I seldom run MacWorks, and an installation of XL 3.0 proceeds normally. MacWorks also boots without trouble. Unfortunately, it's a bit flaky once it does, with occasional crashes. And it gets even worse... MacWorks XL 4.5 (the HFS-modded version of XL that you can find on Macintosh Garden) will give a bomb error when you try to install it onto the drive, and while MacWorks Plus 1.1 will install, it'll start raising errors (and crash eventually) if you try to copy the System Folder onto the simulated hard drive.

Meh, I wonder if the MacWorks issues LisaEm has are timing related too. Hmmmm.

[stepleton]

Ray, do you find that the problems mostly come around when you try to work with the hard drive? In other words, does it run fine from a floppy drive?

[rayarachelian]

Ray, do you find that the problems mostly come around when you try to work with the hard drive? In other words, does it run fine from a floppy drive?

With MacWorks, yes, absolutely. There was some bug introduced shortly after the 1.0 release, where upon returning to the Finder, it would crash, but that was fixed recently back around August with all the CPU fixes.

It now mostly works, though sometimes after installing to the hard drive, it shows a sad mac and other times it works. But not issues with the floppy.

Edit: For 1.2.8, I'm leaning towards rewriting profile.c and that widget.c code that rolled back as a FSM. I found some GPL C Finite State Machine generators on github, there's a few that take a yaml file that describes the states, inputs, and outputs for each state, and generate C code, since profile.c is a big ball of switch/case statements and annoying to debug. I'll likely reuse the code in profile.c in those states after generation, but it will be a bit cleaner.

If it turns out to be a timing thing where the guest OS wants to wait at minimum X ms, I'll likely patch the guest OS to get around the waiting and let it run at full speed.
But yeah, plans for 2.0 or possibly 1.2.8 is to use HLE to bypass all the block read/writes, but that will require more disassembly and reverse engineering.

[stepleton]

It miiiiiiight be the case that my firmware is timing out too soon waiting for MacWorks to toggle the ~PSTRB line during reads and writes (e.g. here). If I had to guess, I'd say that MacWorks is more aggressive than some other operating environments about servicing interrupts while it's sending bytes to and from the drive.

If this hypothesis is right, then when my code gave up on ~PSTRB, it would stop sending bytes, but MacWorks would keep reading from the parallel port data lines. The nonsense values it would read would then have been what caused such a variety of trouble, especially if the values were code that it would then go on to try to execute.

Bumping up the amount of time my code waits on PSTRB seems to improve the stability of MacWorks Plus 1.1 quite a bit.

I'm not certain it's 100% fixed yet, but I think I've made progress.

[rayarachelian]

It miiiiiiight be the case that my firmware is timing out too soon waiting for MacWorks to toggle the ~PSTRB line during reads and writes (e.g. here). If I had to guess, I'd say that MacWorks is more aggressive than some other operating environments about servicing interrupts while it's sending bytes to and from the drive.

Certainly possible.

If this hypothesis is right, then when my code gave up on ~PSTRB, it would stop sending bytes, but MacWorks would keep reading from the parallel port data lines. The nonsense values it would read would then have been what caused such a variety of trouble, especially if the values were code that it would then go on to try to execute.

Bumping up the amount of time my code waits on PSTRB seems to improve the stability of MacWorks Plus 1.1 quite a bit.

I'm not certain it's 100% fixed yet, but I think I've made progress.

That's great news, you're at least getting closer to a working solution.

So, I don't use the PSTRB line at all, AFAIK, it's automatically driven via CB1 (or CB2?) as writes are sent through ORA. Or rather, I should say my code doesn't depend on it in any way, but the effect is the same.

If writes are made to ORANH, PSTRB is not pulsed and the value just shows up on PA0-7 (assuming DDR=255 and RRW=write). However, from the point of view of the emulator, while its pulses are ignored, each write is handled like an event, instead, so there's no duplication, etc. Not sure it matters or not. You could disassemble the VIA 2 ISR and see what it does in detail and if there are any writes to ORANH (I think that's register 15 instead of 0 or something like that).

Most likely the VIA 2 ISR is just going to set a flag that the profile driver will pick up and act on instead of being used to transfer data.

But unless they've done some weird things manually - which could be, basically it should be pulsed once with each write to ORA.

Hmmm, I think perhaps one early change in LisaEm post 1.0 was to decouple the CPU clock from the rest of the system, including the VIAs. I wonder if it was working with MacWorks because it was pinned at 5MHz? It doesn't matter at all for LOS, but perhaps there's something in MW that depends on those having the same timings.

I do decouple the MHz rate for the CPU from the T1/T2 timers (as well as the clock in the COPS), perhaps that's an issue. But then how does the XLerator work? Hmm, maybe it does some modifications to that code as well.

You could (should?) use PSTRB as a clock input to drive reading bytes from the Lisa to the Aphid's buffers? But I'm not sure if the rest of handshaking uses it or not, if any of it doesn't that probably wouldn't work well. i.e. the 55 and other handshaking values, i.e. any code reading from IRANH or writing to ORANH.

I do recall the LOS ProFile code doing some writes to ORANH here and there, but not sure if MW does the same or not. I'm pretty sure it's when they're first talking to each other around the 01/02/55 handshaking stuff, but I think it's meant to leave the port with that value and reset the state machine.

It may well be that if MacWorks is busy with something else, while a transfer is in progress that it would stop in the middle of sending bytes out for some time frame while it handles some other devices, such as the floppy or the serial port, etc. and if you time out waiting for PSTRB too early, you'd fall out of sync with it. So maybe increase that time to a lot longer. Maybe something ridiculous like 1s and then do a binary search dropping it lower by 1/2 until you see errors again and then back off a bit.

[stepleton]

You could (should?) use PSTRB as a clock input to drive reading bytes from the Lisa to the Aphid's buffers? But I'm not sure if the rest of handshaking uses it or not, if any of it doesn't that probably wouldn't work well. i.e. the 55 and other handshaking values, i.e. any code reading from IRANH or writing to ORANH.

Yes, ~PSTRB is the only clock there is when the Apple wants to read or write more than one byte. It's not really negotiable. But...

I do recall the LOS ProFile code doing some writes to ORANH here and there

It sure does. This was the one bug I had in the first test of the Cameo/Aphid stack. I'm actually pretty proud of this --- when I first made Cameo/Aphid, I thought very hard about the software and hardware, and I checked, and double checked, and triple checked, and... to my delight, when I tried it for the very first time with my UsbWidEx, it just worked! That's my memory anyway; in real life I wouldn't be surprised if there were a few software hiccups that I just hadn't remembered. Nothing complicated though, what joy!

Unfortunately things weren't quite so rosy when I tried it with the Lisa (neither the boot ROM nor the Office System), and my mistaken assumption can be understood when you look at Dr. Schaefer's very handy logic analyser traces. Note all the places where it says "answers with $55" --- you don't see PSTRB there! I think I had missed this (so much for triple-checking!) and my recollection is that UsbWidEx does toggle PSTRB in those settings, which is what allowed it to work.

I'm not sure how dependable my memory really is here, but I do remember being annoyed by not having ~PSTRB on $55 for a silly reason. The TI AM3358 ARM-based SOC that powers the PocketBeagle single-board computer (and therefore Cameo/Aphid) is neat because it has two real-time I/O coprocessors called PRUs --- nifty, simple little RISC chips with very predictable timing. I had it all set up where only PRU0 would touch the data lines (and ~PSTRB and ~PPARITY) and only PRU1 would touch the handshaking lines PR/~W, ~PCMD, and ~PBSY. A handsome plan, but PRU0's code is driven by ~PSTRB, and if you don't ~PSTRB, well... that means PRU0 can't load $55 for you and PRU1 has to snoop the data lines to catch that $55 for itself. Oh well.

Cool story. Anyway, now what?

It may well be that if MacWorks is busy with something else, while a transfer is in progress that it would stop in the middle of sending bytes out for some time frame while it handles some other devices, such as the floppy or the serial port, etc. and if you time out waiting for PSTRB too early, you'd fall out of sync with it. So maybe increase that time to a lot longer. Maybe something ridiculous like 1s and then do a binary search dropping it lower by 1/2 until you see errors again and then back off a bit.

Contemplating the way the original ProFile is designed, and how it basically allows the Lisa to clock data in and out of the on-controller RAM at its own leisure, it seems like the correct timeout is... forever. I think most OSs are a bit more impatient than that, and even with MacWorks I could probably just bump up the timeout and call it a day, although you never know if someone's weird "helper" system modification or extension will grab an interrupt sometime and go to the circus for a few seconds. Maybe this is the kind of mischief that LocalTalk could get up to.

Maybe I will just bump up the timeout at first. But, given that I already have PRU0 set up to clock data in and out independently of most of the handshaking, it doesn't seem impossible to make it support an infinite timeout, so long as I have a way to interrupt it if it's time to move different data or move it in a different direction. I'll have to think about it a bit more.

[rayarachelian]

A handsome plan, but PRU0's code is driven by ~PSTRB, and if you don't ~PSTRB, well... that means PRU0 can't load $55 for you and PRU1 has to snoop the data lines to catch that $55 for itself. Oh well.

Cool story. Anyway, now what?

Yeah, exactly, you have to work around it, as you have. Fun times!

Maybe this is the kind of mischief that LocalTalk could get up to.

Indeed as it has the highest IRQ and could be tied up for quite some time depending on what data storm the network has going on.

I'd imagine "reasonable" timeouts would be on an order of 1-5s, but... that's pretty long.

Might be worth looking through these to see what the Widget does with the timeouts if you understand z80 assembly and can figure out the timings: http://bitsavers.org/pdf/apple/disk/widget/firmware/code/

[stepleton]

Indeed as it has the highest IRQ and could be tied up for quite some time depending on what data storm the network has going on.

Oh, thanks, that's a good warning. I think I'll go straight ahead then to try and implement "infinite timeouts" (with interruptability, of course). I've actually already begun and it seems to be making some of the code simpler so far --- that's nice of course, but often beautiful code in your mind turns out not to be suitable. We'll see what it's like when I finish.

Might be worth looking through these to see what the Widget does with the timeouts

I've actually been looking at Dr. Schaefer's ProFile ROM disassemblies instead (also on Bitsavers), and you don't really need to speak Z8 very well for that --- he's left excellent comments as usual (thanks!). Thing is, though, this firmware tells you very little about timeouts for moving bytes over the parallel port, since it doesn't do that at all: all that the MCU does is move block data to and from the controller board RAM, and then the other hardware in the controller DMAs it out over the line on its own. I would expect that this hardware doesn't have any timeouts in it at all.

It's actually pretty similar to my design, I think: instead of PRU0, the ProFile has separate hardware for clocking the data in or out. I suspect the Widget is similar, since as the IDEFile webpage notes, it's a rare MCU (for the time) that can keep up with a 1 MHz ~PSTRB signal. There probably aren't many other practical ways to do it, I don't think. But as always I could be wrong.

[rayarachelian]

I've actually been looking at Dr. Schaefer's ProFile ROM disassemblies instead (also on Bitsavers), and you don't really need to speak Z8 very well for that --- he's left excellent comments as usual (thanks!). Thing is, though, this firmware tells you very little about timeouts for moving bytes over the parallel port, since it doesn't do that at all: all that the MCU does is move block data to and from the controller board RAM, and then the other hardware in the controller DMAs it out over the line on its own. I would expect that this hardware doesn't have any timeouts in it at all.

It's actually pretty similar to my design, I think: instead of PRU0, the ProFile has separate hardware for clocking the data in or out. I suspect the Widget is similar, since as the IDEFile webpage notes, it's a rare MCU (for the time) that can keep up with a 1 MHz ~PSTRB signal. There probably aren't many other practical ways to do it, I don't think. But as always I could be wrong.

Sure, whatever works, yeah, I was pointing to the disassembly as a way to look at how it handles the state transitions and whether there's a timeout. I wonder if it's just going to use the CRSET line or if it just flips CMD when it loses sync with Profile/Widget. Unfortunately the Boot ROM profile read routine is just a very tight loop reading from IRA2 and nothing else. The only useful clue is that it sets CA2 into pulse mode on the positive edge, but zero timing whatsoever, it just assumes the ProFile will be ready to pulse those bytes once it has them and that nothing will stop it.

The only time outs are during handshaking. UniPlus's profile driver is here: http://bitsavers.org/bits/Unisoft/V.1.5+/sys/pro.c it also doesn't seem to do much in the way of timeouts, though there is a mention of a 1ms timeout at line 595, - this seems to happen only for the BSY response to CMD. So perhaps that's something you might be able to use as a guide as well.