LisaList2

General Category => LisaList2 => Topic started by: stepleton on January 17, 2021, 07:15:23 pm

Title: Cameo/Aphid selector
Post by: stepleton on January 17, 2021, 07:15:23 pm
Hi all,

I'm very pleased to announce the near-completion of a project I've been working on for a very long time --- nearly the entire COVID crisis so far! It's a program for my Cameo/Aphid hard drive emulator that allows you to select and manage drive images from an Apple Lisa, and while that may not be very useful for most people (there aren't that many Cameo/Aphid devices out there in the world), some of the parts that went into it may be handy for something you're working on.

The program, which I'll call the Selector, is up on GitHub now. It's what I was "previewing" in that earlier thread (https://lisalist2.com/index.php/topic,142.0.html). I said "near-completion" because there are still some more documents to write and disk images to make, but if you made yourself a Cameo/Aphid at some point --- first of all, why not let me know? --- you can check out the new software at https://github.com/stepleton/cameo/tree/master/aphid/selector (https://github.com/stepleton/cameo/tree/master/aphid/selector). There's also a manual (https://github.com/stepleton/cameo/blob/master/aphid/selector/MANUAL.md) that tells you everything the Selector can do.

I hope you won't mind if I geek out a bit about what I've made; maybe you'll find it interesting.


Up until now, a Cameo/Aphid has basically behaved a lot like an ordinary ProFile: you turn it on, and (once it's ready) it pretends to be a hard drive. This has always seemed like a bit of a waste, since the PocketBeagle computer (https://beagleboard.org/pocket) that powers it is a Linux computer with a 1GHz ARM, a ton of RAM, and as much storage as your microSD card allows. The least it could do is give you a menu to let you switch between drive images now and then, so you could boot the Office System for a while and then play with MacWorks after that without much hassle.

I had a pretty good idea of how to build this kind of feature. The code on the Cameo/Aphid that shuffles block data between the drive image file and the PocketBeagle's I/O processors is just a big-ish Python script (https://github.com/stepleton/cameo/blob/master/aphid/profile.py), so it would be easy to modify it to make some blocks "magical" --- reads and writes to those blocks could do things like create and manage hard drive image files, or really just about anything else you might imagine. So, I gave the script a plugin system (https://github.com/stepleton/cameo/blob/master/aphid/profile_plugins.py), and most of those "magic blocks" behaviours I described in yesterday's thread (https://lisalist2.com/index.php/topic,142.0.html) are these little Python module plugins, like this one (https://github.com/stepleton/cameo/blob/master/aphid/profile_plugin_FFFEFF_key_value_store.py) for the key/value store that my Selector program uses to store configuration information. I implemented this first, since it was easy for me, and while I didn't have any code on a Lisa that could use this new functionality, it was easy to test with the help of Dr. Schaefer's UsbWidEx (http://john.ccac.rwth-aachen.de:8000/patrick/UsbWidEx.htm) gadget.

The next step was harder. I needed to write a bare-metal program that talked to the hard drive, showed text on the screen, and took input from the keyboard, and since I wanted it to work on as many Lisas as possible (and since I also wanted to be able to display lowercase letters), I couldn't use my cheap NeoWidEx trick of jumping into private routines in a specific version of the ROM. I'd have to write everything from scratch.

This is what led me to make the lisa_io library (https://github.com/stepleton/lisa_io) in March and April. Ray egged me on to make an OS, and while I might do that if COVID lasts for too much longer, this is what I was really up to that whole time. I had a good head of steam for a while making some neat UI components, and besides lisa_io, these may be some of the most useful spin-off products of this project. There's
I've forgotten to mention that this is all in 68K assembly language (targeting Ray's EASy68k-derived standalone assembler (https://github.com/rayarachelian/EASy68K-asm)), and like most of my Lisa stuff, it's released into the public domain. I don't trash a lot of registers in this code, so it may not be hard to call it from C or what have you.

Anyway, this was a good start, but I got distracted for a few months by work and by a weird old Whitechapel MG-1 workstation (http://mg-1.uk:31132/) I've had on a (thanks to COVID) very extended loan. As autumn arrived I picked up the project again, and in the usual manner of the last bits stretching out longer than you expect, it eventually reached a state where it could be released. The usual silly feature creep started to take hold, and so now we have
Eventually I stopped adding more bugs by adding more features while I was fixing the bugs, and here we are. I had always hoped to finish it by the time COVID was over, and although I've succeeded, that's only because my plan contained two underestimates instead of just the usual one.


Well, that's it --- thank you if you've read this far! I hope some of this project may be useful; since not many Cameo/Aphid devices exist out there, I'll probably make a video someday for a more vivid show-and-tell than a bunch of text documents. I hope you all have been well for the past months --- here's hoping for a better year this year!
Title: Re: Cameo/Aphid selector
Post by: jamesdenton on January 17, 2021, 08:58:17 pm
Tom - thanks for the write up on this awesome project! I saw the teaser on Twitter the other day and am finally glad to see more about it. Looking forward to kicking the tires on it. Nice job!
Title: Re: Cameo/Aphid selector
Post by: blusnowkitty on January 17, 2021, 09:44:30 pm
I don't think I've asked or seen - can the images created by Aphid be loaded into LisaEm or IDLE, or vice versa? This would be a huge game-changer if they could, especially with the work Ray's been doing to get Workshop able to compile in LisaEm and what Al Kossow has been working on.
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on January 18, 2021, 11:06:03 am
I don't think I've asked or seen - can the images created by Aphid be loaded into LisaEm or IDLE, or vice versa? This would be a huge game-changer if they could, especially with the work Ray's been doing to get Workshop able to compile in LisaEm and what Al Kossow has been working on.

I'm guessing they can be transferred with BLU just like any ProFile, but yeah, if there's documentation about the native format of how the individual ProFile disk images are stored on the SDXC, and maybe access to a sample dd of the SDXC itself, one that has several images on it, I can take a look and write a translator tool, after which you can "sneakernet" the SDXC card to your mac/pc and run a tool to move the data to/from LisaEm, and by extension IDLE (I think it's using either rraw or raw formats for which I have a translator to/from dc42.)

If it's just a bunch of disk image files on a FAT32 file system that makes it easier, then I'd need to know the file format of each image and where the tags are, but if it's a muxing partiton, similar to what the X/ProFile has odd/even stars, it would need slightly more work.
Title: Re: Cameo/Aphid selector
Post by: stepleton on January 18, 2021, 05:05:08 pm
The disk images are not .dc42 disk images, but they will be easy to convert.

They're just ordinary files on a FAT partition on the microSD card. The way you make the files is very simple: you just take block $000000, write its 532 bytes to the file, then take block $000001 and write its 532 bytes after that, then block $000003, then $000004, and so on until you run out of blocks. Like a real ProFile, the drive emulator doesn't know about the structure of the blocks --- tags and data and so on --- it just treats them like 532-byte chunks.

There is no header or footer information --- it's just the drive data and nothing else. There is no interleaving either: all blocks are arranged in sequential order. This program by Ray may already be capable of converting them into .dc42 format:

https://github.com/rayarachelian/lisaem/blob/master/src/tools/src/raw-to-dc42.c

This is a convenient format for the emulator, which just `mmap`s the entire file and does reads and writes to something that looks to Python like a big list (https://github.com/stepleton/cameo/blob/master/aphid/profile.py#L598).

Anyhow, you can pull the microSD card out of the Cameo/Aphid, plug it into a modern computer, and just copy these disk images to and from the FAT partition like any other file.

(ETA: It is good manners to unplug the Cameo/Aphid first.)
Title: Re: Cameo/Aphid selector
Post by: stepleton on January 19, 2021, 04:57:55 pm
Now that my fun software project is basically complete, I have to return to the cold reality that Cameo/Aphid is a piece of hardware that wasn't designed by an EE --- and so there is one hardware bug! I thought it might be good to describe the bug here in case anyone has any thoughts about what might be going on. I have a feeling that the fix will involve adding a component to some empty pads on the Cameo PCB, so hopefully not a bad one as these things go.

Here's the problem: you just can't boot a Cameo/Aphid from the upper port of a parallel interface card.

It is the strangest issue, since it appears only to affect booting and nothing else. BLU will detect and work just fine with a Cameo/Aphid installed on that port. The Office System will format a Cameo/Aphid there and will read and write files on that drive all day long. Also, there is no problem with booting from any other port: the built-in port on a Lisa 1 or 2/5, the Widget parallel cable on a 2/10, the lower port on the 2-port card. Those all work without any issues.

This problem reproduces identically across two parallel interface cards that I own. It's possible that this is a signal quality issue relating to bus termination. I assume this is so because the 100 ohm series resistors that I've put on the signal lines in the current SMD layout, which you can see here:

https://github.com/stepleton/cameo/blob/master/aphid/pics/smd_layout.png (https://github.com/stepleton/cameo/blob/master/aphid/pics/smd_layout.png)

actually make the problem worse. You see, it used to be that I used only 0-ohm jumpers everywhere. In this older configuration, the upper parallel port would sometimes boot successfully, but the Widget parallel cable would not work under any circumstances. Some termination was clearly required, and with the 100-ohm resistors, the Widget cable became reliable for all accesses, while booting (and only booting!) on the second parallel port was broken forever.

Now that the Selector is done, perhaps it's time to get to the bottom of this issue. I have to admit feeling a bit daunted! One bit of good news though is that the "plugboard" that you use on the Cameo (1 (https://github.com/stepleton/cameo/blob/master/pics/plugboard_pads.jpg), 2 (https://github.com/stepleton/cameo/blob/master/pics/plugboard.jpg)) to hook up individual signal lines will likely make it really easy to introduce the components that make up a fix, whatever the fix winds up being. It's almost like I expected to have to deal with a problem like this :)

Some observations that may be important:

- When the boot fails, the error message complains of an error 81. On the boot ROM, this means "DISK NOT READY", which suggests that something may be amiss with the BSY line. But then again, when you boot from the parallel card, it's the parallel card who's complaining, not the boot ROM. So who knows if 81 means the same thing.

- Cameo is built out of fancy-pants TXS0108 bidirectional level adaptor chips to convert between 5V signalling and 3.3V signalling. These things are pretty funky---they detect which side of the connection (the 5V side or the 3.3V side) is driving the signal automatically and then drive the other side with the voltage it expects. There's no way to force a fixed direction, which would be appropriate for the BSY line, which is always driven by the emulator. Maybe there's some noise or voltage on BSY from the parallel card upper port that fools the TXS into thinking the parallel card is in charge on that pin.

(Why fancy TXS0108 adaptors instead of something much simpler? I wanted Cameo to be useful for more than just ProFiles---basically general purpose 5V I/O for the PocketBeagle computer. Maybe you could also do IEEE-488/GPIB/HPIB with it...)
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on January 20, 2021, 12:14:10 pm
- When the boot fails, the error message complains of an error 81. On the boot ROM, this means "DISK NOT READY", which suggests that something may be amiss with the BSY line. But then again, when you boot from the parallel card, it's the parallel card who's complaining, not the boot ROM. So who knows if 81 means the same thing.

Indeed:

Code: [Select]
0000| 0000 0051             DSKBSY          .EQU    81              ;DISK NOT READY
...
1F70| 48E7 3F7E             PROREAD MOVEM.L D2-D7/A1-A6,-(SP)       ; save regs
1F74|                               DISABLE                         ; ensure interrupts off
1F74| 40E7                 #          MOVE     SR,-(SP)
1F76| 007C 0700            #          ORI      #$0700,SR
1F7A| 6174                          BSR.S   PROINIT                 ; init for Profile access and check
1F7C|                                                               ;  if attached
1F7C| 6704                          BEQ.S   CHKBSY                  ; go on if OK
1F7E| 7050                          MOVEQ   #NODSK,D0               ; else set error code and
1F80| 605C                          BRA.S   PROERR                  ; skip if error - no disk attached
1F82|
1F82|                       ;  Now check if Profile ready - wait time presently set for about 100 seconds
1F82|                       ;  to allow enough time for normal Profile startup time of about 80 seconds
1F82|
1F82| 0810 0001             CHKBSY  BTST    #BSY,IRB2(A0)           ; check if Profile ready (not busy)
1F86| 6608                          BNE.S   TRYRD                   ; skip if yes
1F88| 5382                          SUBQ.L  #1,D2                   ; else loop until timeout
1F8A| 66F6                          BNE.S   CHKBSY
1F8C|
1F8C| 7051                          MOVEQ   #DSKBSY,D0              ; set disk busy error code
1F8E| 604E                          BRA.S   PROERR                  ; and go to error exit
1F90|

So this is what PROINIT does, the Boot ROM expects BSY to be in the ready state right after this, perhaps your timing is a little off:
Code: [Select]
1FF0|                       ;----------------------------------------------------------------------------
1FF0|                       ;  Subroutine to init parallel port for Profile access.
1FF0|                       ;  Inputs:
1FF0|                       ;       None
1FF0|                       ;  Outputs:
1FF0|                       ;       D0 cleared for error use
1FF0|                       ;       A0 = VIA base address for parallel port
1FF0|                       ;       CCR zero bit set if cable connected
1FF0|                       ;  Side Effects:
1FF0|                       ;       None
1FF0|                       ;----------------------------------------------------------------------------
1FF0|
1FF0|                       PROINIT
1FF0| 4280                          CLR.L   D0                      ; clear for result use
1FF2| 267C 00FC DD81                MOVE.L  #VIA1BASE,A3            ; get kybd VIA base address                     CHG036
1FF8| 0013 00A0                     ORI.B   #$A0,ORB1(A3)           ; initialize profile-reset and parity-reset     CHG036
1FFC| 002B 00A0 0004                ORI.B   #$A0,DDRB1(A3)          ;  and set lines as outputs                     CHG036
2002| 207C 00FC D901                MOVE.L  #VIA2BASE,A0            ; get paraport VIA base address
2008| 0228 007B 0060                ANDI.B  #$7B,PCR2(A0)           ; set ctrl CA2 pulse mode/positive edge
200E| 0028 006B 0060                ORI.B   #$6B,PCR2(A0)
2014| 4228 0018                     MOVE.B  #$00,DDRA2(A0)          ; set port A bits to input
2018| 0010 0018                     ORI.B   #$18,ORB2(A0)           ; then set direction=in, cmd=false,             CHG036
201C| 0210 00FB                     ANDI.B  #$FB,ORB2(A0)           ;  enable=true                                  CHG036
2020| 0228 00FC 0010                ANDI.B  #$FC,DDRB2(A0)          ; set port B bits 0,1=in,
2026| 0028 001C 0010                ORI.B   #$1C,DDRB2(A0)          ;  2,3,4=out
202C| 0810 0000                     BTST    #OCD,IRB2(A0)           ; check OCD line
2030| 4E75                          RTS                             ; and exit
2032|

So maybe play around with that. Perhaps the LOS driver or whatever else you're testing is not as sensitive to the timing.
Title: Re: Cameo/Aphid selector
Post by: stepleton on January 20, 2021, 04:05:08 pm
Thanks Ray! I think this is a hardware problem, though. The fact that it works on the lower port but not the upper port is just too suspicious, since both ports must use the exact same code from the parallel card ROM --- and while PROINIT is relevant, I haven't checked to see if that code in the Lisa boot ROM (which is only for the built-in parallel port, I think) is the same as the parallel card ROM's ProFile I/O code.

I'm really tempted to disassemble the parallel ROM and see what it does, since after all I am a Software Guy and I would love to try and tackle this problem with software. But I think I gotta do it the hardware way, it just seems too much like a hardware issue. I've got the tools (oscilloscope, breakouts); I should get better at using them.

Last night I hung the scope off of the ~PBSY and observed that as the computer tried to talk to the ProFile, ~PBSY wiggled around a bit and then just stayed low for too long --- definitely the reason for the DISK NOT READY error. Time now to put channel 2 on ~PCMD and see what it's doing. Noise there could throw my emulator's state machine out of whack, causing those funny wiggles. Or maybe the only problem is on ~PBSY after all.

The TXS0108 level translator is a delicate device, and in all likelihood this application (pushing bits down a bus of any length) is probably pushing it to its limits. It's almost too tempting for a newbie like me: just throw it between the 3.3V and 5V devices and don't worry about all of that messy electronics stuff, the magic chip will handle everything on its own. As an older and wiser hobbyist now, I know better: the real solution is TI's even more magical LSF0108 (https://www.ti.com/lit/ds/symlink/lsf0108-q1.pdf) device. Or wait, no, maybe it's the SN74GTL2003 (https://www.ti.com/lit/ds/symlink/sn74gtl2003.pdf).

Just kidding. In the worst case, if I can't get the current device to work, there is definitely a solution that is ProFile-specific and that uses a few more conventional unidirectional parts similar to what you find behind the parallel port in a real ProFile. It's a new board to design, but it'll still be pretty cheap.
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on January 20, 2021, 05:11:58 pm
I think this is a hardware problem, though. The fact that it works on the lower port but not the upper port is just too suspicious

Or it might be that specific port on that specific 2x parallel card went bad or marginal, or it's VIA is failing, etc. If it's working on 2/3 ports, maybe the problem is unlikely to be with your device, but that specific port failing?  Maybe swap the two VIA 6522s on the card if they're socketted? Maybe a cap on that board is marginal?

If you think it's because of noise and debouncing/cleaning it, it would help, maybe add a Schmit trigger on those lines? Or go all out and add a pair of octal tristate Schmitt trigger to cover all the lines?
Title: Re: Cameo/Aphid selector
Post by: stepleton on January 20, 2021, 05:49:38 pm
That's what I'd hoped at first too. Unfortunately the problem shows up on both of the parallel cards that I own, and both cards' upper ports work fine with a real ProFile or with an IDEfile.

Got the 'scope out now, having a look :)
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on January 20, 2021, 06:22:51 pm
That's what I'd hoped at first too. Unfortunately the problem shows up on both of the parallel cards that I own, and both cards' upper ports work fine with a real ProFile or with an IDEfile.

Hmm, so this would imply there's something different about the ports. Been looking at http://bitsavers.org/pdf/apple/lisa/hardware/050-4027-A_Parallel.pdf and I don't see anything too obvious, both seem to have very similar circuits, I see PB7 and the gate it goes to are above the VIA in one drawing and below in the other, but they're the same circuit. Both have resistor packs. Both have the LS280, 245, 09 on PA lines for the parity checker,  both use the phi2 clock, they do share the DTACK circuitry... so what else could it be?

Does it work on both ports if you're not booting from it? i.e. just as a data disk?

I started a disassembly of the 2x parallel ROM here: https://onlinedisassembler.com/odaweb/c2Cu8Ejq/0 I removed the 1st 5 words from the disassembly, but not sure if that's correct or not. The ROM is here: http://bitsavers.org/bits/Apple/Lisa/firmware/341-0193-A.BIN

Not sure what the offsets are supposed to be as it's a bit weird. The 1st 5 words are:  e0 02, 03 fd, 60 08, 60 02,  05 c7
e0 02 - this is the card ID, e0=has icon, has status, is bootable. 02=2x parallel port card id

Code: [Select]
03fd - number of words in the ROM
6008 - this is the offset to the status entry routine
6002 - this is the boot routine
05c7 - icon pointer in packbits format.

But 6002 is outside the size for this card, so it's got to be at 20002 most likely, not sure what the significance of the 60 is.

The first instruction is a branch to 202a2 which checks d0 for the device id and makes decisions from there, likely loading the address of the VIAs and the device to boot from.

Title: Re: Cameo/Aphid selector
Post by: stepleton on January 20, 2021, 08:10:00 pm
Thanks for looking at the ROM! Now I know from my Applenet disassembly that 6008 and 6002 aren't sizes: they're the first words of the status entry and boot routines themselves. Of course there's not much room for code in 16 bits each: you just get one small instruction. So the one instruction in both cases is: jump to the place where the real code is.

Quote
Does it work on both ports if you're not booting from it? i.e. just as a data disk?

It does work as a data disk, which was one of the things that was such a mystery. But thanks to the probing I did just now, I have an answer for why booting alone was a problem. I wonder if I would have figured it out from a disassembly.

For some reason the Lisa parallel port card likes to give the ~PCMD (that's the "HEY HARD DRIVE" signal for folks following along) on both of its ports a good shaking prior to talking to either of the ports on boot: both ports get the business no matter which port you're booting from. I don't know the reason for this, but it also seems to happen during the POST, which must mean that it's happening in the status entry routine. (IIRC, in addition to during POST, the boot ROM runs the status entry routine prior to running the boot routine.)

Now, when Cameo/Aphid is on the lower port, it's no problem: see the first attachment.

The blue trace is ~PBSY, the "We are dealing with your request" line that comes back from the hard drive. Basically, the Lisa stabs ~PCMD down very briefly, then gently pulses it down a couple of times more. This three-wiggle manouvre repeats eight times for some reason.

(The reason ~PCMD doesn't drop to 0, I think, is because I've got a series resistor on the line for termination, and there's a pullup on the Lisa end, so the probe is sitting in the middle of a voltage divider.)

Now let's move Cameo/Aphid to the top port: see the second attachment.

You can see here that our poor guy is a little more confused by all this nonsense: while last time ~PBSY goes high very crisply after a wiggle, now it sometimes takes a little longer. The final wiggle keeps ~PBSY low for nearly 11 seconds, which is way too long for the parallel card boot routine: it times out.

Look at how long it takes ~PBSY to return to high after wiggles 1, 2, and 7. It's very suspicious that the delay seems to be an integer multiple of other obvious intervals --- I'd say it's about 88us given the labeled period there.

My hunch now is that we're not seeing the entire story of the wiggles. There are probably other control lines to check out. I bet we're getting treed up at this point:
https://github.com/stepleton/cameo/blob/master/aphid/firmware/aphd_pru1_control/aphd_pru1_control.cc#L328
waiting on PR/~W, the direction signal from the parallel card. Maybe it's doing something weird.

As you've noted, the upper and lower ports seem pretty similar on the schematic, so any difference between the two may be down to the layout?

http://bitsavers.org/pdf/apple/lisa/pcb_pictures/620-0135-A_Parallel_F.jpg

Those traces for the upper port are longer...
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on January 20, 2021, 08:23:01 pm
So then my guess is that the aphid hardware is a lot faster and notices these ups/down where as a real profile and IDE file use a slower processor which would average them out to a nice flat line. So either a switch debounce circuit would solve this, or you could do something similar via software where you don't act on the CMD going back up to 5V immediately, but wait a bit to see if it gets pulled down again, etc. before flipping BSY?

Edit: This is what I was talking about regarding the Schmitt trigger circuit: https://mansfield-devine.com/speculatrix/2018/04/debouncing-fun-with-schmitt-triggers-and-capacitors/
Title: Re: Cameo/Aphid selector
Post by: stepleton on January 21, 2021, 06:18:25 pm
You may be right about that. The main problem facing any theory though is that it has to account for why there's a problem on the upper parallel port but not the lower one.

I got some traces of PR/~W, which is also wiggling in a confusing way, but there wasn't really much difference there between the two ports that way. The signals seem very similar. It's possible that whatever the problem is, it's being provoked by a different control line.

Can't hold back my "software guy" any longer. To narrow down the possibilities for signals that are triggering the problem, it will help to know what state the Cameo/Aphid state machine is in when it gets stuck for so long. This kind of inspection was a big part of my initial debugging process, which is why the state machine implementation has statements like these:

https://github.com/stepleton/cameo/blob/master/aphid/firmware/aphd_pru1_control/aphd_pru1_control.cc#L318

The heart of the PocketBeagle single-board computer is a TI AM335x SOC, which is an ARM core paired with two real-time I/O processors called PRUs --- they're decent little RISC devices. The PRUs talk to the pins and then the ARM talks to the PRUs. It's handy because if Linux on the ARM wants to steal the CPU away from the disk emulator in order to do some other task, it won't interfere with the timing that the parallel port hard drive protocol requires. Dr. Schaefer notes on his IDEfile page that the Apple can send bytes down the cable (or demand that they flow back up the cable) at a megahertz, and the drive just has to deal with it; there's no flow control like on a serial port, so it can't drop the ball.

Anyway, the state machine code linked above is running on one of the PRUs. It's recording the current state to a shared memory region that's accessible to the ARM, so you can write a nice program for the Linux side that probes that shared memory in /dev/mem and reads that useful debug information. Thank goodness I took notes on where to look:

https://github.com/stepleton/cameo/blob/master/aphid/firmware/aphd_pru1_control/aphd_pru1_shared_memory.h#L95

I used to have such a program, but I seem to have lost it forever. Too bad; I can write another one. Time to remember how to mmap again...
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on January 22, 2021, 12:10:11 am
shmem ops are pretty straightforward, it's almost like using malloc but it's mapped to a "file", you could use that with /dev/shm, and once you get a pointer back, the offset is equivalent to the file offset.
However, you mentioned something yesterday that might be a clue, you've noted that the upper and lower ports have slightly different lengths in their traces. This shouldn't matter at all because normally you'd have a ProFile at the end of a 3 foot 25-conductor cable, right?
So are you hanging the Aphid directly on the port itself? Or is there a cable between it and the port?

Perhaps hanging it off the port is the issue because the wire is too short and it's causing some kind of EM/RF interference from some other noisy thing in the Lisa, or just the right condition to cause a resonance?
Or perhaps the wires are indeed too short and the resistor pack terminating the pins is now slightly out of spec enough due to changed capacitance? I recall seeing something similar with 10B2 and early 10BT ethernet where if your cable was too short you'd have issues.

In terms of the direction of the BSY line and the voltage converter, you could always skip going through that chip and instead use a resistor to drop the voltage from ~5v to roughly 3.3v if you think it's the converter chip. And if you want to ensure direction, use a diode too, which would also drop the voltage by ~.7v. Hmm, or instead of a resistor, if you'd use 3 diodes in series it could drop the voltage from 5v to 2.9v which likely should still ok with the rest of the circuitry.
Title: Re: Cameo/Aphid selector
Post by: stepleton on January 22, 2021, 08:55:00 pm
The upper parallel port boot bug is now fixed!

I was halfway through writing my shared memory prober when a friend who is much better at mental boolean arithmetic pinged me for a video call. I told him the story of my woes and showed him some traces, and he cleverly determined that my code was too picky!

There's a part of the Cameo/Aphid state machine that was waiting for the ~PCMD signal line to be high and the PR/~W signal line to be low. It turns out that valid ProFile transactions usually lower PR/~W before raising ~PCMD, so this change here:

https://github.com/stepleton/cameo/commit/b6bed36619bc3c256e727477c11d96acbedbc3a7

abandons the transaction if it sees ~PCMD go high when PR/~W is already high. Previously the software was just waiting for PR/~W to go low eventually, and in those random wiggles observed during the parallel card's initialisation routine, that can't be relied on to happen, so it was a long wait ending in a timeout (during which the emulator was unusable). The fact that the bug didn't affect the lower port is probably down to minor differences in signal propagation --- in other words, we got lucky there.

There's a chance that some Lisa OSs are sloppy about the order in which they change ~PCMD and PR/~W, which would mean this change makes Cameo/Aphid newly incompatible. Fortunately, the workaround is pretty simple: rather than give up immediately if PR/~W is high, just delay for a little bit to give it a chance to come down. I'll try some more testing tomorrow, but I have a good feeling. If things look good, I'll prepare a new microSD card image to replace what's found on the current release.

Ultimately, it was a software bug all along. I'm very glad that no revisions to the hardware are necessary!
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on January 22, 2021, 11:27:16 pm
The upper parallel port boot bug is now fixed!

Congrats and well done! :)
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 02, 2021, 09:04:05 pm
Hi again. I'm happy to find that the latest system software for Cameo/Aphid (available now (https://github.com/stepleton/cameo/tree/master/aphid)) has shortened boot time to under 27 seconds. This is a long time for a hard drive emulator (IDEfile and X/ProFile are both ready in only a couple seconds), but the critical thing is that it's short enough that if the Lisa and the emulator turn on at the same time, Cameo/Aphid is ready soon enough that the boot ROM can boot from it instead of timing out.

What this means is that if you make a little adaptor that converts from the Widget power supply connector to an ordinary USB Micro-B plug, you can throw a Cameo/Aphid into a Lisa 2/10 drive bay and it should just work.

Now, I still have to make this adaptor for myself (it's easy if you have the connector for the Widget power plug---just hack up an ordinary USB "charging cable"), but I have simulated the setup by turning the Lisa and the emulator on together. To keep myself honest, I gave the Lisa a 3-second head start, and it still worked. I'll probably try to make the cable tomorrow, and if it makes a monkey out of me, I'll let you know. Otherwise, please assume it worked :)


"Fast-enough" startup was the last of the big features I wanted to add in order to make Cameo/Aphid useful to me. There are a few more items to add before I actually stop this round of developing Cameo/Aphid and just start using it:
Does anyone know how slow a hard drive has to be in order for Xenix to work?
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 02, 2021, 11:11:08 pm
Does anyone know how slow a hard drive has to be in order for Xenix to work?

I don't, but there's two ways you could find out:

1. look at what change the patch for Xenix does and what the previous code expects.

from: http://sigmasevensystems.com/xpf_xenix.html
Code: [Select]
c) To assert CMD after BSY interrupt is enabled and accept fast response
 
Sectors:  $22 (#34), $CF (#207), $183 (#387)
Search:   0200 0002 0C00 0002 6680
Replace:  4E71 08EB 0004 0001 6080 (change 9 bytes)

2. time an actual ProFile or Widget drive by reading from it. You could attach a scope to the ProFile while you use BLU to do a dump, or boot an OS and run tar to /dev/null or dd to /dev/null, and see how fast the data streams through between sectors.

3. estimate. Likely the rotational speed between two consecutive sectors is what they're after, and not track seek time. I think they're 3600 or so RPM, likely the Widget ERD has that info also.

a. Maybe divide the RPMs (3600) by number of sectors per track, this gives 57, so you're looking at reading 57 sectors/minute, so somewhat less than 1 sector/second.
I don't know if this applies to the ProFile, but the ST506 when MFM formatted used 63 sectors per track.
Now it's possible to not read sequentially, but instead read in parallel across the heads (I think the ST506 might have had 8 heads, but not sure), so perhaps start with a multiple of 1sector/sec upto maybe 8 or 16 and then slow down from that until it starts to work.

b. Another way to measure: off the top of my head it takes about 30mins or so to transfer a whole ProFile over BLU, I may be off here, but that could be used to divide the number of sectors by that time, however, the question is, is 57.6kbps the limit or the parallel interface? Likely the serial port is the bottle neck, so that's not going to be a great estimate, but it could be a starting point, and then ratchet up from there.

This would be 9728 sectors / ( 30mins*60s)=5.4 sectors/sec?

So that's quite the range from ~1sector/second to 5.4, so the math here is off here somewhere, but yeah, it's a start.

One thought, perhaps it might be best to not have to force the user to rename the hard drive ahead of time - you could detect their presence of UniPlus or Xenix it by reading the boot block, and then, if it's Xenix, figure out where the speed patch gets carried to onto the ProFile hard drive, and detect if it's been patched or not, and adjust the speed accordingly - likely it's in a fixed place after it's installed (possibly the boot loader, and certainly the "xenix" kernel file) and use that value to decide whether or not you should slow down the interface.

X/ProFile doesn't requires patching UniPlus for speed, only Xenix, so likely that's the more sensitive of the two, so you could go at least as fast as the X/ProFile for UniPlus and only slow down if unpatched needed for Xenix, unless your device is significantly faster than the X/ProFile, if in which case: Woo Hoo!
Title: Re: Cameo/Aphid selector
Post by: compu_85 on February 04, 2021, 09:20:30 pm
This is just anecdotal, but both Xenix and UniPlus work fine from a 200mb conner IDE drive I use with an IDEFile. I don't know how much faster this is than a Widget - but it is a noticeable improvement.
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 06, 2021, 09:06:42 pm
Well... Xenix 3.0 works with the emulator with the modifications Ray mentioned (http://sigmasevensystems.com/xpf_xenix.html) in place --- I have to admit that I didn't take a very close look at those mods before, and I see now that there's more to it than just speed. And since there's a fine workaround in the modified disk, that's probably too much to try to accommodate for the time being.

And that's because it's time to forget about Xenix for now, since I have MacWorks problems! I seldom run MacWorks, and an installation of XL 3.0 proceeds normally. MacWorks also boots without trouble. Unfortunately, it's a bit flaky once it does, with occasional crashes. And it gets even worse... MacWorks XL 4.5 (the HFS-modded version of XL that you can find on Macintosh Garden) will give a bomb error when you try to install it onto the drive, and while MacWorks Plus 1.1 will install, it'll start raising errors (and crash eventually) if you try to copy the System Folder onto the simulated hard drive.

So, it looks like it's back to Logic Analyser Land for me. I suspect it's another handshaking hiccup, similar to what the problem was with the upper parallel port.
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 06, 2021, 10:50:02 pm
And that's because it's time to forget about Xenix for now, since I have MacWorks problems! I seldom run MacWorks, and an installation of XL 3.0 proceeds normally. MacWorks also boots without trouble. Unfortunately, it's a bit flaky once it does, with occasional crashes. And it gets even worse... MacWorks XL 4.5 (the HFS-modded version of XL that you can find on Macintosh Garden) will give a bomb error when you try to install it onto the drive, and while MacWorks Plus 1.1 will install, it'll start raising errors (and crash eventually) if you try to copy the System Folder onto the simulated hard drive.

Meh, I wonder if the MacWorks issues LisaEm has are timing related too. Hmmmm.
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 07, 2021, 03:40:48 pm
Ray, do you find that the problems mostly come around when you try to work with the hard drive? In other words, does it run fine from a floppy drive?
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 07, 2021, 05:39:58 pm
Ray, do you find that the problems mostly come around when you try to work with the hard drive? In other words, does it run fine from a floppy drive?

With MacWorks, yes, absolutely. There was some bug introduced shortly after the 1.0 release, where upon returning to the Finder, it would crash, but that was fixed recently back around August with all the CPU fixes.

It now mostly works, though sometimes after installing to the hard drive, it shows a sad mac and other times it works. But not issues with the floppy.

Edit: For 1.2.8, I'm leaning towards rewriting profile.c and that widget.c code that rolled back as a FSM. I found some GPL C Finite State Machine generators on github, there's a few that take a yaml file that describes the states, inputs, and outputs for each state, and generate C code, since profile.c is a big ball of switch/case statements and annoying to debug. I'll likely reuse the code in profile.c in those states after generation, but it will be a bit cleaner.

If it turns out to be a timing thing where the guest OS wants to wait at minimum X ms, I'll likely patch the guest OS to get around the waiting and let it run at full speed.
But yeah, plans for 2.0 or possibly 1.2.8 is to use HLE to bypass all the block read/writes, but that will require more disassembly and reverse engineering.
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 07, 2021, 08:44:11 pm
It miiiiiiight be the case that my firmware is timing out too soon waiting for MacWorks to toggle the ~PSTRB line during reads and writes (e.g. here (https://github.com/stepleton/cameo/blob/master/aphid/firmware/aphd_pru0_datapump/aphd_pru0_datapump.asm#L420)). If I had to guess, I'd say that MacWorks is more aggressive than some other operating environments about servicing interrupts while it's sending bytes to and from the drive.

If this hypothesis is right, then when my code gave up on ~PSTRB, it would stop sending bytes, but MacWorks would keep reading from the parallel port data lines. The nonsense values it would read would then have been what caused such a variety of trouble, especially if the values were code that it would then go on to try to execute.

Bumping up the amount of time my code waits on PSTRB seems to improve the stability of MacWorks Plus 1.1 quite a bit.

I'm not certain it's 100% fixed yet, but I think I've made progress.
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 07, 2021, 09:57:27 pm
It miiiiiiight be the case that my firmware is timing out too soon waiting for MacWorks to toggle the ~PSTRB line during reads and writes (e.g. here (https://github.com/stepleton/cameo/blob/master/aphid/firmware/aphd_pru0_datapump/aphd_pru0_datapump.asm#L420)). If I had to guess, I'd say that MacWorks is more aggressive than some other operating environments about servicing interrupts while it's sending bytes to and from the drive.

Certainly possible.

If this hypothesis is right, then when my code gave up on ~PSTRB, it would stop sending bytes, but MacWorks would keep reading from the parallel port data lines. The nonsense values it would read would then have been what caused such a variety of trouble, especially if the values were code that it would then go on to try to execute.

Bumping up the amount of time my code waits on PSTRB seems to improve the stability of MacWorks Plus 1.1 quite a bit.

I'm not certain it's 100% fixed yet, but I think I've made progress.

That's great news, you're at least getting closer to a working solution.

So, I don't use the PSTRB line at all, AFAIK, it's automatically driven via CB1 (or CB2?) as writes are sent through ORA. Or rather, I should say my code doesn't depend on it in any way, but the effect is the same.

If writes are made to ORANH, PSTRB is not pulsed and the value just shows up on PA0-7 (assuming DDR=255 and RRW=write). However, from the point of view of the emulator, while its pulses are ignored, each write is handled like an event, instead, so there's no duplication, etc. Not sure it matters or not. You could disassemble the VIA 2 ISR and see what it does in detail and if there are any writes to ORANH (I think that's register 15 instead of 0 or something like that).

Most likely the VIA 2 ISR is just going to set a flag that the profile driver will pick up and act on instead of being used to transfer data.

But unless they've done some weird things manually - which could be, basically it should be pulsed once with each write to ORA.

Hmmm, I think perhaps one early change in LisaEm post 1.0 was to decouple the CPU clock from the rest of the system, including the VIAs. I wonder if it was working with MacWorks because it was pinned at 5MHz? It doesn't matter at all for LOS, but perhaps there's something in MW that depends on those having the same timings.

I do decouple the MHz rate for the CPU from the T1/T2 timers (as well as the clock in the COPS), perhaps that's an issue. But then how does the XLerator work? Hmm, maybe it does some modifications to that code as well.

You could (should?) use PSTRB as a clock input to drive reading bytes from the Lisa to the Aphid's buffers? But I'm not sure if the rest of handshaking uses it or not, if any of it doesn't that probably wouldn't work well. i.e. the 55 and other handshaking values, i.e. any code reading from IRANH or writing to ORANH.

I do recall the LOS ProFile code doing some writes to ORANH here and there, but not sure if MW does the same or not. I'm pretty sure it's when they're first talking to each other around the 01/02/55 handshaking stuff, but I think it's meant to leave the port with that value and reset the state machine.

It may well be that if MacWorks is busy with something else, while a transfer is in progress that it would stop in the middle of sending bytes out for some time frame while it handles some other devices, such as the floppy or the serial port, etc. and if you time out waiting for PSTRB too early, you'd fall out of sync with it. So maybe increase that time to a lot longer. Maybe something ridiculous like 1s and then do a binary search dropping it lower by 1/2 until you see errors again and then back off a bit.
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 08, 2021, 08:58:51 pm
Quote
You could (should?) use PSTRB as a clock input to drive reading bytes from the Lisa to the Aphid's buffers? But I'm not sure if the rest of handshaking uses it or not, if any of it doesn't that probably wouldn't work well. i.e. the 55 and other handshaking values, i.e. any code reading from IRANH or writing to ORANH.

Yes, ~PSTRB is the only clock there is when the Apple wants to read or write more than one byte. It's not really negotiable. But...

Quote
I do recall the LOS ProFile code doing some writes to ORANH here and there

It sure does. This was the one bug I had in the first test of the Cameo/Aphid stack. I'm actually pretty proud of this --- when I first made Cameo/Aphid, I thought very hard about the software and hardware, and I checked, and double checked, and triple checked, and... to my delight, when I tried it for the very first time with my UsbWidEx (http://john.ccac.rwth-aachen.de:8000/patrick/UsbWidEx.htm), it just worked! That's my memory anyway; in real life I wouldn't be surprised if there were a few software hiccups that I just hadn't remembered. Nothing complicated though, what joy!

Unfortunately things weren't quite so rosy when I tried it with the Lisa (neither the boot ROM nor the Office System), and my mistaken assumption can be understood when you look at Dr. Schaefer's very handy logic analyser traces (http://john.ccac.rwth-aachen.de:8000/patrick/data/profile_read_cycle.png). Note all the places where it says "answers with $55" --- you don't see PSTRB there! I think I had missed this (so much for triple-checking!) and my recollection is that UsbWidEx does toggle PSTRB in those settings, which is what allowed it to work.

I'm not sure how dependable my memory really is here, but I do remember being annoyed by not having ~PSTRB on $55 for a silly reason. The TI AM3358 ARM-based SOC that powers the PocketBeagle single-board computer (and therefore Cameo/Aphid) is neat because it has two real-time I/O coprocessors called PRUs --- nifty, simple little RISC chips with very predictable timing. I had it all set up where only PRU0 would touch the data lines (and ~PSTRB and ~PPARITY) and only PRU1 would touch the handshaking lines PR/~W, ~PCMD, and ~PBSY. A handsome plan, but PRU0's code is driven by ~PSTRB, and if you don't ~PSTRB, well... that means PRU0 can't load $55 for you and PRU1 has to snoop the data lines to catch that $55 for itself (https://github.com/stepleton/cameo/blob/master/aphid/firmware/aphd_pru1_control/aphd_pru1_control.cc#L337). Oh well.

Cool story. Anyway, now what?

Quote
It may well be that if MacWorks is busy with something else, while a transfer is in progress that it would stop in the middle of sending bytes out for some time frame while it handles some other devices, such as the floppy or the serial port, etc. and if you time out waiting for PSTRB too early, you'd fall out of sync with it. So maybe increase that time to a lot longer. Maybe something ridiculous like 1s and then do a binary search dropping it lower by 1/2 until you see errors again and then back off a bit.

Contemplating the way the original ProFile is designed, and how it basically allows the Lisa to clock data in and out of the on-controller RAM at its own leisure, it seems like the correct timeout is... forever. I think most OSs are a bit more impatient than that, and even with MacWorks I could probably just bump up the timeout and call it a day, although you never know if someone's weird "helper" system modification or extension will grab an interrupt sometime and go to the circus for a few seconds. Maybe this is the kind of mischief that LocalTalk could get up to.

Maybe I will just bump up the timeout at first. But, given that I already have PRU0 set up to clock data in and out independently of most of the handshaking, it doesn't seem impossible to make it support an infinite timeout, so long as I have a way to interrupt it if it's time to move different data or move it in a different direction. I'll have to think about it a bit more.
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 08, 2021, 10:09:28 pm
A handsome plan, but PRU0's code is driven by ~PSTRB, and if you don't ~PSTRB, well... that means PRU0 can't load $55 for you and PRU1 has to snoop the data lines to catch that $55 for itself (https://github.com/stepleton/cameo/blob/master/aphid/firmware/aphd_pru1_control/aphd_pru1_control.cc#L337). Oh well.

Cool story. Anyway, now what?

Yeah, exactly, you have to work around it, as you have. Fun times!

Maybe this is the kind of mischief that LocalTalk could get up to.

Indeed as it has the highest IRQ and could be tied up for quite some time depending on what data storm the network has going on.

I'd imagine "reasonable" timeouts would be on an order of 1-5s, but... that's pretty long.

Might be worth looking through these to see what the Widget does with the timeouts if you understand z80 assembly and can figure out the timings: http://bitsavers.org/pdf/apple/disk/widget/firmware/code/
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 09, 2021, 06:38:10 pm
Quote
Indeed as it has the highest IRQ and could be tied up for quite some time depending on what data storm the network has going on.

Oh, thanks, that's a good warning. I think I'll go straight ahead then to try and implement "infinite timeouts" (with interruptability, of course). I've actually already begun and it seems to be making some of the code simpler so far --- that's nice of course, but often beautiful code in your mind turns out not to be suitable. We'll see what it's like when I finish.

Quote
Might be worth looking through these to see what the Widget does with the timeouts

I've actually been looking at Dr. Schaefer's ProFile ROM disassemblies instead (also on Bitsavers), and you don't really need to speak Z8 very well for that --- he's left excellent comments as usual (thanks!). Thing is, though, this firmware tells you very little about timeouts for moving bytes over the parallel port, since it doesn't do that at all: all that the MCU does is move block data to and from the controller board RAM, and then the other hardware in the controller DMAs it out over the line on its own. I would expect that this hardware doesn't have any timeouts in it at all.

It's actually pretty similar to my design, I think: instead of PRU0, the ProFile has separate hardware for clocking the data in or out. I suspect the Widget is similar, since as the IDEFile webpage notes, it's a rare MCU (for the time) that can keep up with a 1 MHz ~PSTRB signal. There probably aren't many other practical ways to do it, I don't think. But as always I could be wrong.
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 10, 2021, 01:16:26 pm
I've actually been looking at Dr. Schaefer's ProFile ROM disassemblies instead (also on Bitsavers), and you don't really need to speak Z8 very well for that --- he's left excellent comments as usual (thanks!). Thing is, though, this firmware tells you very little about timeouts for moving bytes over the parallel port, since it doesn't do that at all: all that the MCU does is move block data to and from the controller board RAM, and then the other hardware in the controller DMAs it out over the line on its own. I would expect that this hardware doesn't have any timeouts in it at all.

It's actually pretty similar to my design, I think: instead of PRU0, the ProFile has separate hardware for clocking the data in or out. I suspect the Widget is similar, since as the IDEFile webpage notes, it's a rare MCU (for the time) that can keep up with a 1 MHz ~PSTRB signal. There probably aren't many other practical ways to do it, I don't think. But as always I could be wrong.

Sure, whatever works, yeah, I was pointing to the disassembly as a way to look at how it handles the state transitions and whether there's a timeout. I wonder if it's just going to use the CRSET line or if it just flips CMD when it loses sync with Profile/Widget. Unfortunately the Boot ROM profile read routine is just a very tight loop reading from IRA2 and nothing else. The only useful clue is that it sets CA2 into pulse mode on the positive edge, but zero timing whatsoever, it just assumes the ProFile will be ready to pulse those bytes once it has them and that nothing will stop it.

The only time outs are during handshaking. UniPlus's profile driver is here: http://bitsavers.org/bits/Unisoft/V.1.5+/sys/pro.c it also doesn't seem to do much in the way of timeouts, though there is a mention of a 1ms timeout at line 595, - this seems to happen only for the BSY response to CMD. So perhaps that's something you might be able to use as a guide as well.
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 12, 2021, 09:40:38 pm
Found something while looking for the ProFile behavior in the other thread:

Quote
From: https://apple3.org/Documents/Manuals/Profile_Level_2_Manual.pdf page 157:

"CRES - generated by the host - this is used to reset the Z8 back to its initial state. This usually begins at the start of communications, but whenever it happens, it depends on the host."

So there's also an output on the VIA outside the RESET signal that can trigger CRES, but CRES isn't used to reset the ProFile, it's to put the communications state machine back to the first listening state.

This implies there's no need for timeouts because the Lisa can pulse CRES any time to push the state machine back to the idle state.
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 13, 2021, 07:52:31 am
Quote
This implies there's no need for timeouts because the Lisa can pulse CRES any time to push the state machine back to the idle state.

This is true. Unfortunately, I forgot to make any electrical connection at all between the reset line and the PocketBeagle single-board computer, so I can't use this facility. The only workaround for me is absolutely excellent code that will never get stuck anywhere.

I've made a lot of progress on my "infinite timeout" revision, which is a pretty big change, but which I hope can withstand all kinds of interrupts and delays on the Lisa side (especially from MacWorks). I tried it out last night for the first time, and while it works OK for a first run after a major change, it is obviously still not "absolutely excellent" and can get hung up in at least one spot. The work continues.
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 13, 2021, 09:29:49 am
You're probably never going to get every weird uncommon edge case, so don't worry too much about it. If you get compatibility with the most common cases, i.e. boot blocks loading boot loaders, and most of LOS, MacWorks, UniPlus/Xenix OS's working it should be good enough. The only way to get 100% compatibility is to emulate the z8 ROM on either a real ProFile or Widget, but that will also cause problems: limits on disk size, limits on how fast you can go.

It's better to have a device that runs much faster than the original and doesn't have limits, IMO. Worst case, if it even comes to that, you can provide patches for the drivers of those OSs like James MacPhail did for the X/ProFile.

But yeah, the big issue is that the 8530 doesn't have much buffer space for incoming chars so it could drop data if the IRQ isn't serviced in time. Which isn't that big of a deal really. If you're running on LocalTalk, it will eventually retransmit the packet. If you're in the middle of an X/Y/Zmodem transfer, it will retransmit. If you're just connected to a BBS and miss some incoming text, oh well, pick a lower bps rate, or use another terminal.

Hope I'm not being insensitive here, but between dropping bytes on a serial port or causing file system damage or OS crashes, I'd rather avoid the file system damage and OS crashes.
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 14, 2021, 06:12:21 pm
I think I've got the fix done now, and the problem was a bit obvious. You can't get rid of all the timeouts --- if you are willing to wait forever with the ~PBSY (meaning "I'm working, don't bother me") line asserted, then if you get stuck there, Lisa software will usually leave you alone, and you'll stay stuck. If you're holding ~PBSY, you need a timeout to let it go, in other words.

The difference this time is that the "control" coprocessor PRU1 used to be able to wait on the "data pump" coprocessor PRU0 to timeout on transactions that expected to transfer a byte when ~PBSY was low. Now, for the benefit of MacWorks, PRU0 is no longer in the timeout business at all. PRU1 has to take care of those timeouts itself.

Works now with MacWorks Plus, LOS 3.0, and Xenix. I'll update GitHub and compile a new drive image soon.

I was thrown off a bit when testing with MacWorks Plus 1.1h --- I was still seeing boot failures with the Cameo/Aphid installed on the lower port of a parallel port card. Ultimately this may be a problem with MacWorks Plus, since it fails in the same way for an IDEfile. I haven't tried a real Profile yet to compare, and probably won't :)
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 15, 2021, 08:24:03 pm
A small note for anyone interested that the new code continues to work for me. I don't have my own LocalTalk network set up (though I do have the adaptor for the Lisa end --- and none for any other computer). I haven't tried really any applications, just installation of MW+ 1.1h, which is at least a bit of a workout. LOS 1.0 has now been tried as well.

This is still the biggest low-level firmware update since the initial release in 2018, and it should probably be considered even slightly more beta than usual :)

GitHub is up to date, and an updated version of the Debian 10.0-based microSD card image is available.
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 16, 2021, 12:01:47 pm
Yeay!
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 16, 2021, 12:31:22 pm
As an aside, incase you want to make your own Phone Net connectors (RJ11) there's a schematic in glorious ASCII here: http://www.nic.funet.fi/index/mac/doc/phone-net-connectors.txt (http://www.nic.funet.fi/index/mac/doc/phone-net-connectors.txt)

This is a snippet of that:
Code: [Select]

 
A "O" means a connection
 
PIN NUMBERS ARE FOR MAC DB-9.
NUMBERS IN () ARE FOR MINI DIN 8 (MAC PLUS)
 
          CapNet Connector  (last update 8/21/90)
 
 J1                                                  J2
       9 (5)                               C1 .1 uF
RCV(-) ----------------------------O----------][------O
                                   !
       8 (                       !       C2 .1 uF
RCV(+) --------------------------------O------][------O
                                   !   !            ^
                                   !   !           TO
       5 (3)                10 ohm !   !         "PHONE"
TX(-)  -----------O-----------R3----   !          LINES
                  !                    !
       4 (6)      !         10 ohm     !
TX(+)  -----------------O-----R4--------
                  !     !
             1K   R     R  1K
                  1     2
                  !     !
       3 (4)      !     !
GND    -----------O------
 
 
PARTS LIST:
 
QTY    DES      ITEM
2    R1,2      1K 1/8W RESISTORS
2    R3,4      10 OHM 1/8W RESISTORS
2    C1,2      .1 uF THREE LAYER CERAMIC CAPS
1    J1        DB-9 MALE OR MINI DIN 8 CONNECTOR
1    J2        2 PIN BERG OR RJ-11 PHONE CONNECTOR
5    WIRE      (NOT SHOWN) 22 GAUGE STRANDED WIRE 6"
 
 
NOTES:
 
C1,2 can be disk type but ceramic is more stable.
The entire unit can be mounted on a 1" X 1" .100 ctr
bread board or smaller. No printed circuit is needed.
Connect all componets lead to lead and connect cable
wires to leads on bottom of board.
Use heat shrink tubing over entire board as a cover
or dip in RTV sealer or put in a old 35MM film case.
 

In the case of the Lisa, since the Mac is RS422, you'd tie TX- and RX- together to data ground on the 25 pin side.

There's a bigger discussion here: https://mac68k.info/forums/message.jspa?messageID=1141
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 28, 2021, 10:35:31 am
This weekend's Cameo/Aphid improvements  :D
I hope these updates aren't annoying --- this project has been a lot of fun to work on and I'm excited to share progress. There are just a couple more features to add at this point before I think it's at 1.0. I haven't made a microSD card image with these changes yet, but I will before too long.
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 28, 2021, 01:24:33 pm
The "Selector" program that lets you select and manage hard drive images is itself loaded from a hard drive image, and I found it was too easy to overwrite it with (for example) a fresh install of the Office System.

You could also add a read-only flag to an image to prevent writes. So the on the first write attempt it could switch to a default sized blank image with a specific flag?

And also another flag could be the concept of a stationary image, where it starts off as a read only image, say a golden LOS 3.1 install, and the first time it's written to, it gets cloned to a new image, and then subsequent block writes go to that one (now without the stationary flag). That way you could roll back images, or run various experiments, etc.

Hmm, it sounds like when you boot off a floppy the default is to have the selector image up, which makes sense, but then when it starts to format, perhaps that behavior could be detected and it could switch to a new blank image of a default size instead and then that could be named generically until the user changes its name? Or perhaps its boot sector/loader blocks written to the first blocks could be hashed and the hash used to select a name for the image.

If the Beagleboard has an RTC, perhaps the new image could be named with the date as well.
Don't mind me, just thinking outloud, really neat stuff so far! Many thanks!
Title: Re: Cameo/Aphid selector
Post by: stepleton on February 28, 2021, 07:34:56 pm
That's a bunch of interesting ideas --- the copy-on-write idea sounds practical! One thing I want to be the case though is that people who don't care about the Selector, particularly Apple II and Apple III users, but also Lisa users who just want a plug-and-play ProFile replacement, don't have to take any special steps to deal with it. (It's not like there are more Cameo/Aphids in the world than can be counted on two hands, but I like to pretend.)

Talking of other Apples, I would love to know if my device would work with an Apple that isn't a Lisa. I have high hopes, but there's no substitute for testing. Maybe post-pandemic I can find someone suitably-equipped nearby who would let me try it out. (There isn't much room for more computers here, unfortunately, and they're not exactly handing out Apple IIIs these days...)

Unfortunately there's no RTC on a PocketBeagle --- on those devices, unless you connect them to the internet somehow (or maybe just to another computer), it's always 2019 (or whenever the kernel was built).
Title: Re: Cameo/Aphid selector
Post by: rayarachelian on February 28, 2021, 08:16:36 pm
That's a bunch of interesting ideas --- the copy-on-write idea sounds practical! One thing I want to be the case though is that people who don't care about the Selector, particularly Apple II and Apple III users, but also Lisa users who just want a plug-and-play ProFile replacement, don't have to take any special steps to deal with it. (It's not like there are more Cameo/Aphids in the world than can be counted on two hands, but I like to pretend.)

Um, just wait a while. :)

Talking of other Apples, I would love to know if my device would work with an Apple that isn't a Lisa. I have high hopes, but there's no substitute for testing. Maybe post-pandemic I can find someone suitably-equipped nearby who would let me try it out. (There isn't much room for more computers here, unfortunately, and they're not exactly handing out Apple IIIs these days...)

Yeah, from what I know it was available on the ]['s with a rather expensive, somewhat rare card, and also on ///'s. Perhaps it would be possible to modify an emulator for an Apple /// to get an external DB25 parallel port, not sure how you'd go about that, but, maybe there's a way.

Ofc the real issue is there's no way from the point of view of the Cameo/Aphid to tell what it is you're talking to, so it would have to be setup by the user on the SD card.

And who knows maybe owners of ///'s and ]['s (and possibly IIgs?) with the proper card would also like a selector menu to boot from so they can use multiple hard drives, never know. Perhaps someone will make replicas of those cards again.

Just had a thought, another thing you could do is flag the boot drive as having a full spare table and keep the size tiny - to whatever the size of the selector is. So if something attempts to write it, they'll see an invalid drive or a drive that has all the spare sectors filled and should refuse to install on it - instead of a standard 5MB or 10MB drive that does a replace on write or copy on write. So that might be another alternative strategy.

Though between the two personally I'd prefer the copy on write one. :)
Title: Re: Cameo/Aphid selector
Post by: sigma7 on March 08, 2021, 04:03:57 pm
Does anyone know how slow a hard drive has to be in order for Xenix to work?

I don't, but there's two ways you could find out:

1. look at what change the patch for Xenix does and what the previous code expects.

from: http://sigmasevensystems.com/xpf_xenix.html
Code: [Select]
c) To assert CMD after BSY interrupt is enabled and accept fast response
 
Sectors:  $22 (#34), $CF (#207), $183 (#387)
Search:   0200 0002 0C00 0002 6680
Replace:  4E71 08EB 0004 0001 6080 (change 9 bytes)

For posterity:

IIRC, the Xenix "A ProFile can't be so fast" limitation is in how quickly the ProFile asserts BSY once it sees CMD has been asserted.
(ie. all in the controller, not related to how fast the media is)
Title: Re: Cameo/Aphid selector
Post by: patrick on March 09, 2021, 07:57:53 am
Correct. To quote myself from the IDEfile FAQ:
http://john.ccac.rwth-aachen.de:8000/patrick/IDEfaq.htm#oses

"The ProFile driver of Lisa Xenix 3.0 has a bug that may cause handshake errors when used with a device that is significantly faster than the original ProFile: it asserts -PCMD before enabling the interrupt that should be triggered by the falling edge of -PBSY! If the drive is fast enough and has already set -PBSY in the meantime, this falling edge will never be detected, resulting in a timeout or a hanging Lisa."
Title: Re: Cameo/Aphid selector
Post by: stepleton on March 09, 2021, 04:16:25 pm
Thanks for the background on this --- it makes the problem a lot easier to understand. It looks like it's past time for me to make another pass through all the FAQs and other accumulated wisdom out there.

Adding a delay to my gizmo to account for this problem is possible (well, anything is possible, I guess) but annoying, since it requires modifying the firmware that runs on the single-board computer's I/O coprocessors. But another thing that scared me off for now was the remark on the X/ProFile Xenix modification page (http://sigmasevensystems.com/xpf_xenix.html) talking about some commands being sent as just four bytes. It's another thing I could probably deal with, but these two bugs together certainly made the already-extant patch seem like a great option to use instead :)