LisaList2

General Category => LisaList2 => Topic started by: AlexTheCat123 on September 06, 2021, 05:10:36 pm

Title: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on September 06, 2021, 05:10:36 pm
I've been working on an interesting project for around a week or so that I figured you guys would want to know about! I built myself a USBWidEx a month or so ago and I really love the functionality that it provides, but it was pretty hard to find the Renesas processor that it requires and I ended up destroying 3 processors and 2 boards since I suck at such fine surface-mount soldering.

In order to remedy these two issues, I decided to try and build my own USBWidEx-like device using an Arduino Mega and a simple breakout board that only uses through-hole components. This was my first time trying to emulate a communications protocol like this and I was worried that it wouldn't work, but luckily it seems that my fears weren't justified.

My device communicates with the modern computer using a terminal emulator just like USBWidEx does and it uses XMODEM to transmit and receive drive image files. So far, it can read and interpret the spare table, read and write to any block that the user specifies, and backup and restore the entire drive to a file. It works great with my ProFiles and Widget, but random data errors crop up when I try to use it with a Cameo/Aphid for some reason, so I'll need to look into that some more. I've attached some pictures showing the device in action.

The SD card slot on the board is for the next phase of this project, where I hope to make the board emulate a ProFile in addition to communicating with one, but I'm skeptical about the Arduino being fast enough to emulate a drive since the host, not the drive, sets the pace of communication.

Once I add more functionality and get some more of the bugs worked out, I'll make sure to release the (sloppily written but functional) code so that other people can build their own. In case you can't tell, I'm pretty excited about this project! Let me know what you guys think and tell me if you have any tips or suggestions!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on September 06, 2021, 05:19:34 pm
Wow, awesome! Congratulations on this new gizmo!

Quote
It works great with my ProFiles and Widget, but random data errors crop up when I try to use it with a Cameo/Aphid for some reason, so I'll need to look into that some more.

Oh yes, please let me know if you find anything out there. If it's OK with the real drive then any emulator ought to be able to deal with it too! (Unless it's the Xenix ProFile driver --- that's just weird.) I wonder if it's timing out on you --- there was an update to the software on 20 June (http://stepleton.com/cameo_aphid_image.php) that removed some timeout assumptions that had been there previously.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on September 06, 2021, 06:33:43 pm
Thanks! I'm using the latest version of the Cameo/Aphid software, so unfortunately that doesn't fix the issue. I'll keep looking and I'll try to figure out what the problem is! My guess is that it has something to do with the Arduino being slower at communicating with the drive than a Lisa, Apple III, or USBWidEx would be.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: rayarachelian on September 06, 2021, 06:47:12 pm
Very nice!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on September 19, 2021, 05:48:47 pm
I just uploaded the source code and a description of the device to GitHub, so feel free to check it out if you guys are interested!

It still doesn't work with the Cameo/Aphid, so I'll have to keep looking into that issue.

By the way, I tried using the Arduino to emulate a ProFile and it kind of worked, but it just couldn't always keep up with the strobe pulses from the Lisa.

Let me know what you guys think and if you have any suggestions!

Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on September 19, 2021, 05:56:19 pm
Gotta be honest... call me thin-skinned if you like, but being close relations to a few folks who work in child welfare, I'm not a big fan of the name. I'm unlikely to have much to do with it as-is.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: blusnowkitty on September 19, 2021, 07:02:34 pm
As much as I appreciate the hardware and software engineering effort and as much as I like puns and wordplay,

Uh, really?
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on September 19, 2021, 07:26:02 pm
I'm sorry if I offended anybody with the name. That really wasn't my intent. I have a pretty dark sense of humor sometimes and I guess this is a situation where I should just keep it to myself. I've changed the name to ArduinoFile in the interim while I think of a better name. Once again, I really didn't mean to upset anyone with the name and I'm really sorry if I did. Now that I've given it a less offensive name, hopefully people will feel more comfortable using it.

https://github.com/alexthecat123/ArduinoFile
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on September 19, 2021, 08:24:22 pm
No worries, and thanks for making the change. I'm looking forward to checking it out! (It may have to wait until after I move, though.)
Thanks also for sharing the materials.

I wonder if there's a way to keep up with that strobe pin. Maybe you could do it with AVR assembly? I don't know a lot about Arduino programming, so I'm not sure how much headroom there is for improvements over the Arduino C compiler, but maybe there could be enough?
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on September 19, 2021, 09:20:09 pm
Quote
I wonder if there's a way to keep up with that strobe pin. Maybe you could do it with AVR assembly? I don't know a lot about Arduino programming, so I'm not sure how much headroom there is for improvements over the Arduino C compiler, but maybe there could be enough?

That's a good thought! I'm not very experienced with assembly, but I will definitely check it out and see if I can optimize things that way. A faster Arduino like the Due or something like an ESP32 would be more than fast enough, but they use 3.3V logic levels and one of my major goals with this project was to avoid any sort of level shifting for simplicity's sake, so I really hope that your AVR assembly suggestion works out!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on September 23, 2021, 11:35:02 am
Good news! I think I've managed to optimize things enough to keep up with the strobe, at least at the speed that the Lisa and USBWidEx pulse it. I don't know if it's fast enough for the Apple III and I don't have one to test it with, so I guess that will remain unknown. Luckily, I didn't have to resort to assembly at all; optimizing my C code was enough. I haven't had a formal education in computer science yet (I just started college a month ago), so many of the inefficiencies were probably stupid things that would have been obvious to anyone who has more experience with software development. Hopefully I'll have a prototype of the ProFile emulator going in a few days.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on September 23, 2021, 04:56:45 pm
Wow, neat! So is the plan to interface with an SD card reader? All of these things are really very inexpensive!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on September 23, 2021, 08:28:47 pm
Quote
So is the plan to interface with an SD card reader? All of these things are really very inexpensive!
Yep! I got a little microSD breakout board off eBay for $3 and it handles all the logic level conversion for you, so I can just plug it into a header on my PCB and I'm ready to go. My goal with this project is to make it as cheap as possible and I think I'm doing pretty well with that so far!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on September 23, 2021, 08:58:18 pm
Outstanding!

Now, it may be more of a programming exercise than an Arduino is ready to handle, but if you implement the protocol described at https://github.com/stepleton/cameo/blob/master/aphid/selector/PROTOCOL.md, your device will be compatible with the Cameo/Aphid selector program.

You know, you might be able to do it. I think you'd have to lean a lot on the filesystem on the SD card (assuming you choose to go with a filesystem at all): you could implement the key/value store by means of special files containing the data. But the Arduino's 2K of RAM may start to feel pretty cramped --- it looks like the libraries you'd need will consume half a kilobyte all on their own.

But that's probably getting ahead of things. A ProFile emulator made out of an Arduino is plenty awesome all on its own.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on September 24, 2021, 12:02:22 pm
Once I get things up and running reliably, I'll definitely try to get the Selector protocol working! That would be a really nice addition to the emulator.

The Arduino Mega, which is what I'm using for this project, actually has 8K of RAM instead of the 2K of the other Arduino boards, so luckily RAM almost certainly won't be an issue! I originally wanted to go with an Uno or a Nano to make the device a lot smaller, but the ArduinoFile code just wouldn't fit in that limited RAM, so I had to go with the Mega instead. The emulator code should be somewhat less complicated than the ArduinoFile code since we don't have to deal with making a user interface, error-checking user input, and so on, so the emulator might actually be able to run on a 2K board instead of requiring the Mega.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on September 30, 2021, 02:21:30 pm
Great news! I just finished the code for the ProFile emulator and I plugged the device into the Lisa for the first time (I was using a USBWidEx instead of an actual Lisa to test it up until this point) and it was actually able to boot BLU off the SD card! I was guessing that I'd have to fiddle with the timings some more to get the Lisa to communicate with my emulator properly, but it actually worked first try! Now it's just a matter of trying to install LOS on it to see if it still works with a much more complex operating system!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on September 30, 2021, 07:10:09 pm
Did it work?  :)

If not, did you run into the issue where the ProFile sometimes has to see a $55 on the bus from the Apple even though the Apple doesn't cycle ~PSTRB?

If so, that's the same thing I ran into after moving Cameo/Aphid from UsbWidEx over to the Lisa. The boot ROM cycles ~PSTRB for those $55 bytes, but the Office System is a different story...
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on October 01, 2021, 08:11:10 am
MacWorks XL 3.0 and MacWorks Plus worked great when I tested them, but unfortunately the Office System did not. I copied an image of LOS 2.0 from my Cameo/Aphid over to my device and after a few seconds of trying to boot the Lisa just restarts itself. I tried LOS 3.0 as well and it just gives a 10707 (Cannot initialize the File System volume) error when I try to boot it.

I don't think the issue has anything to do with the cycling of STRB with a $55 on the bus because my code isn't paying attention to the strobe when checking for the $55. It only looks at the strobe when receiving the command bytes, sending or receiving data for a block, and sending the status bytes.

I tried booting into the LOS installer and initializing my emulated ProFile, but it always fails with an error saying something to the effect of "The Lisa can't write to the disk." The logic analyzer seems to show that reading from the disk is working fine, but that when writing to the disk the Lisa never lowers CMD after sending the command bytes for some reason. The Lisa then proceeds to try the write operation again two more times, with the same result each try, before giving up and showing the error message.

I also realized that I somehow completely forgot about the parity signal and it just stays low all the time. USBWidEx and the other operating systems that I've tried on the Lisa don't seem to care about this and work fine, but maybe the Office System is different?
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on October 01, 2021, 10:04:51 am
Interesting! Well, I'm sure you'll track down the error soon!

I don't actually know if the Office System is worried about parity when it reads and writes (I've never checked), but the OS does have two obvious error codes (https://lisafaq.sunder.net/single.html#lisafaq-sw-los_error_codes) relating to parity, one for reading and one for writing:

662    Parity error while sending command or writing data to ProFile
663    Checksum error or CRC error or parity error in data read

It's interesting that the ProFile is responsible for computing parity on the bus no matter which device on either side of the cable is putting bytes there. A lookup table is an easy way to compute parity, but I'm guessing that you might find it irritating to spend 256 bytes of your available 8K. For computing parity on data from the Lisa, Cameo/Aphid uses a method described here:

http://graphics.stanford.edu/~seander/bithacks.html#ParityParallel

note the text talking about how that method "may be optimized to work just on bytes" by omitting some operations.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on October 01, 2021, 06:42:16 pm
I figured out why the Lisa would sometimes never lower CMD after sending the command bytes. It appears that the Office System will sometimes start a handshake in order to make sure that a drive is there, but never finishes it once it's satisfied that the drive is alive and well. Also, when the LOS installer reads the spare table to determine the drive type, it only cares about the identifying information at the beginning and not the table of spared blocks itself, so it just stops sending stobe pulses after it gets the data that it wants and expects the drive to time out and get back into its idle state, ready to receive the next command. I had my timeouts set to around one second, so I was waiting way too long and was missing subsequent commands from the Lisa. Unfortunately, making the timeouts a lot shorter didn't fix the "Lisa can't write to the disk" error message, but at least we're not missing commands anymore while waiting around in exorbitantly long timeouts!

At this point, I'm guessing that it probably has something to do with the parity since I don't really see much else that could be wrong.

Quote
A lookup table is an easy way to compute parity, but I'm guessing that you might find it irritating to spend 256 bytes of your available 8K.

Unfortunately, I think a lookup table is going to be my only option here. I'm already significantly pushing the limits of the Arduino's processing power and the overhead of adding a routine to calculate parity would almost certainly cause us to start missing strobe pulses. Heck, even the lookup table might be too slow. If it is, I might have to add an LS280 to my PCB to handle all of the parity stuff for me.

Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: rayarachelian on October 01, 2021, 07:29:11 pm
A lookup table is an easy way to compute parity, but I'm guessing that you might find it irritating to spend 256 bytes of your available 8K.

You could also split the byte into 4 bit nibbles and then use only 16 bytes for the lookup table. You'd then XOR the two values.

Or more efficiently, you could do something like this from C:

Code: [Select]
parity=(!!(d & 128))^(!!(d & 64))^(!!(d & 32))^(!!(d & 16))^(!!(d & 8))^(!!(d & 4))^(!!(d & 2))^(!!(d & 1));

In Python, you can do this - though you'd probably be able to optimize it a bit, and admittedly, looks fugly. But it does work.
You can reverse (0,1) to (1,0) in every expression below to get even parity if (0,1) not compatible with the Lisa's parity circuit.

Code: [Select]
parity = (0,1) [ ((128 & d)!=0)^((64 & d)!=0)^((32 & d)!=0)^((16 & d)!=0)^((8 & d)!=0)^((4 & d)!=0)^((2 & d)!=0)^((1 & d)!=0) ]

Or (still python) if you're fine with a boolean result instead of 0 or 1, this is a bit cleaner:
Code: [Select]
parity = ((128 & d)!=0)^((64 & d)!=0)^((32 & d)!=0)^((16 & d)!=0)^((8 & d)!=0)^((4 & d)!=0)^((2 & d)!=0)^((1 & d)!=0)

This might help too: http://graphics.stanford.edu/~seander/bithacks.html#ParityMultiply if you can use a multiply in that microcontroller. This is even cleaner:
http://graphics.stanford.edu/~seander/bithacks.html#ParityParallel - in the case for the profile you can discard operations on the high bytes.

So just:
Code: [Select]
uint8 parity(uint8 data) {
  uint32 v=data;
  v ^= v >> 4;
  v &= 0xf;
  return (0x6996 >> v) & 1;
}
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on October 03, 2021, 12:36:54 pm
Thanks for all of the parity suggestions! Unfortunately, they were all too slow for the Arduino, so I ended up borrowing an LS280 parity generator from one of my ProFiles instead. Now that we have parity, attempting to boot from a ready-made Office System 3.0 image still gives the same 10707 error as before, but I do get a bit farther when trying to install LOS from floppies. Now the installer is able to make it through the disk erase/initialization process, but it always fails at some point during the actual installation (the farthest it's ever made it is to disk 4). When it fails, it either gives the same "can't write to the disk" error as before or it gives a message stating that there isn't enough room on the disk, which doesn't make any sense to me. It's weird that it can boot MacWorks Plus, MacWorks XL 3.0, the Cameo/Aphid Selector, and BLU just fine, but that LOS doesn't work. Hopefully I can figure this out!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: rayarachelian on October 03, 2021, 02:23:31 pm
Thanks for all of the parity suggestions! Unfortunately, they were all too slow for the Arduino, so I ended up borrowing an LS280 parity generator from one of my ProFiles instead.

Uh, that's a bit disturbing, so the q is, is it going to be fast enough to work as a profile emulator fully, if it can't do a few XORs and shifts?

Now that we have parity, attempting to boot from a ready-made Office System 3.0 image still gives the same 10707 error as before, but I do get a bit farther when trying to install LOS from floppies. Now the installer is able to make it through the disk erase/initialization process, but it always fails at some point during the actual installation (the farthest it's ever made it is to disk 4). When it fails, it either gives the same "can't write to the disk" error as before or it gives a message stating that there isn't enough room on the disk, which doesn't make any sense to me. It's weird that it can boot MacWorks Plus, MacWorks XL 3.0, the Cameo/Aphid Selector, and BLU just fine, but that LOS doesn't work. Hopefully I can figure this out!

Do you know what state in the state machine of the write it failed in?
see: http://john.ccac.rwth-aachen.de:8000/patrick/data/profile_write_cycle.png
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on October 03, 2021, 06:49:57 pm
Quote
Uh, that's a bit disturbing, so the q is, is it going to be fast enough to work as a profile emulator fully, if it can't do a few XORs and shifts?
I'm definitely pushing the limits of the Arduino with this project! Adding parity calculation causes it to start occasionally missing /PSTRB pulses, but it's really reliable as long as it doesn't have to calculate parity.

Quote
Do you know what state in the state machine of the write it failed in?
The weird thing is that it doesn't fail during a write. The Office System just gives up in between writes according to the logic analyzer. When I look at the writes that occured immediately before it gives the error, they all seem to be perfectly normal, so I don't really know what's going on.

During my troubleshooting, I decided to change the spare table so that my device reports itself as being a 10MB ProFile instead of a 5MB ProFile and now the Office System installs and boots just fine! It boots up pretty fast (around 40 to 45 seconds) and it seems to be quite stable. I have no clue why making it a 10MB drive would cause it to start working, so I'll definitely have to look into this more tomorrow!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: rayarachelian on October 03, 2021, 10:08:40 pm
I'm definitely pushing the limits of the Arduino with this project! Adding parity calculation causes it to start occasionally missing /PSTRB pulses, but it's really reliable as long as it doesn't have to calculate parity.

You might be good now, but it means you can't upgrade this thing or add much else to it. So it's gonna be a dead end. Might want to consider the next model up.

Quote
Do you know what state in the state machine of the write it failed in?
The weird thing is that it doesn't fail during a write. The Office System just gives up in between writes according to the logic analyzer. When I look at the writes that occured immediately before it gives the error, they all seem to be perfectly normal, so I don't really know what's going on.

During my troubleshooting, I decided to change the spare table so that my device reports itself as being a 10MB ProFile instead of a 5MB ProFile and now the Office System installs and boots just fine! It boots up pretty fast (around 40 to 45 seconds) and it seems to be quite stable. I have no clue why making it a 10MB drive would cause it to start working, so I'll definitely have to look into this more tomorrow!

This sounds more like LOS thinks the drive is shared with MacWorks, but doesn't have enough free space.
Or maybe the spare table is incorrectly formed for 5MB and reports too many spared blocks?
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on October 04, 2021, 05:31:49 am
Quote
You might be good now, but it means you can't upgrade this thing or add much else to it. So it's gonna be a dead end. Might want to consider the next model up.

I think I'm more optimistic than that!

The Lisa will eat up all your cycles while it's talking to you: ~PSTRB is a pretty demanding boss. But in between some well-defined and limited parts of the read/write cycle, the ProFile can sit there holding down ~PBSY while it thinks, and the Lisa will just have to wait or decide to time out.

The thing is that you've already implemented all of the really time-critical parts by now, and all of the bells and whistles that you might want to add (e.g. Selector compatiblity :D ) are things that the Arduino will do while ~PBSY is asserted and the Lisa is waiting. I think the little microcontroller will be able to accomplish a lot in those intervals. Sure, it'll have limits --- it won't be able to run Doom in the background and send dithered bitmap frames down the parallel cable for the Lisa to render on its display, something which is definitely not a gimmick I've been thinking of doing for months on Cameo/Aphid --- but virtually all "reasonable" extras should be doable, i would think.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on October 04, 2021, 12:33:58 pm
Quote
The thing is that you've already implemented all of the really time-critical parts by now, and all of the bells and whistles that you might want to add (e.g. Selector compatiblity :D ) are things that the Arduino will do while ~PBSY is asserted and the Lisa is waiting.

Exactly! Any additions that I make will take place at times when we can hold BSY low for pretty much as long as we want and I won't be adding anything to the time-critical sections, so as long as the Arduino can reliably handle these timing-intensive sections as they are now, it should be able to handle things like Selector compatibility that I add later.

Quote
You might be good now, but it means you can't upgrade this thing or add much else to it. So it's gonna be a dead end. Might want to consider the next model up.

I was originally planning on using a more powerful microcontroller, like the Teensy or ESP32, but I decided to stick with a slower Arduino Mega because the Mega uses 5V logic levels and all of the faster offerings use 3.3V levels. One of my major goals with this project is to minimize the cost and complexity of the device and eliminating the need for level shifting definitely makes the hardware cheaper and simpler.

Once I figure out the strange 5MB problems with LOS, I'll get started with adding Selector compatibility. It seems like it will be pretty straightforward to implement, although the SD card library that I'm using only supports 8.3 filenames, so I'll have to figure something out there since the Selector lets you use names that are much longer than this. Maybe I'll need to find a different library that can handle longer names!

Quote
it won't be able to run Doom in the background and send dithered bitmap frames down the parallel cable for the Lisa to render on its display, something which is definitely not a gimmick I've been thinking of doing for months on Cameo/Aphid

You should totally do that! It would be so cool to see Doom "running" on the Lisa!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on October 16, 2021, 11:33:25 am
It's finally time for another update on the ArduinoFile! I've implemented almost complete Selector compatibility, which really increases the usefulness of the device. The only things that I don't have working are modification dates when the Selector asks for information about a file (I can't get the getModifyDateTime() function in the SDFat library to work properly for some reason), the uptime counter, and the key-value store (I'm not looking forward to implementing that one).

I should hopefully be able to get the modification date and key-value store working, but the uptime counter is a bit of a lost cause. Since I have to keep interrupts off throughout most of the code to keep things fast enough, I have no way of accurately tracking the passage of time since the millis() function only works when interrupts are enabled. The Selector seems to send emulator status requests once every two seconds, so I've just settled for incrementing the uptime by 2 seconds each time we get a status request. This solution isn't ideal because the uptime will stop incrementing whenever we're not in the Selector (or when the Selector's screensaver activates), but it's better than nothing.

I just put the ProFile emulator code on GitHub if anyone's interested: https://github.com/alexthecat123/ArduinoFile (https://github.com/alexthecat123/ArduinoFile)

I also just finished designing a much more compact PCB that stacks on top of the Arduino, much like the Cameo/Aphid cape for the PocketBeagle. I've added a status LED, a reset button, and the LS280 parity generator to the board and I'll put the new board design on GitHub once I get the boards in the mail and test them.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on October 16, 2021, 04:12:45 pm
Congratulations on your progress, it sounds really impressive! I had wondered how things were going.

The uptime counter is not very important at all --- like the manual (https://github.com/stepleton/cameo/blob/master/aphid/selector/MANUAL.md) says, "It serves little purpose except to look 'geeky' and to demonstrate that the Cameo/Aphid is operating and responsive." If you ask me, since it can't really keep time in a meaningful way, and since keeping time isn't very useful, you ought to make it count backwards ;D Or just animate it some other way, like

Code: [Select]
0d 00:00:01
0d 00:00:10
0d 00:01:00
0d 00:10:00
0d 01:00:00
0d 10:00:00
1d 00:00:00

I think the Selector's rate for pinging the uptime counter is a bit faster than once every 2 seconds, but I've never timed it.

I don't know how to solve the modification time problem, but for the key/value store I'd probably just hijack the filesystem on the SD card in some cheap and cheerful way. You'd have to be careful about some characters that you're not allowed to use, and I guess case sensitivity could be a problem, but otherwise it's kind of a key/value store already...
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on December 16, 2021, 08:20:59 am
Update: I finally got the revision 2 PCBs in the mail from China and they seem to work great, so I've updated the hardware section of readme.md to provide links to all the parts needed for this project and I put the new board files in the PCB folder. Now that there's a parts list and a fully functional PCB, people should actually be able to make one of these!

https://github.com/alexthecat123/ArduinoFile (https://github.com/alexthecat123/ArduinoFile)
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on December 20, 2021, 04:48:07 pm
Congrats on getting this far! It looks like the Apple parallel port hard drive emulation space is really heating up. I think that ArduinoFile uses the most modest microcontroller, which is pretty cool.

ETA: This is probably nonsense. IDEFile uses an 8031, which is far smaller! Perhaps I mean the most modest of the modern "hobbyist class" of MCUs.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on December 20, 2021, 07:34:23 pm
Quote
Congrats on getting this far! It looks like the Apple parallel port hard drive emulation space is really heating up.

Thanks! It feels really good to have contributed something to the Lisa community that might be useful to other people!

I would also like to apologize once again for the original name that I gave the ArduinoFile in some of the earlier posts in this thread. I've felt really bad about that over the past couple months and I want to make sure that everyone knows that the original name was a big misjudgement on my part and that I'm really sorry about it.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on December 21, 2021, 05:29:00 pm
No sweat, try not to let it bother you too much. I think it's water under the bridge now!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: jamesdenton on March 24, 2022, 07:26:53 pm
Just wanted to give a shout out to Alex here, as I successfully built and tested an ArduinoFile and appreciate the quick assistance he was able to provide with a timing problem. Happy to report that the great Selector, LOS, MacWorks, and UniPlus all are working without issue so far.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on March 24, 2022, 10:32:09 pm
Thanks James! I'm glad you got it working and I was happy to help out! As far as I know, you're the first person (other than me) to actually build an ArduinoFile, which is really cool!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on May 07, 2022, 06:39:31 pm
I just uploaded a new version of the ProFile emulator code to GitHub (https://github.com/alexthecat123/ArduinoFile (https://github.com/alexthecat123/ArduinoFile)) with a few changes and bug fixes.

Thanks to James Denton, I was able to fix a timing-related bug that kept the emulator from working on certain people's Lisas that are slower than mine at putting an 0x55 on the parallel bus.

I also fixed a bug where the ready LED sometimes wouldn't come back on after the end of a read or write operation and added enough key-value store functionality to get the Moniker and Autoboot functions in the Selector working. I probably won't ever fully implement the key-value store since these are the only two Selector operations that use it, but I might end up doing it if someone ends up writing an application that actually needs it!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on May 07, 2022, 08:47:04 pm
Probably not super necessary to add full key/value store support. I doubt it'll find additional uses anytime soon, but you never know.

One feature I was thinking of adding someday was caching a value for the clock. It's not clear how desirable the feature would be, but the idea goes like this. On startup, the Selector could check the Lisa's idea of the time. If it looked like a sensible value (as if someone had set the clock and then rebooted the machine), then the Selector would save the current time to the key/value store. On the other hand, if the Selector found that the Lisa's clock was clearly unset (as if someone had just plugged in the Lisa and turned it on), then the Selector would load the saved time from the key/value store and set the clock to that.

The time would be incorrect, sure, but these days the Lisa clock is always incorrect. The upside is that the Office System wouldn't have to keep reminding you to set the clock all the time.


Did you implement the "selector restore" functionality? Users encounter it as described here (https://github.com/stepleton/cameo/blob/master/aphid/selector/MANUAL.md#oops-i-damaged-or-deleted-the-selector), and the protocol that allows it to work behind the scenes is here (https://github.com/stepleton/cameo/blob/master/aphid/selector/PROTOCOL.md#block-fffefc-selector-rescue). I've found it's helped me out a few times now.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on May 08, 2022, 08:21:35 am
Quote
One feature I was thinking of adding someday was caching a value for the clock.
That sounds like a good idea to me! Even if the time isn't correct, at least the Office System would stop bothering you about it!

Over the past few weeks, I've been thinking about making some sort of daughterboard that lets you connect a battery-backed RTC to the COP421 that will automatically update the clock (excluding the year, of course) to the current time whenever the Lisa is plugged in, but it might be a while before I'll get around to this and your Selector-based solution seems like a really good alternative.

Quote
Did you implement the "selector restore" functionality?
Yep! I think I've implemented every Selector feature by now and "selector restore" has come in handy for me a couple of times as well!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: rayarachelian on May 08, 2022, 12:02:11 pm
One feature I was thinking of adding someday was caching a value for the clock. It's not clear how desirable the feature would be, but the idea goes like this. On startup, the Selector could check the Lisa's idea of the time....

This triggers another thought: it might be useful to also save the portion of the 1K floppy RAM that maps to PRAM to the SDXC card and load it back in when the selector profile (or other profile) is loaded. This will likely help with MacWorks, perhaps it can be a switchable option for the end user if desired?
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on May 08, 2022, 12:59:40 pm
This triggers another thought: it might be useful to also save the portion of the 1K floppy RAM that maps to PRAM to the SDXC card and load it back in when the selector profile (or other profile) is loaded. This will likely help with MacWorks, perhaps it can be a switchable option for the end user if desired?

This sounds like a great idea --- the first step of course is to determine what portion of the RAM that is and how to tell when it's initialised vs. uninitialised.

The Selector configuration data structure's feature bitmap (https://github.com/stepleton/cameo/blob/master/aphid/selector/config.x68#L59) is a 32-bit bitmap, and only two bits are used so far. It'll be easy to enable/disable switchable options like these.

Oh hey, speaking of switchable options, I forgot altogether: the Selector has the ability to display a little 128x32 bitmap logo when it starts up, kinda like the little picture you see with old Sun workstations (https://www.leadedsolder.com/2018/05/28/sparcstation-video.html). I forgot that I put that feature in there, and in fact I never tried it. Here's the code that does it. (https://github.com/stepleton/cameo/blob/master/aphid/selector/selector.x68#L178) Who knows, maybe it even works! :)

The plan was (and I guess still is) that besides the Selector (which needs to be fairly small so that it boots up quickly), I'd make an additional set of Selector utilities that you could run from the Selector, including something that lets you edit that little bitmap, which lives in the key/value store.

On that note, @AlexTheCat, the list at the bottom of key_value.x68 (https://github.com/stepleton/cameo/blob/master/aphid/selector/key_value.x68#L227) is the set of all the key/value store elements that the Selector expects to find right now, as you may well know. It includes the entry that holds the logo image data.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: rayarachelian on May 08, 2022, 02:44:59 pm
This triggers another thought: it might be useful to also save the portion of the 1K floppy RAM that maps to PRAM to the SDXC card and load it back in when the selector profile (or other profile) is loaded. This will likely help with MacWorks, perhaps it can be a switchable option for the end user if desired?

This sounds like a great idea --- the first step of course is to determine what portion of the RAM that is and how to tell when it's initialised vs. uninitialised.


I vaguely remember seeing it somewhere in the HWG, but it doesn't say what MacWorks uses, so it's fairly safe to save the whole 1K of RAM, but maybe exclude the lower values that include the IOROM version (incase you move the Aphid from one machine to another.)



Edit: this is what I save in LisaEm, but don't recall where I got the values from:
Code: [Select]
    fwrite(&floppy_ram[0x180/2],(0x200-0x180)/2,1,F);
So the floppy_ram block is presented to the Lisa on the low bytes in words only hence the /2 and looks like the range is 0x180-0x200=0x80 (or 128 bytes) and the address range should then be double that. So it's the high 128 bytes that are saved as PRAM.  From the M68000 point of view, this is the range from 0xFCC301-0xFCC3ff (odd bytes only).

Note: Looks like MacWorks II+ uses double that and has a serial EEPROM for this as part of the PFG so it can support System 7 - https://lisafaq.sunder.net/single.html#lisafaq-sw-mw



The other problem is going to be how to save those at shutdown, so would need a MacWorks MacOS extension/app that runs in the background and on shutdown saves those values to the Aphid.


Oh hey, speaking of switchable options, I forgot altogether: the Selector has the ability to display a little 128x32 bitmap logo when it starts up, kinda like the little picture you see with old Sun workstations (https://www.leadedsolder.com/2018/05/28/sparcstation-video.html). I forgot that I put that feature in there, and in fact I never tried it. Here's the code that does it. (https://github.com/stepleton/cameo/blob/master/aphid/selector/selector.x68#L178) Who knows, maybe it even works! :)

Neat! I am a fan of the old SPARC v8s.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: blusnowkitty on May 09, 2022, 10:18:08 am
I don't have an Aphid and I haven't looked at the code, but could you not make the Selector a chainloader? Effectively, give it 3-5 seconds on a splash screen waiting for user input if they want to load into the config menu, then set up the I/O RAM (Beaglebone's a full-blown ARM SBC, no? How hard would it be to use NTP to set the date/time automatically?), and finally load the Profile image the user wants to boot from (either saved in a config file as a default boot option or whatever image the user selected from the menu).
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on May 09, 2022, 03:59:33 pm
I'm not sure I fully understand the proposal, but something very close to what you propose can happen with the Selector's autoboot feature (https://github.com/stepleton/cameo/blob/master/aphid/selector/MANUAL.md#autobooting): if enabled, you have three seconds (ish) before it

The Selector program itself is fairly simple: when it starts up, it loads itself in its entirety into RAM and never touches its boot volume again. The reason for this is because the Selector can change the Aphid's current hard drive image at any time, so suddenly the data on the hard drive image it booted from may no longer be available.

If you wanted to dynamically load bits of the Selector only when you needed them (apologies if this is not what you're proposing!) then you'd want to do that dynamic loading using commands to the Aphid that are different from regular hard drive reads and writes. This is because the Aphid doesn't know whether it's the Selector talking or some other Lisa program or OS, and you probably only want the Selector to load up bits of the Selector (other Lisa software has no use for it and would only be confused by it --- or worse, it might just try to overwrite it or something). The key/value store would be the easiest option for some kind of "out of band" loading, but as Alex has noted, not all implementations of the Aphid key/value store (https://github.com/stepleton/cameo/blob/master/aphid/selector/PROTOCOL.md#block-fffeff-durable-keyvalue-store) are feature complete :-)

BeagleBone is a line of SBCs, but some are bigger and more feature-packed than others, and the one I've gone for is the smallest of them all: the PocketBeagle (https://beagleboard.org/pocket). Size is the main feature there; many other conveniences are missing. To talk to an NTP server, you'd need to get a network connection out of it, and there are certainly options for that. There is no built-in ethernet port or WiFi on the board itself, though, so the easiest option is to use the USB cable and tunnel TCP/IP over that (something the board sets up out of the box --- you can SSH into your Cameo/Aphid that way).

But if you don't want to use another computer as a network gateway for your tiny computer, you'll need to do something like adding an ethernet port or a USB host that you can plug a network dongle into. The PocketBeagle's SOC has built-in support for such ports if you attach their connectors to some of the I/O pins, but you'd need to hope that the I/O pins that those features use aren't the same as the I/O pins that the Cameo board uses. Could I change the I/O pins? Yes, but I don't want to --- the 16 that I chose are the ones most directly connected to the SOC's built-in I/O coprocessors. Plus I'd have to respin the Cameo board itself :-)

Other architectures may not have the same constraints that the PocketBeagle does --- maybe ArduinoFile can work differently, or maybe you can use one of those WiFi-enabled ESP32 things for yet another emulator. I saw somewhere that someone had made a ProFile emulator using a Raspberry Pi Pico... yes, here it is (https://github.com/BwanaFr/pico-profile). Coming up with your own emulator is a fun project and I suspect Patrick's excellent documentation is a big part of what's made it possible.

Anyway, without a way of getting the time from the outside world, it's more-or-less Groundhog Day for the PocketBeagle: it boots up and the time is whatever time it was when the kernel was built. At least I think that's how it works; there may be a cached time sort of scheme like the mechanism I've proposed for the Selector, but I forget whether that's the case.

Not sure if this is useful information, but there it is!
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: blusnowkitty on May 09, 2022, 05:26:57 pm
Sounds to me like Aphid Selector already does what I was thinking then.
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: stepleton on May 09, 2022, 05:30:42 pm
But wait, there's more! You can even set a password on the autoboot feature so that visitors to your computer museum won't monkey with your hard drive images! :D

(They'd have to come to the museum with their own floppy disk with the Selector on it to work around the password.)
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: rayarachelian on May 09, 2022, 08:14:09 pm
But wait, there's more!

Sigh... you went Billy Mays instead of Steve Jobs. "One more thing..." :)
Title: Re: An Arduino-Based Parallel Port Hard Drive Troubleshooter
Post by: AlexTheCat123 on May 10, 2022, 10:40:37 am
Quote
maybe you can use one of those WiFi-enabled ESP32 things for yet another emulator

I've actually been thinking about trying this for a couple of months! I bet it could make my emulator a little cheaper since an ESP32 costs around $5 less than an Arduino Mega. Also, I could probably get rid of the LS280 and do parity generation right on the ESP32 itself since it's way faster than an Arduino. Due to the ommission of the LS280 and the small size of the ESP32, I could probably make the whole thing a good bit smaller as well while still using only through-hole parts, which was a major goal of the ArduinoFile project. And plus, as you mentioned, the WiFi functionality could make it really easy to update the Lisa's clock to the current time!