Print

[AlexTheCat123]

I've finished working out all the issues at this point, except for one really weird one where the newly-compiled SYSTEM.OS fails to boot the Lisa with a 10730 (bad SYSTEM.OS) error. Not sure what's happening here because I'm pretty sure everything is identical to my original working build image, but I'm trying to get it figured out.

I'm at the beach for the weekend, so I might be a bit slower than usual to get this done, but I brought a Lisa with me to keep working on everything. Sorry for the delay; I promise I'll get the final working version of everything onto Github as fast as I possibly can!

[AlexTheCat123]

It turns out that my disk image was corrupted in some way, and that's what was causing the 10730. It compiles and boots just fine on a second image. So I've put all the fully tested and working scripts on Github now.

If you've already downloaded the LOS Compilation Base disk image in the past, be sure to redownload the new version before continuing!

Detailed instructions are in the readme, but here's the gist of it.
1. Run the patch_files.py script with your Lisa_Source directory as the argument, and it'll apply all the needed patches to your source files. Make sure it says it patched them all!
2. Connect a serial cable between your modern computer and your Lisa. If you're using a USB to serial adapter, make sure it supports DTR/DSR handshaking!
3. Run the ALEX/TRANSFER.TEXT EXEC script on your Lisa (hit R and then type "<ALEX/TRANSFER") and wait for the Lisa to switch to the alt console.
4. Run the lisa_serial_transfer.py script with your serial port and Lisa_Source directory as the arguments, and hit enter when it asks if you've run ALEX/TRANSFER.TEXT.
5. Wait for the transfer to complete. It'll take about 34 hours, but it's completely automated, so it doesn't feel too bad.
6. Once it's done, shut down your Lisa and MAKE A BACKUP OF YOUR DISK IMAGE!!! Then boot back up and run the ALEX/MAKE/ALL_NOFLOP EXEC script to build everything.
7. Building everything will take about 8 hours. Times might vary depending on the speed of your ProFile emulator. Make sure to reboot immediately after the build has finished!

Please let me know if you encounter any problems (you shouldn't, but you never know) and enjoy!

[jamesdenton]

Please let me know if you encounter any problems (you shouldn't, but you never know) and enjoy!

Thanks for making this available, Alex!

[ried]

+1 Thank you, Alex. Outstanding. 

[snua12]

Hello,

I'm currently trying to transfer the LisaOS source files to my Apple Lisa using the serial transfer script from this GitHub repo:
https://github.com/alexthecat123/LisaSourceCompilation/blob/main/scripts/lisa_serial_transfer.py

The machine is a Lisa 2/5 with 1MB of RAM.

However, during the transfer, the Lisa crashes with a Level 7 Interrupt and a CHK RANGE ERROR. I've attached a photo of the screen at the time of the crash (see below).


I'm wondering if anyone here has seen a similar issue or can help me interpret the crash and register dump. It seems to be failing during a MOVE.W instruction involving A5, possibly trying to read an invalid address.

Any insight or suggestions would be greatly appreciated!

Thanks, Ales

[AlexTheCat123]

So it looks like you're getting an error 643 when trying to read some data from the serial port. I haven't been able to figure out what 643 means exactly, but I do know that the 640 range is reserved for serial port errors, so clearly something's going wrong with the port. And then the failed read is causing something (array index or variable value, not sure) to go out of bounds, leading to the CHK RANGE ERROR.

So we just need to figure out what's causing this read error on your serial port. Have you tried the transfer multiple times, and if so, does it give the error every time? Does it always occur when transferring the first file, or are you sometimes able to make some progress before it errors out?

[stepleton]

643 is "Unexpected RS-232 interrupt", not to be confused with 647 a.k.a. "Unexpected RS-232 timer interrupt". I wonder if the sending computer or the serial adaptor is doing something funny with e.g. hardware handshaking lines.

[sigma7]

643 is "Unexpected RS-232 interrupt"

Where did you find that? It is missing from a few versions of the various workshop manuals that I examined.

Am testing now with 1MB of RAM, seems ok; but I suppose APLD/PRT.TEXT is reached after quite a few transfers.

[AlexTheCat123]

Where did you find that? It is missing from a few versions of the various workshop manuals that I examined.

Yeah, that's what I'm wondering too! I looked everywhere, including in the source code, and couldn't find it.

Am testing now with 1MB of RAM, seems ok; but I suppose APLD/PRT.TEXT is reached after quite a few transfers.

I've noticed that there can be some variance in how Python walks the directory structure on different people's computers, so it's possible that APLD/PRT.TEXT comes earlier or later in @snua12's transfer than in yours and in mine. In fact, looking at the log output to the alt console, it appears as if it might be their very first file, unless the LisaBug output covered up previous logs.

Assuming I'm right about it being the very first file, then I bet this points to some sort of weird serial configuration issue like @stepleton said. Not sure about Windows, but on macOS and Linux, I've noticed that many of my USB to serial adapters show up as 2 different devices and only one of them supports hardware handshaking. So you just have to try both and pick the one that doesn't break things partways into the transfer.

[snua12]

I really appreciate all your advice so far!

I’m running the transfer script on Linux with a PL2303 USB–RS232 adapter. In the past, the internal battery leaked and I had to repair a damaged trace with a bodge wire. I also replaced the AM26LS32ACN line receiver because Port B RxD was completely dead before, so there’s a chance the serial hardware might still have marginal issues.

The failure (error 643: “Unexpected RS-232 interrupt”) happens mostly on certain LisaOS source files. Lowering the transfer speed from 28800 bps to 14400 bps doesn’t help. For testing, I use a directory with only a single file. When I replace the file’s contents with repeated test123 strings to make it large, it transfers fine every time. This suggests the crash might be triggered by specific byte patterns in the data stream rather than file size or total length.

Has anyone seen RS-232 errors caused by certain data patterns on the Lisa, especially when hardware might be slightly out of spec?

[stepleton]

Maybe I was playing fast and loose 
To get directly to one of those lists of error codes, I Googled my favourite Lisa OS error code:

1162 "Power is already off"

(I would love to know the circumstances that could raise an 1162; in any case, I could not find an instance of the Lisa OS using it, though it does seem to define a constant called "pwr_already_off" in a few places.)

Anyway, the first Google hit was this 1982 document, and scrolling up to PDF page 44 brings us to a list of RS-232 errors. I suppose there is a chance that the list is out-of-date by version 3 of the OS, of course --- it's funny that the 643 code is undocumented in later resources like this one.

We can also do some sleuthing. The file LISA_OS/OS/source-rs232.text.unix.txt defines a constant called errbase with the value 640, and raises errors offset from that constant in several locations. The two places where the offset is 3 seem to involve copying data out of the "typeahead" buffer, though I find the code pretty difficult to understand at a glance if I'm being completely honest! There is mention of interrupts near the first offset-3 place.

I'll have to stop my investigations there. But if I can suggest a diagnostic experiment to @snua12, what if you do the following. Instead of having repeated "test123" strings, can you try having strings of varying lengths and with runs of spaces of varying lengths in them? I say this because the Lisa's text file format handles these things in interesting ways. Runs of spaces are "compressed" to save space, and the Lisa may need to do extra processing to do that kind of text encoding. Additionally, Lisa text files have a block structure, and a line of an inconvenient length at the right time may force the Lisa to close the current text file block and start a new one --- meaning it might be getting distracted at a critical moment while data continues to stream in. (Citation: PDF pages 37-38 here)

If a more interesting "artificial" file can cause this kind of trouble, we might get a clue from it, particularly if the text data (outside of the spaces and newlines) continues to be something boring like "test123".

[sigma7]

Where did you find that? It is missing from a few versions of the various workshop manuals that I examined.

Yeah, that's what I'm wondering too! I looked everywhere, including in the source code, and couldn't find it.

APLT/REC.TEXT contains this:

Code: [Select]

        if RecUartLen<>0 then                  {Any UART receives?}
          CheckReceive:=True
        else
          if OnLine then begin
            Read_Data (eCode, RefNum, Ord (@RecUartSeq), PaocSize, NrRead, Sequential, 0);
            if eCode <> 0 then begin
              if eCode = 643 then begin  {input missed}
                RecUartSeq [NrRead+1] := Chr(ParityChar);
                NrRead := NrRead + 1;
              end else begin
                Close_Object (eCode, RefNum);
                Unmount (eCode, UartPath [PortA]);
                UartOpen:=False;
                OnLine:=False;
                GiveAlert (HTerm, AlrtNote, ALUart, SerialName [PortA], TheTitle, '');
                NrRead:=0;
              end;
            end;
            RecUartLen:=NrRead;
            if NrRead>0 then
              CheckReceive:=True;
          end;

Is {input missed} a useful comment? I'm not sure.

edit: seeing as it then adds a filler character, I'm supposing this means receive buffer overrun.

Perhaps Ales's computer or python implementation is particularly fast?

[AlexTheCat123]

seeing as it then adds a filler character, I'm supposing this means receive buffer overrun.

Perhaps Ales's computer or python implementation is particularly fast?

The Python program checks the state of DTR before sending each byte, so we shouldn't have any buffer overruns here unless handshaking isn't functioning properly. An easy way for @snua12 to check: does the Python program ever pause transmitting in the middle of a file and start flashing the "Waiting for Lisa to catch up" message at the top of the terminal window? We expect it to pause at the end of a file regardless of whether or not handshaking is set up, but pauses in the middle of larger files (anything greater than about 8K) will indicate that handshaking is working properly.

Runs of spaces are "compressed" to save space, and the Lisa may need to do extra processing to do that kind of text encoding. Additionally, Lisa text files have a block structure, and a line of an inconvenient length at the right time may force the Lisa to close the current text file block and start a new one --- meaning it might be getting distracted at a critical moment while data continues to stream in.

We shouldn't have to worry about the space compression here; don't 100% quote me on this, but I believe that the Pascal reference manual states somewhere that it doesn't automatically do this compression when you write data to a text file. It can read compressed text from a file and convert it to a regular string, but can't take a string and compress it into a file. I guess you have to write your own text file interface routines in order to take advantage of the compression.

The closing and opening of a block messing up timing might be a valid concern, although I'd assume that the SCC would keep us safe in this situation too. I'm guessing it would just keep receiving until its internal buffer filled up (not sure how big that is; I've never read the 8530 datasheet) and then deassert DTR to pause things until the OS finishes opening the new block and comes in to clear the buffer.

My best guess right now is that this is a problem with handshaking. If you're not getting the pauses in the middle of file transfers, then it might be a good idea to probe the DTR pin on Serial B and make sure that it's toggling. If it toggles after about 8K of data and then toggles again 30-45 seconds later, then your Lisa should be asserting it properly. If you're getting the toggling but not getting pauses in the Python program, then something's probably up with your USB to serial adapter.

[sigma7]

checks the state of DTR

On my setup (not using Python to send data), I observed that toggling CTS was what worked for handshaking, DTR & DSR didn't throttle the data being sent to the Lisa. So I wired DSR from the Lisa to CTS on the USB-Serial adapter, and then handshaking worked in the driver. ie. I no longer needed to watch DTR or DSR, just turn on the driver's hardware handshake.

Runs of spaces are "compressed" to save space, and the Lisa may need to do extra processing to do that kind of text encoding. Additionally, Lisa text files have a block structure, and a line of an inconvenient length at the right time may force the Lisa to close the current text file block and start a new one --- meaning it might be getting distracted at a critical moment while data continues to stream in.

My observation is that:
- the compression of spaces occurs only at the beginning of a line of text (not elsewhere in a line/paragraph)
- uploading using 'copy' from -keyboard or -console, implements that compression, however using Alex's Receive program, the compression does not occur
- the compression (and slowdown) when uploading using 'copy' does not occur while collecting it into the receive buffer, it happens later when when saving the file. I think this is also the point at which text blocks would be arranged so that lines are not split across blocks (assuming that is still the case in Workshop 3.0).
- charcount seems much faster when the file has not experienced compression (such as when saved by Alex's Receive program)

it would just keep receiving until its internal buffer filled up (not sure how big that is)

The 8530 SCC has a 3 byte receive buffer. The pin compatible 85230 ESCC has a 5 byte buffer.

then deassert DTR to pause things

To be clear, the SCC doesn't do the handshaking automatically, the 68k software needs to observe that it is running low on it's own RAM receive buffer and do the handshake. The size, low and high threshold settings are for the block of RAM the Lisa driver sets aside for the receive buffer.

Since the SCC buffer is barely large enough for an interrupt handler to get the bytes before losing some, once in the interrupt handler, it will keep checking for and collecting data, essentially locking out other processing.

[AlexTheCat123]

Reading back over my post, I realized that I said DTR a few times when I meant DSR. Sorry about that! So when you're checking the signal coming out of the Lisa, make sure that you're checking DSR not DTR!

On my setup (not using Python to send data), I observed that toggling CTS was what worked for handshaking, DTR & DSR didn't throttle the data being sent to the Lisa. So I wired DSR from the Lisa to CTS on the USB-Serial adapter, and then handshaking worked in the driver. ie. I no longer needed to watch DTR or DSR, just turn on the driver's hardware handshake.

I noticed the same thing; the serial driver wouldn't pay any attention to DTR and DSR no matter what I did. But I decided to use DSR anyway by manually polling it instead of connecting it to CTS so that people wouldn't have to make custom cables to do their transfers.

To be clear, the SCC doesn't do the handshaking automatically, the 68k software needs to observe that it is running low on it's own RAM receive buffer and do the handshake. The size, low and high threshold settings are for the block of RAM the Lisa driver sets aside for the receive buffer.

Right, I understand that the big RAM buffer is completely managed by the serial driver and not the SCC itself. I just figured that, if the SCC's internal 3-byte buffer filled up and the 68K didn't come along to clear it, that it would automatically deassert DSR to tell the other computer to stop sending data until the 68K has the time to empty the SCC buffer. But I guess you're saying it's not that intelligent!