LisaList2

Unfortunately I am confused now My immediate interpretation of what you're saying is that Xenix is managing to read 532 bytes from a floppy disk sector that has only 524 bytes!

Is it possible that the raw-to-dc42 tool is assuming 20-byte tags? Checking the source code for the tools that shipped with LisaEm and... hmm, if you're using that, I may have sent you down a garden path, since that does seem to be for ProFiles, not floppies.

I think that Xenix may actually be using the full 524 bytes per sector since you don't seem to be missing any of your data. The easiest way to confirm that I can think of is to use dd to copy the entire floppy into a file. Look at the size of the file and divide by 800: that's the block size. If it's 524, that's Xenix using the full block.

Just a quick response/correction - the 20 bytes doesn't eat into the 512 preceding bytes, it's just 20 bytes instead of 12. So, 0x0000-0x01FF is still intact and the tag is the next 0x14 bytes (0x0200-0x0213), followed by the next 512+20 bytes starting at 0x0214.

Sector tags on floppies are 12 bytes, 20 bytes on hard drives, except for Priam hard drives, where they're 24. (I have experience with the first two but not the Priams --- I get that fact from the BLU manual.) The explanation that they're treating floppies like hard drives makes the most sense to me, but if you're not going to go for the full 524 bytes, why you'd choose anything besides 512 (504 bytes??) is a big mystery to me.

I'm almost wondering if block device access on Xenix has some tricky behaviour --- I suggested the dd argument bs=1024 as an easy way to get 4 kilobytes of dumped data, but maybe block devices need to receive reads that are some correct size. Maybe some experimentation would confirm this: trying dd if=/dev/<floppy> of=/tmp/foo count=1 bs=X | ls -l /tmp/foo with X values of 400, 504, 512, 524, and 600 should result in /tmp/foo files that have those same sizes, and if they don't, we'll know that the floppy block device needs special treatment.

If you have the hard drive space for it and the patience, then I wonder what the size of a full floppy image on Xenix would wind up being. If it's also 800*504 = 403200 bytes, then that confirms it. dd if=/dev/<whatever> of=/tmp/weird bs=??who knows?? to see.

Anyway, thanks very much warmech for running the experiment you've run!

It'll be a while before I'm anywhere near north Texas, but you never know! 

I'm not super knowledgable about Lisa floppy formatting, but I do know a decent bit about the ProFile. And the ProFile uses 20 tag bytes per block (each block is 532 bytes, so you still have 512 left over for data), so maybe Xenix is treating floppy sectors like ProFile blocks or something? Maybe other operating systems like LOS only use 12 tag bytes per floppy sector, but Xenix treats floppies and hard drives identically?

Let us know how it goes if you try method #2. Like I say, I expect the tag bytes issue will complicate things. I put even odds on Xenix using the first 512 bytes of each sector and discarding the remainder, so when you make your raw disk image, you may need to introduce twelve-byte gaps between every 512 bytes of data that you wish to copy. This should be pretty easy to detect, though. In case folks are following along, I'll spell things out.

I'd experiment with the help of a raw disk image that you generate with a Python program like this:

#!/usr/bin/python3
with open('/tmp/raw_image.bin', 'w') as f:
  for _ in range(200):
    for ch in 'abcd':
      f.write(ch * 512)
      f.write(ch.upper() * 12)

Copy the data in /tmp/raw_image.bin onto a disk or convert it to a .dc42 (I think utilities to do this are bundled with lisaem). Then, on the Lisa, dd a few kilobytes from the floppy in Xenix, e.g. dd if=/dev/<floppy device> of=/tmp/sample bs=1024 count=4. Investigate the resulting /tmp/sample file somehow, perhaps with od -c /tmp/sample | more (if od is present on Xenix):

• If you never see any uppercase letters, then Xenix ignores the last 12 bytes of each sector.
• If you see runs of 512 lowercase letters interleaved with runs of 12 uppercase letters, then Xenix uses the full sector.
• If you see runs of 500 lowercase letters interleaved with runs of 12 uppercase letters, then Xenix ignores the first 12 bytes of each sector.

If, for some inexplicable reason, you should ever find yourself in north Texas... let me know because I owe you a beer. It took some fiddling around with the sector size, but this worked swimmingly. Xenix does not appear to discard 12 bytes, though; it instead appears to discard 20. On attempt number two, I added an additional four capital letters to the first instance in the file (so, AAAAAAAAAAAAAAAA) and found that it was still discarding them. On attempt three, I incremented the hex values of the characters after the 16 AAAAs and checked for the character that did show up, which ultimately turned out to be the 21st byte in the sequence. I'm not adept or knowledgeable enough with filesystems at the elemental level to know if 20 bytes is reasonably expected or bananas crazy, but it looks like we have a winner. Maybe you could shed some light on whether or not that's "normal"?

Today was a long day at work, so I had just enough energy left in me to try this little experiment and call it a day. I'll take another swing at this tomorrow and see if I can get some tarred data moved over (like the C source for a tiny kermit client) and update with my results. Thank you again for the advice and test code - both were absolutely invaluable!

Update: I found a Xenix drive image tucked away on one of my Cameo/Aphid SD cards. As you said, there's no uuencode, but there is od. I didn't carry out the floppy experiments, though

Let us know how it goes if you try method #2. Like I say, I expect the tag bytes issue will complicate things. I put even odds on Xenix using the first 512 bytes of each sector and discarding the remainder, so when you make your raw disk image, you may need to introduce twelve-byte gaps between every 512 bytes of data that you wish to copy. This should be pretty easy to detect, though. In case folks are following along, I'll spell things out.

I'd experiment with the help of a raw disk image that you generate with a Python program like this:

#!/usr/bin/python3
with open('/tmp/raw_image.bin', 'w') as f:
  for _ in range(200):
    for ch in 'abcd':
      f.write(ch * 512)
      f.write(ch.upper() * 12)

Copy the data in /tmp/raw_image.bin onto a disk or convert it to a .dc42 (I think utilities to do this are bundled with lisaem). Then, on the Lisa, dd a few kilobytes from the floppy in Xenix, e.g. dd if=/dev/<floppy device> of=/tmp/sample bs=1024 count=4. Investigate the resulting /tmp/sample file somehow, perhaps with od -c /tmp/sample | more (if od is present on Xenix):

• If you never see any uppercase letters, then Xenix ignores the last 12 bytes of each sector.
• If you see runs of 512 lowercase letters interleaved with runs of 12 uppercase letters, then Xenix uses the full sector.
• If you see runs of 500 lowercase letters interleaved with runs of 12 uppercase letters, then Xenix ignores the first 12 bytes of each sector.

I don't think uuencode/decode exists, unfortunately; I'll check when I'm back home this evening, but I think Lisa Xenix might predate these.

I've tried slowing the baud rate down to molasses-slow speeds (1200 is as low as I can get my USB/Serial converter and PuTTY to drop to; 600 just produces garbage and I dare not try anything below that, lol) and using a character/line delay then pasting into ed, but it still mangles the text buffer pretty badly. I may need to try something older running XP/98 with HyperTerminal and an actual physical serial port. I've got Linux boxes, but never used minicom/ascii-xfr - I'll absolutely take a look!

As for idea two, that's brilliant! It's probably also the lowest cost in terms of effort and time and, therefore, the most appealing, lol. I will absolutely give this a try tonight and update with the results. I really don't see why I couldn't wrap all my C files up in a tarball and make that the disk image itself.

Xenix does come with a C compiler, but it's K&R C and not ANSI, apparently; I suspect I'll have to tweak some things no matter what, but I wouldn't mind trying to get a version of one of those tools running on Xenix to simplify future file transfers.

Thank you for the recommendations!

I don't know how to use cu/uucp, but if they exist in Xenix, chances are that uuencode/uudecode do as well.

So, quick ideas before I run into work:

Copying a text file onto the Lisa: `cat > file.txt` on the Lisa and then rate-limited paste of the text data (hopefully the terminal program can do that? many can. If you have a Linux box and minicom, look into the program called `ascii-xfr`).

Copying a binary file onto the Lisa, idea 1: uuencode the file on the modern computer, `uudecode` on the Lisa, then rate-limited paste or raw upload of the uuencoded text data.

Copying a binary file onto the Lisa, idea 2: pad the binary file out to the size of a 400k diskette, convert this new "raw disk image" into a dc42 image, "burn" it to a real floppy (or use a floppy emu), then use dd on the Lisa to copy raw bytes off of the floppy and into a file. No need to worry about "format" or filesystems if you're copying raw bytes.

(Not sure if idea 2 will work effortlessly --- it may depend on what Xenix does with the disk's tag bytes.)

I forget whether xenix comes with a C compiler, but if it does, I'd look into using these methods to copy on an ancient version of Kermit (if you can find one somewhere) or some old implementation of xmodem and then compiling and using that.

So, I've been banging my head against my desk for a few days now trying to figure out the best way to get data on to a Xenix instance and I'm hitting some roadblocks. Vi doesn't seem to want to cooperate over a terminal connection for direct text input and my complete unfamiliarity with ed isn't helping either. Xenix obviously doesn't have xmodem or kermit as options, which leaves cu/uucp, but I am also completely unfamiliar with how cu on Lisa Xenix is supposed to work. The manual is less than clear about how cu connections over serial should work exactly; I'd have hoped it would have included an example but, eh. I've thought about going the floppy disk route, but I'm uncertain if that's even viable. If I formatted a floppy disk/image (via FloppyEmu), would it just be the standard SYSV format or did Lisa Xenix use some other format? I can't find any direct indication as to what it would format the disk as, so it's a mystery to me at the moment how Linux would even handle that. I've so far been unsuccessful at trying to get Linux to mount the ProFile image as a SYSV volume (which it should be able to do).

Does anyone have any proper experience working with serial connections in Lisa Xenix to more modern systems beyond a direct terminal connection or, alternatively, moving files via physical/virtual media? If anyone's got some practical experience with cu/uucp on the Lisa that could throw some pointers/advice my way, I'd be incredibly grateful. Or, like I said, if anyone's managed to use media to move data back and forth between Xenix and more modern systems that would be fantastic knowledge as well. Either way, I'd prefer not to have to type a rather large amount of C...

Edit - Also, had a RAM card go bad and cause a kernel panic while trying to get cu to work, so that was an interesting experience getting to see a KP happen in a 40 year old Unix distro, lol.