LisaList2

Advanced search  

News:

2019.06.07 fixed NChat for the "Curve" theme, will eventually move it to its own page and add it to the default theme as well. Other plugins are next. see post in the Meta board for details

Pages: [1] 2   Go Down

Author Topic: Monitor format for ProFiles  (Read 809 times)

stepleton

  • Sr. Member
  • ****
  • Karma: +44/-0
  • Offline Offline
  • Posts: 154
Monitor format for ProFiles
« on: May 09, 2020, 08:46:18 pm »

This is a long-shot question: does anyone know anything about the hard drive format used by the Monitor?

(For those not aware: the Monitor is the internal-only* operating system and development environment that Apple used when creating the Lisa OS and applications that it released to the public. The Monitor is a bit like a cross between the Pascal Workshop and the UCSD p-System, perhaps being closer to the p-System. There are a few Monitor floppy disk images on Bitsavers, as well as some user documentation. There is also a source listing that may answer my question, but I haven't examined it too closely yet.)

Without documentation about Monitor internals (that I'm aware of), you might not expect to have much to go on when making guesses about how the filesystem is organised. As it turns out, though, the floppy images use the same filesystem as the p-System. If you extract all of the sector data from a disk image and place it in a single file, you can read it just fine with utilities from this project. The disks make no use of sector tags (except to mark sector 0 as bootable). Additionally, the affordances of the Monitor file manager and hints from the user documentation suggest that a hard drive would also use a p-System-like filesystem.

With that in mind, I have tried using the filesystem utilities linked above to make a fresh 5 megabyte p-System filesystem image, then giving that image some blank sector tags and loading it into my ProFile emulator. This does change the behaviour of the Monitor version I tried it with---it seems to make it crash more! (More than it did with a hard drive image with BLU on it.) There are many possible reasons for this behaviour, but it may be because the Monitor is trying to read the filesystem and is getting some unexpected data (zeroes, in fact) back instead---that is, my formatting is incorrect.

There are many additional diagnostic and trial-and-error things to attempt to figure out what's going on, but I thought I'd ask just in case---does anyone just happen to know how the Monitor filesystem was arranged on ProFiles? If so, I'd love to hear about it...


* Internal-only: technically LisaTest was an externally-released Monitor disk, but it booted straightaway into the LisaTest GUI and didn't let you use the p-System-like user environment.
Logged

rayarachelian

  • Administrator
  • Sr. Member
  • *****
  • Karma: +23/-0
  • Offline Offline
  • Posts: 421
  • "But what's puzzling you is the nature of my game"
    • LisaEm
Re: Monitor format for ProFiles
« Reply #1 on: May 09, 2020, 09:42:07 pm »

I've never seen monitor installed on a profile, so that's pretty cool. I'd ask to try to figure out what the maximum sector number it searched for was? This may help figure out what the internal limits are. i.e. is it 16 bits, 12 bits? some other thing?

How is to booting? Did you find a bootloader for Profiles for Monitor or did you use your own? If you used your own, how does monitor know where it's file system starts and the boot loader end?

This might also help: http://ucsd-psystem-fs.sourceforge.net/ucsd-psystem-fs-1.22.pdf but it's way too modern, and it does mention interleave, so not sure if the one from 77 of whatever the Lisa used would have it too.

This might help: http://bitsavers.org/pdf/apple/lisa/pascal_monitor/Monitor_11_Listing_Sep82.pdf

Not to be a one trick pony and say that this is a nail, but I'd approach this by enabling tracelog in LisaEm like so: `./build.sh --with-debug --with-tracelog` - with 1.2.6 I'd enable the define that turns on tracelog on start, on 1.2.7, I'd add `--with-debug-on-start` and then boot monitor from a profile as you've set it up. The tracelog it would generate would produce many gigs of output but it would contain everything Monitor did and tried to do, but failed.

Next I'd ask if it's smart enough to do some sort of interleave on the hard drive, but I'd guess it likely wouldn't.
Logged
Fate whispers to the warrior, 'You can not withstand the storm.'  The warrior whispers back, 'I am the storm.'

rayarachelian

  • Administrator
  • Sr. Member
  • *****
  • Karma: +23/-0
  • Offline Offline
  • Posts: 421
  • "But what's puzzling you is the nature of my game"
    • LisaEm
Re: Monitor format for ProFiles
« Reply #2 on: May 09, 2020, 10:15:11 pm »

So in http://bitsavers.org/pdf/apple/lisa/pascal_monitor/Monitor_11_Listing_Sep82.pdf on page 21 at address 0B12 there's a mention of cluster size and another comment saying #$100 (256) is the start of the volume table.

On page 23 there are comments about profiles with and without headers, not sure what the difference is. page25 has mentions of corvus and pipin and "cable disk" - interesting.

Bingo! page 29 at 110C there is an 9:1  interleave table. Likely that's what you need to make this work

I wonder what this Marksman interface was (page 32-33), there's mention of a 4 port PIA (Parallel Interface?)

At 2924 on page 63 we see the 1st block is 0000 and the last is 7fff. So that gives you the max size of the disk to 32768 blocks or 16384K or 16MB, but don't know how clusters work here so that would change the size to much larger.

At page 90 3C42 mentions macsbug (which is also on the mac - but the mac in macsbug does not stand for macintosh, rather it comes from motorola as a 68000 debugger). Warms my heart to see it mentioned here. Good times. :)

« Last Edit: May 09, 2020, 10:38:53 pm by rayarachelian »
Logged
Fate whispers to the warrior, 'You can not withstand the storm.'  The warrior whispers back, 'I am the storm.'

stepleton

  • Sr. Member
  • ****
  • Karma: +44/-0
  • Offline Offline
  • Posts: 154
Re: Monitor format for ProFiles
« Reply #3 on: May 10, 2020, 06:44:07 pm »

Thanks for the insights!

I think I wasn't clear in my initial post---I'm not booting from a ProFile, or even accessing a ProFile successfully at all. I'm still starting Monitor from Twiggies, and then trying to get it to do anything with the ProFile besides fail to access it. I'd like to have a bootable Monitor ProFile someday, but am not even certain how possible this might be. On the other hand, it may not be too hard, and you might even get my bootloader to do it with some tweaks...

Anyway, your thoughts inspired me to spend several hours this afternoon poring over the listing and experimenting.


1. The first thing I tried is just filling every block on the disk image with an empty p-System disk catalog. Forget interleaving---if it reads anything on the disk, it's going to find that the disk is formatted and has no files on it. Sadly, no joy. The system seems to carry on as if no drive were plugged in---well, except for randomly crashing into MacsBug with an unserviced level 5 interrupt all the time. Not sure why it does this.


2. Now, looking at PDSKRD in the monitor listing ($121C on), it seems like there's a simple disk block checksum mechanism at work during reads. If the read is configured to use disk headers, then the checksum is computed over the entire 532-byte block; otherwise it only does a checksum of the 512 data bytes (and then ignores them; statements $12B8 and $12BC). I think you can bypass the checksum check in "header mode", though, if the seventh byte of the header is positive (statements $12BE and $12C2). So I tried filling every byte of every header with the obviously positive byte $21. Still didn't work.


3. Although LisaEm would surely have been effective for peeking behind the scenes, I decided to investigate the Monitor's disk activity with my hard drive emulator---it's easy to do, since it's a tiny Linux computer, and you can just log in and watch the log messages scroll by. On boot (and when you attempt to mount a ProFile in the System Manager as well), the Monitor attempts to access blocks $000003 and $000007, which, if I've reversed the interleave scheme correctly, means that the Monitor was asking for blocks 7 and then 11... but only if we were in header mode! (Statements $11F0 and $11F4; in "non-header mode", it looks like you don't interleave.

Unfortunately none of these blocks make sense to me. On floppies, block 3 is the second block of the file directory., block 7 is the second block of the first file on the disk, and block 11 is the sixth block of the first file (assuming it extends that far). I'm not sure---maybe the filesystem layout is different in hard drives?

I guess the main thing to figure out then is why it tries to access these blocks. I still haven't been able to pinpoint the spot in the code where it does this. Hmm...
Logged

stepleton

  • Sr. Member
  • ****
  • Karma: +44/-0
  • Offline Offline
  • Posts: 154
Re: Monitor format for ProFiles
« Reply #4 on: May 25, 2020, 11:39:14 am »

Well, an update---it turns out that I was silly and missed something important. You can initialise a ProFile in the Monitor after all---just use the Z(ero command in the S(ysMgr. Reads and writes to the ProFile work just fine after that.

There is however an additional problem, which is that if you restart the computer (and boot the Monitor from a Twiggy again), the Monitor will now hang at some point midway through the startup process. Fortunately the Monitor has installed MacsBug prior to the freeze. I've started to use it to investigate the hangs, but have some ways to go. It seems that the Monitor is trying to report an I/O error of some kind and is waiting for the user to acknowledge this by typing the space bar, but no message appears on the screen and pressing the space bar does nothing. It's possible that this communication is actually going to somewhere besides the screen, though I haven't had any success yet from listening in on the serial ports.

The freezing occurs only when an initialised ProFile is attached to the parallel port---if you have an uninitialised ProFile ready, then it will still boot. This freezing behaviour occurs even when the ProFile has an Office System 1.0 filesystem installed (instead of a Monitor filesystem---I'm not sure how similar these filesystems are---or if whether might be the same). It doesn't seem to matter whether you're using a genuine ProFile or an emulator, though I haven't tried initialising my real ProFile (I don't want to blow away my Office System install on that disk right now).

Other notes:

After reading Monitor documentation on Bitsavers, I haven't seen any evidence that you could install and boot the Monitor off of a ProFile. There's discussion of using the ProFile as a working drive, but nothing about how to install a bootloader or anything like that that I can recall. It's possible that Apple folk just always booted the Monitor off of floppies or from the connected Apple machine (the documentation often discusses having an additional "Apple" of some kind as part of your workstation).

I've determined that the unserviced Level 5 interrupts that I had mentioned upthread were probably being raised by my AppleNet card. Removing the AppleNet card seems to have got rid of these.
Logged

jamesdenton

  • Full Member
  • ***
  • Karma: +22/-0
  • Offline Offline
  • Posts: 40
  • ArcaneByte
    • ArcaneByte
Re: Monitor format for ProFiles
« Reply #5 on: February 18, 2021, 07:28:19 pm »

Well, an update---it turns out that I was silly and missed something important. You can initialise a ProFile in the Monitor after all---just use the Z(ero command in the S(ysMgr. Reads and writes to the ProFile work just fine after that.

Do you recall what the ProFile showed up as in the Devices list? I am using Monitor 12.3 on a Lisa2, and all I get are DRIVE1 and DRIVE2, with DRIVE2 being the 'working' device. Any attempt to communicate with DRIVE1 just hangs the system. That said, there does not appear to be a way to unmount the 'working' floppy device to load additional resources. Access to a ProFile would be handy here.

I assume the guide you're referring to is this? http://www.bitsavers.org/pdf/apple/lisa/pascal_monitor/The_Pascal_Development_System_Manual_Feb82.pdf

Page 57 mentions using a ProFile, but not booting from one. But, it does mention 'Transfer all of the files on the female boot diskette (to the profile)', so I dunno.

Anyway, I can't see my ProFile to even Zero it out.
Logged

stepleton

  • Sr. Member
  • ****
  • Karma: +44/-0
  • Offline Offline
  • Posts: 154
Re: Monitor format for ProFiles
« Reply #6 on: February 18, 2021, 07:57:34 pm »

I'm afraid I don't remember much that's useful here --- I guess I just let this project fall by the wayside! I have a distant memory rattling around in my head that you have to mark the drive as being attached to the system somehow (something like mounting in Unix), but I really can't recall anything useful. Sorry about that!
Logged

stepleton

  • Sr. Member
  • ****
  • Karma: +44/-0
  • Offline Offline
  • Posts: 154
Re: Monitor format for ProFiles
« Reply #7 on: February 18, 2021, 08:21:59 pm »

Rereading, I do remember copying the system files to the ProFile at one point with high hopes that it might do something to enable booting from the hard drive, but it didn't work in my experiments.

I'd forgotten about the odd gendered terminology that the documentation uses: there's the word "male" for little-endian (and hence the 6502-powered Apple II is "male") and "female" for big-endian, which, well, "Lisa" I guess. It was important to talk about these things because you could apparently have Monitor setups where a Lisa and II could be connected together. It sounds like you would use the II to get the ball rolling, which I suspect was a more common thing to do as the Lisa hardware was still under development.

PDF page 4 also drops unhelpful hints:

Quote
To boot from a diskette based Apple II, first power up the Apple II with the male boot diskette in drive #4:. Insert the female boot in drive #5: and power up the Lisa. The female boot volume can also reside on a hard disk. SYSTEM.STARTUP on the male boot volume automatically executes MONBOOT, the program that starts up the Monitor on the Lisa. If you type space during the boot process, MONBOOT is not executed. If you type 'D' during the boot process, the debugging version of the Monitor is booted.

It still sounds to me like disks on the Lisa side are just used for data storage here and not booting.
Logged

jamesdenton

  • Full Member
  • ***
  • Karma: +22/-0
  • Offline Offline
  • Posts: 40
  • ArcaneByte
    • ArcaneByte
Re: Monitor format for ProFiles
« Reply #8 on: February 18, 2021, 08:37:22 pm »

I have a distant memory rattling around in my head that you have to mark the drive as being attached to the system somehow (something like mounting in Unix), but I really can't recall anything useful
...
It still sounds to me like disks on the Lisa side are just used for data storage here and not booting.

That's OK with me, as long as I can change the working device to the ProFile vs floppy. I need to play around with it and see where that attachment point happens. Thanks for the tip!
Logged

blusnowkitty

  • Sr. Member
  • ****
  • Karma: +32/-0
  • Offline Offline
  • Posts: 95
Re: Monitor format for ProFiles
« Reply #9 on: February 19, 2021, 10:48:58 am »

I am using Monitor 12.3 on a Lisa2, and all I get are DRIVE1 and DRIVE2, with DRIVE2 being the 'working' device. Any attempt to communicate with DRIVE1 just hangs the system.

Is it possible that since 12.3 is a point release, and seems to be the only one for Sony drives, that DRIVE1/DRIVE2 refers to the old Twiggies and that 12.3 was a quick and dirty hack to add Sony compatibility? I seem to remember some piece of software somewhere that still refers to the Sony drive as the "lower floppy" which is where the second Twiggy drive was in the Lisa 1.
Logged
You haven't lived until you've heard the sound of a Sony 400k drive.

D.Finni

  • Sr. Member
  • ****
  • Karma: +27/-0
  • Offline Offline
  • Posts: 91
Re: Monitor format for ProFiles
« Reply #10 on: February 19, 2021, 11:54:24 pm »

This is a long-shot question: does anyone know anything about the hard drive format used by the Monitor?
The Monitor file system, at least on the floppy disks, is the UCSD P-System, same as on the Apple II Pascal system, except it's big-endian. I documented the format and wrote an interpreter for it just a few months ago.
« Last Edit: February 20, 2021, 12:05:15 am by D.Finni »
Logged

stepleton

  • Sr. Member
  • ****
  • Karma: +44/-0
  • Offline Offline
  • Posts: 154
Re: Monitor format for ProFiles
« Reply #11 on: February 20, 2021, 08:19:17 am »

Oh interesting! As I was trying to format a ProFile for the Monitor, one of the things I did (or at least I think I did it) was to create a disk image in Apple II Pascal in an emulator and see if that image would work when transferred directly to a ProFile emulator. The endianity issue could be one reason it didn't go anywhere :)

Have you put any of your work online? But maybe more importantly --- is there something that would suggest to you that the format would allow you to boot from the disk? For example, one clue might be that the first few blocks are not used, making room for a bootloader.

Logged

rayarachelian

  • Administrator
  • Sr. Member
  • *****
  • Karma: +23/-0
  • Offline Offline
  • Posts: 421
  • "But what's puzzling you is the nature of my game"
    • LisaEm
Re: Monitor format for ProFiles
« Reply #12 on: February 20, 2021, 11:31:17 am »

*bump* yes pls, give us a pointer to this. Not committing to anything, but perhaps I could add support for this fs in lisafsh-tool, or something else eventually.
Logged
Fate whispers to the warrior, 'You can not withstand the storm.'  The warrior whispers back, 'I am the storm.'

D.Finni

  • Sr. Member
  • ****
  • Karma: +27/-0
  • Offline Offline
  • Posts: 91
Re: Monitor format for ProFiles
« Reply #13 on: February 20, 2021, 06:13:45 pm »

I attached two useful text files to this post. The first is an except from Apple's Monitor documentation, from a PDF on Bitsavers. It explicitly says "After the Monitor is installed [on the ProFile], you will still boot with the Monitor Boot disk."

So there's your answer from the horse's mouth regarding booting from a hard disk.

The second text file is the Apple Pascal 5.25" disk format for Apple II. You can also find a description of Apple Pascal disk format in the Call-APPLE publication called "All About Apple Pascal." It's documented in a few other sources too.

Like I said earlier, Lisa Monitor's format is exactly the same (so far as I know) as Apple II Pascal, except that word-length values are in big-endian order.

As far as code, I think that you could take DiscImageChef's Pascal FS module and modify it. Within the Apple II community, you should also find one or two projects which can read Apple II Pascal formatted disks.
Logged

D.Finni

  • Sr. Member
  • ****
  • Karma: +27/-0
  • Offline Offline
  • Posts: 91
Re: Monitor format for ProFiles
« Reply #14 on: February 20, 2021, 06:27:18 pm »

Oh, and by the way, last November I added Monitor format to my Vault archive site so you can click through and browse the contents of Lisa Monitor disks. And you can link directly to files within the disks. The URLs will never change.

This is neat. Take a look at some examples:

https://macgui.com/spyglass/r/7e8a7a533c8dfb56
https://macgui.com/spyglass/r/c65d7e778b0d2b79
https://macgui.com/spyglass/r/0f87d6f7c1cb821d

LisaTest is a Monitor-format disk. The feature is called "look inside with Spyglass" in the Downloads section of my site.
Logged
Pages: [1] 2   Go Up