LisaList2

General Category => LisaList2 => Topic started by: rayarachelian on October 14, 2019, 05:54:46 pm

Title: LisaEm 1.2.7-RC4 support bug reports
Post by: rayarachelian on October 14, 2019, 05:54:46 pm
2022.04.01 - pushed to unstable + master branches. https://github.com/rayarachelian/lisaem/tree/unstable

Windows Packages:


    Skinny Packages:

    Full macos package:

Title: Re: LisaEm 1.2.7-Alpha support bug reports
Post by: rayarachelian on October 15, 2019, 03:52:08 pm
Found what was causing crashes on x86_64 macos X:
There might be more. Haven't yet done a git push as I'm fixing these, but will do so once I fix every instance of these.

Edit: fixed up code is up on github, and here's a DMG: https://lisaem.sunder.net/downloads/LisaEm-1.2.7-20191015-ALPHA-x86_64-only-macosX10.11.dmg
Title: Re: LisaEm 1.2.7-Alpha support bug reports
Post by: Lewton on October 21, 2019, 08:04:29 am
How often are you pushing code up on github, Rayarachelian?
Title: Re: LisaEm 1.2.7-Alpha support bug reports
Post by: rayarachelian on October 21, 2019, 10:36:14 am
Quote
How often are you pushing code up on github, Rayarachelian?
Since I'm the only one working on LisaEm, right now pushes are around once a week or so - mostly on weekends, and only when something is fixed. I generally don't want to push in-flight work-in-progress incomplete/non-functional code that doesn't compile or work.

Currently I'm working on new features such as the hqx upscaler, using the hq3x upscaler with a double-X lens to fix the rectangular pixel issue, so won't be pushing that until it works or at least mostly works. And if that turns out to not work, or look terrible, I'll remove it as a feature.

I suppose once 1.2.7 reaches prod-release, then I'll stop making changes and start work on 1.2.8 features, and once enough of 1.2.8 works, I'll push that.
After 1.2.8 I suppose there'll be long breaks again while I consider what I want to do for 2.0, etc.

Title: Re: LisaEm 1.2.7-Alpha support bug reports
Post by: rayarachelian on November 11, 2019, 08:13:03 pm
New Alpha, fixed some bugs, added the HQX scaler as a new display method, Drag and Drop of disk images for floppies and text to clipboard.
https://github.com/rayarachelian/lisaem-1.2.x and https://lisaem.sunder.net/downloads/LisaEm-1.2.7-20191111-ALPHA-x86_64-only-macosX10.11.dmg

Title: Re: LisaEm 1.2.7-Alpha support bug reports
Post by: D.Finni on November 29, 2019, 03:57:50 pm
I would like to maintain Mac OS X 10.5.8 support for LisaEm, but wxWidgets seems to be the weak link and is causing most of the trouble.

My solution is to keep the old wxWidgets 2.8.x interface from old LisaEm, and combine with emulator bug-fixes and other improvements from your latest GitHub source. The result is here: LisaEm 1.2.7-Alpha for Mac PowerPC 10.4 (https://lisalist2.com/index.php/topic,55.0.html)
Title: Re: LisaEm 1.2.7-Alpha support bug reports
Post by: rayarachelian on December 01, 2019, 07:34:05 pm
Well done!
There is some infection point between the wx3.11 and wx3.12 where it stops working on G4/G5. I have a build on my G5 iMac that works. I'll post it up when I get a chance. Newer versions than that commit hash won't build, but it's compatible enough.  I suppose wx3.0.4 should build fine too.
I'll probably do a dual release for the release version of 1.2.7 where it's a fat binary for classic macs that supports 10.4 PPC-32, 10.5 PPC-64, 10.5 i386 and then a more modern 10.11+ for just x86_32|64.

I've got a 10.11 El Capitan Vbox VM that mostly works well enough to build on.

My old 2009 core2 intel 15" mbp w/10.9.5 doesn't seem to be able to build proper binaries and the lldb debugger is terrible there as backtrace and list don't work too well to track down bugs, not sure why. Binaries built on that box all crash.

But binaries built on 10.11VM do work on 10.12, and 10.14 and 10.15 VMs, and on 10.11, clang + lldb work pretty well and I can debug most things, it's just that the display isn't GPU accelerated so I can't tell how well it would do on real hardware, though I do have a 10.14 on a fairly recent Mac that works well.
I wish I could get a working build for as low as 10.7 as supposedly wxWidgets 3.x does go that low. But... it is what it is.
I'll likely test it on FreeBSD, and OpenIndiana (and maybe OpenBSD) as well before the final 1.2.7 release (or maybe I'll leave that for 1.2.8 ), but I don't expect too many issues with those OSs. I tried to get the latest Oracle Solaris 11.4 in a VM, but it's quite crashy, so not going to bother with that. Might be nice to get it working on AIX 4 or 5 as well, but that would be more of a curiosity than anything anyone would likely care about (assuming I can even get wx to compile on those beasts)

Anyway, I've just rewritten the build system using the scripts from 1.3.0 which use a file system layout where the directories are organized like the Lisa's hardware - that was the original intent from the very first versions in 1998, but once I got to ~2007 I gave that up. So bringing it back now. So what's gonna happen next is that the 1.2.x repo on github will go into deprecated mode and I'll have a new repo called just "lisaem" without the lame ass version number. I was saving that for when 1.3.0 was working, but, I can do that in a few days once this stuff is stable.

But more urgently, I gotta write the Cygwin scripts for building on windows. I'm using this VM for that: https://developer.microsoft.com/en-us/windows/downloads/virtual-machines so I have to finish that bit before the end of the month or it will expire.  :P They're usually pretty good about releasing new versions a couple of weeks after they expire, but who knows when they'll end that program. Unfortunately on the pre-built windows versions, the wx-config command doesn't exist, so I'll have to deduce the wx-config --libs and --cflags and --cxxflags outputs from Linux and macos and deal with a bunch of cygpath translations to make shit work right. Originally I relied on wxdsgn off sourceforge, but that's been abandoned ~2011 or so. Too bad, as it was a nice IDE and had pre-built wxWidgets for Windows.

I did manage to script a couple of batch files that do an unattended Cygwin install with the mingw32/64 compilers, so there's that at least.

Title: Re: LisaEm 1.2.7-Alpha support bug reports
Post by: D.Finni on December 02, 2019, 10:26:42 am
Well done!
There is some infection point between the wx3.11 and wx3.12 where it stops working on G4/G5. I have a build on my G5 iMac that works. I'll post it up when I get a chance. Newer versions than that commit hash won't build, but it's compatible enough.  I suppose wx3.0.4 should build fine too.
I can tell you more about it from my experience this weekend:
2.8.12 is fine for 10.4 and 10.5
2.9.5 killed 10.4 support, but supports 10.5
3.12 killed support for both 10.4 and 10.5.

I spent probably 3 hours trying to get 2.9.5 to compile on 10.4, ripping out and modifying the code rather ham-fistedly when compile errors came up. These were generally for libraries not in 10.4, like CoreGraphics and CoreText. Managed to get a successful build, but then linking LisaEm with it failed. And probably I've broken one or the other from my attempts.

I noted that removal of support for older OS X operating systems seemed arbitrary and probably could have been maintained in wxWidgets.

Quote
I'll probably do a dual release for the release version of 1.2.7 where it's a fat binary for classic macs that supports 10.4 PPC-32, 10.5 PPC-64, 10.5 i386 and then a more modern 10.11+ for just x86_32|64.
My proposed solution is to maintain two wxui directories. One links to 2.8.12 and is for PPC/Intel 10.4 and 10.5 builds. The other wxui directory links to 3.x and is for everyone else.

So long as nothing you do in cpu68k/, generator/, or lisa/ breaks older wxWidgets support, this seems like a reasonable compromise to me.

I volunteer to maintain 10.4 and 10.5 PPC/Intel builds.


My old 2009 core2 intel 15" mbp w/10.9.5 doesn't seem to be able to build proper binaries and the lldb debugger is terrible there as backtrace and list don't work too well to track down bugs, not sure why. Binaries built on that box all crash.

And early gotcha that I ran into is that OS X back till at least 10.4 comes with a wxWidgets installation, but it's usually out of date. You need to check /usr/bin/wx-config because that's what LisaEm will link with if you don't update it. I used a symlink to point to my newer build of 2.8.12.

But maybe you already know about this....



I wish I could get a working build for as low as 10.7 as supposedly wxWidgets 3.x does go that low. But... it is what it is.
I have a Mac running 10.8 that I could build on. I don't have any 10.7 machines, however. I also have a Mac Pro running 10.11 and another Mac running 10.12 Sierra.

I have a fairly good spread of OS X versions across PPC and Intel.  8)
Title: Re: LisaEm 1.2.7-Alpha support bug reports
Post by: rayarachelian on December 02, 2019, 07:51:39 pm
I can tell you more about it from my experience this weekend:
2.8.12 is fine for 10.4 and 10.5
2.9.5 killed 10.4 support, but supports 10.5
3.12 killed support for both 10.4 and 10.5.

I spent probably 3 hours trying to get 2.9.5 to compile on 10.4, ripping out and modifying the code rather ham-fistedly when compile errors came up. These were generally for libraries not in 10.4, like CoreGraphics and CoreText. Managed to get a successful build, but then linking LisaEm with it failed. And probably I've broken one or the other from my attempts.

I noted that removal of support for older OS X operating systems seemed arbitrary and probably could have been maintained in wxWidgets.
Quote

Agreed. I really hate this idea that we have to throw out backwards compatibility. I haven't spent too much time in 10.4 on the G5 so maybe I should revisit 10.4 and seeing if I can get things going there.

My proposed solution is to maintain two wxui directories. One links to 2.8.12 and is for PPC/Intel 10.4 and 10.5 builds. The other wxui directory links to 3.x and is for everyone else.

So long as nothing you do in cpu68k/, generator/, or lisa/ breaks older wxWidgets support, this seems like a reasonable compromise to me.

I volunteer to maintain 10.4 and 10.5 PPC/Intel builds.

Um, ok, so... "Tag you're it" :-D I mean, "I dub thee maintainer of PPC" backports.


And early gotcha that I ran into is that OS X back till at least 10.4 comes with a wxWidgets installation, but it's usually out of date. You need to check /usr/bin/wx-config because that's what LisaEm will link with if you don't update it. I used a symlink to point to my newer build of 2.8.12.

But maybe you already know about this....

Oh yes, I always set PATH=/usr/local/wx-3.x.x/bin:$PATH to avoid issues. But even 3.1.2 is a pain. On linux they've decided to do some sort of automated rescaling so a 4K display reports back as 1080p and it undoes all the scaling stuff I've added. 3.1.1 works great. I'm probably not calling the proper new interface to get the actual size of the display or something, but it moved far away from the way 3.0 and 2.8 did things. So I'm mostly sticking to wx3.1.1

I haven't bothered with symlinks there because I keep a bunch of different ones on hand there. Currently there's 5 of them on my linux box.


I wish I could get a working build for as low as 10.7 as supposedly wxWidgets 3.x does go that low. But... it is what it is.
I have a Mac running 10.8 that I could build on. I don't have any 10.7 machines, however. I also have a Mac Pro running 10.11 and another Mac running 10.12 Sierra.
I have a fairly good spread of OS X versions across PPC and Intel.  8)

Sort of the same here, though as a new macos comes out every year I've given up trying to keep a machine for each, it's a losing battle, so I now just have a VM for every few versions.
Title: Re: LisaEm 1.2.7-Beta support bug reports
Post by: rayarachelian on March 15, 2020, 11:06:56 am
I renamed this thread to Beta to match the current release state.

Just cut a beta release 2020.03.14 and pushed source to github last night, and uploaded to the downloads page:Most of the non-display bugs are fixed, but lots of display bugs are still there. Once those are fixed, there's one or two more features to add, and then will move to release. I can't say when that will be as the display bugs are many and fugly.

On macos x the display refreshing is really badly broken, so I need to do a lot more work there. The good news is that I was able to create a Mountain Lion ( 10.8 ) VirtualBox VM with XCode 4.6.3 - I had to use macports as brew.sh (aka homebrew) does not support it.
In theory I used the 10.7 SDK, so it might work on macos x 10.7, but as I don't have a Lion machine to test it on, I can't yet tell. This uses wxWidgets 3.0.4 as 3.1.x will not build there.
I think for the final version I'm going to do some shell scripting tricks on MacOS X to glue several binaries together - without using lipo. Instead of launching a Mach-O binary for LisaEm, I'll run a shell script which will detect the OS and what binaries are available and run the latest one that the current OS will support. So if you're on PowerPC it will launch the G4/G5 version. If you're on an older intel it will try to run one that works. If you're on catalina it will launch one with wxWidgets 3.12 or 3.13, etc.
That is assuming the next macos x doesn't remove the shell altogether.  :P
I probably need to write some automation, scripts as well, to cut releases like these, (creating the DMG and populating it, pushing source to gitlab and binaries to the download page) as doing them by hand is getting annoying.
Title: Re: LisaEm 1.2.7-Beta support bug reports
Post by: rayarachelian on April 11, 2020, 03:40:13 pm
New Beta release is pending. Watch this thread, likely this weekend, or Monday the latest with a focus on macos x This will include the new macos version selector script as described above, and a rebuilt script for creating wxWidgets on macos. I guess you can go on a virtual easter egg hunt, where the eggs are bugs and report them here.  ::)
Title: Re: LisaEm 1.2.7-Beta support bug reports
Post by: rayarachelian on April 12, 2020, 11:51:07 pm
Phrew, that was a marathon of Yak Shaving. This is named 2020.03.31 because that's when the final source code changes were made, it's released today because I spent about two weeks shaving a very big Yak - namely (mostly) automating the dmg creation process and setting up a bunch of macos Virtualbox VMs to be able to create obese binaries for macos x.
This release contains binaries for macos x 10.8, 10.10, 10.11, 10.12 and 10.15 x86_64. On startup it will run whichever matches the OS most closely. This doesn't contain i386 binaries, nor PPC binaries, but the final release likely will. It's likely this will run on 10.7, as it was compiled on 10.8 with min macos SDK 10.7, but I don't have a way to test it there as I don't have a 10.7 VM.
While I do have a 10.9 VM, compiling under it does not produce viable binaries - LisaEm compiles successfully there but immediately crashes on startup, and I'm not sure why. I've peeled off some layers of that onion, but got nowhere, so I'll just rely on 10.8 binaries working on 10.9.x.

It's not necessary to make a bespoke binary for every os, but it's likely a good idea to build one for every major change so that the best combination of binary + wxWidgets library is provided. For example, the 10.8 in this release uses wx3.0.4, but the 10.15 one uses wx3.1.3, so it should pick up the latest fixes. (wx3.1.3 does not compile on macos 10.8 )

To mostly fix the worst display artifacts I had to switch to a double buffered wxDC, which causes display refreshing to be very slow. I'll hack on this some more until I find a better solution, so hopefully this is temporary, but it gets rid of the nasty flashing on macos x.
Next, I'll be continuing on the display artifacts issue, and then will add a couple more features, and that will lead to the final 1.2.7 release if all goes well.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on May 31, 2020, 09:21:43 pm
Edit: 2020.06.05 updated with new links, source + dmg.
RC1 is ready for download here: https://lisaem.sunder.net/downloads/ (https://lisaem.sunder.net/downloads/) - here are the relevant direct links:

* https://lisaem.sunder.net/downloads/lisaem-1.2.7-2020.05.27.tar.xz* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC1-2020.05.27-macos-10.10-x86_64.pkg
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC1-2020.05.27-macos-10.11-x86_64.pkg
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC1-2020.05.27-macos-10.12-x86_64.pkg
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC1-2020.05.27-macos-10.15-x86_64.pkg
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC1-2020.05.27-macos-10.5-i386.pkg
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC1-2020.05.27-macos-10.5-ppc.pkg
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC1-2020.05.27-macos-10.7-x86_64.pkg
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC1-2020.05.27-macos-10.8-x86_64.pkg
* https://lisaem.sunder.net/downloads/LisaEm_1.2.7-RC1_2020.05.27-macosx.dmg

The packages (pkg) will install both the LisaEm.app to /Applications as well as the command line tools to /usr/local/bin/ but these are skinny versions of LisaEm - that is they only hold the executable built from a specific macos x machine. They should run on machines with macos x of the same version or newer.

The DMG has the new "obese" "swole" binaries (on 2nd thought swole is a better word than obese), so if you install from there, you'll be able to move it around between systems, but the binary will be much larger 38MB rather than the 7~12MB skinny versions.

The above are small versions with only one macosx binary each. If you try to run one that's not suitable for your system, it will fail to start. ie. trying to run the 10.5-ppc versions on 10.15 which only wants x86-64.

The command line tools inside the Command Line Tools folder are packages that only install the CLI tools, but you'll need to specifically install the right one for your os. The packages will not block you from installing the wrong versions as there's no test script in them (yet).
These were built using a master script that powers on a vbox vm, waits for it to come up, ssh's into it and builds for that os, then shuts it down and repeats until all are done. The final one also builds the DMG. Since the 10.5-ppc was built on a 10.5 intel VM, and I haven't yet tested it, it might not work.I was hoping that the 10.5 version would work on 10.4, but sadly it does not. I've not been able to get a wxWidgets 3.x to compile properly on 10.4.  :-\

Similarly for some reason I couldn't compile i386 versions on 10.7 and above, only x86_64 builds properly. There's some linking issue vs wxWidgets 3.0.2 and 3.0.4.But this is ok as the last of the actual 32 bit mac os x was 10.6, and while you could run 32 bit binaries all the way to 10.14, it doesn't offer much of an advantage to do so over a 64 bit binary.
Enjoy.  :D
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: jamesdenton on June 01, 2020, 10:43:09 am
Hi Ray,

Trying to boot off an existing ProFile image returns the following error:

"I'm sorry, the emulation has aborted due to a fatal error

cpu68k_makeipclist: But! ipc is null!
Stopped at cpu68k.c:cpu68k_makeiplist:1071 with code :501"

Also, browsing to select a ProFile seems to want to only create a new profile image, potentially overwriting an existing file.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 01, 2020, 11:41:31 am
Hi Ray,

Trying to boot off an existing ProFile image returns the following error:

"I'm sorry, the emulation has aborted due to a fatal error

cpu68k_makeipclist: But! ipc is null!
Stopped at cpu68k.c:cpu68k_makeiplist:1071 with code :501"

Ouch, is this on 10.15? Does it also happen with the same image on older betas or 1.2.6?

Also, browsing to select a ProFile seems to want to only create a new profile image, potentially overwriting an existing file.

I think that's normal, although misleading as that field allows you to just set the location, it won't actually overwrite it. If there's no profile at that path, it will prompt you on power on to select what size to create.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: D.Finni on June 05, 2020, 02:40:47 pm
*  https://lisaem.sunder.net/downloads/lisaem-1.2.7-rc1-20200527-for-macos-x-10.5-i386.pkg (https://lisaem.sunder.net/downloads/lisaem-1.2.7-rc1-20200527-for-macos-x-10.5-i386.pkg)
This will not immediately work on 10.5 because the .plist file specifies minimum Mac OS X version 10.7.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 05, 2020, 04:33:52 pm
This will not immediately work on 10.5 because the .plist file specifies minimum Mac OS X version 10.7.

Right you are sir, fixed in the next pending release (soon, maybe today.)

Fun bit of trivia, 10.5 doesn't have a pkgbuild command, so I had to script it so the building of the 10.7 package picks up the 10.5 binaries and packages them for 10.5, but I borked it a bit and the Info.plist picked up 10.7.

I'm trying to get it to slap on the background images in the DMG so the version + release date is in the there, but but half the time it doesn't do it. Meh.

Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: D.Finni on June 05, 2020, 09:47:46 pm
By the way, any update on the CPU bug that's breaking scroll bars, Workshop, etc? This seems like the highest-priority thing...
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 05, 2020, 10:12:10 pm
By the way, any update on the CPU bug that's breaking scroll bars, Workshop, etc? This seems like the highest-priority thing...

I agree, it's also been the hardest to fix and so far no luck, I've got some plans to attempt resurrecting the 1.3.0 CPU engine which might have those fixed - or might introduce new bugs, not sure yet.

I've got 2-3 more features to add, I know, not supposed to add them to Release Candidates, but I think it's worth having them.

Also, I've edited the above post with new Rc1 links and also pushed the newer source code to github.

Have to retry compiling and testing on Linux and also making Windows binaries.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 08, 2020, 10:51:33 am
attempt to build on 10.11 almost works but I'm getting

ld: warning: object file (/usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib/libwxjpeg-3.1.a(wxjpeg_jidctfst.o)) was built for newer OSX version (10.12) than being linked (10.11)
ld: warning: object file (/usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib/libwxtiff-3.1.a(wxtiff_tif_fax3sm.o)) was built for newer OSX version (10.12) than being linked (10.11)
ld: warning: object file (/usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib/libwxtiff-3.1.a(wxtiff_tif_predict.o)) was built for newer OSX version (10.12) than being linked (10.11)
Undefined symbols for architecture x86_64:
  "_iconv", referenced from:
      wxMBConv_iconv::wxMBConv_iconv(char const*) in libwx_osx_cocoau-3.1.a(monolib_strconv.o)
      wxMBConv_iconv::ToWChar(wchar_t*, unsigned long, char const*, unsigned long) const in libwx_osx_cocoau-3.1.a(monolib_strconv.o)
      wxMBConv_iconv::FromWChar(char*, unsigned long, wchar_t const*, unsigned long) const in libwx_osx_cocoau-3.1.a(monolib_strconv.o)
      wxMBConv_iconv::GetMBNulLen() const in libwx_osx_cocoau-3.1.a(monolib_strconv.o)
     (maybe you meant: __ZN14wxMBConv_iconvC2EPKc, __ZTV14wxMBConv_iconv , __ZNK14wxMBConv_iconv5CloneEv , __ZTI14wxMBConv_iconv , __ZN14wxMBConv_iconvD0Ev , __ZN14wxMBConv_iconv14ms_wcNeedsSwapE , __ZTS14wxMBConv_iconv , __Z18new_wxMBConv_iconvPKc , __ZN14wxMBConv_iconvD1Ev , __ZNK14wxMBConv_iconv7ToWCharEPwmPKcm , __ZN14wxMBConv_iconv16ms_wcCharsetNameE , __ZN14wxMBConv_iconvC1EPKc , __ZNK14wxMBConv_iconv6IsUTF8Ev , __ZN14wxMBConv_iconvD2Ev , __ZNK14wxMBConv_iconv9FromWCharEPcmPKwm , __ZNK14wxMBConv_iconv11GetMBNulLenEv )
  "_iconv_close", referenced from:
      wxMBConv_iconv::~wxMBConv_iconv() in libwx_osx_cocoau-3.1.a(monolib_strconv.o)
  "_iconv_open", referenced from:
      wxMBConv_iconv::wxMBConv_iconv(char const*) in libwx_osx_cocoau-3.1.a(monolib_strconv.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
see: /tmp/slot.72108.1.sh /tmp/slot.72108.1.sh.failed for details
failed command was: g++ -o bin/LisaEm obj/floppy.o obj/profile.o obj/unvars.o obj/vars.o obj/glue.o obj/fliflo_queue.o obj/cops.o obj/zilog8530.o obj/via6522.o obj/irq.o obj/mmu.o obj/rom.o obj/romless.o obj/memory.o obj/symbols.o obj/lisaem_wx.o obj/LisaConfig.o obj/LisaConfigFrame.o obj/LisaSkin.o obj/hq3x-3x.o obj/imagewriter-wx.o src/lib/libGenerator/lib/libGenerator.a src/lib/libdc42/lib/libdc42.a -lstdc++.6 -L/usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib -framework IOKit -framework Carbon -framework Cocoa -framework AudioToolbox -framework System -framework OpenGL /usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib/libwx_osx_cocoau-3.1.a -L/opt/local/lib -lSDL2 -framework WebKit -L/opt/local/lib -lSDL2 -lwxtiff-3.1 -lwxjpeg-3.1 -lwxpng-3.1 -lwxregexu-3.1 -lwxscintilla-3.1 -lwxzlib-3.1 -framework Security -lpthread -liconv
from: /Users/aek/git/__rayarachelian/lisaem
---------------------------------------------------------------

/Users/aek/git/__rayarachelian/lisaem/waitq: Aborted due to a failed job! See /Users/aek/git/__rayarachelian/lisaem/obj/build-warnings.txt
* Exiting
make: *** [build] Error 9


--

don't quite understand the undefined symbols, or why it decided to build for 10.12
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 08, 2020, 11:11:10 am
That smells like a linking ordering issue, can you try running this?
Code: [Select]
g++ -o bin/LisaEm -lpthread -liconv obj/floppy.o obj/profile.o obj/unvars.o obj/vars.o obj/glue.o obj/fliflo_queue.o obj/cops.o obj/zilog8530.o obj/via6522.o obj/irq.o obj/mmu.o obj/rom.o obj/romless.o obj/memory.o obj/symbols.o obj/lisaem_wx.o obj/LisaConfig.o obj/LisaConfigFrame.o obj/LisaSkin.o obj/hq3x-3x.o obj/imagewriter-wx.o src/lib/libGenerator/lib/libGenerator.a src/lib/libdc42/lib/libdc42.a -lstdc++.6 -L/usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib -framework IOKit -framework Carbon -framework Cocoa -framework AudioToolbox -framework System -framework OpenGL /usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib/libwx_osx_cocoau-3.1.a -L/opt/local/lib -lSDL2 -framework WebKit -L/opt/local/lib -lSDL2 -lwxtiff-3.1 -lwxjpeg-3.1 -lwxpng-3.1 -lwxregexu-3.1 -lwxscintilla-3.1 -lwxzlib-3.1 -framework Security

and see if it links it?

If it helps, my 10.11 is 10.11.5.
xcode-select -p shows the full XCode 7.2 install, i.e.:
/Applications/Xcode.app/Contents/Developer

Which is likely where the 10.12 stuff came from, not sure why wxWidgets was linked against the later 10.12 SDK.

 gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.5.0
Thread model: posix

Just tried building it against wx3.1.2, and in obj/build-errors.txt this is the linking text I get:

Code: [Select]
---------------------- /tmp/slot.881.39.sh.done ----------------------- g++ -o bin/LisaEm obj/floppy.o obj/profile.o obj/unvars.o obj/vars.o obj/glue.o obj/fliflo_queue.o obj/cops.o obj/zilog8530.o obj/via6522.o obj/irq.o obj/mmu.o obj/rom.o obj/romless.o obj/memory.o obj/symbols.o obj/lisaem_wx.o obj/LisaConfig.o obj/LisaConfigFrame.o obj/LisaSkin.o obj/hq3x-3x.o obj/imagewriter-wx.o src/lib/libGenerator/lib/libGenerator.a src/lib/libdc42/lib/libdc42.a -lstdc++.6 -L/usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib -framework IOKit -framework Carbon -framework Cocoa -framework AudioToolbox -framework System -framework OpenGL /usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib/libwx_osx_cocoau-3.1.a -llzma -framework WebKit -lwxtiff-3.1 -lwxjpeg-3.1 -lwxpng-3.1 -lwxregexu-3.1 -lwxscintilla-3.1 -lwxzlib-3.1 -llzma -framework Security -lpthread -liconv -llzma
in: /Volumes/shared/code/lisaem-1.2.7
!!* Linked ./bin/LisaEm

Now, if I run the linking command you've pasted, I get an error saying that /opt/local/lib doesn't exist (it doesn't in my VM, which is fine). I'm guessing that somehow that snuck in somewhere and possibly one of the libraries in /opt/local/lib got included which may be causing an issue, not sure, but that might be something to look at as well. Is that from Darwin Ports? I'm using brew on this vm, but ports on older macos:

Code: [Select]
elcapitan:lisaem-1.2.7 ray$ g++ -o bin/LisaEm obj/floppy.o obj/profile.o obj/unvars.o obj/vars.o obj/glue.o obj/fliflo_queue.o obj/cops.o obj/zilog8530.o obj/via6522.o obj/irq.o obj/mmu.o obj/rom.o obj/romless.o obj/memory.o obj/symbols.o obj/lisaem_wx.o obj/LisaConfig.o obj/LisaConfigFrame.o obj/LisaSkin.o obj/hq3x-3x.o obj/imagewriter-wx.o src/lib/libGenerator/lib/libGenerator.a src/lib/libdc42/lib/libdc42.a -lstdc++.6 -L/usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib -framework IOKit -framework Carbon -framework Cocoa -framework AudioToolbox -framework System -framework OpenGL /usr/local/wx3.1.2-cocoa-x86-macOS-10.11-clang-sdl/lib/libwx_osx_cocoau-3.1.a -L/opt/local/lib -lSDL2 -framework WebKit -L/opt/local/lib -lSDL2 -lwxtiff-3.1 -lwxjpeg-3.1 -lwxpng-3.1 -lwxregexu-3.1 -lwxscintilla-3.1 -lwxzlib-3.1 -framework Security -lpthread -liconv
ld: warning: directory not found for option '-L/opt/local/lib'
ld: warning: directory not found for option '-L/opt/local/lib'
ld: library not found for -lSDL2
clang: error: linker command failed with exit code 1 (use -v to see invocation)
elcapitan:lisaem-1.2.7 ray$

If I remove /opt/local/lib from that command, instead, I get complaints about lzma, and removing  -lwxzlib-3.1 throws other errors.
Code: [Select]
Undefined symbols for architecture x86_64:
  "_lzma_code", referenced from:
      _LZMADecode in libwxtiff-3.1.a(wxtiff_tif_lzma.o)
      _LZMAPostEncode in libwxtiff-3.1.a(wxtiff_tif_lzma.o)
      _LZMAEncode in libwxtiff-3.1.a(wxtiff_tif_lzma.o)
  "_lzma_end", referenced from:
...

Not sure how to get closer to your setup. I'd say check to ensure xcode-select -p shows XCode 7.2, worst case try to rebuild wxWidgets maybe tweaking the build-wx3.1.2-macos script disabling any libs you don't need. If you run ../configure --help in the wxWidgets build directory and look for both with and without options, see what else can be disabled, or set to builtin.

The one I shipped in rc1 is linked against wx3.1.3 - you can modify the build-wx3.1.2 to build both 3.1.2 and 3.1.3 like so at line 15:
Code: [Select]
for VER in 3.1.3 3.1.2; do

Pls let me know what winds up working for you (or not).

Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 08, 2020, 12:25:26 pm
I use ports, which is no doubt where the /opt/local/.. stuff comes from

Then, I have a different tool chain for building MAME

sigh..

Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 08, 2020, 12:28:24 pm
just noticed at the start of the build

--

aek$ ./*modern*
MIN_MACOSX_VERSION: 10.12

Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 08, 2020, 01:15:17 pm
https://stackoverflow.com/questions/57734434/libiconv-or-iconv-undefined-symbol-on-mac-osx
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 08, 2020, 02:04:21 pm
Can you check that script for this snip? I think it's there since it does produce that output, but want to make sure.
It should ask XCode for what it supports. Might be it needs a slightly earlier XCode version that targets for 10.11 instead.

Code: [Select]
#!/usr/bin/env bash

OSVER="$( sw_vers -productVersion | cut -f1,2 -d'.' )"

export MIN_MACOSX_VERSION="$(  xcodebuild -showsdks 2>/dev/null | grep macosx10 | cut -d'-' -f2 | sed -e 's/sdk macosx//g' | sort -n | head -1 )"
[[ -z "$MIN_MACOSX_VERSION" ]] && export MIN_MACOSX_VERSION="$( basename $(ls -1d $(xcode-select -p )/SDKs/* | grep -i macosx10 | sort -n | head -1 ) | sed -e 's/.sdk$//g' -e 's/[Mm]ac[Oo][Ss][Xx]//g' )"
[[ -z "$MIN_MACOSX_VERSION" ]] && export MIN_MACOSX_VERSION="$OSVER"

OSVER="macOS-$( sw_vers -productVersion | cut -f1,2 -d'.' )"

echo "MIN_MACOSX_VERSION: $MIN_MACOSX_VERSION"
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 08, 2020, 03:25:58 pm

"Might be it needs a slightly earlier XCode version"

Likely. My configuration is very different

aek$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

--

I don't think they could they have made backwards development compatibility any more difficult.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 08, 2020, 07:36:59 pm
I don't think they could they have made backwards development compatibility any more difficult.

Yeah, it's on purpose, so you keep your systems up to date until they no longer support them, and then have to buy newer machines.

If you have access to older XCode from an older OS, you should be able to copy the SDK files for macos x in the same directory and use those to compile against. It mostly should work: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs

You could also rename the XCode.app file and download an older XCode and then install it and move its SDK that way, if you're a registered developer with Apple, which is a free process.

The wxWidgets configure command will accept something like -min-macosx=10.10 or similar, but has to have that SDK available.

There's a good chart here: https://en.wikipedia.org/wiki/Xcode#Version_comparison_table  that shows which XCode will run on which macos and more importantly produce proper binaries for. So you'd want XCode 7.2 or 7.3. Worse, you'll have figure out which command line tools to download as well and have that match, which is yet another annoyance to deal with.

It's possible, for example, to get a new XCode that happily runs on 10.11 but only has the 10.12 SDK and doesn't build binaries for 10.11.


In practice the binaries will most likely still work, but you'll get warnings on linking as you saw.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 09, 2020, 11:12:42 am
it would be nice if you could explicitly add which libiconv lisaem is expecting
anyone using macports is going to trip over this
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 09, 2020, 12:15:06 pm
it would be nice if you could explicitly add which libiconv lisaem is expecting
anyone using macports is going to trip over this

I'm not directly using any iconv, I'm using wxWidgets, which in turn will make calls against whatever libiconv its configure script decided to link against. That decision shouldn't be changed between compiling wxWidgets and LisaEm, but I can't control for that. That's the point of a cross platform GUI framework and statically linking it - not having to specify such things. I'm expecting a mostly vanilla os, though yes, on elder macos such as 10.5-10.7 I do use ports and on newer ones I use brew - but this is mostly for updating tools, and less for libs to be used by LisaEm.

Infact, I go out of my way to specify builtin in the wxWidgets scripts for things like png, jpg, tiff, etc. just to avoid such situations. It's not that my system doesn't have libjpg or whatever, but more I don't want have to depend on that version when distributing binaries.

Since you're building for yourself, it doesn't matter if it's static or dynamic wxWdigets, but then that copy of LisaEm can't be moved to another system that has a different setup.

In your case, I think the issue is that wxWidget's configure decided to pick up a specific libiconv that LisaEm's build script did not see or have in it's library path, causing the issue, or somehow wx-config did not pass the appropriate -L -l flags which then build.sh passed to gcc/ld. Possibly because port inserts its own paths first, etc., and I'm sure brew does things like that too.

You can fix this by forcing wxWidgets to use a specific iconv like so by changing the script/build-wx3.1.2-modern-macos.sh to pass these:

../configure --help | grep -i iconv
  --with-libiconv         use libiconv (character conversion)
  --with-libiconv-prefix=DIR  search for libiconv in DIR/include and DIR/lib

And hopefully wx-config --libs might then give the right libs and fix the issue.

I can add those in the next push/rc, but I'll first have to ensure they work on all systems - or insert logic  to detect which to use, which is painful.

There's some discussion here: https://wiki.wxwidgets.org/Distributing_WxWidgets_Applications-Distributing_WxMac_Programs - (but this is from a very old wxWidgets version in that discussion as unicode is now mandated in wx3.x, but he runs into the same kind of issues.)
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 09, 2020, 10:26:55 pm
Looks like all macosx'en have this path: /usr/lib/libiconv.2.dylib so hopefully that will do the trick.

Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 11, 2020, 01:56:40 pm
should there be a separate thread for build problems?

it built, but running the LisaEm file fails to start

aek$ lldb ./LisaEm
(lldb) target create "./LisaEm"
2020-06-11 10:55:43.216 lldb[60281:1979433] Metadata.framework [Error]: couldn't get the client port
Current executable set to './LisaEm' (x86_64).
(lldb) run
Process 60751 launched: './LisaEm' (x86_64)
Process 60751 stopped
* thread #1: tid = 0x1e35e5, 0x00007fff91c9a09e libsystem_c.dylib`flockfile + 4, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x68)
    frame #0: 0x00007fff91c9a09e libsystem_c.dylib`flockfile + 4
libsystem_c.dylib`flockfile:
->  0x7fff91c9a09e <+4>:  movq   0x68(%rdi), %rdi
    0x7fff91c9a0a2 <+8>:  addq   $0x8, %rdi
    0x7fff91c9a0a6 <+12>: popq   %rbp
    0x7fff91c9a0a7 <+13>: jmp    0x7fff91ce344a            ; symbol stub for: pthread_mutex_lock
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 11, 2020, 06:01:41 pm
should there be a separate thread for build problems?

it built, but running the LisaEm file fails to start

aek$ lldb ./LisaEm
(lldb) target create "./LisaEm"
2020-06-11 10:55:43.216 lldb[60281:1979433] Metadata.framework [Error]: couldn't get the client port
Current executable set to './LisaEm' (x86_64).
(lldb) run
Process 60751 launched: './LisaEm' (x86_64)
Process 60751 stopped
* thread #1: tid = 0x1e35e5, 0x00007fff91c9a09e libsystem_c.dylib`flockfile + 4, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x68)
    frame #0: 0x00007fff91c9a09e libsystem_c.dylib`flockfile + 4
libsystem_c.dylib`flockfile:
->  0x7fff91c9a09e <+4>:  movq   0x68(%rdi), %rdi
    0x7fff91c9a0a2 <+8>:  addq   $0x8, %rdi
    0x7fff91c9a0a6 <+12>: popq   %rbp
    0x7fff91c9a0a7 <+13>: jmp    0x7fff91ce344a            ; symbol stub for: pthread_mutex_lock


Please build it with --debug and then when this happens try to do run the list command and/or bt (backtrace) to see where it's crashing so we can find out more about what caused this crash.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 11, 2020, 07:36:19 pm
(lldb) bt
* thread #1: tid = 0x233677, 0x00007fff91c9a09e libsystem_c.dylib`flockfile + 4, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x68)
  * frame #0: 0x00007fff91c9a09e libsystem_c.dylib`flockfile + 4
    frame #1: 0x00007fff91ca3368 libsystem_c.dylib`vfprintf_l + 28
    frame #2: 0x00007fff91c9b885 libsystem_c.dylib`fprintf + 186
    frame #3: 0x000000010003330e LisaEm`LisaEmFrame::LoadImages() + 3118
    frame #4: 0x000000010004505d LisaEm`LisaEmFrame::LisaEmFrame(wxString const&) + 28445
    frame #5: 0x0000000100035484 LisaEm`LisaEmApp::OnInit() + 3396
    frame #6: 0x00000001001cbabd LisaEm`wxApp::CallOnInit() + 173
    frame #7: 0x000000010015c599 LisaEm`wxEntry(int&, wchar_t**) + 121
    frame #8: 0x000000010003415f LisaEm`main + 15
    frame #9: 0x00007fff938b35ad libdyld.dylib`start + 1
    frame #10: 0x00007fff938b35ad libdyld.dylib`start + 1


--

should everything be in the app bundle at that point?
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 11, 2020, 10:19:14 pm
(lldb) bt
* thread #1: tid = 0x233677, 0x00007fff91c9a09e libsystem_c.dylib`flockfile + 4, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x68)
  * frame #0: 0x00007fff91c9a09e libsystem_c.dylib`flockfile + 4
    frame #1: 0x00007fff91ca3368 libsystem_c.dylib`vfprintf_l + 28
    frame #2: 0x00007fff91c9b885 libsystem_c.dylib`fprintf + 186
    frame #3: 0x000000010003330e LisaEm`LisaEmFrame::LoadImages() + 3118
    frame #4: 0x000000010004505d LisaEm`LisaEmFrame::LisaEmFrame(wxString const&) + 28445
    frame #5: 0x0000000100035484 LisaEm`LisaEmApp::OnInit() + 3396
    frame #6: 0x00000001001cbabd LisaEm`wxApp::CallOnInit() + 173
    frame #7: 0x000000010015c599 LisaEm`wxEntry(int&, wchar_t**) + 121
    frame #8: 0x000000010003415f LisaEm`main + 15
    frame #9: 0x00007fff938b35ad libdyld.dylib`start + 1
    frame #10: 0x00007fff938b35ad libdyld.dylib`start + 1


--

should everything be in the app bundle at that point?

Yes, the LisaEm.app folder structure should be created, the skins and the images it's trying to load should be inside it at this point, but weirdly it's dying on fprintf, so since debug is enabled, it's trying to call ALERT_LOG which is going to say it entered LoadImages, and then what image it's loading.

Not sure what caused the segfault there, likely something was NULL, possibly there's a failure to convert the wxString to a cstring in the string concat on pngfile:
 
Code: [Select]
 
    #define LOADBITMAP(XpngX,XresnameX)                                                                  \
    { pngfile=skindir + skin.XresnameX; if (!XpngX) XpngX    = new wxBitmap(pngfile, wxBITMAP_TYPE_PNG); \
      ALERT_LOG(0,"Opening %s filename",(const char *)pngfile);                                          \
      if (!XpngX->IsOk()) EXIT(0,0,"something went wrong with loading %s",(const char *)pngfile);        \
    }                                                                                                       
It's possible skindir isn't right, or the bundle hasn't been created properly by the build.

If you do frame return (I think) it will rise up in the stack frame one level, repeating this until you get to the LoadImages method might give you more info, and you can try list to see where exactly it went wrong.

Some other things to try - I'd see about deleting your LisaEm preferences - if you've got some from 1.2.6, they may cause issues.

Another thing is to check that you see a skins directory under LisaEm.app/resources and inside is another directory called default (for default) skin, and it has a config file with details about the skin, along with the pngs and wavs:

Code: [Select]
└── LisaEm.app
    └── Contents
        ├── MacOS
        └── Resources
            ├── resources -> .
            └── skins
                └── default

Since there's questions around iconv, and iconv is involved in UTF, I wonder if it would help you if I tarred up my wxWidgets dir from my install of macos x 10.11 to make things simpler? if so try this: https://lisaem.sunder.net/downloads/wx-widgets-macosx-10.11-elcap.tar.xz (https://lisaem.sunder.net/downloads/wx-widgets-macosx-10.11-elcap.tar.xz) - it expects to be extracted to /usr/local from /

Also, when you compile with debug enabled, it will automatically run lisaem within lldb and set it to power on immediately.
I'd say also unset LD_LIBRARY_PATH and DYLD_LIBRARY_PATH before running build.sh.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 12, 2020, 12:01:45 pm
The issue is with my build of wx3.1.3

your wx3.1.3-cocoa-x86-macOS-10.11-clang-sdl works and builds with -liconv

also it creates LisaEm.app, with my build of the library, replacing -liconv with /usr/lib/libiconv.2.dylib it creates LisaEm

I will leave finding the root cause for what the mismatches in my building of the library for another day..
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 12, 2020, 01:00:23 pm
The issue is with my build of wx3.1.3
...
I will leave finding the root cause for what the mismatches in my building of the library for another day..

Yeah, I treat wxWidgets as yet another lib, and not part of LisaEm, though experience tells me wx usually needs a custom build, and OS shipped wxWidgets almost always don't work - the only exception I've seen so far is for the wx3.0 that ships with Ubuntu 19.10, but I prefer later releases as they've fixed some of their bugs and have better features.

I've had plenty of issues with wxWidgets, but for now I'm stuck with it until things get better with Qt - there's possible talk of forking it to get rid of the licensing issue, we'll see. see: https://www.phoronix.com/scan.php?page=news_item&px=More-Interest-Possible-Qt-Fork

There's not many other choices. SDL is fine for full screen games and emus, but terrible at UIs. I didn't like Qt in the past because what it created didn't look like a native app - not sure about newer ones. GTK might be a possibility - I've been told it's actually portable to Windows/macos x, but don't know that much about it.

I'm certainly not about to support 3 different GUI libraries and write native windows, cocoa and GTK UI's that would be a waste of time.

One of the (many) goals for 2.0 is to separate out the UI from the core so that alternative UIs can be used.

But for now I've got a lot of core features to fix and new ones to implement.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 12, 2020, 01:13:44 pm
Thanks for all of you efforts

Cross-platform GUI development is a nightmare

The whole point of building my own version of Lisaem is to try to get a development platform with a source debugger for the Lisa OS sources so that people could do something with them. It's been stalled for months becuase I had been waiting for your release to try gluing in a debugger, and the MAME Lisa emulation still doesn't get through self-test.

Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 12, 2020, 01:38:38 pm
Thanks for all of you efforts

Cross-platform GUI development is a nightmare

The whole point of building my own version of Lisaem is to try to get a development platform with a source debugger for the Lisa OS sources so that people could do something with them. It's been stalled for months becuase I had been waiting for your release to try gluing in a debugger, and the MAME Lisa emulation still doesn't get through self-test.



One thing to be aware of, you're going to run into issues with the LPW linker throwing either bus/address errors or throwing "Bad intrin patch" - I've not been able to solve that, but suspect it's the same as the Desktop menu bug, or the scrollbars bug. :( I'm trying to finish off 1.2.7 by the end of June so I can work on resurrecting the 1.3.0 CPU core in order to fix this and get to the other 2.0 features after July.
It's either it's own bug, or some opcode that behaves differently between the 68000 and 68040 math/shift/bit ops, or possibly some address mode - I didn't test any complex addressing modes in the 68000 tester I build on the NeXT slab because I didn't have a good way to do that yet, but I think I have an idea of how to do that next.


Is the source debugger a part of LOS or are you adding it to LisaEm itself? If it's for LisaEm itself, would you release that also?

And many thanks for working on getting the Lisa sources released!
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: stepleton on June 12, 2020, 03:51:31 pm
The whole point of building my own version of Lisaem is to try to get a development platform with a source debugger for the Lisa OS sources so that people could do something with them.

Hooray!

I can't think of what I could do to help this effort, but if there's any way to pitch in, just name it!
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 12, 2020, 04:23:55 pm

> Is the source debugger a part of LOS or are you adding it to LisaEm itself? If it's for LisaEm itself, would you release that also?

It would be for LisaEm, and I'd be releasing it.
The intent is to have a symbolic debugger that knows about LisaOS segmentation so you could see how the system works.

Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 12, 2020, 04:26:33 pm
> if there's any way to pitch in, just name it!

getting the Pascal Workshop tools running standalone under Linux would be nice, as has been done for MPW but it isn't a trivial undertaking.

Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 12, 2020, 05:22:56 pm

> Is the source debugger a part of LOS or are you adding it to LisaEm itself? If it's for LisaEm itself, would you release that also?

It would be for LisaEm, and I'd be releasing it.
The intent is to have a symbolic debugger that knows about LisaOS segmentation so you could see how the system works.


Fantastic! I look forward to it.

In terms of the CPU bug, I think I have a possible starting point. I didn't test some of the lesser used addressing modes with MOVE, and I remember having to change a few when I ported the CPU core from Generator over to LisaEm because it didn't support some at all. So those weren't tested at all. I did have some disassembly errors in some of these as well for some opcodes that were working, but disassembled incorrectly, so it's a fuzzy area where there may be more errors.

But, I think instead of creating giant tables in memory, they can be tested by using LEA, stuff like offset off PC, and such. They're possibly used as indices into tables, such as the menu list inside the Desktop menu, I suspect it's returning just the first one instead of adding an index or multiplying properly and so the same item shows up 8x instead of 8 different ones. Possibly such an opcode is used in the SANE lib on lookup tables, and certainly they'd be used to deference a specific file in a directory.

I don't think I can do the same for BRAnch/jmp/jsr, but if the addressing modes are good for LEA they should be good there too.

Going to see if I can fire up the good ole NeXT slab to test this theory. Will take weeks to test properly, but if that's what it is, it should be solvable.

I don't think they're all broken, I think they're broken under certain edge cases or else a lot more of the emulator would be broken.

Code: [Select]
|Addressing Mode|Mode|Register
|-----------------------------
|   (d16,PC)    |111 |  010   
|-----------------------------
|   (d8,PC,Xi)  |111 |  011   
|-----------------------------
|   (bd,PC,Xi)  |111 |  011   
|-----------------------------
|([bd,PC,Xi],od)|111 |  011   
|-----------------------------
|([bd,PC],Xi,od)|111 |  011   


I'm making a ton of assumptions as I write this and I might be chasing my own tail, but if they pass, and the bug isn't found at least it would indicate that I should have a lot more confidence in those addressing modes. So at least it's a starting hypothesis to test.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 12, 2020, 05:35:21 pm
> if there's any way to pitch in, just name it!

getting the Pascal Workshop tools running standalone under Linux would be nice, as has been done for MPW but it isn't a trivial undertaking.


Here's another thing I've been thinking of, I want to build a Lisa interface app, not a full tool with stationary, but something more akin to the clock or the settings panel that can sit in the background and wait for commands from the host side of LisaEm. (Think Virtual Box Guest Additions, or VMWare tools)

Al, and Tom, if you'd like to help this to move forward, here's what I propose:
What I'd need it to do is to sit in a loop and accept commands that do the following things:

1. fetch the current directory listing with full attributes for all files and details and send it over to the host side of LisaEm. (i.e. directory at a specific location)
2. open a file handle and either read the whole file over to LisaEm or write a new entire file sent by LisaEm to the file system. (It can be a block level protocol where it does one block at a time and has some handshaking to say block sent, ok, get next block, etc. now close the file handle and go back to the idle loop)
3. enumerate devices such as inserted floppies or change directories. (i.e. one floppy inserted, what's it's volume name, 3 profiles, what are their paths)
4. optional - send the Lisa Clipboard over to LisaEm, or the other way.
5. optional - launch some other process/app

I can use the F-Line HLE (High Level Emulation) interface that I already have for ROMless mode to do this, so basically if this were ran on a real Lisa it would cause it crash with an F-Line exception, but on LisaEm it would work, but perhaps we can do something else with the NOP opcode in some weird pattern, like maybe:

Code: [Select]
NOP
NOP
NOP
JMP (PC+2)
JMP (PC+2)
NOP
NOP
NOP

which would do nothing on a real Lisa, but I could detect that stream of opcodes in LisaEm and use it as an interface.

It would need to use some embedded assembly so I can put values directly in either A or D registers to pass back and forward. I'd need it to send me a pointer to a block for some operations, and then I could read/write the data in the block on the LisaEm side. I pinged David T. Craig about this a few years ago and he shared how to write embedded assembly in pascal, so it's pretty doable, but I just didn't spend cycles on this.

Might be a bit more complicated for LPW, but if that could run in the background in LPW it could be used to allow us to control the compiler/linker and use an external IDE like VSCode/codium, etc. with some scripting. I've wanted to make LisaEm scriptable for a very long time now, so this would be a great excuse for that. (So I can easily add stuff like send a set of keystrokes or move the mouse to a specific location, or detect a button and click it, etc. sort of like QuickKeys.)

So if both of these things are done we can then add some git hooks or just have a make file that initiates a bunch of actions in LisaEm to press certain keys or move some files around which could invoke the compiler or linker, etc. as the end result, but you'd be editing in a comfy modern editor which LisaEm running in the background.

If I can get some help on this, I can finish off 1.2.7 in parallel and start implementing some of these features (which I was planning for 2.0 anyway) now which can help Al.

License wise it would need to be compatible with the GPL so I can include it.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: Al Kossow on June 12, 2020, 06:20:29 pm
if you want to look at a core other than generator, the one in MAME has been pretty thouroughly tested.
there also is a 68k instruction test suite out there.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: stepleton on June 12, 2020, 09:26:55 pm
Here's another thing I've been thinking of, I want to build a Lisa interface app, not a full tool with stationary, but something more akin to the clock or the settings panel that can sit in the background and wait for commands from the host side of LisaEm. (Think Virtual Box Guest Additions, or VMWare tools)

...

License wise it would need to be compatible with the GPL so I can include it.

This is probably a bit more than I can chew at the moment (and the Linux-hosted Workshop is way out of my league, alas), but what's described here sounds like it may be able to use parts from this old project: https://github.com/stepleton-xx/lisabbs (https://github.com/stepleton-xx/lisabbs). The license is public domain, so you can do whatever you like with it. There's no inline assembly code, but there is non-inlined assembly that someone could use as a pattern.

BTW, if your NeXT isn't working, if Al's resources aren't appropriate, and if you can't use the Previous emulator to do what you need (which would be a roundabout way of comparing two 68k emulators), I have several NeXTs of my own. Conceivably I could establish a way for you to log into one of these machines remotely if you don't need the GUI.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 13, 2020, 11:21:37 am
what's described here sounds like it may be able to use parts from this old project: https://github.com/stepleton-xx/lisabbs (https://github.com/stepleton-xx/lisabbs).

Yeah, looks like you've got a lot, if not all, of the needed pieces in there to build the implant tool. I can do the rest of the stuff on the outside, such as finishing off scriptability in LisaEm and the HLE traps to communicate with it.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on June 13, 2020, 11:23:36 am
Looks like there's a bug with screenshots when it builds the list of file filters i.e. *.png, *.jpg, etc. when more than one file type is available to the build. Fixing that now. It will be a part of the next release candidate along with other fixes/features.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on July 13, 2020, 01:37:38 pm
For the last two weeks I've been vetting the LisaEm CPU core and found a minor issue in the CHK opcode, and a false positive in MOVEFUSP (really MOVE USP,d0) and the STOP opcodes. Fixing the CHK one did not fix the Desktop menu, scrollbars, or LPW linker issue.

This leads me to believe the bug either happens much earlier on in the boot process, or it's part of something outside of the CPU such as IRQ handling, timing, or I/O.

Edit: I've been using ODA as a sanity check to the libGenerator disassembler, it's not perfect, but it works and it's pretty helpful. Here's a sample with the CHK opcode, incase anyone is curious or wants to play with it: https://onlinedisassembler.com/odaweb/G6w3LNQS/0
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: D.Finni on July 14, 2020, 01:25:19 pm
For the last two weeks I've been vetting the LisaEm CPU core and found a minor issue in the CHK opcode, and a false positive in MOVEFUSP (really MOVE USP,d0) and the STOP opcodes.
Nice work finding those gremlins!
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on July 22, 2020, 09:49:07 am
Last night found another bug via CPU core testing, so before 1.2.7 betas I found out that I shouldn't be filtering out the MSB of the A regs, but I looks like I had missed some
Code: [Select]
regs_pc &=0x00ffffff; or similar filter somewhere that strips the MSB of the PC.

(Background the 68000 cannot access 32 bits of address space, so the top byte is filtered out, however, it seems LOS, like early Mac OS stores flags there. This is why the 32-bit enabler was needed for classic macos when the switch to 030 CPUs was introduced, and why machines like IIcx were not 32-bit clean, but the IIci and IIsi were.)

(forgive the weird disassembly output here)
Code: [Select]
3/a0220294 JMP.L      {PDIS:00220b26}=(PC+#$0892)

This results in PC going to 00220b26 rather than a0220b26, however the offending code isn't the JMP opcode, but somewhere else that I haven't yet located, possibly entering/leaving the emulation timing loop. Not sure what this, if anything will fix, but it is an actual bug.

Basically there's a huge JMP table in this address space used by various OS routines or perhaps libraries, and the "a0" is meaningful as this is also an A-Line trap that maps to exactly that address - it's actually a JMP anywhere trampoline. So if you execute opcode a0220b26 it will effectively through some gymnastics wind up being a JMP to the a0220b26 address, which is a relative JMP to the actual routine.

[So this is a compatibility layer that allows the OS or this library to be updated, but to keep the same ABI. A similar thing happens with the boot ROM but to a lesser level and without the A-line trap. The A-line trap also turns on the S flag, so this potentially could be used to gain supervisor access to the OS if you find a way to fool the trap to return without clearing the S-flag]
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on July 22, 2020, 11:13:52 am
Another thing I've noticed is that inside the OS, I do see it using the CLR opcode. I remember recently reading some workshop docs that says that the assembler will internally replace CLR opcodes with MOVEQ 0, but I do see CLR used while I test, and as usual it does do the read before the write, which is inefficient.

So either this portion of code was created outside of that, or was form a much earlier assembler/code generator, or the pascal compiler doesn't do that.

And yeah, the PC here should be a02206ac as mentioned above

Code: [Select]
read at 1/00f7d912(0011b512 phys).L=dce800f7
write at 1/00f7d912(0011b512 phys).L=00000000
core cache [-16+16]: 0x0022069c:6704 5340 4219 3200 e448 6700 000a 5340 4299 51c8 fffc 0801 0001 6702 4259 0801  | g$S.B92 dHg  *S.B9QH.|(! !g"BY(!
0/002206ac CLR.L      (A1)+
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: D.Finni on July 22, 2020, 09:22:14 pm
Another thing I've noticed is that inside the OS, I do see it using the CLR opcode. I remember recently reading some workshop docs that says that the assembler will internally replace CLR opcodes with MOVEQ 0, but I do see CLR used while I test, and as usual it does do the read before the write, which is inefficient.
By coincidence, I was reading those Workshop docs an hour ago, and they say the opposite: MOVEQ #0 is replaced by CLR. This optimization causes some problems when accessing hardware because CLR causes two memory accesses.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on July 23, 2020, 08:38:24 am
By coincidence, I was reading those Workshop docs an hour ago, and they say the opposite: MOVEQ #0 is replaced by CLR. This optimization causes some problems when accessing hardware because CLR causes two memory accesses.
[/quote]

Hmm, I possibly misread/misremembered it then.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on August 01, 2020, 11:00:38 pm
So it's worse than I though, this entire addressing mode is messed up, but now I've narrowed down its location.

Code: [Select]
                lisa_ww_ram(0x20000,0x33b0); lisa_ww_ram(0x20002,0x0002); lisa_ww_ram(0x20004,0x1004); test_an_opcode_a(0x30000, 0x400000,d0,d1,k);
                lisa_ww_ram(0x20000,0x47F1); lisa_ww_ram(0x20002,0x1004); test_an_opcode_a(0x30000, 0x400000,d0,d1,k); // 47F1 1004   lea  4(A1,D1.W),A3
                lisa_ww_ram(0x20000,0x47F0); lisa_ww_ram(0x20002,0x1002); test_an_opcode_a(0x30000, 0x400000,d0,d1,k); // 47F0 1002   lea  2(A0,D1.W),A3
                lisa_ww_ram(0x20000,0x47F1); lisa_ww_ram(0x20002,0x0008); test_an_opcode_a(0x30000, 0x400000,d0,d1,k); // 47F1 0008   lea  8(A1,D0.W),A3
                lisa_ww_ram(0x20000,0x47F0); lisa_ww_ram(0x20002,0x000A); test_an_opcode_a(0x30000, 0x400000,d0,d1,k); // 47F0 000A   lea  10(A0,D0.W),A3

All of these replace the D1 register with D0. It properly picks the A register as A0 or A1, but the D register is always D0.
I'd guess if I chose D7 or D2, it would show up as D0 anyway, and the disassembly is wrong too. In the 2nd code block below it should be D1 not D0

Code: [Select]

       regs D 0:00000004  1:000000cc  2:deadbeef  3:deadbeef  4:deadbeef  5:deadbeef  6:deadbeef  7:deadbeef  .S..z.. imsk:3
            A 0:00030000  1:00400000  2:deadbeef  3:0040000c  4:deadbeef  5:deadbeef  6:deadbeef  7:00040000  SP:00040000  PC:00020004  IR:47f1

 read at 1/00020012(000a0012 phys).W=00004efa
core cache [-16+16]: 0x0001fff0:ffff ffff ffff ffff ffff ffff ffff ffff 47f1 0008 1004 4e71 4e71 4e71 4e71 4e71  | ................Gq (0$NqNqNqNqNq
0/00020000 LEA.L      8(A1+D0.W),A3

ipc-opcode:47f1 used/set:00/00 wordlen:2 src:dst: 00000008 00000000 reg:0000
iib: mask/bits:f1f8/41f0 len:2 size:3 s/dtype:6/1 s/dbitpos:0 immvalue:00000009/0 cc:0
iib: priv:0 endblk:0 notzero:0 used:0 set:0, fn:0^^------------------------------------------------------------------------------------------------------------vv

Actually, yeah, if I change the opcode to use D7, it still shows up as D0. That's the bug, it's not decoding the data register portion in this address mode properly.
It's dt_Aidx in the code.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on August 02, 2020, 11:32:16 pm
bug has been fixed, linker works now, scrollbars and desktop menu are good.
Will take me a few days to rollback the pc24->pc32 changes and test if it breaks anything, the will push to github + push another RC, etc.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: jamesdenton on August 03, 2020, 07:37:31 am
Fantastic work! Congrats!
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: stepleton on August 03, 2020, 02:22:12 pm
Congratulations!! This will be huge for people who want to try making Lisa software. It's going to be a lot easier to experiment with the ToolKit now.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on August 03, 2020, 02:38:39 pm
Thanks guys. I'll clean up the source after work and start rolling back the 32 bit to 24 bit tonight, if it still works fine in 24 bit mode, I won't pursue fixing the high byte of PC anymore, though it might turn out that uniplus or xenix might be using that, I think macworks should be since it's not 32 bit clean, but the unix ones don't work due to profile/floppy emulation anyway, so I won't mess with that until after 1.2.7 production is released (likely mid-end of this month the way things are going.)

After that, going to take a break before starting on 2.0 as I've got a huge backlog of physical Lisa projects such as power supply recapping and fixing NiCAD damaged boards, etc.

But yeah, will see about picking up things to make it easier to automate using LPW without directly interacting with it as one of the early features. Some of those will likely make it into a 1.2.8 before I start bringing back the libGen core from 1.3.0 and rerunning the CPU tests against it.

Took a month to get through this batch of CPU tests, but I'm glad it worked out, and now I can reuse them on the 1.3.0 core which will add a lot more features for debugging as well, which might be useful in reversing LOS - unless ofc, AEK releases the sources before I finish that off, etc. :) But yeah, I think the odds are that AEK will be able to use 1.2.7 for his efforts before 2.0 gets shipped since there are a lot of features there, like splitting up the emulator proper from the GUI.
This should also open up other fun avenues like a maybe a QT or native macos versions or a wasm version of lisaem eventually.

Did you guys see these - they're desktop emus made from emscripten recompiles of Basillisk II (not sure about the windows one), but talk about coming full circle, from desktop emu in C to web emu and back to desktop app. :)

* https://github.com/felixrieseberg/macintosh.js (https://github.com/felixrieseberg/macintosh.js)
* https://github.com/felixrieseberg/windows95/releases (https://github.com/felixrieseberg/windows95/releases)
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: D.Finni on August 03, 2020, 03:37:12 pm
Really fantastic news! And this opens up a lot of opportunities for the future, as you well know.  ;D
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on August 03, 2020, 08:57:08 pm
So I was curious to see if fixing "The Bug" would help MacWorks, and it did, somewhat.

It Macworks started to work for a bit, I tried 3.0 and 4.5, they now both format the hard drive and install on it, *however* trying to boot from the hard drive causes a sad mac with f0004 or something like that, so there is some improvement, but not enough to get it working again. I'll revisit this later.

Booting from the boot disk after the hard drive is installed also causes the same sadmac.

Also trying to format a disk larger than 10MB causes it to run a very long time and estimate 0mins and 0 seconds so perhaps earlier versions of MacWorks don't work too well with larger drives, but that's less of a concern.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on August 04, 2020, 08:01:56 pm
I've pushed the RC2 source code up at github, I couldn't yet push the macos x binaries.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: D.Finni on August 06, 2020, 12:11:11 pm
I couldn't yet push the macos x binaries.
I couldn't wait.  ;D I back-ported your Generator changes to an older version of LisaEm and compiled it on my Intel Mac mini with OS X 10.5.

Suffice it to say, your addressing mode fix fixed a lot more than just those two or three main problems. There are several other little things that are now working correctly too.

You and this story should be on Slashdot.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: stepleton on August 06, 2020, 01:39:39 pm
I've compiled my own copy on my Linux laptop. Things went pretty well during building (I changed things to put the wxWidgets build in a subdirectory of my local repo---I am happy to work out of just this one directory and don't expect to need multiple LisaEms), but the screen updates at about a 1Hz refresh rate (regardless of the menu setting in Display). The Lisa mouse pointer does not appear to track the system mouse well---it positions itself halfway between the screen's top-left corner and the system mouse pointer location in the emulator window. Has anyone encountered this behaviour before? I hope I'm not missing a FAQ...
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on August 06, 2020, 02:53:44 pm
I couldn't yet push the macos x binaries.
I couldn't wait.  ;D I back-ported your Generator changes to an older version of LisaEm and compiled it on my Intel Mac mini with OS X 10.5.

Suffice it to say, your addressing mode fix fixed a lot more than just those two or three main problems. There are several other little things that are now working correctly too.

You and this story should be on Slashdot.

Glad it helped. So here's what happened, I finished this code on Sunday night and yelled "Woo Hoo" quietly so as not to wake the kids and went to bed as it was nearing midnight.

Monday I finished cleaning up the code and started compiling macos binarinies and I have them on the  big laptop, but didn't finish vetting them so didn't push them up.

Tuesday around 2pm the hurricane hit and lost power, and I've been without power since. I managed to copy all the stuff off to the chromebook which has galliumos on it and can run a very long time before losing the battery on the big laptop but have no power at home other than charging off the car and no internet - just wifi hotspot off the cellular network. so not sure how long this chromebook will last. right now it says it's got 8hrs left, so hope to make good use of that. i have some solar 5v power banks that we've been using to charge our cell phones off, but that's it. PSE&G says they expect to fix power for 85% of the outages by tomorrow night, so that sucks. :(

luckily when I went on vacation a couple of years before i built a step up 5v to 19.5v transformer using COTS parts off amazon so I can use the phone charger to charge up the chromebook, but it drains them very quickly. right now I've got two of those outside charging from the sun and a couple more I plugged into a car battery so hope that will do until power comes back.

the 2nd biggest challenge now is that we have to finish off all the fresh food in the fridge b4 it goes bad. luckily the house has a gas stove and water heater so we can at least cook and shower. last night was very rough, no sleep bc it was 95f in the bedroom and the noise from the neighborhood generators was also painful. was nice and cool this am so managed to get some more sleep.

I guess another pain to add to the list of how much 2020 sucks, right? we've been through these things before, and did have a lot of UPSs, but ofc they have limited run time.

the bug itself is one i introduced bc when I added support for those address modes to generator back in 2006. I forgot to add source vs destination and felt pretty lame when I saw that's what it was. so it was confusing the source operand with the destination for this move.l I posted about earlier. so that's a 13 year old bug. :)

If you see other stuff it fixed that you recall, pls report it here so I don't wind up chasing stuff that's fixed later on.

I'm on a cell phone when writing this, so it's hard for me to reply, I had to dig up the pass for my lisalist2 account so wasn't fun.

and thx for your kind words.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on August 06, 2020, 03:03:21 pm
I've compiled my own copy on my Linux laptop. Things went pretty well during building (I changed things to put the wxWidgets build in a subdirectory of my local repo---I am happy to work out of just this one directory and don't expect to need multiple LisaEms), but the screen updates at about a 1Hz refresh rate (regardless of the menu setting in Display). The Lisa mouse pointer does not appear to track the system mouse well---it positions itself halfway between the screen's top-left corner and the system mouse pointer location in the emulator window. Has anyone encountered this behaviour before? I hope I'm not missing a FAQ...

Sorry Tom, I'm sure I introduced this the last time I tried to fix macos x off the top of my head this is all in lisaem_wx.cpp, if you diff the last last two RCs when it was working vs now you should see where the changes to the wxui were. lkely in the onpaint_skin and onpaint_skinless methods. possibly a change in the type of bitmap for my_lisabitmap and related.

there were other changes around mouse mapping with zoom perhaps being that the big laptop has a 4k display the mapping between the mouse and display and zoom levels is the issue. I think I put in some tables for each OS that help scale for the display. it's likely that my 4k laptop running GTK behaves very differently than yours, I guess I'll notice it on gallium next time I compile and hopefully will fix it. I don't know if that was the right thing to do or not but it seemed to work on the 3-4 physical machines I tried the last time as well as the macos vms I used.

You can try cheating by replacing lisaem_wxui.cpp from the one that work and see if that fixes it for you for now. but there's a couple of other bugs I fixed there - I think screenshot file naming that were previously broken in the one you last tried.

also wx3.1.4 is out, and I wanted to test against it too - not sure if it will help or cause more issues.
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on August 06, 2020, 03:08:43 pm
oh, one more thing I do remember changing some stuff around refreshing because previously it was refreshing constantly like hundreds of times per second, but I don't recall where that code was or what it was. I'll try to dig, that's possibly for the refresh rate. on macos x it was blitting a blank white screen with just the mouse when LOS moved the mouse around, plus some pixels underneath the mouse, and then alternating between that and the screen refresh, but I guess on GTK it was covering that up with a full repaint way too often. so when I fixed it on macosx I broke it on GTK.

Isn't it fun how even if you have a cross platform GUI framework you still need to be picky and careful for each OS?
that's another thing for the future, what to replace wxwidgets with? or if I'm doing it wrong, how do I do it right?
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: stepleton on August 06, 2020, 06:40:59 pm
oh no! Hope you get the power back soon. Thanks for the advice---will have to try some other time if you don't fix it first!
Title: Re: LisaEm 1.2.7-RC1 support bug reports
Post by: rayarachelian on August 11, 2020, 09:11:37 pm
RC2 2020.08.03 binaries/packages for macos x is now available for download.

Skinny packages:
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.5-i386.pkg (https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.5-i386.pkg)
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.5-ppc.pkg (https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.5-ppc.pkg)
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.7-x86_64.pkg (https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.7-x86_64.pkg)
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.8-x86_64.pkg (https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.8-x86_64.pkg)
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.10-x86_64.pkg (https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.10-x86_64.pkg)
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.11-x86_64.pkg (https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.11-x86_64.pkg)
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.12-x86_64.pkg (https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.12-x86_64.pkg)
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.15-x86_64.pkg (https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-macos-10.15-x86_64.pkg)

Universal:
* https://lisaem.sunder.net/downloads/LisaEm_1.2.7-RC2_2020.08.03-macosx.dmg (https://lisaem.sunder.net/downloads/LisaEm_1.2.7-RC2_2020.08.03-macosx.dmg)

Source: (also on https://github.com/rayarachelian/lisaem/ (https://github.com/rayarachelian/lisaem/) via git)
* https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-src.tar.xz (https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC2-2020.08.03-src.tar.xz)
If you're on 10.15.6, note I repushed lisaem-1.2.7-RC2-2020.08.03-macos-10.15-x86_64.pkg but not the DMG to compiled with a newer version of XCode - the old one was causing several crashes around null IPC errors, as reported by James Denton (thanks!)
Title: Re: LisaEm 1.2.7-RC2 support bug reports
Post by: rayarachelian on August 16, 2020, 11:17:35 am
About the video/updates/mouse issues Tom reported:

At first I added back the video HZ refresh display and saw it was way off, but this is because most of the time there isn't much to update on the display except when a dialog box, or window changes or the mouse moves. I also added a checkbox to force the display update at all times which flags all video RAM as dirty, forcing a full repaint each time.

At some point it jumped from something like 2Hz to several KHz, so that wasn't right either, and it doesn't seem to care what refresh rate you select from the menu, so there's a disconnect there.

Next, I've been playing around with adding a 2nd wxTimer for video updates, which isn't turning out too well either. I've tied this to the refresh rate in the display menu, so when you change between say, 60Hz to 20Hz, it sets the timer to fire that often and then call Refresh and Update. I've found two things, even though Refresh is called, it doesn't really create an OnPaint event as often, and worse, it's pinning the CPU so badly that it prevents the pull down menus (the usual File, Edit, Display, Throttle, etc.) from coming down. I also added menu options for 120Hz and 100Hz, but it's behaving badly even at 20Hz, consuming far more CPU than available, so these are unlikely to remain.

Code: [Select]
void LisaEmFrame::OnDisplayTimer(wxTimerEvent& event)
{
    paintcount++;

    if (force_display_refresh) videoramdirty=32768;

    Refresh(false,NULL);
    if (force_display_refresh) Update();
}

The original code uses a single timer to call the emulation loop and intersperses calls to Refresh/Update. So perhaps I'm going to return to this. The issue there is that the rate of this timer is tied to the CPU clock, so I'd have to figure out some gymnastics to keep the display refresh rate true.

On the other issue of the mouse, I added a checkbox to disable the or enable the mouse scaling. The big issue is that when using wx3.1.x+ GTK on a 4K display with GTK scale set to 2.0, getting the display resolution returns 1920x1080 while earlier versions of wx3.0.x return the physical resolution of 3840x2160. However, I found there is a GetPPI call that on the 4k display returns 256x256 - not sure this is correct and certainly it's incomplete as there doesn't seem to be a method to get the physical size of the display, so you can't tell the actual physical resolution. So not sure what would happen, on say a 56" 8K TV vs a 4k 17" monitor, etc. Alternatively I'd have to write some direct GTK code to ask GTK what the physical resolution is, which I'm not sure if it will return a scaled or actual version. Meh.

I'll move the code to an older linux laptop and a chromebook to test the mouse scaling and GetPPI() call there to see what it returns there and what the mouse scaling should be.
Title: Re: LisaEm 1.2.7-RC2 support bug reports
Post by: rayarachelian on August 18, 2020, 10:35:12 pm
Looks like wx3.0.5 builds and works on macos x 10.5, so that should bring in some nice fixes.

Looks like GetPPI is ~130x166 on the chromebook and disabling the mouse scaling there works, so that's the fix for the GTK mouse issues.
Title: Re: LisaEm 1.2.7-RC2 support bug reports
Post by: D.Finni on August 20, 2020, 12:16:54 pm
I found crashes. I'm running Intel 10.5.8. There is a text file attached to this forum post with details of the crash.

Also the status line at bottom of window has a tendency to show multiple messages concatenated together.
Title: Re: LisaEm 1.2.7-RC2 support bug reports
Post by: rayarachelian on August 20, 2020, 06:22:13 pm
Interesting, James Denton reported the same kind of thing against 10.15: crash and concated status messages - turned out to be an issue with the XCode version in my VM. He was on 10.15.6, I was on a 10.15.0 in the VM with an older XCode toolchain.

I'm upgrading to wx3.0.5, and where possible 3.1.4 for the next RC3, so that might help. I'll give it a test on 10.5.8 before I publish it.
Code: [Select]

Exception Type:  EXC_BAD_ACCESS (SIGBUS)
Exception Codes: KERN_PROTECTION_FAILURE at 0x0000000000020006
Crashed Thread:  0

Thread 0 Crashed:
0   lisaem-i386-10.05-wx3.0.4        0x0009fb49 wxConfigBase::Read(wxString const&, wchar_t const*) const + 206425

Unfortunately since debug wasn't compiled in, the crash report is almost worthless. Almost, however there is a clue that it has to do something with the config file. As a work around you might try renaming the config file - both the main one and the Lisa specific one config and trying it again, there's a clean-lisaem-prefs-macosx.sh script under scripts which will delete them, but renaming them will allow you to recover if that doesn't fix it. Alternatively, you can try:

Code: [Select]
cd lisaem-1.2.7
./build.sh clean build --debug --tracelog

And see what errors/messages turn up before the crash.

Or just wait a few days for RC3 :) Speaking of, I redid the video refreshes again, previously it was relative to the CPU speed, so if you picked 60Hz, and were at 5MHz CPU speed, it would  refresh as needed upto 60Hz. Now it's relative to the host clock. There's new two checkboxes under the Display Menu, now, one to force repaints whether it's needed or not and another to disable the mouse scaling.

I might remove the force-repaint one, not sure if it's really useful, it certainly will eat a lot of host CPU cycles, which may also limit the maximum 68k CPU speed.
I've also added back the video refresh rate meter in the status bar. The "D" next to it indicates vidram was dirty on the last pass, but the rate is no longer the actual number of refreshes/second, but the attempted refreshes, so if you say 60Hz, it'll be close to 60Hz and vary +/- some timing delta based on how busy the machine is and whatever else is going on.

There's a new default preference that checks the PPI of the display as a proxy for testing hidpi/retina displays, if it's less than 200 and on GTK, it disables mouse scaling, which should fix the other issue Tom reported, of the mouse being way off, but it might be useful elsewhere. However, if you switch between hidpi and lower res displays, the setting will stick to whatever was saved in preferences...

I'm working in parallel on x/profile disk image support, so if you've taken a disk image of an X/ProFile CF card with dd you should be able to use it with a future LisaEm, but it won't be ready for RC3, though it'll likely be in the release 1.2.7.
Title: Re: LisaEm 1.2.7-RC2 support bug reports
Post by: D.Finni on August 20, 2020, 09:11:13 pm
Unfortunately since debug wasn't compiled in, the crash report is almost worthless. Almost, however there is a clue that it has to do something with the config file. As a work around you might try renaming the config file - both the main one and the Lisa specific one config and trying it again, there's a clean-lisaem-prefs-macosx.sh script under scripts which will delete them,
Yeah, I thought this might be something to do with my old config file from version 1.2.6. I will wait for RC3.

The screenshot I posted is a separate problem. That dialog box won't go away even if I click OK. I had to force quit the application.

I had the mouse problem that Tom reported too.
Title: Re: LisaEm 1.2.7-RC2 support bug reports
Post by: rayarachelian on August 20, 2020, 10:08:51 pm
I had the mouse problem that Tom reported too.

Is this on 10.5.8 or elsewhere?
Title: Re: LisaEm 1.2.7-RC2 support bug reports
Post by: D.Finni on August 21, 2020, 11:12:26 am
I had the mouse problem that Tom reported too.

Is this on 10.5.8?
Yes, on 10.5.8.
Title: Re: LisaEm 1.2.7-RC3 support bug reports
Post by: rayarachelian on August 23, 2020, 12:27:20 am
Released RC3:Skinny packages:Source:[Rather than only posting at the end of the thread for release notices, I'll update the top message as well to make it easy to find the latest release. Bug reports/replies will go to the end of the thread as usual, this way I'll avoid polluting the rest of LisaList2 with bug reports/release reports.]
Title: Re: LisaEm 1.2.7-RC3 support bug reports
Post by: D.Finni on August 23, 2020, 05:14:43 pm
Here's another crash for you. This happens on 10.12.6.

ROMless Boot -> Choose ProFile Hard Drive -> click OK
Hard Drive Size? -> Click Cancel -> crash

This sequence however does not cause a crash when using a Lisa ROM file.

 Text file attached below.
Title: Re: LisaEm 1.2.7-RC3 support bug reports
Post by: rayarachelian on August 23, 2020, 07:09:52 pm
Here's another crash for you. This happens on 10.12.6.

ROMless Boot -> Choose ProFile Hard Drive -> click OK
Hard Drive Size? -> Click Cancel -> crash

This sequence however does not cause a crash when using a Lisa ROM file.

 Text file attached below.

Great, thanks for finding this, thank you!
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on August 24, 2020, 08:46:12 pm
Released RC3a:

    https://lisaem.sunder.net/downloads/LisaEm_1.2.7-RC3a_2020.08.24-macosx.dmg

Skinny packages:

    https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC3a-2020.08.24-macos-10.10-x86_64.pkg
    https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC3a-2020.08.24-macos-10.11-x86_64.pkg
    https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC3a-2020.08.24-macos-10.12-x86_64.pkg
    https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC3a-2020.08.24-macos-10.15-x86_64.pkg
    https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC3a-2020.08.24-macos-10.5-i386.pkg
    https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC3a-2020.08.24-macos-10.5-ppc.pkg
    https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC3a-2020.08.24-macos-10.7-x86_64.pkg
    https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC3a-2020.08.24-macos-10.8-x86_64.pkg

Windows 10:

    https://lisaem.sunder.net/downloads/LisaEm-1.2.7-RC3a-2020.08.24-win64.zip

Source:

    https://lisaem.sunder.net/downloads/lisaem-1.2.7-RC3a-2020.08.24-src.tar.xz
    and github as usual: https://github.com/rayarachelian/lisaem/
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on September 17, 2020, 08:12:00 am
So been having fun implementing pty support for the serial ports. Some weird things:

0. Serial port A hangs LOS. I haven't been able to figure out exactly why, but regression testing shows this has been the case since at least LOS 1.0 beta. I've also found a bug in the fliflo (fifo+lifo library) that broke some serial access which I've fixed.

1. LisaTerm (or perhaps the OS itself) will chop off output after X characters of input, that is it will accept the first ~64 chars of input, and then discard the rest until it stops coming in and then accept more later. I haven't worked out the exact timing of these, so working on a bit of code, where if LOS is the active OS (I don't yet have a way to detect just LisaTerminal), it will slow the input down. I don't know how Xenix/UniPlus will behave, will revisit that once those OS's boot.

I tested this by running seq 100 200 - the output stops at "112" right before outputting the "2" character. Why 100-200 instead of 1 1000? Because this way, the output is exactly 5 bytes per line. The 3 number chars + cr+ new line. So this adds up to roughly 60 chars.

On another attempt I tried to output a string of text that was around 80 chars via echo and it stopped exactly at the 64th one. So that seems to be the size of the buffer. Changing the baud rate up doesn't affect this at all.

But wait! It gets worse!

2. ANSI color codes confuse it. Modern linux/macos  implementations insist on sending color, so setting TERM=vt100 is not enough. I can limit colors for things like ls and grep upto a point by setting environment variables before the execve bash, but not for everything.

As an example, ls -l itself will bold the output of some filenames right after the date/size, and LisaTerm discards the output immediately before the file name. Ditto for colorized grep.

3. Forget about using vi, it's horribly broken, the vt100 implementation in LisaTerm doesn't allow it to work properly. It can sort of work, but not quite. ed, or jed, haven't tried emacs, are your best bet.

4. I'm considering writing a small ANSI/xterm/vt100 escape sequence filter command line tool to remove color sequences and potentially any other things that break LisaTerminal by causing it to abort outputting text. It would have to do it's own fork/exec pty allocation. This way it can sit outside of LisaEm and be invoked by the user via preferences when pty is used.

I can build this into to LisaEm, but perhaps it might be more useful as a standalone unix cli filter tool.

I suppose this is analogous to ancient web browsers not understanding SSL, or modern HTML 1.x and requiring a filtering proxy in the form of a Raspberry Pi that connects to the outside world over wifi, and then presents TCP/IP traffic over SLIP/PPP for things like the MacPlus/SE or even MacWorks. :)

5. Using the default cooked output causes the pty to freeze, so the fix is to switch to raw before doing the execve, but then, the terminal is all messed up and I have to run stty sane to get echoing and other things to behave.

I'm adding back some of the options one by one of cooked/sane mode to the code to figure out which tcsetattr option is the culprit, but there's dozens of them.

I also seemed to have broken a couple of things in this effort:

1. BOOT ROM now throws an I/O error, so some of the z8530 behavior got messed up, and I'll need to fix that.

2. Mouse behavior in LOS, but not the BOOT ROM behaves as if everything is dragging, it's as if the mouse up event isn't being received even though it's sent and accepted by LOS, not sure what's going on there.

3. Still have XproFile/BLU/dc42 work to complete before RC4's release.


I think after I'm done with LisaEm 2.0, I'll work on a replacement LisaTerm program... one with XModem as well.

(https://lisaem.sunder.net/lisaem-pty.png)

(https://lisaem.sunder.net/lisaem-pty-pref.png)
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: stepleton on September 17, 2020, 02:39:48 pm
Thanks for the hard work as usual. I'm tempted to connect my own 2/10's serial port to my laptop and see what happens if I run getty. I may report back if I do...

Can you elaborate on "LOS 1.0 beta"? Do you mean a LisaEm beta, or do you have an unreleased LOS kicking around?  :)

I know that LisaTerminal wasn't available until Lisa Office System 1.2, so it's not surprising to hear that serial stuff is a bit strange in Office System <=1.0...
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: stepleton on September 17, 2020, 04:58:16 pm
Things seem to work OK for me on real hardware. The only caveat is that you have to enable XOn/XOff software handshaking under "Computer Compatibility..." or else the computer can't even keep up with 2400 bps.

With that on, though, 19200 works all right. I've set $TERM to vt100 and `ls -l` will boldface certain files, but no usage issues. vi works fine as well, although it helps to resize the LisaTerminal window just a bit. See attached photo...
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on September 18, 2020, 10:02:28 am
Things seem to work OK for me on real hardware. The only caveat is that you have to enable XOn/XOff software handshaking under "Computer Compatibility..." or else the computer can't even keep up with 2400 bps.

Thanks for trying on real hardware, Tom.
Unfortunately, even with software handshaking on in LisaTerm, it still trims after 64 chars. I'm sending a lot quicker from the pty than real hardware ever could, so now it has to be slowed down to the point where LOS/LisaTerm doesn't discard the buffer, and I haven't yet found that exact threshhold.

I suppose the rationale for 64 bytes is that since the Lisa was so slow that it can't even do 2400bps as you've noticed, that 64 bytes were enough, and this is an issue caused by running LOS on too modern a system under emulation.

With that on, though, 19200 works all right. I've set $TERM to vt100 and `ls -l` will boldface certain files, but no usage issues. vi works fine as well, although it helps to resize the LisaTerminal window just a bit. See attached photo...

Oh I see, I'm guessing you weren't connected to a linux system in that photo or had already disabled color in vi itself.
It seems that in Linux, vt100's termcap is considered color vs BSD's termcap which correctly sets vt100 as mono. see this thread: https://unix.stackexchange.com/questions/58982/disable-colours-on-terminal-and-ssh#comment81353_58992 (https://unix.stackexchange.com/questions/58982/disable-colours-on-terminal-and-ssh#comment81353_58992), and the screenshot below, as well as about grep and ls not honoring even xterm-nocolor further in down this reply.

There's some details on disabling color in various CLI tools, here as well as a proposed new variable to disable color, which various tools might|not honor: https://no-color.org/ (https://no-color.org/)

I've fixed the "stty sane" issue, now that I'm setting these termios after cfmakeraw. That fixed a lot of the issues, including the hanging on startup when cfmakeraw isn't invoked:

Code: [Select]
  new_term_settings.c_iflag |= ICRNL | IXON;
  new_term_settings.c_oflag |= OPOST;
  new_term_settings.c_lflag |= ECHO | ECHOCTL  | ECHOE | ECHOK | ECHOKE |ICANON | ISIG;
...
    setenv("TERM","vt100",1); // LisaTerm, Xenix, UniPlus aren't going to understand modern xterm-color256 here.
    setenv("NO_COLOR","nocolor",1);
    setenv("LS_COLORS","none",1); // color output breaks LisaTerm.
    setenv("GREP_OPTIONS","--color=never",1);
    setenv("LC_ALL","C",1);

I guess it depends on which vi, you're using. There's, vim, nvi, elvis, and ofc the original BSD vi written by Bill Joy (which is exceedingly rare now). Default on ubuntu is vim. Telling vim to not use color is a setting that has to be done via vimrc or command line before your launch it - So, I can't set it via a var, nor can I set aliases before doing a fork - at least I'm not aware of a way.
It's also likely that vim would work, but because it might be sending more than 64 chars on startup in a single burst, it's causing LisaTerm to discard the rest and leaving the terminal display in a bad state. I'll still have to fix that pacing issue before I'll know for sure.

In terms of disabling color, it's actually a big mess.
Setting TERM=xterm-nocolor doesn't work for everything either, for example, neither grep nor ls honor that if their default options were set to do color.
ls doesn't remove colors unless LS_COLORS is unset. and
Code: [Select]
alias grep='grep --color=auto' being the default setting for modern linux.

The bolding done by ls and grep inplace of color, when color is turned off doesn't seem to be an issue for LisaTerm. So at least that's good.

It seems the solution is to build a .bashrc/.bashprofile specific for use with LisaTerm with aliases for things like vim, grep, etc. and have the user use that with LisaEm.
Also, while GREP_OPTIONS apparently work to disable color, but grep now throws a warning saying it's deprecated  >:( , so I'll have to remove that one:
Code: [Select]
ray@predator:~/code/lisa/lisaem-1.2.7/bin$ export GREP_OPTIONS="--color=never"
ray@predator:~/code/lisa/lisaem-1.2.7/bin$ alias grep
alias grep='grep --color=auto'
ray@predator:~/code/lisa/lisaem-1.2.7/bin$ unalias grep
ray@predator:~/code/lisa/lisaem-1.2.7/bin$ grep lisa pty.log                 
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
src/lisa/io_board/z8530-pty.c:init_child:126 port:-25024 slave_orig_term_settings: c_iflag: 00000500
Perhaps the thinking, if there was any, is that it's 2020, and you're unlikely to be using a monochrome terminal, and if you are these tools will rely on your terminal ignoring color codes rather than behaving badly...

I suppose I could implement the "authentic" thing and see what baud rate is being set and pace it to that limit, that might be a good option.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: blusnowkitty on September 18, 2020, 12:06:59 pm
For what it's worth, I watched a video by curiousmarc not too long ago where they interfaced an antique Baudot TTY to a modern Linux PC with the help of an Arduino to do the vt100 to Baudot conversion, and also to act as a buffer so they wouldn't overwhelm the TTY. There was probably also stuff in there to strip modern-day colour and control codes too. Maybe that could be an option for LisaEm - buffer modern input and strip colour before it gets to the Lisa?
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on September 18, 2020, 02:23:10 pm
For what it's worth, I watched a video by curiousmarc not too long ago where they interfaced an antique Baudot TTY to a modern Linux PC with the help of an Arduino to do the vt100 to Baudot conversion, and also to act as a buffer so they wouldn't overwhelm the TTY. There was probably also stuff in there to strip modern-day colour and control codes too. Maybe that could be an option for LisaEm - buffer modern input and strip colour before it gets to the Lisa?

That's exactly the plan. It might just be the modern 256/24 bit color xterm stuff that confuses LisaTerm, but right now the biggest issue is pacing. If color is still an issue after I slow things down and find an acceptable rate to avoid tripping that 64 char limit, I'll write a small filter program to strip out color codes as well. I might also build the color stripping into LisaEm itself as that'll likely be more efficient than having to allocate a second pty, and/or provide an external program to do that also - in that thread there seems to be interest in such a tool and it might be useful for other projects too.

Maybe I'll add a checkbox to the Serial Ports to honor guest bps rate, if that's checked it should slow down I/O to whatever LisaTerm sets, but if it's not it should only slow it down whenever LOS is running, and only to whatever the maximum is before it drops characters.

So more experimentation to do to figure out what works and what's possible.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: D.Finni on September 18, 2020, 02:40:23 pm
I don't know if it's a coincidence, or if they referred to the same code, but the ROM serial drivers for classic Macintosh also default to a 64-byte input buffer. There's a device driver call that allows you to define your own buffer of arbitrary size.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: stepleton on September 18, 2020, 03:53:35 pm
Code: [Select]
tom@ostepletron:~$ uname -a
Linux ostepletron 4.15.0-115-generic #116-Ubuntu SMP Wed Aug 26 14:04:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
tom@ostepletron:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic
tom@ostepletron:~$ vi --version | head -n 1
VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Mar 18 2020 18:29:15)

Definitely connected to a Linux system, with TERM set to vt100 and no other special measures taken to disable colour in vim or elsewhere. Looks like I'm using the 3.0 version of the Office System for some reason; not sure why.

In unrelated news but talking of vintage vi, my investigations of the obscure Whitechapel MG-1 workstation (http://mg-1.uk:31132) led me to learn about its direct ancestor em (https://www.coulouris.net/cs_history/em_story/), the "editor for mortals". It doesn't say so on that page, but apparently people didn't appreciate that the command to edit a file was so close to the command to delete the file.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on September 23, 2020, 08:53:25 am
Definitely connected to a Linux system, with TERM set to vt100 and no other special measures taken to disable colour in vim or elsewhere. Looks like I'm using the 3.0 version of the Office System for some reason; not sure why.

Right you were sir, it looks like after (somewhat) fixing the buffering issues, colors don't cause issues. Tested the normal 16 color fore/background colors, 256 colors, and even 24 bit RGB colors, LisaTerm mostly ignores them properly. The only weird side effect was caused by this shade of green, which enabled blinking due to the ;5; inside it:

Code: [Select]
printf "\x1b[38;5;106m 256 green\n"

I used the values in this answer to test: https://stackoverflow.com/questions/4842424/list-of-ansi-color-escape-sequences/33206814#33206814

I also fixed the error 56 in the Boot ROM - this was caused by enabling the pty too early, which sent the bash prompt to the Boot ROM before it's test and it wasn't expecting any data on the port. I added a bit of code that split up the initialization in two passes. On power up it connects the serial ports to nothing, then about ~20 seconds (relative to the 5MHz CPU clock, so if you run at 10MHz, it's 10s, etc.) after power on it connects the real device.

However, now I ran into error 55, so even though there's nothing on serial port A, it fails the tiny test the Boot ROM does on port A. This is likely from my attempts to fix serial port A from locking up the OS/

It seems the LOS/LisaTerm 64 byte buffer can be increased by picking a higher baud rate such as 19.2kbps, however, there is still some limit and my attempts to limit the number of bytes didn't quite work out.

If I now run `seq 100 300` it gets as far as 2nd character of "202", so it's getting a lot further.

Enabling Xon/off handling sort of helped as well, but as I'm buffering internally before it gets to LisaTerm, it's not fast enough to tell bash to stop sending, so there's some more to do. It's likely vim should also work once the buffer cut off bug is fixed. I could watch the LisaTerm send Xon and then stop sending on my side rather than let Xon/off make it to the pty and bash instead.

The downside is that this requires Xon/Xoff to always be set, which is probably ok if the OS happens to be LOS as LisaTerm doesn't do hardware handshaking. (And would require another checkbox added to preferences to select it).
But it's probably better to get the timing just right.

It's possible that LOS is using the VIA timers or the actual clock instead of CPU cycles, I haven't explored that and have been limiting based on CPU clock cycles, so that's a possible route as well. Likely I'll try this next and use the wall clock rather than the CPU clock to rate limit the bytes.

The next remaining issue seems to be cursor keys don't work right. Control-A, Control-E bash editing works right, cursor left/right don't work at all. Cursor up/down send - and , respectively. These likely will turn out to be a keyboard translation issue and not a LisaTerm issue.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on October 18, 2020, 10:54:55 am
Trying to boot Xenix on LisaEm 1.2.7-RC3a setup as a 2/10 w/10MB internal widget (profile really) results in it trying to write to block 16456 and then read from the same block. After this it displays "Xenix pfcmd bad state 0306" and hangs.

Not quite sure why yet, been tracing the accesses to the profile code. I've also taken a RAM dump and captured the CPU addresses where the profile is being accessed so should be able to find and disassemble the profile driver. So either I'll find where that error comes from, or I'll find the entry points into the driver state machine and be able to intercept them,

Code: [Select]
33846686:src/storage/profile.c:do_profile_write:394:ProFile write request block #16456 0x00004048 deinterleaved:16456 0x00004048
34126893:src/storage/profile.c:ProfileLoop:1025:step6: Reading block#16456,0x004048 - buffer: 00.00.00.00(00 )[00 40 48]:0a:04 00 00 00 00 00 00| 19:34:09.2 15462333758

Looks like it's timing out at some of the states, and so it drops out of the state machine. (This isn't a complete protocol analysis, it's only showing the state transitions, not the data transfered back and forth but shows where we drop out, so I can go and look at those state transition points and look at the disassembly there.)

Code: [Select]
31336384:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:33:59.4 15455367045
31336402:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:33:59.4 15455367045
31336415:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:33:59.4 15455367073
31336433:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:33:59.4 15455367073
31336446:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:33:59.4 15455367097
31336478:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:33:59.4 15455367135
31336526:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:33:59.4 15455367187
31336545:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:1 && P->last_cmd:1 if non zero, should see state transition.| 19:33:59.4 15455367187
31336566:src/storage/profile.c:ProfileLoop:786:In state 2 - wasting while waiting for 55, got 01, last_a_accs:0 before turning BSY to 0: left:0000000000000012| 19:33:59.4 15455367219
31336602:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:0  - BSY is now 1| 19:33:59.4 15455367265
31336638:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:0  - BSY is now 1| 19:33:59.4 15455367305
31336663:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:0  - BSY is now 1| 19:33:59.4 15455367331
31336681:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:0  - BSY is now 1| 19:33:59.4 15455367331
....
31690210:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:1  - BSY is now 1| 19:34:00.7 15455866919
31690250:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:1  - BSY is now 1| 19:34:00.7 15455866977
31690290:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:1  - BSY is now 1| 19:34:00.7 15455867035
31690330:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:1  - BSY is now 1| 19:34:00.7 15455867093

31690370:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:1  - BSY is now 1| 19:34:00.7 15455867151

looks like we've timed out here waiting for 55 and lost!
31690449:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:34:00.7 15455867267
31690523:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:34:00.7 15455867357
31690541:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:34:00.7 15455867357
31694581:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:34:00.7 15455877698

...
33825423:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:34:08.2 15461543969
33832481:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:0 && P->last_cmd:1 if non zero, should see state transition.| 19:34:08.2 15461563787
33832500:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:1 && P->last_cmd:1 if non zero, should see state transition.| 19:34:08.2 15461563787
33832516:src/storage/profile.c:ProfileLoop:786:In state 2 - wasting while waiting for 55, got 01, last_a_accs:0 before turning BSY to 0: left:0000000000000016| 19:34:08.2 15461563815
33832536:src/storage/profile.c:ProfileLoop:786:In state 2 - wasting while waiting for 55, got 01, last_a_accs:0 before turning BSY to 0: left:0000000000000016| 19:34:08.2 15461563815
33832588:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:0  - BSY is now 1| 19:34:08.2 15461563903
33832710:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:0  - BSY is now 1| 19:34:08.2 15461564105
33832739:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:0  - BSY is now 1| 19:34:08.2 15461564137
33832757:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=01, last_a_accs:0  - BSY is now 1| 19:34:08.2 15461564137
33832779:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=55, last_a_accs:1  - BSY is now 1| 19:34:08.2 15461564181
33832790:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=55, last_a_accs:1  - BSY is now 1| 19:34:08.2 15461564197
33832809:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=55, last_a_accs:1  - BSY is now 1| 19:34:08.2 15461564197
33832858:src/storage/profile.c:ProfileLoop:847:In state 4 - waiting for command| 19:34:08.2 15461564279
33833025:src/storage/profile.c:ProfileLoop:847:In state 4 - waiting for command| 19:34:08.2 15461564541
33833052:src/storage/profile.c:ProfileLoop:847:In state 4 - waiting for command| 19:34:08.2 15461564587
33833077:src/storage/profile.c:ProfileLoop:847:In state 4 - waiting for command| 19:34:08.2 15461564633
33833102:src/storage/profile.c:ProfileLoop:847:In state 4 - waiting for command| 19:34:08.2 15461564679

and we miss the command here, so we start over

34125776:src/storage/profile.c:ProfileLoop:718:In state 0 now - idle - CMDLine is:1 && P->last_cmd:1 if non zero, should see state transition.| 19:34:09.2 15462332018
34125798:src/storage/profile.c:ProfileLoop:786:In state 2 - wasting while waiting for 55, got 01, last_a_accs:0 before turning BSY to 0: left:0000000000000032| 19:34:09.2 15462332018
34126030:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=55, last_a_accs:1  - BSY is now 1| 19:34:09.2 15462332400
34126049:src/storage/profile.c:ProfileLoop:804:In state 2 - waiting for 55, last PA=55, last_a_accs:1  - BSY is now 1| 19:34:09.2 15462332400
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on October 31, 2020, 05:15:57 pm
So been playing with trying to get Xenix to boot this week, I think I've discovered a routine that does the disk access, it seems to pass some variable along with the sector number, but what I don't see is a pointer to memory where the data is either written to the drive from, or read from the drive into RAM, so I'm unsure about what this actually does,  likely it's some kind of queue mechanism.

I think I also see the Interrupt Service Routine (IRQ handler) checking various function pointers, and if not-null executes them, so that was pretty interesting to see the inner workings of that.

I did find the routine that prints a single character to the Lisa's CRT as well as the routine that draws the cursor by highlighting it - similar to the LPW routine, it uses EOR #$ff to highlight a character, but the font size is different and also the console size in Xenix seems to be different. It's something like 88 chars x 33 lines. And I've also found a printf equivalent (kprintf I suppose) that calls the single char output routine. So this along with a keyboard interface can be used to turn the Xenix console into a text one and control it via telnet or pty, etc. so automation will be possible.

I'm not seeing the same timeouts, last night I tried again and manually walked through something like 4GB of tracelogs and see a successful write of the same block # above followed by a state 0306 error followed by a successful read of the same block #. So not sure where this is actually breaking. I've played with extending the various timings for the states, but that didn't seem to do anything to the pfcmd state 0306 error.

There is some interesting code that merges in a bunch of status stuff or state steps into what the 0306 is, figuring that out might help.

At some point it might be worth disassembling the full xenix kernel and playing around with it. The tracelogs provide some good info as how these functions are being called/used. Mapping the executed opcode patterns to the "/xenix" file would be a good start to figuring out the addresses of these, when disassembling, and then the values passed and what the code does when I/O happens will help map which code does what... A fully disassembled kernel might be useful for projects such as adding ppp/slip/ethernet and TCP/IP (I think UniPlus might have some support for some of that stuff, but not sure if Xenix does.) Unixen of this age usually had a monolithic kernel without separate drivers - they were compiled into the kernel.

Looking at Dr. Patrick's traces of the ProFile (been using that since I wrote the original profile.c code), which can be found here:

* http://john.ccac.rwth-aachen.de:8000/patrick/idefile.htm
* http://john.ccac.rwth-aachen.de:8000/patrick/data/profile_write_cycle.png
* http://john.ccac.rwth-aachen.de:8000/patrick/data/profile_read_cycle.png

I see all the state machine steps are being met, so this might turn out to be a VIA issue or timing, or something like that. Or perhaps going back to state zero something's not quite right or not generating the right IFR flags, not sure yet. It's hard to say because of the disconnect between the various events

No error is returned to Xenix in the status bytes, the Xenix state machine does each step as expected.

* Even though Xenix doesn't write the full command bytes, once CMD is flipped and the handshake acknowledges the command (read/write) as correct, the buffer pointer is correct, I though perhaps there was some indexing issue, but doesn't appear to be the case. The right number of bytes are written at the right places in the profile buffer despite Xenix skipping the last byte (spare count) or so for some operations.

I've also tried the patched boot disk meant for the X/ProFile, however that threw a lot more errors than the normal one. Perhaps that's worth spending time on as it throws a lot more error messages, so it likely exposes more issues, but ideally I'd like to support Xenix without having to rely on a patched OS.

Alternatively if I can suss out the full exact details for read/write sector calls, and find their buffers (I need read/write command, device, and buffer space), I can then switch to HLE and just intercept those calls instead, but that's a quick last resort. It would run much faster, but would be less authentic. (I already do this kind of thing for the floppy controller, so it's not that big a deal.) The only tricky piece would be signalling the operation is done, so would also need to patch the ISR (Interrupt Service Routine) to make it work.

I'll keep playing around with this, but if I don't make more progress in a few days, it'll go on the backburner, and revist it when I rework the widget code later, as it's delaying the 1.2.7 RC4 release. (I still need to fix the serial port pty issues and work on a few more features I want to include in 1.2.7.) If I can get Xenix booting, that would be a huge bonus, but I don't want to stretch this out too long, want to release 1.2.7 this year.


If you guys are more interested in getting Xenix working on LisaEm, than getting the serial port connectivity finished off, please let me know and I'll keep going on it. I'll continue on this through the weekend and then after Monday if I don't figure out a solution, I'll decide if I go back to other tasks or continue on this.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: D.Finni on November 01, 2020, 05:46:59 pm
I vote continue with Xenix-- you're clearly on a roll! :-)
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 04, 2020, 11:23:38 pm
Yeah, so it looks like after the write happens VIA #2, timer 1 gets fired, it causes an IRQ, and then it prints the bad state message when it sees that it's last request is completed. So either T1 shouldn't have been fired, or perhaps the emulated profile is way too fast. I'll have to write some more debug code to log where T1 is being set and for how long and what mode T1 is in and figure out whether or not it's supposed to actually fire or not.

I can't tell if it's supposed to fire or not yet, only that right before it goes into the code that checks the profile state, it looked at whether or not the Profile disk enabled line is on, and then it looked at the IFR for VIA2 saw that IFR bit 7 was on, and that the T1 fired bit was also on, and right after that it prints pfcmd bad state: 0306 via the printf call.

A quick hack might be to disable T1 after the last status byte is read (but reading from the port should have cleared the IRQ anyway, so that too might be the bug here.)

So not sure if it's set T1 what mode it set it in, and whether or not it should have fired, or if in Xenix it should have fired first to allow it to read the status rather than wait for BSY to flip.

(fixing this won't mean that Xenix will boot, there might be onion layers worth of bugs, or it might just be this one. Will find out soon enough.)
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 06, 2020, 08:16:22 am
Far as I can tell, VIA #2 Timer 1 is NEVER set anywhere in the tracelog - all the regs stay at zero - which means it should be disabled, and the cpu clock expiration for it is -1, so it should never be fired, however the counters for it say it was fired multiple times, so there's a bug somewhere in the code that calculates timing of when this was fired, and worse, the IFR flag for it isn't cleared on read.

So for sure this is a bug in the VIA 2 timer handling code, or in irq.c

Code: [Select]
1497875:via2 T1 Control: 00 Timed IRQ each time T1 Loaded, PB7 Disabled
1497904:via2 reg[4]=00 T1CL     
1497905:via2 reg[5]=00 T1CH     
1497906:via2 reg[6]=00 T1LL     
1497907:via2 reg[7]=00 T1LH     
1497931:via2 T1 cpu_68k clk expiration:ffffffffffffffff cpu68k_clocks:0000000058b2bb0b t1_fired:8 times
...
5536358:VIA2 IFR: (fired IRQ's): CA1 TIMER1
5536421:VIA2 IFR: (fired IRQ's): CA1 TIMER1
5538925:VIA2 IFR: (fired IRQ's): CA1 TIMER1
5538982:VIA2 IFR: (fired IRQ's): CA1 TIMER1
5539051:VIA2 IFR: (fired IRQ's): CA1 TIMER1
5541541:VIA2 IFR: (fired IRQ's): CA1 TIMER1
5541598:VIA2 IFR: (fired IRQ's): CA1 TIMER1
5541661:VIA2 IFR: (fired IRQ's): CA1 TIMER1
5544185:VIA2 IFR: (fired IRQ's): CA1 TIMER1
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 06, 2020, 07:45:23 pm
Yup, Xenix will be an onion of bugs, not a single layer, I fixed the IFR issue that was incorrectly tagging TIMER1 when TIMER1 was actually off, however, now it just sits at the "Entering System Maintenance" text, no shell is shown, no menu is shown, I don't think it gets far enough to run init.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: D.Finni on November 06, 2020, 07:59:56 pm
Yup, Xenix will be an onion of bugs, not a single layer
Yeah, I think we were all suspecting that would be the case, but didn't want to spoil the mood. :-P

But don't worry: the cause of all bugs is down to a single 68000 instruction. You only need to find which particular instruction is at fault, take correction, and move on to the next bug! :-)
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 06, 2020, 09:44:56 pm
Yeah, I think we were all suspecting that would be the case, but didn't want to spoil the mood. :-P

But don't worry: the cause of all bugs is down to a single 68000 instruction. You only need to find which particular instruction is at fault, take correction, and move on to the next bug! :-)

Eh, not likely to be a CPU issue, most likely something to do with the VIA or IRQ code. If there's one bug there, there's likely others.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 06, 2020, 10:00:00 pm
While testing the changes to the via, I thought I'd give UniPlus another try. I noticed it was trying to execute an illegal opcode, so rather than stopping the emulator, I modified it to allow for illegal opcodes and allow it to follow the ILLEGAL trap. Not sure if it's supposed to execute an illegal instruction or not, but thought I'd give it a shot and was rewarded with the double panic in the screenshot below.

So not much progress, but some progress is still progress. Minor win, yeay!
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: D.Finni on November 07, 2020, 12:27:49 pm
I noticed it was trying to execute an illegal opcode, so rather than stopping the emulator, I modified it to allow for illegal opcodes and allow it to follow the ILLEGAL trap. Not sure if it's supposed to execute an illegal instruction or not, but thought I'd give it a shot and was rewarded with the double panic in the screenshot below.
This shows that there's a strong "anything goes" attitude, and you should expect the 3rd party operating systems to use different mechanisms/exception vectors/etc that Apple did not use, and hence may not (yet) be fully emulated in LisaEm.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 07, 2020, 03:55:57 pm
This shows that there's a strong "anything goes" attitude, and you should expect the 3rd party operating systems to use different mechanisms/exception vectors/etc that Apple did not use, and hence may not (yet) be fully emulated in LisaEm.

Too early to say until I go through the whole 4GB tracelog, but I'm leaning towards agreeing with you here.

I don't see any TRAP calls, nor A-Line calls, but it's too early to tell, I'll know more once it gets outside of the kernel and starts executing processes. (This is true of Xenix as well, though I know TRAP #0 is used there.)

This could be some sort of obfuscation to deter disassembly, I've seen that kind of thing before with GEOS 1.3 on the C64 - well actually there it was self modifying code. (There I was able to freeze it at the right time on a C128 and use the monitor mode it had to capture all memory and then fix the self-modifying code to be able to get it to restart again after the copy protection. I wasn't able to do that with GEOS 2.0, but that's not relevant.)

This could also be (maybe more likely) some loading bug that bad data is loaded and that's not supposed to be an illegal opcode, but by accident invoking the ILLEGAL trap it is and going around whatever lockup and it causes it to at least turn the screen black and print the kernel banner. Then again, I did see the full set of bytes including this illegal opcode on the disk using lisafsh-tool, and have confirmed using ODA that is is infact not a valid 68000 opcode, so perhaps it is an intentional thing.

I did notice the boot loader complain (with tracelog on because it flashes by too quickly otherwise) that it couldn't find the superblock and that there were profile state errors.

So next step here is for me to map out the print calls and the profile calls. There's some source code for the kernel and drivers up on bitsavers so that will help a bit as we don't have that for Xenix.

I've seen Xenix do some really weird things. Now both UniPlus and Xenix are based on Bell Labs code, but different versions and different compilers, and I'm sure there was plenty of customization too. Perhaps they follow some ABI standard, that would be nice if binaries on one are compatible with the other, but it's doubtful.

Xenix does some weird things early at startup like doing a lot of 32 bit multiplies for some reason that I've not figured out why. Could be some kind of checksum or perhaps the rotor machine, but it doesn't look like a rotor cypher kind of thing, so I'm unsure what the purpose of that is - especially since James MacPhail was able to make modifications of the profile driver inside Xenix to make it compatible with the X/ProFile, and there was no mention of a checksum correction.

Between the two, Xenix has a lot more software available for it (Lyrics, Multiplan, Informix, Cobol) - it's possible these exist for UniPlus but haven't been rescued yet, I don't know, but UniPlus has some source code available and I think it's a newer OS - System V vs System 7, not sure. And at least UniPlus doesn't have the retarded way of locking the size of the profile down by the I/O ROM version as Xenix does...

But overall I'm not going to spend too much more time on this. I'll go through this 4GB log once or twice and see if I can suss out the putchar routine and maybe printf and maybe a bit of the profile handling code and see if I can map it out to the source code and take lots of notes like I did for Xenix.

After that, I'd like to fix up the open bugs in the serial code and wrap up 1.2.7 and cut a full release rather than an RC. I can revisit Xenix and UniPlus after that as I plan to revisit the widget code so that can be done together with the widgety multi-block transfer protocol it uses.

I want to get 1.2.7 out the door so I can start on the much bigger 2.0 features before the year ends. I might backport features to a 1.2.8 release while I work on 2.0, or maybe phase features from 2.0 to the 1.2.x as I work my way through the list, but there are going to be some pretty huge changes there.

I've mentioned the widget code already, but the biggest will be using shmem for all the storage including RAM, floppy, profiles and state so we can pause and save and resume states, another huge one would be resurrecting the 1.3.0 CPU core and porting all the fixes from 1.2.7 to it, and another would be separating the UI from the backend. There'll be some smaller things like adding a Daisywheel Printer emulation too. But running out of runway for 2020 and there's plenty of bugs to fix.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 07, 2020, 05:45:48 pm
Looking at the tracelog for UniPlus, looks it did run away and hit an F-Line opcode by accident, that's one of the two exceptions mentioned.
Somehow it started executing from address 0 which ofc is where the exception vectors live.
This happened immediately after an RTS opcode which means that it pulled a 00000000 off the stack and set PC to it.
Wheeee!

Hmmm, unfortunately I don't see a set of 8-10 EOR opcodes in a row, too bad, I used this to find the cursor routines in both Xenix and LPW.

I do see it set the video latch to 3b which maps video RAM to physical address 001d8000 which helps a bit, but will take more work to locate as the MMU will change this address to some other one.
I don't see a push followed by a JSR where the push contains a single character to print to the display as in Xenix, so too bad there as I have a nice grep expression to find that and could have spotted the text that's printed to the console that way.

However I did find an ADDA.W     #$005a,A5 where #$5a is 90, which is the number of bytes per video row, so that code is likely involved in writing to the display and that code is in a DBRA loop which is another strong indicator. I also see that the DBRA loop has 7 iterations, which would fit 8 lines by 8 pixels across which strongly indicates a font character being copied from a font table to the video memory. And that 7 is hardcoded via a MOVE.

Code: [Select]
268944
  268945 1/0006338a (0 0/0/0) : 7a07                       : z.       : 1064 : MOVE.L     #7,D5  SRC:clk:00000000c31b9bb5 +12 clks
  268946 D 0:00000298 1:0000001b 2:000fffff 3:0000000b 4:0000ffff 5:00000007 6:0000001b 7:0000000b .S..... imsk:7 pnd:2 (0/0/normal cx:1)SRC:
  268947 A 0:00062296 1:0006566a 2:0006d600 3:00064410 4:00064de0 5:0015a2e9 6:00068a48 7:00068a34 SP:00000000 PC:0006338c SRC:
  268948
  268949
  268950
  268951 1/0006338c (0 0/0/0) : 4ab9 0006 48b4             : J...H.   :  714 : TST.L      $000648b4  SRC:clk:00000000c31b9bc1 +20 clks
  268952 src/lisa/cpu_board/memory.c:dmem68k_fetch_long:458::::::READ LONG 00000000 '    ' from @1/000648b4 (@000648b4) using 17-ram| 11:50:03.0 3273366465
  268953 D 0:00000298 1:0000001b 2:000fffff 3:0000000b 4:0000ffff 5:00000007 6:0000001b 7:0000000b .Sz.... imsk:7 pnd:2 (0/0/normal cx:1)SRC:
  268954 A 0:00062296 1:0006566a 2:0006d600 3:00064410 4:00064de0 5:0015a2e9 6:00068a48 7:00068a34 SP:00000000 PC:00063392 SRC:
  268955
  268956 1/00063392 (0 0/0/0) : 6710                       : g.       : 1054 : BEQ.B      $000633a4  SRC:clk:00000000c31b9bd5 +8 clks
  268957 D 0:00000298 1:0000001b 2:000fffff 3:0000000b 4:0000ffff 5:00000007 6:0000001b 7:0000000b .Sz.... imsk:7 pnd:2 (0/0/normal cx:1)SRC:
  268958 A 0:00062296 1:0006566a 2:0006d600 3:00064410 4:00064de0 5:0015a2e9 6:00068a48 7:00068a34 SP:00000000 PC:000633a4 SRC:
  268959
  268960
  268961
  268962 1/000633a4 (0 0/0/0) : 1a9c                       : ..       :  248 : MOVE.B     (A4)+,(A5)  SRC:clk:00000000c31b9bdf +12 clks
  268963 src/lisa/cpu_board/memory.c:dmem68k_fetch_byte:406::::::READ BYTE 00 ' ' from @1/00064de0 (@00064de0) using 17-ram| 11:50:03.0 3273366495
  268964 src/lisa/cpu_board/memory.c:dmem68k_store_byte:484::::::WRITE BYTE 00 ' ' to @1/0015a2e9 using 17-ram| 11:50:03.0 3273366495
  268965 D 0:00000298 1:0000001b 2:000fffff 3:0000000b 4:0000ffff 5:00000007 6:0000001b 7:0000000b .Sz.... imsk:7 pnd:2 (0/0/normal cx:1)SRC:
  268966 A 0:00062296 1:0006566a 2:0006d600 3:00064410 4:00064de1 5:0015a2e9 6:00068a48 7:00068a34 SP:00000000 PC:000633a6 SRC:
  268967
  268968 cpu68k.c:cpu68k_ipc:612:dt_ImmW @ 000633a8 target:0000005a opcode:dafc| 11:50:03.0 3273366507
  268969 1/000633a6 (0 0/0/0) : dafc 005a                  : ...Z     : 1466 : ADDA.W     #$005a,A5  SRC:clk:00000000c31b9beb +12 clks
  268970 D 0:00000298 1:0000001b 2:000fffff 3:0000000b 4:0000ffff 5:00000007 6:0000001b 7:0000000b .Sz.... imsk:7 pnd:2 (0/0/normal cx:1)SRC:
  268971 A 0:00062296 1:0006566a 2:0006d600 3:00064410 4:00064de1 5:0015a343 6:00068a48 7:00068a34 SP:00000000 PC:000633aa SRC:
  268972
  268973 cpu68k.c:cpu68k_ipc:491:i_DBcc/DBRA @ 000633aa target:000633a4 opcode:51cd| 11:50:03.0 3273366519
  268974 1/000633aa (0 0/0/0) : 51cd fff8                  : Q...     : 1031 : DBRA.W     #$633a4,D5  SRC:clk:00000000c31b9bf7 +10 clks
  268975 D 0:00000298 1:0000001b 2:000fffff 3:0000000b 4:0000ffff 5:00000006 6:0000001b 7:0000000b .Sz.... imsk:7 pnd:2 (0/0/normal cx:1)SRC:
  268976 A 0:00062296 1:0006566a 2:0006d600 3:00064410 4:00064de1 5:0015a343 6:00068a48 7:00068a34 SP:00000000 PC:000633a4 SRC:
  268977

So there we go, that's part of the drawing code. Now I'll have to scroll up to see how this routine was entered and what was passed to it - the entry point should contain some ASCII character value at some point, and depending on how it was coded, might be a TRAP or a LINEA or just a JSR.

Once I have that, I can write a bit of grep-fu to locate this call and its parameters and map out what's printing at what line in the log. This can help me figure out what error messages are printed where and thus, get a hint of what the code is before those messages are.

Indeed, there's a few wrapper functions that strip off chars less than 32 and above 127 etc. before it looks up the font bytes, but eventually I find out that.... yup, it is just a JSR after all as you can tell by the values passed:

Code: [Select]
$ grep -n --before-context=4 'JSR.L      $00062d0c'  lisaem-output.001-00062140.00000000c3117f1b.txt | grep 'WRITE'
259291-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 0000000d '   N' to @1/00068aec  using 17-ram| 11:50:03.0 3273344773
260232-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 0000000a '   K' to @1/00068aec  using 17-ram| 11:50:03.0 3273346615
266728-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000073 '   s' to @1/00068a6c  using 17-ram| 11:50:03.0 3273361637
267751-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 0000006e '   n' to @1/00068a6c  using 17-ram| 11:50:03.0 3273363769
268705-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000073 '   s' to @1/00068a6c  using 17-ram| 11:50:03.0 3273365901
269681-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000074 '   t' to @1/00068a6c  using 17-ram| 11:50:03.0 3273368033
270704-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000072 '   r' to @1/00068a6c  using 17-ram| 11:50:03.0 3273370165
271658-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000061 '   a' to @1/00068a6c  using 17-ram| 11:50:03.0 3273372297
272612-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000074 '   t' to @1/00068a6c  using 17-ram| 11:50:03.0 3273374429
273650-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000065 '   e' to @1/00068a6c  using 17-ram| 11:50:03.0 3273376561
274604-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000067 '   g' to @1/00068a6c  using 17-ram| 11:50:03.0 3273378693
275558-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000079 '   y' to @1/00068a6c  using 17-ram| 11:50:03.0 3273380825
276581-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 0000003a '   :' to @1/00068a6c  using 17-ram| 11:50:03.0 3273382957
277535-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000020 '    ' to @1/00068a6c  using 17-ram| 11:50:03.0 3273385089
278489-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000062 '   b' to @1/00068a6c  using 17-ram| 11:50:03.0 3273387221
279512-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 0000006e '   n' to @1/00068a6c  using 17-ram| 11:50:03.0 3273389353
280466-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 0000003d '   =' to @1/00068a6c  using 17-ram| 11:50:03.0 3273391485
282342-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000032 '   2' to @1/00068a0c  using 17-ram| 11:50:03.0 3273396789
283414-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000035 '   5' to @1/00068a24  using 17-ram| 11:50:03.0 3273399461
284554-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000030 '   0' to @1/00068a3c  using 17-ram| 11:50:03.0 3273402133
285625-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000032 '   2' to @1/00068a54  using 17-ram| 11:50:03.0 3273404805
286615-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000020 '    ' to @1/00068a6c  using 17-ram| 11:50:03.0 3273407017
287653-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000072 '   r' to @1/00068a6c  using 17-ram| 11:50:03.0 3273409149
288607-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000065 '   e' to @1/00068a6c  using 17-ram| 11:50:03.0 3273411281
289561-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000073 '   s' to @1/00068a6c  using 17-ram| 11:50:03.0 3273413413
290584-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000069 '   i' to @1/00068a6c  using 17-ram| 11:50:03.0 3273415545
291538-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000064 '   d' to @1/00068a6c  using 17-ram| 11:50:03.0 3273417677
292492-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 0000003d '   =' to @1/00068a6c  using 17-ram| 11:50:03.0 3273419809
294243-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000035 '   5' to @1/00068a24  using 17-ram| 11:50:03.0 3273424475
295448-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000031 '   1' to @1/00068a3c  using 17-ram| 11:50:03.0 3273427147
296584-src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 00000032 '   2' to @1/00068a54  using 17-ram| 11:50:03.0 3273429819
...

So yeah, 'JSR.L      $00062d0c' is PUTCHAR, but only in the boot loader.  I'll have to continue this until I find out where the equivalent code is in the kernel.

But yeah, and that's how this kind of sausage is made. :)
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 08, 2020, 11:20:13 am
So I'm glad I went down this rabbit hole. There's either a bug in UniPlus or in LisaEm, or a much earlier bug related to setting the MMU incorrectly.

There's this code here (the line numbers, which come from vim, won't match since this is a new run:

Code: [Select]
43162051
43162052 1/00010986 (0 0/0/0) : 4eb9 0002 452c             : N...E.   :  756 : JSR.L      $0002452c  SRC:clk:000000006bc6bc8c +20 clks
43162053 src/lisa/cpu_board/memory.c:dmem68k_store_long:531::::::WRITE LONG 0001098c ' BJM' to @1/00fa07b4  using 17-ram| 20:00:06.2 1808186508
43162054 D 0:0003c000 1:00000009 2:000fffff 3:0000000b 4:0000ffff 5:00000073 6:00000010 7:000001dd .S..... imsk:7 pnd:2 (0/0/normal cx:1)SRC:
43162055 A 0:0003be00 1:00065669 2:0006d600 3:0000017c 4:00157e00 5:0003c000 6:00fa07c0 7:00fa07b4 SP:00000000 PC:0002452c SRC:
43162056

This JSR pushes the next PC to return from (0001098c) to the stack at address 1/00fa07b4. When this routine is called, it zeroes out memory. This is important because there's no write to 1/00fa07b4 during that routine at all. However, when we get to the RTS, we get:

Code: [Select]
43162412 1/000245a2 (0 0/0/0) : 4e75                       : Nu       :  749 : RTS          SRC:clk:000000006bc6be1a +20 clks
43162413 src/lisa/cpu_board/memory.c:dmem68k_fetch_long:458::::::READ LONG 00000000 '    ' from @1/00fa07b4 (@00fa07b4) using 17-ram| 20:00:06.2 1808186906
43162414 reg68k.c:reg68k_external_execute:1768:pc=0 LastPC24=00011284 pc24=00000000 abort_opcode:0| 20:00:06.2 1808186926
43162415 D 0:00000000 1:00000000 2:00000000 3:00000000 4:00000000 5:00000000 6:00000000 7:00000000 .Szx... imsk:7 pnd:2 (0/0/normal cx:1)SRC:
43162416 A 0:0003c100 1:00000000 2:00000000 3:00000000 4:00000000 5:00000000 6:00000000 7:00fa07b8 SP:00000000 PC:00000000 SRC:

So here, a zero is read from @1/00fa07b4 - and yet, nothing in the tracelog shows a write to that address which clobbered the return address and replaced it with a zero. Either something has clobbered LisaEm's Lisa RAM array, or there's some MMU segment that points to the back end physical RAM that got overwritten by the zero fill routine. I'll have to compile in some extra one-off code in LisaEm to track this down.

But I suspect this is a legitimate bug somewhere in my code that hasn't been exposed before, possibly related to the way the stack is handled in the MMU.

Edit: Yup confirmed, one of the zero out writes has a physical address that overlaps with the physical address of the stack space.
Title: Contrast bug report
Post by: D.Finni on November 10, 2020, 11:26:50 am
New bug report: high end settings of screen contrast will turn the screen yellow. This is in the Lisa Monitor system settings. I think it allows a greater range than the Workshop's contrast.

This yellow is from increase.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 10, 2020, 01:31:28 pm
New bug report: high end settings of screen contrast will turn the screen yellow. This is in the Lisa Monitor system settings. I think it allows a greater range than the Workshop's contrast.

This yellow is from increase.

Nice one! Thanks!
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: D.Finni on November 10, 2020, 04:46:06 pm
New bug report: high end settings of screen contrast will turn the screen yellow. This is in the Lisa Monitor system settings. I think it allows a greater range than the Workshop's contrast.

This yellow is from increase.

Nice one! Thanks!
Yeah, that didn't seem like correct behavior to me, so I thought I ought to report it.  :P

Also I have a feature request:

If there is a DC42 image that is longer than the DC42 header says it should be, LisaEm throws an error about this case. Could you modify DC42 to ignore the extra data, and work within the DC42 header length instead? Maybe just show a warning, like you do for a checksum mismatch, but allow the user to continue using the disk.

For example, disk images made with BLU often have $1A padding at the end because no one stripped it off before uploading to Bitsavers.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 10, 2020, 05:52:37 pm
Yeah, that didn't seem like correct behavior to me, so I thought I ought to report it.  :P

Also I have a feature request:

If there is a DC42 image that is longer than the DC42 header says it should be, LisaEm throws an error about this case. Could you modify DC42 to ignore the extra data, and work within the DC42 header length instead? Maybe just show a warning, like you do for a checksum mismatch, but allow the user to continue using the disk.

For example, disk images made with BLU often have $1A padding at the end because no one stripped it off before uploading to Bitsavers.

Sure, thanks for the reports, those are appreciated I'll fix these.

I suspect something over/underflowed from the contrast over the color values.

Yeah, the size detection change will have to be in libdc42 ofc as you've noted. I had to add a workaround for short images a year or two ago as a few images were shorter than 400k, so I guess it should be less strict the other way too.

Edit: I've made changes to libdc42 for this, hopefully it will take care of the issue, this will need testing.
working on fixing stack MMU segments now, will revisit the monitor issue after that - tried monitor last night and it immediately crashed so either the MMU changes broke it, or it's a broken image. Will retest this again when I've got the MMU working again.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: D.Finni on November 11, 2020, 04:11:18 pm
tried monitor last night and it immediately crashed so either the MMU changes broke it, or it's a broken image. Will retest this again when I've got the MMU working again.
I used the Monitor 12.3 image that's a 400K floppy. None of the Twiggy Monitors would boot. Got a hang with an hourglass cursor.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 12, 2020, 08:02:10 am
I used the Monitor 12.3 image that's a 400K floppy. None of the Twiggy Monitors would boot. Got a hang with an hourglass cursor.

Ok thanks, will retry. i was using that same one so must be the MMU changes that's breaking it.

So there's a bit of a puzzle I'm trying to solve with the MMU. The docs on the surface seem pretty forward, but they're not when it comes to the stack. I wrote a small test program that fills RAM with the same value as the address, and then I set MMU segment 125 to be a stack segment of length 32. And this should be mapped from the top. So segment 125 maps to the range 0x00fa000000-0x00fbffffff.

For normal RAM segments the space is allocated to the start, that is fa000000. But if it's a stack, it starts at 0x00fbffff and grows downward by SLR pages. each 512 bytes.

So the goal here is to get LisaEm in service mode to match what this code does on a real Lisa and then retry UniPlus and see if it gets any further.
The bug there for those who missed it was that there's a memory fill routine that because of the incorrect stack mmu segment setting overwrites it's return address, which then winds up executing from address 0. After looking at it, I saw that this was because two MMU segments pointed to the same physical space, and realized LisaEm's stack segment implementation is incorrect.

The mmu test code looks like this incase anyone's curious. It just fills memory, sets segment 125, and goes back to the boot ROM, which throws an error, likely I should have cleared D0, but the error is good because then I can hit Apple-S to go to service mode and poke around memory:

So I put this on a CF card and fed it to the floppy emu, and booted off it, and then went in service mode to poke around memory. I then modified that 720 and 600 and reran it, and no matter what I do, I always get the same window of memory that works. Changing the SOR from 600 to 400 or 500 works and I see a different set of values there. But any other addresses outside that single page cause a bus error which means the size value in SLR isn't honored at all.


Code: [Select]
00020000 Starting Address
Assembler used: ASy68K Assembler v5.15.04
Created On: Thu Nov 19 21:37:07 2020


00000000                             1  ;------------------------------------------------------------------------------------
00000000                             2  ; mmu-stack tester - tiny program to fill most of Lisa RAM with it's own address
00000000                             3  ; and then set mmu segment 125 so we can see the physical addresses and test
00000000                             4  ; access via Service Mode.
00000000                             5  ;------------------------------------------------------------------------------------
00000000                             6 
00000000                             7 
00020000                             8    ORG     $20000     ; starting address note spaces at start
00020000                             9 
00020000                            10  START:
00020000  203C 00020080             11    MOVE.L #$20080,D0  ; start ~0x40 bytes after this code.
00020006                            12  FILL:
00020006  2040                      13    MOVE.L D0,A0
00020008  2080                      14    MOVE.L D0,(A0)
0002000A  5880                      15    ADDQ.L #$04,D0
0002000C  B0B8 0110                 16    CMP.L ($110),D0   ; fill RAM until screenbase
00020010  6FF4                      17    BLE FILL
00020012                            18    ;--------------------------------------------------------------------------------------------------------------------------------
00020012                            19    ; uniplus sets mmu like this:
00020012                            20    ;
00020012                            21    ;    mmu[1][122].slr:0c00,sor:0000  00f40000-00f5ffff::-->(00000000)  type::unused:0c00
00020012                            22    ;    mmu[1][123].slr:0c00,sor:0000  00f60000-00f7ffff::-->(00000000)  type::unused:0c00
00020012                            23    ;    mmu[1][124].slr:0c00,sor:0000  00f80000-00f9ffff::-->(00000000)  type::unused:0c00 v- so is this a bug somewhere else?
00020012                            24    ;    mmu[1][125].slr:07fc,sor:05dd  00fa0000-00fbffff::-->(0003ba00)  type::rw_mem:07fc <- this is normal RAM, not stack!!!
00020012                            25    ;    mmu[1][126].slr:0900,sor:0000  00fc0000-00fdffff::-->(00000000)  type::io_spc:0900
00020012                            26    ;    mmu[1][127].slr:0f00,sor:0000  00fe0000-00ffffff::-->(00000000)  type::siospc:0f00
00020012                            27    ;
00020012                            28    ;   2x of 7d (125)=0xfa => 0x00faxxxx == sor:00fa8008 slr:00fa8000
00020012                            29    ;   400=rostack, 500=romem, 600=rwstack, 700=rwmem, 900=iospace, c00=bad page, 0f00=sio/rom
00020012                            30    ;--------------------------------------------------------------------------------------------------------------------------------
00020012  247C 00FA8008             31    MOVEA.L #$00fa8008,A2       ;MMUSADRB,A2    ;SET MMU BASE REG PTR - use fa here as fc will cause crashing in ROM  - fa8008 is SOR
00020018  267C 00FA8000             32    MOVEA.L #$00fa8000,A3       ;MMUSADRL,A3    ;SET LIMIT REG PTR                                                    - fa8000 is SLR
0002001E  303C 0300                 33    MOVE.W  #$0300,D0           ;SET BASE VALUE should be 300*200=60000
00020022  323C 0600                 34    MOVE.W  #$0600,D1           ;MEMLMT,D1      ;SET to stack, only 32 pages.
00020026  287C 00FE0084             35    MOVE.L  #$00fe0084,A4       ;return value is monitor entry
0002002C  4EF9 00FE008C             36    JMP      $00fe008C          ;WRTMMU and return to A4
00020032                            37 
00020032                            38    END START                   ;need spaces before end keyword for it to work.

No errors detected
No warnings generated


SYMBOL TABLE INFORMATION
Symbol-name         Value
-------------------------
FILL                20006
START               20000

When I went to try this on another Lisa that has the "H" ROM, to eliminate whether this is an issue with the "F" ROM not honoring SLR properly, unfortunately either the display on my floppyemu died, or the entire floppy emu died. Grrrr. So now, worst case, I'll have to either enter these bytes by hand on the keyboard or make a physical floppy... Or I can use the 2nd floppy emu that I got for a Mac SE instead...

Funnily enough I do have a few of the nokia LCD screens that it uses because when I got the floppy emu, it came with a dead display so I ordered a few of them, but I'll have to go solder the header pins on another... Meh....  :-\ ::) This is the exact reason I wrote this emulator - because I got tired of dealing with failing Lisa hardware and irony of ironies, the floppy emu I got so I don't have to deal with failing floppies also failed...

Limiting the stack to a single page and trying UniPlus with LisaEm now causes a different crash on an undocumented opcode - similar but different to the last time, which I'll investigate further as well. Hmmm, come to think of it, I should check that opcode on a real Lisa to see if it also causes a crash - maybe that's actually not supposed to crash. Wouldn't that be quite the find!

Somewhat in parallel I'm working on the xon/off (software flow control) SCC handling and mouse issues in LOS. I haven't yet attempted to fix the contrast issue.

Edit 2020.11.12 8pm:

I manually typed in the above (now slightly cleaned up code) into my H-ROM 2/10 and got exactly the same results. I confirmed that it only allocates a single 512 byte page of stack and no more. It's not the F-ROM vs H-ROM, either the above code is wrong, or the Lisa MMU can only allocate a single 1 page stack for a segment. Edit: this was caused by an error in the tester code.

Tried that illegal 4e7a opcode that UniPlus tries to execute and got an error 48 (Illegal Instruction), so not quite sure what it's for, but throwing an illegal instruction trap is the right thing to do.



Edit 2020.11.19: so going over this code again, it turns out I've swapped the pointers to SLR and SOR, and that's what it wasn't working properly. I've modified the tester code, and optimized it a bit, and tried it again, and got proper results that match what the HWG's say. However, trying the same in LisaEm resulted in physical memory underflows, which are errors in emulation, which I'll fix. So this exposed a pretty huge bug.

The good news is that it allocates the right number of pages at the right virtual addresses. The bad news is that the physical location of the RAM is wrong.

The weird thing, the SLR that was used by UniPlus is incorrect, it's 7xx, which is not a stack segment, but rather a normal RAM segment, even though it was used as a stack. So that will require a lot more investigating, That could be on purpose or caused by some other issue elsewhere, will find out after I fix this.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 23, 2020, 03:02:25 pm
Still fixing some more MMU related code, in the last week, I've written three alternate MMU handlers, two of them worked, the other did not, however with the new ones that worked, it broke LOS, so there's some more fixes to apply. One of the MMU handlers completely gets rid of the address ea caching and calculates the RAM address right off SOR on each access. The other uses math as before, but now has a lot of int32 casts around the addition code which seems to be making the big difference.

The ones that worked also fixed Monitor, after which I fixed the yellow contrast bug.

It also seems to be allowing MacWorks 3.0 to install to the hard drive. It didn't work on the first pass, but after I enabled tracelog to try to find the issue, it worked the 2nd time around - this might be some timing bug since TRACELOG slows it down to about 500KHz on my machine, but it did work and allow me to copy the System folder over to the hard drive. At least now I have a 5MB ProFile drive that has MacWorks installed on it for what it's worth.


You still have to boot off the boot floppy (just the same as with lisaem <=1.0) because it will throw error 75 when trying to boot off the hard drive. MacWorks XL 4.5 still throws a sad mac. (On a real Lisa, MacWorks 3.0 does self-boot off the hard drive without the boot floppy, so that's yet another bug somewhere, likely with ProFile/via emulation.)


One thing might be the serial number, not sure about this, but was using it with the magic-zero serial number on the first attempt and then switched to one of my Lisa's serial numbers - perhaps it cares, but not sure why it would, but I did see a message saying it read the serial number after it started booting.

One clue might is that all of a sudden after the new MMU fix, the vectors I use to detect the OS in glue.c have moved for MacWorks and Monitor.
Perhaps these new address changes have something to do with LOS failing. LOS now crashes by executing code that doesn't exist - that is it branches or JMPs or something to a region where no code has been loaded which causes the emulator to bail since there's no basic block for it to work on.

But obviously Monitor crashing on boot was a bug that got fixed so both somewhat closer, and further away. UniPlus behaves exactly the same as before, with a double trap panic, etc. however Xenix is now crashing in the same way that LOS crashes, so that's another clue that the MMU isn't quite there yet.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 24, 2020, 06:09:04 pm
I've fixed LOS from crashing, nothing else has changed/improved than MacWorks 3.0. Going back to the pty issues and mouse click-release issues.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on November 27, 2020, 12:03:28 pm
Managed to fix the pty code last nigh, xon/xoff worked well, the output isn't getting cut off anymore.
Serial port A is still broken, however, looking at it now.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on December 09, 2020, 12:20:11 pm
Didn't manage to fix serial port A, so disabled it instead. Added host serial port support (i.e. USB to serial adapters: /dev/ttyUSB0, or wired serial ports /dev/ttyS0 - names on macos and FreeBSD will vary.).

On Linux you might not have permissions to your USB serial port, so you either have to add yourself to the right group (dialout) and then logout and log back in, or use sudo chmod o+rw on /dev/ttyUSB0 (or whatever) as the device will be owned by root and you won't be able to open it.

The Serial code in LisaEm only connects at "power on" - if for example you choose Shell (PTY) and then type in exit, you won't be able to use that connection until you tell Lisa Office System to power off and then power back on. Perhaps in future releases I'll add a way to reconnect or even switch the backend device. But for now this will suffice as a lame, but somewhat acceptably usable thing.

I've not tested this on macos yet. I've mostly used POSIX methods of opening the pty and serial port, so in theory it should work, testing may reveal it will not. Will find out soon enough.  ;D
This will be part of 1.2.7-RC4 release.

As you might have noticed, I've renamed some of the options, so PTY is now "Shell" as that's more human friendly, and the parameter would be either nothing, or something like /bin/bash or /bin/zsh.
For Serial, meaning host physial serial port, you'll need to figure out the device name. If it's a USB serial port, it will create a new character device in /dev with the time/date of the insertion, so you can use ls -Flatr /dev/tty* to find it, or perhaps the output of dmesg:
Code: [Select]

$ dmesg -T

...[Wed Dec  9 12:26:22 2020] keyspan_pda 1-1:1.0: Xircom / Entrega PGS - (prerenumeration) converter detected
[Wed Dec  9 12:26:22 2020] usb 1-1: USB disconnect, device number 6
[Wed Dec  9 12:26:22 2020] keyspan_pda 1-1:1.0: device disconnected
[Wed Dec  9 12:26:24 2020] usb 1-1: new full-speed USB device number 7 using xhci_hcd
[Wed Dec  9 12:26:24 2020] usb 1-1: New USB device found, idVendor=06cd, idProduct=0104, bcdDevice=ab.89
[Wed Dec  9 12:26:24 2020] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Wed Dec  9 12:26:24 2020] usb 1-1: Product: ACME USB serial widget
[Wed Dec  9 12:26:24 2020] usb 1-1: Manufacturer: ACME usb widgets
[Wed Dec  9 12:26:24 2020] usb 1-1: SerialNumber: 47
[Wed Dec  9 12:26:24 2020] keyspan_pda 1-1:1.0: Keyspan PDA converter detected
[Wed Dec  9 12:26:24 2020] usb 1-1: Keyspan PDA converter now attached to ttyUSB0  <<<--- here it is

$ ls -Flatr /dev/tty*

....

crw-rw-rw-   1 root dialout   188,   0 Dec  9 11:37 /dev/ttyUSB0 <<-- and here.

....


As you can see, the device is owned by root:dialout, so adding your user to the dialout group, followed by logging out and logging back in should get you access to the port, if not, or you don't feel like logging out, you can sudo chown it or sudo chmod it as needed.
The preferences string for the port I used is "/dev/ttyUSB0|9600,N,8,1" - the first part is the device name. This is absolutely required or you won't be able to use the port.

The | character is a separator to the parameters. If you omit it, LisaTerminal will be able to set the baud rate, number of bits, parity, and stop bits.
If you provide it, you can lock the port to the settings you've provided, and no matter what's set in Lisa Terminal the port will remain at that speed/settings.
You can also add ",hw" to enable hardware handshaking, or ",-hw" to disable it. ",xon" (as well as checking the Xon/Off checkbox will enable software handshaking on the Linux side of the port. ",-xon" will force disable software handshaking on the Linux side. Note that Lisa Terminal will still send ^S/^Q characters to signal xoff/xon through the port.

For LOS and LisaTerminal it's pretty much required to use Xon/Xoff software handshaking to prevent data loss, even though I've built in some code to detect LOS and slow down the serial port pacing, but since LisaTerminal lacks file transfers, there's not much disadvantage to enabling software handshaking.

Since Serial A is broken, it's now set to Nothing, and all the options are disabled. Sadly this will remain until I can get it to work such as LOS doesn't hang when Serial A is accessed. There's some bug in the z8530 code somewhere that prevents it from working properly.

Neither the PTY(Shell), nor the Serial port code will work on Windows. I'll eventually write Windows specific equivalent code to allow it to work there, but likely this won't be in 1.2.7.

Most likely I'll release 1.2.7 final before the end of this year, and then start on 2.0. I'll back fill some of the features into a 1.2.8 and possibly 1.2.9 - i.e. the widget emulation, and debugger features.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: D.Finni on December 09, 2020, 04:25:43 pm
Since Serial A is broken, it's now set to Nothing, and all the options are disabled. Sadly this will remain until I can get it to work such as LOS doesn't hang when Serial A is accessed. There's some bug in the z8530 code somewhere that prevents it from working properly.
Isn't this the port whose baud rate is limited to 9600? Does that have something to do with the problem? Earlier this summer we discovered that one port can go up to 57600, but the other is limited to only 9600.


Also, have you committed to Github recently? I'm trying to checkout the latest source, but I don't see anything new.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: blusnowkitty on December 09, 2020, 11:54:41 pm
There any way you could change the Double X/Triple Y display scaler to pull up a bypassable warning instead of a hard stop? Just a minor thing really that's kinda bothered me since one of my monitors is only 100 pixels short; let me play with it anyway darn it :P
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on December 10, 2020, 10:14:35 am
Isn't this the port whose baud rate is limited to 9600? Does that have something to do with the problem? Earlier this summer we discovered that one port can go up to 57600, but the other is limited to only 9600.

That's unrelated and has to do with the design of the I/O board and which clock is presented to which z8530 timing pin. That said, it is possible for Serial A on actual Lisas to go higher than 9600 with MacWorks (without the PFG), and it's possible to user Serial B for LocalTalk (230KBPs).

The weird thing is that early on, LisaEm passed the LisaTest of the serial ports where a cable that takes data from port A and sends it to port B and vice versa is used. So it's something in the way LOS uses it that the z8530 emulation isn't quite providing. I've tried older LisaEm's all the way down to 1.0 on macos x and they all lock up when LOS accesses Serial port A.

I'll have to retry LisaTest again to see if it still works or if it throws an error and see if I can get a clue of what's going wrong with port A.

Also, have you committed to Github recently? I'm trying to checkout the latest source, but I don't see anything new.

Nope. I only push to github when there's a release and things are stable. I'm working on adding yet another feature, so depending on how that goes the next one may be as early as next weekend, or much later.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on December 10, 2020, 10:16:48 am
There any way you could change the Double X/Triple Y display scaler to pull up a bypassable warning instead of a hard stop? Just a minor thing really that's kinda bothered me since one of my monitors is only 100 pixels short; let me play with it anyway darn it :P

Eh, I suppose I could, but then you might get stuck in that mode without a way to function properly. 100 pixels would be enough to cut off the bottom of the screen where the icons get minimized to, or the top of the screen where the menu bar is, so do you think you'd enjoy scrolling up and down of almost every operation like that? It might be fun for about 5 minutes and then will be annoying.

edit: if a window shows up too high (or too large in Lisa), you could install https://www.spectacleapp.com/ on OS if you get stuck, but sadly this isn't being supported anymore. That said it seems to work all the way up to big sur.
Ubuntu seems to implement windows key + left|right to resize the window. I'd rather not disable these, but if you're hell bent on it you could disable those dialogs yourself.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: Al Kossow on December 22, 2020, 03:49:51 pm
FYI, someone just mentioned SDL 2.0.14 has been released
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on December 22, 2020, 05:41:27 pm
FYI, someone just mentioned SDL 2.0.14 has been released

LisaEm uses wxWidgets. SDL is only used to provide sound because of the sound class requires SDL:

from: https://docs.wxwidgets.org/3.0/classwx_sound.html
Code: [Select]
Currently this class is implemented on Windows and Unix (and uses either Open Sound System or Simple DirectMedia Layer).

I'll look at add the new SDL to new macos releases.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on December 22, 2020, 05:50:40 pm
Incase folks are curious, I'm adding the https://github.com/jeremysalwen/TerminalWx.git class to LisaEm, it will be used for various purposes including having a terminal that can be used with the emulated serial ports. Once I get it working with that, I'll also add xmodem and support for an actual serial port so  you could use LisaEm to directly talk to BLU so you can image your ProFile/Widget.

Once Xenix or UniPlus start up, it could also be used as an alternative console so you can do copy/paste, etc. and eventually for LPW.

But right this minute the goal is to add it as another endpoint for Serial B.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on January 18, 2021, 12:06:19 pm
I've implemented most of the TerminalWx (which is a vt100 terminal emulator window for wxWidgets) and have gotten it to compile, but the Edit Menu doesn't do much, I couldn't figure out how to enable drag select text on it.

However I was able to get text in/out of Serial B to the terminal window. Setting Serial B to "Terminal" will open a new window with the TerminalWx widget, and then whatever you type in it will show up in LisaTerminal, and vice versa. This could be useful in the future for interacting with the Lisa unixens.

Currently implementing file transfer protocols (upload/download text and xmodem.) The idea here is to attach this to a physical USB serial port, and then you could connect that to a physical Lisa and transfer disk images to/from BLU. I suppose we could build an xmodem command line for the unixens as well.

Once this works and is debugged, I'll release it as RC4 and within a few weeks after testing the 1.2.7 RELEASE. I intended to release this last year, but working in C++ is much slower.

Future plans after this are code freeze for 1.2.7 except bug fixes, which may trigger versions such as 1.2.7.1 1.2.7.2, as needed, etc. The left over stuff I intended for 1.2.7 such as Widget emulation and updates to libdc42, and possibly the debugger, will make it to 1.2.8, but once 1.2.7 is done.

Once 1.2.7 RELEASE is done, I'll be starting work on the 2.0 features, in parallel with 1.2.8, starting with revisiting the 1.3.0 CPU core (which I can test via automation now), then UI vs core separation. This is a much longer term project that could be a year or much more.

Most of the new serial port features (physical port access, shell, BLU interface) won't be available for Windows 10 in 1.2.7 since it has different ways of doing things than POSIX style systems, but I'll likely add them to 1.2.8.

At least that's the plan at the moment.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: compu_85 on February 08, 2021, 04:46:40 pm
.... Added host serial port support ...

Sweet! I'll have to try it with the daisy wheel and Imagewriter!

-J
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: stepleton on February 20, 2021, 09:07:18 am
(I've been using the last Windows release I've been able to get my hands on, which is RC3a-2020.08.24. So this may be old news.)

As a feature request (or maybe just advice on how to use a feature that's already there): is there a way to disable the emulator's requirement that there be boot media, and that the boot media look bootable?

I've got a copy of the ROM and I'd like to boot into the ROM now and then, in order to get into service mode and poke around, even if I don't have a disk attached anywhere. It's useful for experimenting with short machine code snippets.

Also, for some of my homebrew operating environments (which boot off of file images that use my bootloader_hd bootloader (https://github.com/stepleton/bootloader_hd/blob/main/bootloader_hd.x68#L239)), the emulator doesn't think the drive is bootable, even though I mark the first tag with $AAAA in the usual spot.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on February 20, 2021, 11:29:45 am
(I've been using the last Windows release I've been able to get my hands on, which is RC3a-2020.08.24. So this may be old news.)

As a feature request (or maybe just advice on how to use a feature that's already there): is there a way to disable the emulator's requirement that there be boot media, and that the boot media look bootable?

Nope that's not been addressed. I'm pretty sure I put those guards in on purpose, but they should only be for ROMless. Can you clarify a bit? Where are you seeing this error message and for what? The ProFile, or a floppy?
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on February 20, 2021, 12:12:44 pm
So it's been a while, and I might as well mention some status bits.

Newly discovered bugs in lisafsh-tool:

Yesterday was talking to James Denton and he noticed that LisaFSH tool seems to chop off the first few bytes off a file - I suspect this is because they're in the meta data block. i.e. the metadata is the first 0x34 bytes and the rest is the actual start of the file, or something like that. (This is similar to unixen file systems where if your file data is small enough it's included in a portion of the inode, but in this case it's not in the extent file, but rather in the metadata block.)

There's another bug in LisaFSH tool where it doesn't see all the directory blocks (I think I just fixed this one, but need to test it some more.) It will still extract all the files, but names will be file-xxxx instead, and doing dir will not show the whole disk.

There's a few other old issues such as the extracted files have junk at the end because when I wrote this, I didn't know the exact file size, though I'm pretty sure Natalia Portillo has discovered the size fields and I need to add those to the code.

Bigger problems with LisaEm itself:

But there's a bigger issue that's been blocking me for about two months now. I've mostly implemented XModem code for the TerminalWx code bit. (Unfortunately I should have either pushed an RC before this or opened a branch, so now it's kinda difficult to segregate this code and do an RC release without finishing it.)

Potentially the Lisa has a maximum need for something like 16 terminal consoles. (2 for Serial A, B, 3x expansion slots, each with the possibility of 4 serial ports for the Quad Tecmar Serial Port Card or its modern Sapient clone, one more for interfacing with BLU and yet another as a console for Xenix/UniPlus/LPW.)

So I implemented the TerminalWx as a pair of 16-member arrays that use null pointers (one set for the TerminalWx window, the other for the wxFrame). I won't actually open 16 windows, but rather as one is needed a terminal is constructed at these pointers are populated with a pointer to the relative object.

Event handling:

But this causes an issue for the event handling. Events in wxWidgets are handed via macros like so:

Code: [Select]
wxBEGIN_EVENT_TABLE(TerminalWxFrame, wxTerm)
EVT_TERMINAL_INPUT(TerminalWx::OnTerminalInput)
EVT_MENU(wxID_CUT, wxTextCtrl::OnCut)
EVT_MENU(wxID_COPY, wxTextCtrl::OnCopy)
EVT_MENU(wxID_PASTE, OnPaste)
EVT_MENU(wxID_SELECTALL, wxTerm::SelectAll)   
EVT_MENU(ID_FileCapture,OnFileCapture)
EVT_MENU(ID_TextUpload, OnTextUpload)
EVT_MENU(ID_XMODEM_UP,OnXmodemUpload)
EVT_MENU(ID_XMODEM_DN,OnXmodemDownload)
EVT_TIMER(ID_Timer,OnTimer)
wxEND_EVENT_TABLE()

So how do I tell it which of the 16 window frame event handler is to be registered? I think I might be able to extract a window ID from the event and use that as a sector, and call back the member from one of the objects in the array, but not 100% sure yet.

That's one issue.


OnXModemUpload member function is defined, but not recognized:

But this more immediate compile time error one is very maddening. It's saying it doesn't have a member function OnXmodemUpload even through it's both in the header where the class is defined and in the cpp file where it's implemented! I've even copypasta'ed the member name, and it still throws the same error. Anyone seen anything like this? What's the fix?

In the .h file I have
Code: [Select]
class TerminalWxFrame: public wxFrame
{
    public:
        TerminalWxFrame(wxWindow* parent,wxWindowID id = -1, int port=-1);
        virtual ~TerminalWxFrame();
...
        void OnFileCapture(wxCommandEvent& event);
        void XModemSendBlock(void);
        void XModemReceiveBlock(void);
        void OnTextUpload(wxCommandEvent& event);
        void OnXmodemUpload(wxCommandEvent& event);
        void OnXModemDownload(wxCommandEvent& event);
...
}

And in the .cpp file I have the actual implementation of the OnXModemUpload method (I used to have the unused macro there but removing it didn't help):
Code: [Select]
void TerminalWxFrame::OnXModemUpload(wxCommandEvent& event) //WXUNUSED(event))
{
        if (xferproto) {wxMessageBox(_("Cannot start XModem upload as another file transfer operation is in progress"),_("Cannot Upload now"), wxICON_INFORMATION | wxOK); return;}

        wxString openfile;
        wxFileDialog open(this, wxT("Upload File with XModem:"), wxEmptyString, wxEmptyString,
                                "BLU files (*.dc42;*.blu;*.bin)|All (*.*)|*.*)",
                                (long int)wxFD_OPEN,wxDefaultPosition);

        if  (open.ShowModal()==wxID_OK)  {
            wxString filename=openfile.GetPath();
            upload=fopen(CSTR(filename),"rb");
            if (!upload) {wxMessageBox(_("Could not open file for upload"),_("Cannot Upload"), wxICON_INFORMATION | wxOK); return;}
...


However, gcc (gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0) is throwing this, and I've no idea how to go about fixing it.

Code: [Select]
src/host/wxui/z8530-terminal.cpp: At global scope:
src/host/wxui/z8530-terminal.cpp:652:6: error: no declaration matches 'void TerminalWxFrame::OnXModemUpload(wxCommandEvent&)'
  652 | void TerminalWxFrame::OnXModemUpload(wxCommandEvent& event) //WXUNUSED(event))
      |      ^~~~~~~~~~~~~~~
src/host/wxui/z8530-terminal.cpp:652:6: note: no functions named 'void TerminalWxFrame::OnXModemUpload(wxCommandEvent&)'
In file included from src/host/wxui/z8530-terminal.cpp:81:
src/host/wxui/include/terminalwx_frame.h:11:7: note: 'class TerminalWxFrame' defined here
   11 | class TerminalWxFrame: public wxFrame

So... uh, wtf's going on there?
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: stepleton on February 20, 2021, 01:41:47 pm
Nope that's not been addressed. I'm pretty sure I put those guards in on purpose, but they should only be for ROMless. Can you clarify a bit? Where are you seeing this error message and for what? The ProFile, or a floppy?

Hey, sure... I've got a Windows box with a pretty conventional install of RC3a-2020.08.24. It's configured to look for the H boot ROM in a particular location, to pretend to be a 2/5, and also to have a ProFile .dc42 image hanging off of the parallel port. I've attached my lisaem.conf to this email.

Now, the ProFile .dc42 image itself is homebrew, but I know it's good (for booting at least) because LisaEm 1.2.6.2 over on my Linux box will boot it. I made the image by using the drive image generator script (https://github.com/stepleton/bootloader_hd/blob/main/build_bootable_disk_image.py) that comes with my hard drive bootloader. (My Lisas can boot the drive image too.) The drive image puts $AAAA in the right place inside the first sector's tag, so I don't know why the check in romless.c (https://github.com/rayarachelian/lisaem/blob/master/src/lisa/cpu_board/romless.c#L549) doesn't like it.

Precise steps:
1. Start up LisaEm
2. Select "Power Button" under "Key"
3. Lisa pops up a "ROMless Boot" dialogue asking for a boot device (even though I have a ROM file listed in the config), and I select the ProFile on the parallel port
4. A new alert box says "No bootable OS found" etc., and after that, an alert saying that the power-on failed, and that's the end of the line

I can't share the drive image I'm using just yet (hopefully soon!), but I can share the first two blocks of it, which is just the bootloader. I've attached a hexdump of the raw image data (not the .dc42), so 2*532 bytes, with the tags in the first 20 bytes of each 532-byte half.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on February 20, 2021, 02:20:48 pm
Hey, sure... I've got a Windows box with a pretty conventional install of RC3a-2020.08.24. It's configured to look for the H boot ROM in a particular location, to pretend to be a 2/5, and also to have a ProFile .dc42 image hanging off of the parallel port. I've attached my lisaem.conf to this email.

The most likely issue is that if it went to the romless code, it means the path to the ROM isn't being recognized somehow, so it should never go there - it's possible this is a logic bug, and I'll look through the code to see if that's the case, but it shouldn't ever complain about the ProFile from there if the ROM exists.

Now, if you have a recognized ROM, and the storage file for ProFile doesn't exist, it will prompt you to create it. But if it exists, and even if it's not bootable, it shouldn't care at all and it will turn on.

I might be wrong and I promise I will look, but from what's in L1 cache (i.e. wetware brain memory cells), most likely something's not right about the path to that ROM.

In terms of the ROM, if it's a split ROM dump it should rename it and stitch it back together into a single file.
I see the filename for your ROM is 341-0175-H.BIN - but I suspect this is just the "low" side of the ROM, if so, it's not going to recognize the high part and therefore won't do the right thing. I see bitsavers has ROMs with this convention. It also would need a matching 341-0176-H.BIN for the high part.

So the solution for now would be to rename those files to something like "boot-h.hi" and "boot-h.lo" as in here: https://github.com/rayarachelian/lisaem/blob/09afe9e998752d4458d8c9d1fdab2c562a4c628e/src/lisa/cpu_board/rom.c#L421 - the critical thing is the extension should end with ".lo" and ".hi" and then it will derive the name from that and merge them together into boot-h.bin.

(Bitsavers didn't exist when I wrote this code, so there's no handler for -0175/176-X and I don't think the CPU boards I had had the part number on them - or maybe they did but I didn't notice.) :)

I'll make a note of this and add some code for this case.

I can't share the drive image I'm using just yet (hopefully soon!), but I can share the first two blocks of it, which is just the bootloader. I've attached a hexdump of the raw image data (not the .dc42), so 2*532 bytes, with the tags in the first 20 bytes of each 532-byte half.

Yeah, no worries, I understand not wanting to release vaporware/unstable things, and also the need to secrecy. ;)
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: stepleton on February 20, 2021, 02:38:22 pm
D'oh, you called it! I just had half of a ROM sitting there in that file. No wonder! It works fine with the ROM from my Linux installation. Sorry for wasting your time!

(using it now) This is brilliant! It's nice to have the emulator working on this other machine.

Yeah, no worries, I understand not wanting to release vaporware/unstable things, and also the need to secrecy. ;)

Much more boringly --- it's just because I need to get an IP release from my employer first. Shouldn't take too long, I hope!
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on February 20, 2021, 03:46:31 pm
No worries. :)
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on February 26, 2021, 04:26:15 pm
So I think there may have been some unicode fuckery that snuck into that class somehow, maybe an invisible non-space space char or something similar. I retyped the method name a few days ago by hand, exactly as it was before, but no copy paste, and into both the .h as well as the .cpp files and it compiled.

More importantly I found a couple more bugs around UniPlus and the vertical retrace. UniPlus very early on tries to run a 68010 only MOVEC opcode which needed special handling in LisaEm to work properly with the illegal opcode. Once that was fixed, I saw the kernel locking up in a very tight loop turning on the VTIR, then checking to see if the vertical retrace happened.

Originally, because it said so in the HWG, when the VTIR is enabled, I reset the video state machine timing, so it would always be in the same state when the VTIR was set, however, this was causing uniplus to hang. So I added a guard variable around it - so if you set VTIR repeatedly within something like 100 CPU cycles again, it will ignore it and let the state machine run its course.

That allows UniPlus's kernel to get a bit further, but now it fails with a profile state error. So looks like it doesn't like my profile emulation. Oh well.

Anyway for now I'll finish off the terminal stuff and testing and then release RC4. I'll come back to the profile emulation in 1.2.8 and hopefully that will get UniPlus going. Not sure what's up with Xenix after the kernel starts up. It sort of works and shows the console and cursor, but doesn't get any further than that. If you type stuff in, it will echo it as it should, but that's it. So it's waiting for something, not sure what. Not VTIR I expect.

Now with UniPlus since the kernel source is available, it will be somewhat easier to debug, and also a lot easier to intercept the pro.c code in the kernel with high level emulation code.

So that's where we are for now.


Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: jamesdenton on February 26, 2021, 04:40:27 pm
Awesome progress, and I appreciate your attentiveness to some of the issues I brought up recently. Thanks!
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on March 05, 2021, 02:16:25 pm
For the impatient, I've copied the current code into an unstable branch here: https://github.com/rayarachelian/lisaem/tree/unstable

This isn't a release, it's a nightmare with big nasty sharp teeth and big claws.

Do the usual git clone and then do
Code: [Select]
git pull --all; git checkout unstable and then build it yourself.

It's more of a checkpoint before I start working on getting the UniPlus HLE stuff started on. After that's done, I'll come back to the TerminalWx stuff (which you shouldn't expect to work though it will compile and you can use as a console to talk to LisaTerminal.)
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: pcamen on March 07, 2021, 05:54:04 pm
I'm contemplating upgrading from macOS Catalina to Big Sur on my primary Mac.  I haven't seen any discussion here about Big Sur support for LisaEm, and it isn't starting for me presently on Big Sur (it just doesn't start).  Is Big Sur support for LisaEm in the works possibly? 
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on March 07, 2021, 11:13:55 pm
I'm contemplating upgrading from macOS Catalina to Big Sur on my primary Mac.  I haven't seen any discussion here about Big Sur support for LisaEm, and it isn't starting for me presently on Big Sur (it just doesn't start).  Is Big Sur support for LisaEm in the works possibly? 

Just made a bunch of  fixups to the unstable branch and was able to compile this in a macos 11.0 big sur virtualbox vm and get it to run the H Boot ROM.
I haven't yet setup an 11.1 VM, but hopefully that won't matter.

If you want to go through the trouble of compiling it yourself from the unstable branch, it should come up for you. That said RC3 or RC3a will likely not work as is.
The future RC4, or a build from this branch should work with Big Sur.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on March 12, 2021, 07:00:13 pm
About UniPlus, I've previously added a patch to do the right exception on the 68010 MOVEC opcode it uses to test if it's on a 68010. It then threw profile synchronization error messages up, which I've addressed by patching the profile drive to skip some assertions about CMD/BSY timing - which is very tightly controlled based on CPU cycles and would break when running faster than 5MHz if the VIA timers don't match the CPU exactly. It currently does read a couple of sectors from the profile and then just stops.

There was also some VTIR synchronization it did, also with very tight timing loops that I've previously fixed.

On top of that, last week I added High Level Emulation code for UniPlus when it tries to read or write from the ProFile. I've tested the reads and they work the same as without the HLE code.

In both cases at some point it switched MMU contexts to context 2 and got stuck executing some opcode which Generator is unable to disassemble. It shows up like so:

Code: [Select]
A 0:0000002a 1:00000026 2:0006d600 3:00fe1d0e 4:00fe0000 5:00007ff8 6:00068b38 7:000000fc SP:00fa07f8 PC:00000020 SRC:

2/00000020 (1 1/0/0) : 60fe 0000                  : ....     : 1048 : dc.l $60fe0000  SRC:clk:000000000a0400d3 +8 clks
D 0:00000014 1:00000022 2:000fffff 3:0000000b 4:0000ffff 5:00000073 6:00000010 7:000001d9 ......c imsk:0 pnd:3 (1/0/normal cx:2)SRC:
A 0:0000002a 1:00000026 2:0006d600 3:00fe1d0e 4:00fe0000 5:00007ff8 6:00068b38 7:000000fc SP:00fa07f8 PC:00000020 SRC:

2/00000020 (1 1/0/0) : 60fe 0000                  : ....     : 1048 : dc.l $60fe0000  SRC:clk:000000000a0400dd +8 clks
D 0:00000014 1:00000022 2:000fffff 3:0000000b 4:0000ffff 5:00000073 6:00000010 7:000001d9 ......c imsk:0 pnd:3 (1/0/normal cx:2)SRC:
A 0:0000002a 1:00000026 2:0006d600 3:00fe1d0e 4:00fe0000 5:00007ff8 6:00068b38 7:000000fc SP:00fa07f8 PC:00000020 SRC:

But it turns out that this is actually an on-purpose infinite loop. I thought perhaps this was some bug with the MMU, or CPU or some other thing, it's not. When I ran that opcode through ODA, it said this was bra.s to itself. I verified this in EASY68k by assembling this, and got the same opcode.

So it thinks this is a 4 byte opcode, but really it's a two byte opcode that's actually just a short branch to itself.

Code: [Select]
00020000  60FE                       2  START: bra.s START

So this is on purpose, for whatever reason. It could just be the idle loop of the kernel when it doesn't yet have any spawned processes to exec (i.e. init), however there is a bug in Generator - at least in the disassembly side where it doesn't properly disassemble this opcode.

I did see similar loops, but only after panic messages were sent to printf in the pro.c code, and they use the same 60 FE opcode (these are emitted by "while(1);" C code. So this is unlikely to be a panic.
Another reason is that pressing the power switch when looping like this, it prints "Shutting down" and then shuts down, though that's probably in an ISR somewhere.

I've added some code to detect these infinite loops and increase cpu clocks, which has the effect of causing the next timer interrupt to fire sooner.

I also saw uniplus copy a lot of data using MOVEM, and in one case using a direct A register with an offset but no pre-decrement nor post-increment, but looking at the code, it appears to be doing the right thing, though found a bug in the debug output where it would print the wrong register, so that was a false positive for a bug. i.e.:
Code: [Select]
255787 1/000245de (1 1/0/0) : 48e8 fcfc 0004             : H.....   :  687 : MOVEMRM.L  {0xfcfc}: D2-D7/A2-A7,$0004(A0)  SRC:clk:000000000ceba933 +16 clks

So not sure what else is wrong with emulating uniplus yet. It's closer, but not yet there.
The only odd thing I see is that the keyboard type interrupts to floppy size message, it should say Microdiskette with 1 head, but the Keyboard Type message interrupts it. Not sure if this is an issue or not, nor why it shows up there.  (see screenshot below).
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: anonymous on March 16, 2021, 10:10:43 am
I'm contemplating upgrading from macOS Catalina to Big Sur on my primary Mac.  I haven't seen any discussion here about Big Sur support for LisaEm, and it isn't starting for me presently on Big Sur (it just doesn't start).  Is Big Sur support for LisaEm in the works possibly?

I have found a way to get LisaEM working in big sur fairly easily. First, download this version of LisaEM 1.2.7 beta: https://lisaem.sunder.net/downloads/LisaEm-1.2.7-Beta-2020.02.25-x86_64-only-macos-X-10.12-and-higher.dmg (https://lisaem.sunder.net/downloads/LisaEm-1.2.7-Beta-2020.02.25-x86_64-only-macos-X-10.12-and-higher.dmg). You can test this while on Catalina by replacing your LisaEM app with this new app. In my experience, it works with minor display glitches, but nothing major, and the experience is still great. After replacing and testing the beta app, you can upgrade to big sur. However, it will not work immediately. You need to install sdl2 on Big Sur. The easiest way is using homebrew: run these three commands in the mac terminal. If an error comes such as "brew not a command," then you need to install brew: https://docs.brew.sh/Installation (https://docs.brew.sh/Installation).

brew install SDL2

brew install SDL2_image

brew install SDL2_ttf

Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: anonymous on March 16, 2021, 10:14:32 am
I'm contemplating upgrading from macOS Catalina to Big Sur on my primary Mac.  I haven't seen any discussion here about Big Sur support for LisaEm, and it isn't starting for me presently on Big Sur (it just doesn't start).  Is Big Sur support for LisaEm in the works possibly?

I have found a way to get LisaEM working in big sur fairly easily. First, download this version of LisaEM 1.2.7 beta: https://lisaem.sunder.net/downloads/LisaEm-1.2.7-Beta-2020.02.25-x86_64-only-macos-X-10.12-and-higher.dmg (https://lisaem.sunder.net/downloads/LisaEm-1.2.7-Beta-2020.02.25-x86_64-only-macos-X-10.12-and-higher.dmg). You can test this while on Catalina by replacing your LisaEM app with this new app. In my experience, it works with minor display glitches, but nothing major, and the experience is still great. After replacing and testing the beta app, you can upgrade to big sur. However, it will not work immediately. You need to install sdl2 on Big Sur. The easiest way is using homebrew: run these three commands in the mac terminal. If an error comes such as "brew not a command," then you need to install brew: https://docs.brew.sh/Installation (https://docs.brew.sh/Installation).

brew install SDL2

brew install SDL2_image

brew install SDL2_ttf
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on March 16, 2021, 02:05:08 pm
brew install SDL2

brew install SDL2_image

brew install SDL2_ttf

I'm unsure why SDL2 would be required. wxWidgets only relies on SDL for sound, and then only on GTK systems. It might be that I installed SDL2 on the mac VM on which that specific beta release was built on, and then the wxWidgets I compiled there saw it and started using it, but it should work without SDL2 being installed on macos, assuming you compile wxWidgets yourself using the build-wx3.1.4-modern-macos.sh script inside the scripts directory, and then have added your path to it, and then built LisaEm yourself using build.sh

I would advise you to build RC3a (or unstable) yourself instead if you're going to go through that much effort. :)
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on March 16, 2021, 02:24:02 pm

So it thinks this is a 4 byte opcode, but really it's a two byte opcode that's actually just a short branch to itself.

Code: [Select]
00020000  60FE                       2  START: bra.s START

So this is on purpose, for whatever reason. It could just be the idle loop of the kernel when it doesn't yet have any spawned processes to exec (i.e. init), however there is a bug in Generator - at least in the disassembly side where it doesn't properly disassemble this opcode.


Turns out this is pebkac on my part. :) Thanks to James Denton, who pointed me at http://www.bitsavers.org/pdf/unisoft/UniPlus+_System_V_Installation_Instructions.html

I needed to type in "copy" on the boot disk, and not hit enter. What's happening is that hitting enter boots the unix kernel off the disk, but this disk doesn't contain a file system, so when it tries to do fork/exece of '/etc/init' - there's no / directory, let alone a /etc directory and so exece fails with ENODIR and drops in this loop.

The loop can be found in the kernel source code in v1.5/sys/main.c. This is a tiny tiny, but valid assembly code uniplus program which just calls exece to run /etc/init. but apparently this needs to exist as a process in memory, before it can run fork/exec, (actually this is special as it's started via a different call: newproc(0) not actually fork.)

Weirdly, in UniPlus programs can start at address 0 - normally used for the trap table - but since this runs in user mode, it's fine. (This is something which the Generator disassembler doesn't like one bit! But it does emulate it properly as far as I can tell.)

This guy first sets up the stack pointer, then argc/argv, and environment variables, then calls trap #0 with 59 (exece) which tries to replace this process with init, and then fails, so it falls through and gets stuck in that "bra ." loop, but D0 is set to ENODIR - and that's what I saw happening at address 2/00000020 previously.

Fun stuff, and got a quick bit of "Ah Ha!" about how unix starts up. I had no idea it would create a fake stub program before loading init into memory, but I guess exec can't do that unless one exists as there's probably no "load this binary" function in the kernel, and so it can only use the exec class of syscalls. So it does that as a hack.

Code: [Select]
36 /*
 37  * Icode is the bootstrap program used to exec init.
 38  */
 39 short icode[] = {
 40                                 /*       .=USTART               */
 41         0x2E7C, HIGH, LOW+0x100,/*       movl   #USTART+0x100,sp*/
 42         0x227C, HIGH, LOW+0x26, /*       movl   #envp,a1        */
 43         0x223C, HIGH, LOW+0x22, /*       movl   #argp,d1        */
 44         0x207C, HIGH, LOW+0x2A, /*       movl   #name,a0        */
 45         0x42A7,                 /*       clrl   sp@-            */
 46         0x303C, 0x3B,           /*       movw   #59.,d0   exece */
 47         0x4E40,                 /*       trap   #0              */
 48         0x60FE,                 /*       bra    .               */
 49         HIGH,   LOW+0x2A,       /* argp: .long  name            */
 50         0,      0,              /* envp: .long  0               */
 51         0x2F65, 0x7463, 0x2F69, /* name: .asciz "/etc/init"     */
 52         0x6E69, 0x7400
 53 };
 54 int szicode = sizeof(icode);
 55

I'm in the process of adding patches for the profile handshaking code in v1.5/stand on the boot-a disk  (or boot blocks on ProFile/Widget media) to make the copy command work. Will report back after that.

Also, I've added HLE ProFile acceleration for the Boot ROM itself, which the boot code will use (it's now failing on attempts to write to the ProFile, so there, I need to patch the handshaking.) Hopefully this will help speed up anything that uses the Boot ROM call.

This was basically a freebie since I had code for it already in romless, but until now if you use the Boot ROM, it would do the full ProFile handshaking/byte for byte transfer under emulation. So now the entire JSR to that ROM routine happens in native code, so that should speed up almost all OS booting.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on March 18, 2021, 08:58:20 am
Some more progress, I've added patches for the uniplus loader (aka standalone boot), so the :copy command now works, however there's some issue with the floppy after it ejects the disk, where it doesn't recognize the floppy drive anymore after the disk is ejected. This happens in both the standalone boot loader, as well as the kernel, once it comes up.

Likely this is some very tight timing CPU loop that falls out of sync. Meh. I wonder if the UniSoft driver guys though that the Lisa would never get a faster CPU, or newer versions of the ProFile which might react faster.
However, restarting and booting off boot-a-hacked, the next step :sunix worked, but then locked up at address 2/0000020 again, later after asking for which hard drive to install on and complaining about swap vs RAM. Likely this is the same NODIRENT error due to some access issue with the floppy drive.

I've found a really weird bit of behavior in the loader (the v1.5/stand pro.c file) where after it sends the 6 command bytes to the ProFile for a block write, it fails to flip the !CMD line, but goes into the prochk function which loops forever waiting for BSY to change. This caused the LisaEm profile driver to fall out of sync with the code in the loader.

I wonder if on a real ProFile, the ProFile flips BSY some time after the 6 command bytes are passed *without* first seeing !CMD flip in real life. This may be an issue for @stepleton and the Cameo/Aphid. Or not. I don't recall seeing this behavior in the /unix kernel.

Next I have to figure out the issues with the sony.c driver, and also whether /sunix is a separate kernel or a hard link to /unix on the boot disk. (Because if there's more than one and has drivers in different locations in RAM, I'll also need to patch it as well.) and obviously will need to figure out if mkfs has some disk size limit in it, and whether the install script passes a specific size to mkfs and patch that as well with the size of the profile.
Actually looking at this screenshot again, I see sunix has v1.1 on it and 1983 as the date, while before when I was booting unix, it says v1.4 and 1985. So for sure there are two different kernels at play here as well as the standalone loader.
Likely this is the reason patching the v1.4 kernel for a different sized profile did not work - because that doesn't patch the sunix v1.1 kernel.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: stepleton on March 18, 2021, 04:12:05 pm
FWIW: Cameo/Aphid doesn't seem to have any problems to speak of with UniPlus. Works fine for me if you use standard hard drive sizes.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on March 23, 2021, 10:29:30 am
I think I found a situation where the BSY transition (BSY is tied to CA1 on the parallel port VIAs, where CA2 is used for handshaking), was not sending IRQs. Still working on this bit, however, it doesn't seem to affect LOS at all.

As an aside was looking to add HLE ProFile read/write support and seeing that LOS is doing some sort of XOR checksum on both reads and writes. Not sure if this is related to the document checksum protection/password thing, but it's interesting anyway. The strange bit is that it doesn't seem to do a whole sector in one shot, or even all the tags together in one shot, but rather in chunks. For example, for tags it does 2-3 tags at a time and I see several repeated calls to this code for a single sector.

I'm unsure about this chunking thing I'm seeing, but, it is what it is.

This read routine does 8x reads per pass (looks like manual loop unrolling) and in between it's doing an EOR to D1 (I removed the register and other outputs to clean it up a bit):
(A2) is the read from the IRA port on the VIA.

It may well be that this checksum is written to the tags. But it's different from other checksums that we see for example in the Boot ROM code, where it would also mix in some shifts to deal with single bit errors, so not as robust. Likely this code was written by someone else who wasn't aware of the code in the Boot ROM.

Code: [Select]

1 102470 1/00c08a88 (0 0/0/0) : 1012                       : ..       :  239 : MOVE.B     (A2),D0  SRC:clk:00000000038eb7ad +8 clks
102472 src/lisa/io_board/via6522.c:lisa_rb_Oxd800_par_via2:1806:reading from register 1 (I/ORA    )| 14:50:38.6 59684781

102476 src/lisa/io_board/via6522.c:via2_ira:572:profile.c:READ 00 from ProFile, pc24:00c08a88  tag:profile state:10| 14:50:38.6 59684781
102482 1/00c08a8a (0 0/0/0) : b101                       : ..       : 1278 : EOR.B      D0,D1  SRC:clk:00000000038eb7b5 +4 clks
102486 1/00c08a8c (0 0/0/0) : 10c0                       : ..       :  225 : MOVE.B     D0,(A0)+  SRC:clk:00000000038eb7b9 +8 clks
102487 src/lisa/cpu_board/memory.c:dmem68k_store_byte:486::::::WRITE BYTE 00 ' ' to @1/00ad5700 using 17-ram| 14:50:38.6 59684793

2 102491 1/00c08a8e (0 0/0/0) : 1012                       : ..       :  239 : MOVE.B     (A2),D0  SRC:clk:00000000038eb7c1 +8 clks   2nd read
102494 src/storage/profile.c:ProfileLoop:777:ProFile access at PC:1/00c08a8e event:1 read IRA state:10 SEND_DATA_AND_TAGS_STATE idxr,idxw: 529,4 bsy:0 cmd:0 rrw:0| 14:50:38.6 59684801

102503 1/00c08a90 (0 0/0/0) : b101                       : ..       : 1278 : EOR.B      D0,D1  SRC:clk:00000000038eb7c9 +4 clks
102507 1/00c08a92 (0 0/0/0) : 10c0                       : ..       :  225 : MOVE.B     D0,(A0)+  SRC:clk:00000000038eb7cd +8 clks

3 102512 1/00c08a94 (0 0/0/0) : 1012                       : ..       :  239 : MOVE.B     (A2),D0  SRC:clk:00000000038eb7d5 +8 clks
102524 1/00c08a96 (0 0/0/0) : b101                       : ..       : 1278 : EOR.B      D0,D1  SRC:clk:00000000038eb7dd +4 clks
102528 1/00c08a98 (0 0/0/0) : 10c0                       : ..       :  225 : MOVE.B     D0,(A0)+  SRC:clk:00000000038eb7e1 +8 clks

4 102533 1/00c08a9a (0 0/0/0) : 1012                       : ..       :  239 : MOVE.B     (A2),D0  SRC:clk:00000000038eb7e9 +8 clks
102545 1/00c08a9c (0 0/0/0) : b101                       : ..       : 1278 : EOR.B      D0,D1  SRC:clk:00000000038eb7f1 +4 clks
102549 1/00c08a9e (0 0/0/0) : 10c0                       : ..       :  225 : MOVE.B     D0,(A0)+  SRC:clk:00000000038eb7f5 +8 clks

5 102554 1/00c08aa0 (0 0/0/0) : 1012                       : ..       :  239 : MOVE.B     (A2),D0  SRC:clk:00000000038eb7fd +8 clks
102557 src/storage/profile.c:ProfileLoop:777:ProFile access at PC:1/00c08aa0 event:1 read IRA state:10 SEND_DATA_AND_TAGS_STATE idxr,idxw: 532,4 bsy:0 cmd:0 rrw:0| 14:50:38.6 59684861
102566 1/00c08aa2 (0 0/0/0) : b101                       : ..       : 1278 : EOR.B      D0,D1  SRC:clk:00000000038eb805 +4 clks
102570 1/00c08aa4 (0 0/0/0) : 10c0                       : ..       :  225 : MOVE.B     D0,(A0)+  SRC:clk:00000000038eb809 +8 clks

6 102575 1/00c08aa6 (0 0/0/0) : 1012                       : ..       :  239 : MOVE.B     (A2),D0  SRC:clk:00000000038eb811 +8 clks
102587 1/00c08aa8 (0 0/0/0) : b101                       : ..       : 1278 : EOR.B      D0,D1  SRC:clk:00000000038eb819 +4 clks
102591 1/00c08aaa (0 0/0/0) : 10c0                       : ..       :  225 : MOVE.B     D0,(A0)+  SRC:clk:00000000038eb81d +8 clks

7 102596 1/00c08aac (0 0/0/0) : 1012                       : ..       :  239 : MOVE.B     (A2),D0  SRC:clk:00000000038eb825 +8 clks
102599 src/storage/profile.c:ProfileLoop:777:ProFile access at PC:1/00c08aac event:1 read IRA state:10 SEND_DATA_AND_TAGS_STATE idxr,idxw: 534,4 bsy:0 cmd:0 rrw:0| 14:50:38.6 59684901

102608 1/00c08aae (0 0/0/0) : b101                       : ..       : 1278 : EOR.B      D0,D1  SRC:clk:00000000038eb82d +4 clks
102612 1/00c08ab0 (0 0/0/0) : 10c0                       : ..       :  225 : MOVE.B     D0,(A0)+  SRC:clk:00000000038eb831 +8 clks

8 102617 1/00c08ab2 (0 0/0/0) : 1012                       : ..       :  239 : MOVE.B     (A2),D0  SRC:clk:00000000038eb839 +8 clks
102629 1/00c08ab4 (0 0/0/0) : b101                       : ..       : 1278 : EOR.B      D0,D1  SRC:clk:00000000038eb841 +4 clks
102633 1/00c08ab6 (0 0/0/0) : 10c0                       : ..       :  225 : MOVE.B     D0,(A0)+  SRC:clk:00000000038eb845 +8 clks

102639 1/00c08ab8 (0 0/0/0) : 51ca ffce                  : Q...     : 1031 : DBRA.W     #$c08a88,D2  SRC:clk:00000000038eb84d +10 clks


Edit: seeing the same kind of read + xor in MacWorks XL 3.0 as well. Since MacWorks is built on top of Monitor OS and Monitor OS was built before LOS, I suspect this is the original source of this behavior, and likely the XOR is used with one of the tags, perhaps as a way for the scavenger program to detect corrupted data.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on April 12, 2021, 11:16:50 pm
Eh, so 6 weeks forwards, 3 weeks back.

Had to roll back some of the profile HLE code. In my zeal to get UniPlus working, I hadn't noticed for quite a few days that I broke LOS from booting, and then had to do a lot of bisecting to figure out exactly where the bugs were, and roll them back. Pushed to unstable.

The new HLE (High Level Emulation) checkbox in preferences works for the Boot ROM phase of loading LOS (before the Welcome screen).

I'll also be adding LOS+MacWorks HLE support in the next few days/weeks. Will re start again on UniPlus, but I guess I'll have to keep testing against LOS.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on April 16, 2021, 11:24:49 pm
I've added HLE ProFile hard drive acceleration to both MacWorks XL 3.0 and LOS 3.1, both seem to be working.
Pushed to the unstable branch on github.

Next up, restarting on ProFile code tests of the BLU image of UniPlus that Jason sent my way, to get it to boot up all the way.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on April 22, 2021, 08:29:14 am
So seeing some weird issues in both UniPlus and Xenix around the stack. In one case when booting off Xenix, IRQ#2 gets triggered from the COP, but rather than use an RTE to get out of the Interrupt Service Routine, the last thing that unwinds the stack is an RTS, this causes the value of 2100 to be loaded into the high word of the PC and 0001 into the low word of the PC - these come from the SR being pushed on the stack during the IRQ. RTE would have successfully restore the stack properly.

This happens after the kernel has started up and reported about most of the hardware, but before

So either Xenix isn't ready at this point to receive an IRQ from VIA1/COPS421, or the wrong value has been loaded into the IRQ vector.

Using the ROM HLE code works fine with LOS and MacWorks, but in UniPlus, instead of showing the boot loader ":" prompt, it throws error 75, but looking at the tracelog, I see another odd address error issue come up, similar to the one that happens in Xenix.

I've verified that MOVEM, LINK, UNLK opcodes and other mechanisms are working as expected. It's possible/likely there's some side effects from the HLE ProFile routine, but not sure yet. Maybe it's relying on some register values that are normally thrown away, or an IRQ happens when it's not expected (but in Xenix case above, the IRQ mask was 1 and the IRQ triggered was 2, so this indicates to the "hardware" that the OS is ready to receive an interrupt.

In the UniPlus, during the loader startup (standalone/ boot prompt) when HLE is enabled, I see the stack overwriting some code around 0060000, and ofc when that's executed, the emulator crashes. This doesn't happen with HLE turned off, so certainly something's up with A6/A7 which are used in stack frames. I suspect debugging this will be easier since I can turn tracelog on, record what happens, turn HLE on, repeat, and diff and see where it varies. Fixing whatever this is might help on the Xenix side too.

Obviously all this stuff works on a real physical Lisa, so it's something that's different in the emulator than in the hardware that's at fault, as usual. Most likely the symptoms are showing up way after the thing that's different happened, so it's hard to track down (for now.) It's probably some stupid unused register being used side effect, or an off-by-one, etc.

I'll probably timebox this to another week or two of debugging/exploring; if I make progress there, great, if not, I'll go back to the terminal window stuff, etc.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on April 24, 2021, 10:06:45 pm
UniPlus, during the loader startup (standalone/ boot prompt) when HLE is enabled, I see the stack overwriting some code around 0060000, and ofc when that's executed, the emulator crashes. This doesn't happen with HLE turned off, so certainly something's up with A6/A7 which are used in stack frames. I suspect debugging this will be easier since I can turn tracelog on, record what happens, turn HLE on, repeat, and diff and see where it varies.
The ROM Profile read function D1 is the sector number to read, looks like D1 is actually preserved by the ROM call if the sector is read successfully. UniPlus's loader uses the old value in D1 to load the next sector, LOS's does not, so instead it wound up reading the same sector over and over again, and of course this led to a later crash. :)
tl;dr don't rely on the comments in the source code. they're not always completely true.

One dumb bug fixed, 99+/-whatever more to go.
Code: [Select]
0|                       ;-----------------------------------------------------------------------------
1F70|                       ;  First initialize and then ensure disk is attached by checking OCD line.
1F70|                       ;  Assumes ACR and IER registers of VIA set up by caller.  For boot, these
1F70|                       ;  are cleared by power-up reset.
1F70|                       ;  Register usage:
1F70|                       ;    D0 = scratch use           A0 = VIA address for parallel port interface
1F70|                       ;    D1 = block to read         A1 = address to save header
1F70|                       ;    D2 = timeout count         A2 = address to save data
1F70|                       ;    D3 = retry count           A3 = scratch
1F70|                       ;    D4 = threshold count       A4 = unused
1F70|                       ;  Returns:
1F70|                       ;    D0 = error code (0 = OK)
1F70|                       ;    D1 = error bytes (4) <--- not true, this returns the same D1 value (sector #) when there's no error.
                                     I was setting this to zero thinking error bytes are zero to signal no error along with D0.
1F70|                       ;    D2 - D7 and A1 - A6 are preserved
1F70|                       ;-----------------------------------------------------------------------------
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on May 12, 2021, 10:35:19 pm
More work on MacWorks around allowing 4MB of RAM - there's now a preferences checkbox, but likely this should be kept off for now (at least in this version).

Added more code to the uniplus partition resizer.

Added a few more disk conversion tools, plus added options to turn on/off interleave5/deinterleave5 options for some of them.
Code: [Select]
raw-to-dc42 -d aphid.image           # convert a Cameo/Aphid image to LisaEm, likely
dc42-to-raw -i lisaem-profile.dc42  # copy a LisaEm image to Cameo/Aphid maybe.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on June 06, 2021, 02:37:19 pm
After much tweaking of the profile code, I managed to get UniPlus booting off a pre-installed image (this was converted from BLU to a dc42).
Most likely installing from scratch won't work yet because part of the install process uses another kernel, version 1.1 called "sunix" (see: http://www.bitsavers.org/pdf/unisoft/UniPlus+_System_V_Installation_Instructions.html ), versus the normal kernel called "unix" which is 1.4. So that's the next step, trying to get it working.
I'll push to "unstable" once I get the sunix step working.
Title: Re: LisaEm 1.2.7-RC3a support bug reports
Post by: rayarachelian on September 04, 2021, 12:40:23 pm
Just pushed to the unstable branch, this should fix a bunch of bugs, though it does introduce a few new ones.

It now builds for windows, but as @Neelix57 on github noted, on Windows, the conversion of wxString to CString does not properly work for UTF strings - this affects Profile disk paths. Tested by adding an omega character in the profile path and it failed to open. This works properly on macos.

If you're interested in playing with UniPlus you'll need to first install it on a real Lisa, download it to your computer using BLU and XModem, and then run blu-to-dc42 to convert it.

This not-quite-a-release includes several new tools including the video state ROM decoder.

Let me know if you'd like binaries.
https://github.com/rayarachelian/lisaem/tree/unstable
Title: Re: LisaEm 1.2.7-UNSTABLE support bug reports
Post by: D.Finni on September 05, 2021, 07:30:26 pm
Nice work. I'll be sure to check it out!
Title: Re: LisaEm 1.2.7-UNSTABLE support bug reports
Post by: rayarachelian on October 22, 2021, 04:37:29 pm
Just added package building functions in the bashbuild system for win10/NullSoft Installer System, Linux: deb/rpm/snapcraft, freebsd pkgng - these will be available the next time I push to unstable (probably in a week or so after more testing.) - these are only marginally tested, but they do produce installable packages.

I'll try to build a ports* for it in FreeBSD as well, and maybe an OpenIndiana/Solaris package, but these are far lower priority - if you care about these, please let me know, if not I might put these on the back burner and go back to UniPlus sunix v1.1 (fixing the profile code) and then back to TerminalWx instead which are next to do. These are the last barriers to the 1.2.7 release +/- a few other minor features.

I probably will skip NetBSD/OpenBSD/Dragonfly BSD packages for now.

I took a look at HelloSystem and I really like it and want to use it on some of my machines, so I'd support it with a package and it's own LisaEm.app bundle, but I couldn't get it to install on a virtualbox hard drive - only seems to work as a live image, so this will also be skipped for now until a more mature release comes along. (Tried 0.50 and 0.60.)
* Since FreeBSD doesn't install bash by default (and this is required by bashbuild), and I also prefer to use a custom, statically linked wxWidgets rather than the official one which is several versions older, I might just skip the ports dir for FreeBSD, will see how much of an uphill battle it is.There currently is a FreeBSD port for 1.2.6.2, however the maintainers switched it to regular borne shell, and rewrote-out the bash specific parts of the much older build script. That build script has evolved into its own package/library at https://github.com/rayarachelian/bashbuild (https://github.com/rayarachelian/bashbuild) )  Once all this package code is happy I'll update bashbuild with the new code as well.
Title: Re: LisaEm 1.2.7-UNSTABLE support bug reports
Post by: rayarachelian on October 23, 2021, 07:06:36 pm
Just added classic Solaris packages as well as IPS ones to the bashbuild scripts, only took a single day of satisfying scripting on a Saturday

(or would that be better alliterated as csh'elling on a Caturday - but then again, it was bash after all?)  ;)
Title: Re: LisaEm 1.2.7-UNSTABLE support bug reports
Post by: rayarachelian on January 11, 2022, 11:33:35 am
Minor update: fixed some of the copy/paste bugs in TerminalWx code, so this helps us on the road to a future where we'll have a mostly usable terminal window for things like UniPlus, and eventually Xenix, and LPW, as well as a built in interface for BLU to connect LisaEm to an actual physical Lisa. Terminal for things like the unixes and LPW would be very useful for me so I can write code without being limited by a bitmap screen and lack of clipboard passing between host and guest. Eventually (maybe 2.0?) the idea is to have the ability to call LisaEm headless and compile things form a host side Makefile, sort of like that modern day MPW package.

Eventually will add preferences for the Terminal window to select terminal size (80x25 for now), font (Courier New), font size (12 for now), colors, etc. (Likely I'll add a preferences notebook page for this in 1.2.8.)

TerminalWx is not full featured, but it's good enough, lacks a lot of features (most notably double click and triple click don't select properly, no scrollback, etc.), but, it's better than nothing. The alternative would be to try to interface with an external program by creating a pty, but that would be problematic with Windows. If there's interest in future releases.

Still need to fix UniPlus installs (sunix kernel v1.1 profile bugs) before RC4. Hopefully by end of this month if all goes well.
Title: Re: LisaEm 1.2.7-UNSTABLE support bug reports
Post by: andrew on February 14, 2022, 12:14:56 am
I'm not sure if this is an appropriate place to mention this, but since this involves discussion on the upcoming release, I might as well. I made a couple of Big Sur style icons for LisaEm. I can post them here if anyone is interested; I also uploaded them on macosicons. If you want to use them for future versions of LisaEm, you are welcome to.
Title: Re: LisaEm 1.2.7-UNSTABLE support bug reports
Post by: rayarachelian on February 14, 2022, 05:04:47 am
I'm not sure if this is an appropriate place to mention this, but since this involves discussion on the upcoming release, I might as well. I made a couple of Big Sur style icons for LisaEm. I can post them here if anyone is interested; I also uploaded them on macosicons. If you want to use them for future versions of LisaEm, you are welcome to.

Sure, please do. Zip them up or tar them up with .tar.xz and post here.
Title: Re: LisaEm 1.2.7-UNSTABLE support bug reports
Post by: andrew on February 14, 2022, 05:52:31 pm
I attached them here. I did one for the Lisa 1, and another for the Lisa 2.
Title: Re: LisaEm 1.2.7-UNSTABLE support bug reports
Post by: rayarachelian on February 15, 2022, 07:19:28 am
I attached them here. I did one for the Lisa 1, and another for the Lisa 2.

Thank you, much appreciated.
Title: Re: LisaEm 1.2.7-UNSTABLE support bug reports
Post by: rayarachelian on April 03, 2022, 10:05:40 pm
2022.04.01 - pushed to unstable + master branches. https://github.com/rayarachelian/lisaem/tree/unstable (https://github.com/rayarachelian/lisaem/tree/unstable)

Windows Packages:


    Skinny Packages:

    Full macos package:
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: rayarachelian on May 11, 2022, 01:58:57 pm
Recently, Ubuntu had a new LTS, 22.04, which switches the sound system to PipeWire. This is in the long term a good thing, however, previous versions used pulseaudio, and before that OSS, and before that ALSA.

With PulseAudio, an OSS proxy package was provided that allowed applications that used OSS's dev tree muxes/dsp end points to allow applications expecting them to work on PulseAudio.

Unfortunately there's some issue/bug/other with the current 22.04 release. After installing OSSPD (the proxy package), I've noticed that the sound implementation in wxWidgets will lock up a mutex permanently when a wave file is being played in a loop. Unfortunately this is the method I use to generate sound on LisaEm.

So the effect is that whenever the Lisa plays a sound, LisaEm will permanently lock up.

A bunch of this is related to wxWidgets not supporting anything else, and on GTK requiring SDL for wave playback. There's a note that says wxWidgets requires SDL for this, which is less than ideal as it requires me to link in yet another library that's not quite large, but misused as SDL also does graphics and other things that are unused.

Sound seems to be a second class citizen in wxWidgets currently, and for backwards compatibility this is also an issue. All it can do is play a single wave file at a time, optionally asynchronously, and optionally looping the same sound. If another sound play call is issued the current sound stops playing. This works ok for windows and macos without SDL (I think), but never for GTK+ (Linux, *BSD, *Solaris). There were some discussions in the wxWidgets forums about this, but nothing has been implemented. So I'm leaning either for a workaround, or a replacement.

Unfortunately since my current daily driver development machine runs Linux this is a blocker.

Since I'm getting ready for a prod release, this is a problem as it requires a new, mostly untested, solution.

There's what I'm looking at as strategies:

* change the wave file size I'm using to several seconds instead of using looping - this would require more memory use and a work around, but it might work, haven't tested to see if I can stop the sound early successfully if it's something like 5-10 seconds of a wave file.
* directly talk to SDL and see if I can produce working sound that way.
* switch to OpenAL or SoundVox, and a few others - of the two, OpenAL is the smallest and most widely compatible - leaning to this as well. There's a small snag here, OpenAL is no longer provided for modern macos, however it does still compile there, and can run fine. No telling if it will still work in the future, but it's no different than X11 or OpenGL support. As part of Apple's war on all things GPL (including bash, and gcc) and closing off source code they've removed this as well in favor of proprietary frameworks.
* directly targeting the PipeWire calls is not an option, as I'd like to avoid writing platform specific code as much as possible and worse, elder Linux and other OS's that may use ALSA, pulseaudio, OSS, JACK, etc. would then also need yet another exception to work.

Ideally I need something that can play multiple wave files in the background (floppy, widget, imagewriter, etc), but also optionally constantly play sound data preferably without having to create a wave file either in RAM or on disk and then calling the play function. Something like a circular buffer that I can feed new values from the shift register of the VIA to generate sound - like a mixer or the way classic macos does.

The sound output of the VIA is just 1 bit, so it's an 8 cycle (8 bits in SR) timed square wave pattern at a specific volume so perhaps a synth output would be a good shortcut so I don't have to manually generate a large repeating buffer in memory with it.
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: rayarachelian on May 20, 2022, 04:46:47 pm
Quote
* change the wave file size I'm using to several seconds instead of using looping - this would require more memory use and a work around, but it might work, haven't tested to see if I can stop the sound early successfully if it's something like 5-10 seconds of a wave file.

Nope, this doesn't work. I set the wav file to 10s, but when Sound.Stop() is called, it locks up on the mutex the same as LOOPING audio, so the real issue is with the Stop and not the LOOP.
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: rayarachelian on July 26, 2022, 07:35:08 pm
So, RC4 has some bugs on MacOS and Windows in regards to hard drive/floppy images, likely I screwed up something in libdc42. see: https://github.com/rayarachelian/lisaem/issues for the bug reports.

I might have fixed that, not sure as I've not tested on Windows/macos yet, but also haven't pushed to github yet. Will do so. I'm considering a rewrite of libdc42 to add new features that were meant for 1.2.8 that will be necessary.

I'm currently finishing off the move to the OpenAL (Open Audio Library) from wxSound. LisaEm will continue to use wxSound if OpenAL is not installed on the system it's compiled on. Note that Apple in its war against GPLed software, and things like OpenGL has removed it from recent macos versions, though it's possible to compile it there anyway.

This will be released as RC5 once tested, etc.
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: greniu on July 27, 2022, 09:57:09 am
I have tried to install it on fresh Windows 10, macOS Catalina 10.15 and always when trying to boot from a floppy drive I am receiving this error:

 "Sorry, the emulation aborted due to a fatal error. Have a cow man! ipc=NULLL
  Stopped at cpu68k.c:cpu68k_makeipclist:1096 with code: 54
   LisaEm will now quit".

 It seems that there is a bug in the code. Please help to run LisaEM.

Do you know which version of OS is stable to run LisaEM 1.27-RC4?
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: rayarachelian on July 27, 2022, 12:24:27 pm
Most likely the linux version will work, but I have made some changes since that release. I'll test this once I'm done with the OpenAL migration which should be in a few days. I'll then push to the unstable branch on github before cutting RC5.
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: greniu on July 27, 2022, 01:32:06 pm
Most likely the linux version will work, but I have made some changes since that release. I'll test this once I'm done with the OpenAL migration which should be in a few days. I'll then push to the unstable branch on github before cutting RC5.

ok. Thanks. I will wait for stable version. Good luck with programing and please let us know about new release:)
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: rayarachelian on September 30, 2022, 10:42:08 am
Added a livedev branch to the repo, fri0701 contributed some fixes to lisafsh-tool and deserialize tools, I merged them.

The purpose of this branch is to capture changes I've recently made, so the work isn't lost if/when I suddenly become incapacitated or die, and also to allow others who wish to contribute to be able to get at the very latest code. This is more unstable than the unstable branch.
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: rayarachelian on October 06, 2022, 07:07:49 pm
Looking at the LOS 2.0 installer issue, looks like the bug is with ProFile block writes, the write actually completes and get to State machine step 12, which then times out, so the bug is somewhere around there, likely around BSY or CMD detection. Will dig further.
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: rayarachelian on October 14, 2022, 05:02:07 pm
fixes for LOS2.0 installer fail pushed to livedev, but needs regression testing for all OSs to ensure something else didn't break.
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: patrick on October 17, 2022, 09:41:54 am
Recently my Apple collection got an over 20 years younger addition - a Mac Mini A1283 with Intel Core Duo CPU. This is my first macOS 10 machine, until now my favorite was macOS 6.0.8. Now it took me two days to get macOS 10.11 El Capitain running on this machine. Skip the next paragraph for the LisaEM bug report.

Apple sees no reason to create macOS installation media from Windows or Linux. There is no ISO file available, instead you have to run a program on a second Mac from the appropriate era to create a bootable USB drive. And it will boot. But macOS will not install because the "installation files cannot be verified". Yes, they use certificates for the EFI secure boot. And those certificates expired in 2019. So first I had to get Snow Leopard 10.6, which doesn't use Secure Boot yet, install it on the Mac Mini, and apply all available updates. Then I set the date to October 24, 2019, downloaded the El Capitain installer from Apple's support site, unplugged the LAN cable, and finally started the installation. The combination of download date, machine time, and signatures in the installer file is critical and must match.  >:(


But now I have one good use for this modern machine in my museum: running LisaEM! Installation worked fine, configuration worked fine, but every time I want to boot anything, I get the fatal error "cpu68k_makeipclist: But! ipc is null! Stopped at cpu68k.c:cpu68k_makeipclist:1064 with code:501"

This is 1.2.7-RC4_2022.04.01 running on MacOS 10.11.6 (Core 2 Duo 2 GHz 8 GB). First I tried the dedicated installer for 10.11, then the fat one. Both installations behave the same.


---
Update: same applies to RC3a, but line 1085 instead of 1064.
Title: Re: LisaEm 1.2.7-RC4 support bug reports
Post by: rayarachelian on October 17, 2022, 10:30:58 am
Recently my Apple collection got an over 20 years younger addition - a Mac Mini A1283 with Intel Core Duo CPU. This is my first macOS 10 machine, until now my favorite was macOS 6.0.8. Now it took me two days to get macOS 10.11 El Capitain running on this machine. Skip the next paragraph for the LisaEM bug report.

Apple sees no reason to create macOS installation media from Windows or Linux. There is no ISO file available, instead you have to run a program on a second Mac from the appropriate era to create a bootable USB drive. And it will boot. But macOS will not install because the "installation files cannot be verified". Yes, they use certificates for the EFI secure boot. And those certificates expired in 2019. So first I had to get Snow Leopard 10.6, which doesn't use Secure Boot yet, install it on the Mac Mini, and apply all available updates. Then I set the date to October 24, 2019, downloaded the El Capitain installer from Apple's support site, unplugged the LAN cable, and finally started the installation. The combination of download date, machine time, and signatures in the installer file is critical and must match.  >:(

Yes, it's a mess what they did in terms of older machines and operating systems, this is why I want to continue supporting these machines anyway. I don't agree that older machines are useless or should be abandoned.


But now I have one good use for this modern machine in my museum: running LisaEM! Installation worked fine, configuration worked fine, but every time I want to boot anything, I get the fatal error "cpu68k_makeipclist: But! ipc is null! Stopped at cpu68k.c:cpu68k_makeipclist:1064 with code:501"

This is 1.2.7-RC4_2022.04.01 running on MacOS 10.11.6 (Core 2 Duo 2 GHz 8 GB). First I tried the dedicated installer for 10.11, then the fat one. Both installations behave the same.

---
Update: same applies to RC3a, but line 1085 instead of 1064.

I'm very sorry, but yes, RC4 is very much broken.

I didn't realize RC3a is broken in this way as well. Thank you, if that's confirmed as a bug I'd have to back much earlier.

You could try the binary for an earlier MacOS as well, maybe 10.8 or 10.7 and see if it does anything different, that would indicate a compiler/optimization issue, I suppose.