I assume you are using a RasPi model B(+) with Ethernet since you mentioned the 4 USB ports. For me, Ray's emulator does work fine at full (Lisa 5 Mhz) speed, but only if you disconnect the network cable! Servicing the network seems to have a higher priority and saturates the Pi's CPU when running the emulator. This should be no big deal, since Lisa does not "need" the network connection anyway.
Also, while I agree mini vMac is fantastic, I don't agree that it has any place here. Emulating a Mac running early versions of the MacOS is not the same thing as the MacWorks Environment. Mini vMac cant not emulate the Lisa's screen resolution or aspect ratio, hard disk support, RAM size, etc. If you are that keen on just using Mini vMac, why not just build a Mac Semi-Replica.
Rick