Welp, I've made some progress!
Looking through the 10MB LLF ROM disassembly, it seems as if the drive only copies over the first seven command bytes from the command buffer to the Z8's registers, so anything after that seventh byte gets ignored. And all three commands that I was struggling with were longer than seven bytes once you add on the length byte at the beginning and the checksum at the end. So it's just impossible to send any command with a length byte of seven or greater!
I was able to get around this limitation with the Read Header command since I could just omit the retry count byte at the end. For this command, the retry count just changes how the sectors are numbered and I was able to use an interleave table to convert between the two numbering schemes, so that command works just fine now!
But the other two commands require you to specify the number of sector marks or headers that you want to write in the retry count, so omitting the retry counts just won't work since that would result in no sector marks or headers actually being written. As far as I can tell, there's no easy way to get around this, so Write Sector Marks and Write Headers just aren't going to work. I guess it's really not a huge deal since I can't think of any situations in which you'd want to run each of those commands individually instead of just doing a full format of a track. But at least the most useful command of the three, Read Header, is functional!
I doubt that anybody really cares about this right now, but I figured that I'd post it anyway in case anybody else ever has the same problem in the future!