Midge and the Robico Advanced Parser

[davidb]

Looking at the Robico adventures recently, I noticed that they announce their use of "Midge, The Message Compression System" (Project Thesius) and "Midge 3 and the Robico Advanced Parser" (Myorem). Village of Lost Souls announces that it was programmed using Amulet, though I understand that it was originally a Magus adventure.

CASA only says this about Midge:

The internal system used for most of Robico's adventures. It uses a system called MIDGE to compress the text using tokens. The engine seemed to change significantly as the adventures got bigger.

Very little is known about the internal workings - only what has so far been reverse engineered.

Is there any other information online about these systems?

[lurkio]

I don't know anything about Midge, but it's intriguing. Have you tried contacting the CASA peeps to see if they know more, especially about the reverse-engineering?

[davidb]

I was really only a little curious about it and simply wondered if someone on these forums had looked into it. I sent a mail to the contact address.

[iamaran]

Not much help, but from the 'Myorem' section of the 'Rick Hanson Trilogy':

"Myorem has been written with the aid of Midge III, our updated message compression system. Midge III achieves an astounding 59% compression percentage on Myorem's location descriptions and has enabled us to create our best memory-based adventure to date!
The result of using Midge III is plain for all to see - Myorem contains a lot of textual description!"

[iamaran]

Have you seen this? It was available to purchase.

http://everygamegoing.com/landing/items ... ystem.html

[davidb]

I noticed that you are listed as an editor on CASA and I thought you might also have been one of the people here who might know a bit more about the Robico games.

I hadn't seen that item on Every Game Going but I did briefly look at the instructions for Myorem which I found at the Museum of Computer Adventure Game History.

[iamaran]

Not much I'm afraid. I do know that Village of Lost Souls was definitely initially a Magus release.

[davidb]

According to Jacob at CASA, tautology is the person we need to ask about this.

[lurkio]

Yes, tautology has, in the past, analysed the file formats for GAC (see his utility program "grackle") and the Acornsoft text adventures, so it wouldn't surprise me if he's hacked Midge too!

Btw, both GAC and ALPS also use text-compression systems, but while GAC's is automatic, I think ALPS requires the game designer to specify a list of the most frequent bigrams, trigrams, etc., that appear in the text in the game.

[fuzzel]

I've just noticed this article in A&B Computing about adventure writing systems.
Apparently Midge was available to purchase from Robico,

[myelin]

I've been working on reverse engineering Village of Lost Souls. Have it running in a mini emulator on a Mac, and am starting to document the bytecodes for its internal runtime. Watch this space!

[davidb]

Sounds intriguing!

I thought you were more focused on hardware projects. Should we expect a Midge processor in an FPGA at some point?

[myelin]

I thought you were more focused on hardware projects. Should we expect a Midge processor in an FPGA at some point?

I like a lot of things Still meaning to do a CPLD expansion for the 1MHz bus, so I can do a fast serial port / SD card interface like what I have hacked together using one of my Elk PiTubeDirect cartridges, but otherwise I'm taking a break from hardware for now, to indulge my current obsession with decoding the runtime for this adventure game that I loved as a kid.

Some notes so far on VoLS:

V1E (&B00-10CF) is the runtime; this is all assembly code. It loads instructions from a table starting at &10D4 (inside V2E, which is loaded at &10D0) and executes them. The instruction code is in bits 7-4 of the first byte, and the rest is instruction-dependent. Instructions are 1-3 bytes long (there's a lookup table to get the instruction length at &0F09), and the address of the code to handle each instruction comes from two tables: low bytes at &0F19 and high bytes at &0F29. Instruction B0 just calls JSR with the address in the next two bytes, and B1 is an indirect JSR -- it looks up the target address from the location in the next two bytes. 90 and 98 are relative jumps. Everything else is complicated. I'm slowly digging through it all to figure out what it does -- there's some weird CRC-like calculations going on (perhaps decompression?) and tons of table lookups.

I'm guessing that V2E is all bytecode and V3E is the actual adventure data. V4E is something I don't understand yet.

[lurkio]

... my current obsession with decoding the runtime for this adventure game that I loved as a kid.

I haven't played this one. What was it about the game that got you hooked?

Some notes so far on VoLS ...

Very interesting. Good luck with the rest of the excavation!

[myelin]

... my current obsession with decoding the runtime for this adventure game that I loved as a kid.

I haven't played this one. What was it about the game that got you hooked?

I think it was just my first *big* adventure game. I'd played smaller ones, but this was the first one with any detail. I don't think I ever finished it, but I feel like it was something I came back to a bunch of times.

[tautology]

My code for Robico stuff is all on my github:
https://github.com/tautology0/textadventurestuff

I've got a few more notes; I pretty much dumped everything but the logic from Rick Hanson and Project Thesius.

Note, I wrote most of this 10+ years ago and haven't really touched it since.

[myelin]

Thanks! I'll have to upload some of my decoders. Taking a look at your Robico decoders, it looks like Village of Lost Souls code is a bit more obfuscated than Robico's other work. There's an interesting compression system for the string tables, and a whole bytecode interpreter in there too. I left that project hanging a little while back

[tautology]

Argh; the exits.txt file on the github was ancient (you really don't want to see the [lack of] organisation of my code archive).

I've updated it to the one where I started trying to work out other bits.

I'm interested in the bytecode in VoLS and whether it's similar as I couldn't work out how it did logic.

[tautology]

Oh I found a program on one of the PD disks which did the basic MIDGE encoding which gave me some hints about how to parse the text. I'll try and look back through my archive and locate it.

I've just found code to decode Blood of the Mutineers too and some notes about Island of Xaan. I'll upload these tomorrow.

[lurkio]

Oh I found a program on one of the PD disks which did the basic MIDGE encoding which gave me some hints about how to parse the text. I'll try and look back through my archive and locate it.

I've just found code to decode Blood of the Mutineers too and some notes about Island of Xaan. I'll upload these tomorrow.

Hi, tautology! Long time no see!

This is all most interesting. Could you also please upload or link to the PD disc you mention?

[tautology]

Found it on A&B disc 3-7, called MIDGET. It's an Adventurescape -> Midge convertor (well, only the text).

The basic program is below:

Code: Select all

   10 REM Adventurescape-Midge Converter
   20 REM For the BBC B with disc drive
   30 :
   40 REM By Robert O'Leary
   50 REM Copyright (c) 1986
   60 :
   70 ON ERROR GOTO 850
   80 :
   90 MODE7
  100 DIM M$(4)
  110 C%=&C00
  120 PRINTCHR$157;CHR$132;CHR$141;"ADVENTURESCAPE TO MIDGE CONVERTER"
  130 PRINTCHR$157;CHR$132;CHR$141;"ADVENTURESCAPE TO MIDGE CONVERTER"
  140 VDU28,0,24,39,2
  150 PRINT''"Location Text..........................L"''"General Text...........................G"
  160 PRINT'"Input your choice: ";
  170 A$=GET$
  180 IF A$="L" THEN offset%=270 ELSE IF A$="G" THEN offset%=240 ELSE VDU7:GOTO170
  190 PRINTA$
  200 PRINT''"Adventurescape filename"
  210 INPUT"(7 characters maximum): "F$
  220 IF LEN(F$)>7 OR F$="" THEN200
  230 :
  240 CLOSE#0
  250 F=OPENUP(F$)
  260 INPUT#F,max%,number%
  270 midge%=(number%-1) DIV64+1
  280 FOR N%=1 TO midge%
  290 PROCmidge(N%)
  300 NEXT
  310 PROCOSCLI("SPOOL "+M$(1))
  320 file%=1
  330 FOR count%=1 TO number%
  340 fileno%=(count%-1) DIV64+1
  350 IF fileno%=file% THEN 400
  360 IF fileno%<=,odge% THEN PRINT"*";M$(fileno%)
  370 *SPOOL
  380 PROCOSCLI("SPOOL "+M$(fileno%))
  390 file%=fileno%
  400 PTR#F=10+offset%*(count%-1)
  410 IF offset%=270 THEN FOR J%=1TO6:INPUT#F,D%:NEXT
  420 INPUT#F,message$
  430 PROCconvert(message$)
  440 NEXT
  450 :
  460 return%=number% MOD 64
  470 IF return%>0 THEN FOR J%=1 TO 64-return%:PRINT:NEXT
  480 PRINT"***"
  490 *SPOOL
  500 END
  510 :
  520 DEF PROCmidge(N%)
  530 PRINT''"Midge filename ";N%
  540 INPUT"(7 characters maximum): "M$(N%)
  550 IF LEN(M$(N%))>7 OR M$(N%)="" THEN530
  560 ENDPROC
  570 :
  580 DEF PROCOSCLI($C%)
  590 X%=C% MOD 256:Y%=C% DIV 256
  600 CALL&FFF7
  610 ENDPROC
  620 :
  630 DEF PROCconvert($C%)
  640 J%=0
  650 REPEAT
  660 L%=J%?C%:L$=CHR$L%
  670 L1$=CHR$(?(J%+C%-1)):L2$=CHR$(?(J%+C%-2))
  680 IFL%>64 AND L%<91 THEN PROCcapital
  690 IFL%>96 AND L%<123 THEN L%=L%-32:L$=CHR$L%
  700 IFL$="," OR L$="." OR L$="!" OR L$=":" OR L$="?" THEN PROCpunctuate
  710 IF L%>31 PRINTL$;
  720 J%=J%+1
  730 UNTIL J%>LEN$C%
  740 PRINT
  750 :
  760 ENDPROC
  770 DEF PROCcapital
  780 IFL1$="." OR L2$="." OR L1$="?" OR L2$="?" OR L1$="!" OR L2$="!" THEN ENDPROC
  790 PRINT"@";
  800 ENDPROC
  810 DEF PROCpunctuate
  820 IF ?(J%+C%+1)=32 THEN J%=J%+1
  830 ENDPROC
  840 :
  850 REM Error routine
  860 CLOSE#0
  870 REPORT
  880 PRINT" at line ";ERL

[tautology]

I've uploaded my scraps of notes and basic hacked code for Island of Xaan to my github (https://github.com/tautology0/textadven ... obico/xaan)

It took me a while longer as I started to disassemble and reverse the code. So far I've done about 50% of the file $.XAAN, which contains the parsing, input and output routines.

[tautology]

It looks like Village of Lost Souls is a totally different system; with the other games, raw 6502 is used to handle commands, e.g.

Code: Select all

.waitcmd
{
            bne waitnoun      ; if there is a noun say I don't understand
            ldx #&04          ; message 4 - "You pause for a while"
            jmp prtmsg
.waitnoun   ldx #&00          ; message 0 - "I don't understand"
            jmp prtmsg
}

Island of Xaan has some interesting bits, including self modifying code to alter the colour of some status messages:

Code: Select all

; &10921
.printmsg
{           
            lda #&86          ; this instruction can be altered
            jmp oswrch        ; VDU 134 (Colour = cyan)

Code: Select all

.havenoun   jsr osnewl
            lda #&86          ; changes the colour in the printmsg instruction to cyan
            sta colourinst    ; &922; self modifying code; tsk

[tautology]

Starter blog post on disassembling Island of Xaan:
https://tautology.org.uk/blog/2017/11/0 ... d-of-xaan/

[myelin]

Nice work! Yeah, the system in Village of Lost Souls is really complicated; there's a whole bytecode interpreter in there that I got quite a way through decrypting. Time to start on my own blog post, I guess

[lurkio]

Starter blog post on disassembling Island of Xaan:
https://tautology.org.uk/blog/2017/11/0 ... d-of-xaan/

Excellent!

Nice work! Yeah, the system in Village of Lost Souls is really complicated; there's a whole bytecode interpreter in there that I got quite a way through decrypting. Time to start on my own blog post, I guess

You guess right!

[tautology]

I plan to dive deeper into the code for the next post; but as this one took me a few weeks to write, it could take a while...