Python ADFS experiment goes wrong

discuss general risc os software applications and utilities
Related forum: adventures


Post Reply
User avatar
marnanel
Posts: 5
Joined: Fri May 13, 2022 6:33 pm
Contact:

Python ADFS experiment goes wrong

Post by marnanel »

I'm clearly misunderstanding something about ADFS. Can someone throw me a clue?

I'm messing around with one of the Domesday Discs-- Community South. I've decoded the laserdisc image using ld-archive's ld-decode and ld-process-efm tools, and now I have a 264MB file with all the data in it. The domesday86 project's copy of the source code is helping quite a bit.

Other than a bit of blank space, the image begins with what's clearly an ADFS image. I could have used an existing tool to read it, but I wanted to see how it all fits together, so I wrote a Python script to split out all the files.

This is the script, and this is the output. You see that it's reading the root directory mostly correctly. But there are two weird things:
  • The specs I found say that the start addresses are measured in sectors (256 or 1024 bytes long), but the start addresses here are clearly measured in bytes.
  • some of the files (e.g. GAZETTEER, HELP, and HELPTEXT) begin 0x4800 bytes before the offset given in the root directory structure. But others (!BOOT) don't. I've run out of ideas on this one, except that I read somewhere that some versions of ADFS use an allocation map, so maybe that's the difference?
Any thoughts are appreciated! Thank you.
User avatar
geraldholdsworth
Posts: 1140
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Python ADFS experiment goes wrong

Post by geraldholdsworth »

Can you post the disc image here, so I can take a peek myself?

The file addresses will be in number of sectors, not bytes, if it is Old Map ADFS. If the image has some blank space before it, then this needs to be discounted.

Of course, if it is New Map ADFS, then this is a different story and is read differently from those specs. Have a peek at this.
User avatar
marnanel
Posts: 5
Joined: Fri May 13, 2022 6:33 pm
Contact:

Re: Python ADFS experiment goes wrong

Post by marnanel »

Thanks for the document: it's a lot more comprehensive than the one I've been working from. I wrote a script based on it, to search for things that looked like volume titles and parse the possible record around them accordingly, but there was nothing that looked right.

The full image is 264MB long, but I've uploaded the first megabyte here for now: https://marnanel.org/stuff/sample.dat, which should be enough, because it contains everything up to the root directory at 0x5EA00 and beyond. If you want the whole thing, let me know.

Thank you for looking!
User avatar
geraldholdsworth
Posts: 1140
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Python ADFS experiment goes wrong

Post by geraldholdsworth »

OK - it certainly is an Old Map ADFS, so the guide you've been using is what you're looking for. However, the disc image data starts at offset 0x5E800 and all addresses given are from this point. I.e., !BOOT is given as being at sector 0x18. This would be an offset of 0x1800 (sector size is 0x100), but from 0x5E800. So the actual offset is 0x60000 (0x1800+0x5E800).
User avatar
BeebMaster
Posts: 5269
Joined: Sun Aug 02, 2009 5:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Python ADFS experiment goes wrong

Post by BeebMaster »

This is very interesting, I haven't seen an image of a Domesday disc done this way before. The data from &4F800 seems to be some sort of lead-in with some disc and date information towards the end of this section, and then at &5D000 looks to be the LV-DOS volume table described in the VP415 operating instructions, which the LV-ROM controller uses to access the volumes stored on the disc.

After that the VFS/ADFS part begins at &5E800 as Gerald says. It's exactly the same as 8-bit ADFS with one special rule, which is that files always begin at sectors with multiples of &18. This is because of the way the LV-ROM data is stored on the disc - the data is stored in the picture frames (actually in the audio track of picture frames) and so the location of data is always made with reference to the picture number. LV-ROM stores 3 sectors per picture frame, but the LV-DOS spec is for sectors of 2048 bytes, so 2048/256 bytes for Acorn sectors x 3 = 24 or &18.

More on this correlation here:

https://www.beebmaster.co.uk/Domesday/LVROMMap.html
https://www.beebmaster.co.uk/Domesday/LVROMFrames.html

This would also help to explain why, when mounting a Domesday disc and requesting the current picture number, it's not 1 as might be expected for the "start" of a laserdisc, but 13 or 14 depending on the disc.
Image
User avatar
marnanel
Posts: 5
Joined: Fri May 13, 2022 6:33 pm
Contact:

Re: Python ADFS experiment goes wrong

Post by marnanel »

That makes sense. The particular problem I'm having is that although the first few files on the image are in the correct places, the later ones (such as GAZETTEER) are slightly off. I've triple-checked everything, but I don't know where the error has crept in. It could be in the decoding of the audio signal, or in merging the decoded parts into one, or in the ADFS reader itself, or something else I haven't thought of.

Given the information in this thread, I'm pretty sure I now understand well enough how the disc is structured. So it must be something else.

If I put the full 264MB file on dropbox or similar, could one of you take a look?

If either of you has access to the files on Community Disc 1 South, could you provide me with hashes of some kind? The MD5s of the files I have are

Code: Select all

d02f2dc5f5b860e80768be969a379132  area
fa0b592f667a8f246bee3ac2ed4414fc  !boot
edf35e9bbcc83a5511031362d81051ed  chart
3eb2d04e36ac012f86d692e537275f24  cnmauto
dfbedcd77c1fb39ede50c78d82409d3f  cnmcorr
a75e97037954bdaab58d3b8cf33ef31a  cnmdetl
cac8114c457a7e9363e32610b6e9e587  cnmdisp
6b747e13d5ce8d7231b453f88102181c  cnmlink
a78eec11319190b80ade90798fab2966  cnmmanu
9b67b85563dcfd44446cf00cdc263120  cnmretr
6fcf5742048d584b023f721a4ae97c60  cnmwind
cf5d74050e7ef3a022980beed49f5790  cnmwrit
cac3f033b3c2df67c5a9999fc99c54d7  contents
d68b455e7fce63209c4d2aa704ff9fb9  data1
bd25b73219f5a31ba98da329ac41455a  data2
4186e3ed590c36140acd628aa2ea35e0  film
232f75475961a009f3c0ef301cc34e42  find
d35001c0de4f68f830f9586abf39ae06  font2
056698b61eb640e3777510c055b3d38e  gazetteer
2ddfe775e3e45b1e5fcb161e12fd07fe  help
b9faf4064af5bfd0a590da871a2990ca  helptext
0f29022acd765968fbe3b8ab21c3af26  index
571b031e3bde2990fd2f7a862a246199  init
e47d7d644c277088f8ca9915c4038f53  map
54dc13ce53bee65de2b4ea37038e9605  mapdata1
89b8aa49e9ba251e10af4a747772083e  mapproc
703339600c34d7d79296b2f9db459022  names
347e99f287c537244906130daed8f95f  natfind
a62583d5e97a51be089c3750005f38f2  photo
912708088e31a56eb941d901087fab7a  phtx
6e8defac605a08275f1bc54a27b0eb20  rmlhelpt
8f822fca7078632792f78fd8c948c214  root
7fc43a8171012f82f69468de0d801f4a  stinit1
5250415b6ae81526fb3229c3dc08b87a  stinit2
f90fa0321a3fd70f795e9d85792c468e  text
640862569e36794eb4dbf107d574d80e  userkern
e36e1564ae6d2fc0a56b7aa0f9e8ed31  walk
0ab5202f859beb9e8c8705f57e26ef49  words
(When I say "merging the decoded parts into one": decoding the full disc takes days, and kept crashing. So I split it into several sections of about 50MB, which overlapped by a few MB each way. Then I wrote a script to check the overlaps matched and reconstitute the full image.)
User avatar
geraldholdsworth
Posts: 1140
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Python ADFS experiment goes wrong

Post by geraldholdsworth »

marnanel wrote:
Tue May 17, 2022 2:59 pm
If I put the full 264MB file on dropbox or similar, could one of you take a look?
I'm happy to run it through Disc Image Manager.
By the way, the ADFS header has the number of sectors making it closer to about 316MB than 264MB (331,720,704 bytes, to be precise).
Coeus
Posts: 2761
Joined: Mon Jul 25, 2016 12:05 pm
Contact:

Re: Python ADFS experiment goes wrong

Post by Coeus »

marnanel wrote:
Tue May 17, 2022 2:59 pm
(When I say "merging the decoded parts into one": decoding the full disc takes days, and kept crashing. So I split it into several sections of about 50MB, which overlapped by a few MB each way. Then I wrote a script to check the overlaps matched and reconstitute the full image.)
Why does it crash? Could that be introducing some corruption?
Post Reply

Return to “32-bit acorn software: other”