DFS - Get sector for a file

discussion of beeb/electron applications, languages, utils and educational s/w
User avatar
simonm
Posts: 166
Joined: Mon May 09, 2016 2:40 pm
Contact:

DFS - Get sector for a file

Postby simonm » Wed Nov 16, 2016 10:36 pm

Is there some easy way to get the sector info for a given filename? I cant seem to find anything with osfile, osword or osargs. OSFILE 5 only seems to provide load/exec/length info.
Surely I dont have to read in the catalog and do this work myself?!

My aim is to load a file sector by sector for speed, but I need to get its catalog sector info first before I can call OSWORD &72. OSGBPB is dog slow!

TIA!

User avatar
jgharston
Posts: 2756
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: DFS - Get sector for a file

Postby jgharston » Wed Nov 16, 2016 10:53 pm

simonm wrote:Is there some easy way to get the sector info for a given filename? I cant seem to find anything with osfile, osword or osargs. OSFILE 5 only seems to provide load/exec/length info.
Not with any filing system I know of except HADFS where I specifically implemented OSFILE &FD specifically to do this because it was such a pain not having such a call.

What I end up doing is doing a *INFO then reading the data from the screen with OSBYTE 135 and parsing the string.
If you look in MkMap in DEFPROCfile there are some lines that start With ADFS need to peek output of *INFO command

Parse the displayed string into space-separated words. That almost gives you:
DFS name load exec length sector
ADFS name access cycle load exec length sector

...but not exactly. With ADFS 'access' could be three characters and so have no space between it and cycle, eg:

Code: Select all

TREECOPY   LWR(47)  FFFFFB55 67723700 00001FBD 000023
TREECOPY   WR (47)  FFFFFB55 67723700 00001FBD 000023

With DFS the access character is a space or a letter, eg:

Code: Select all

TREECPY     FFFB55 723700 001FBD 0023
TREECPY  L  FFFB55 723700 001FBD 0023


so what I do is: default access to "", scan until space - this gives filename - scan until next non-space, if hex character this is the first address, so continue using normal space-separated parsing. If not a hex character it's the start of an access string, then scan to space, scan to non-space, if this is not a hex digit it's a cycle number, so scan to next space and to next non-space to get to the first address, then continue using normal space-separated parsing.

Does that make sense?

simonm wrote:OSGBPB is dog slow!
Depends on the filing system. On DFS 1 and later, NFS and HADFS OSGBPB is very close to as fast as *LOAD, and is much faster than multiple BGETs. OSGBPB on DFS 0.xx was dog-slow as it just did a loop of multiple calls to OSBGET. It should be similarly fast on ADFS as OSGBPB just does bulk sector loads whenever there is a full 256-byte aligned block to load.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
sweh
Posts: 1847
Joined: Sat Mar 10, 2012 12:05 pm
Location: New York, New York
Contact:

Re: DFS - Get sector for a file

Postby sweh » Wed Nov 16, 2016 11:54 pm

simonm wrote:Is there some easy way to get the sector info for a given filename? I cant seem to find anything with osfile, osword or osargs. OSFILE 5 only seems to provide load/exec/length info.
Surely I dont have to read in the catalog and do this work myself?!

My aim is to load a file sector by sector for speed, but I need to get its catalog sector info first before I can call OSWORD &72. OSGBPB is dog slow!

TIA!

I'm confused; you say "DFS" but then say OSWORD &72, which is an ADFS call. OSWORD &7F would be the DFS call to load a sector.

If you want to be portable across filesystems (DFS, ADFS, NFS, RFS, CFS, HADFS...) then you should use OSGBPB. Even within DFS systems you'd have to worry about different formats (eg Solidisk multi-catalogue, Opus virtual drives) if you wanted portability.
Rgds
Stephen

User avatar
simonm
Posts: 166
Joined: Mon May 09, 2016 2:40 pm
Contact:

Re: DFS - Get sector for a file

Postby simonm » Thu Nov 17, 2016 12:21 am

Not with any filing system I know of

Thanks, good to have that confirmed then. If necessary I'll probably just fetch the first two sectors to get the catalog and hand crank it.

OSGBPB is very close to as fast as *LOAD, and is much faster than multiple BGETs.

That's what I hoped for.

OSGBPB on DFS 0.xx was dog-slow as it just did a loop of multiple calls to OSBGET.

I'm running on Acorn 1770 DFS with a Master.

It should be similarly fast on ADFS as OSGBPB just does bulk sector loads whenever there is a full 256-byte aligned block to load.

I'm asking for 256 bytes at a time and loading to page aligned memory, so I thought it would do the right thing and fetch me a sector at a time, but it's roughly 5x slower than a straight osfile load.

I'm confused; you say "DFS" but then say OSWORD &72, which is an ADFS call. OSWORD &7F would be the DFS call to load a sector.

I plucked &72 from the advanced disk user manual - my bad. Didn't realize there was a difference (and I havent coded this up yet to test).

If you want to be portable across filesystems

I should say I'm writing really hacky demo code for a specific machine setup, so portability isn't a consideration atm.

User avatar
hoglet
Posts: 6623
Joined: Sat Oct 13, 2012 6:21 pm
Location: Bristol

Re: DFS - Get sector for a file

Postby hoglet » Thu Nov 17, 2016 7:02 am

Hi Jonathan,
jgharston wrote:
simonm wrote:OSGBPB is dog slow!
Depends on the filing system. On DFS 1 and later, NFS and HADFS OSGBPB is very close to as fast as *LOAD, and is much faster than multiple BGETs. OSGBPB on DFS 0.xx was dog-slow as it just did a loop of multiple calls to OSBGET. It should be similarly fast on ADFS as OSGBPB just does bulk sector loads whenever there is a full 256-byte aligned block to load.

My experience with OSGBPB in MMFS (which is based on the Master's DFS 2.24) is that it is also dog slow. This was over the tube, to read a 103KB Life Turning Machine pattern. Probably 10x slower than normal file loading.

Looking at the code, it seems it is still using individual OSBGET calls:
https://github.com/hoglet67/MMFS/blob/m ... .asm#L3346

The original DFS 2.24 seems to have the same code at 9F3D:
http://mdfs.net/Info/Comp/BBC/DFS/DFS224.zip

Am I missing something here?

Dave

User avatar
jgharston
Posts: 2756
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: DFS - Get sector for a file

Postby jgharston » Thu Nov 17, 2016 1:20 pm

hoglet wrote:Looking at the code, it seems it is still using individual OSBGET calls:
https://github.com/hoglet67/MMFS/blob/m ... .asm#L3346
The original DFS 2.24 seems to have the same code at 9F3D:
http://mdfs.net/Info/Comp/BBC/DFS/DFS224.zip
You're right, and I've just checked the DFS 1.20 code as well. Odd, the Advanced Disk User Guide says "Unlike the NFS OSGBPB, which goes a long way towards minimising packet overhead, DFS 0.90 OSGBPB invokes OSBGET/OSBPUT in its implementation. Thus ordinary data I/O via OSGBPB is no faster than the equivalent number of OSBGET/OSBPUT calls". I've always read that as saying that later DFSs don't use multiple BGET/BPUT calls, otherwise why single out DFS 0.90.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
simonm
Posts: 166
Joined: Mon May 09, 2016 2:40 pm
Contact:

Re: DFS - Get sector for a file

Postby simonm » Thu Nov 17, 2016 2:10 pm

Out of interest, I presume the DFS at some point loads the first two catalog sectors anyway when loading a file so that it can do what I'm trying to do. I wonder if these are stashed in memory somewhere? Maybe between &E00 and &1900?

User avatar
jgharston
Posts: 2756
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: DFS - Get sector for a file

Postby jgharston » Thu Nov 17, 2016 5:52 pm

simonm wrote:Out of interest, I presume the DFS at some point loads the first two catalog sectors anyway when loading a file so that it can do what I'm trying to do. I wonder if these are stashed in memory somewhere? Maybe between &E00 and &1900?
The DFS directory buffer is at &E00 in low memory or &C000 in high memory and is loaded (if the disk is not still spinning) whenever there's a catalog access. So, you could do, making sure you're running on the I/O processor, something like:

IF DFS then:
A%=FNfile(5,file$):REM Load catalog
IF A%=0 THEN =0:REM Not found
IF LEFT$(file$,1)=":":file$=MID$(file$,3)
IF MID$(file$,2,1)<>"." THEN file$="$."+file$
file$=MID$(file$,3)+STRING$(9-LENfile$," ")+LEFT$(file$,1)
$name%=file$
name%!0=name%!0 AND &5F5F5F5F
name%!4=name%!4 AND &5F5F5F5F
A%=0:dir%=&E00:IF ?&FFDD=&4C:dir%=&C000
REPEAT
A%=A%+8
UNTIL (mem%!A% AND &5F5F5F5F)=!temp% AND (mem%!(A%+4) AND &5F5F5F5F)=temp%!4
=dir%?(A%+256+7)+256*dir%?(A%+256+6)
ENDIF

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
hoglet
Posts: 6623
Joined: Sat Oct 13, 2012 6:21 pm
Location: Bristol

Re: DFS - Get sector for a file

Postby hoglet » Thu Nov 17, 2016 5:55 pm

jgharston wrote:
hoglet wrote:Looking at the code, it seems it is still using individual OSBGET calls:
https://github.com/hoglet67/MMFS/blob/m ... .asm#L3346
The original DFS 2.24 seems to have the same code at 9F3D:
http://mdfs.net/Info/Comp/BBC/DFS/DFS224.zip
You're right, and I've just checked the DFS 1.20 code as well. Odd, the Advanced Disk User Guide says "Unlike the NFS OSGBPB, which goes a long way towards minimising packet overhead, DFS 0.90 OSGBPB invokes OSBGET/OSBPUT in its implementation. Thus ordinary data I/O via OSGBPB is no faster than the equivalent number of OSBGET/OSBPUT calls". I've always read that as saying that later DFSs don't use multiple BGET/BPUT calls, otherwise why single out DFS 0.90.

Hmm, that's a shame.

Can you think of any speedy way to read a large (100K+) file over the Tube from a DFS-like file system?

I guess there's always OSWORD &7F.....

Dave

User avatar
jgharston
Posts: 2756
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: DFS - Get sector for a file

Postby jgharston » Thu Nov 17, 2016 6:03 pm

hoglet wrote:Can you think of any speedy way to read a large (100K+) file over the Tube from a DFS-like file system?
I've got some code somewhere. I've been wallpapering, once I get out of these mucky clothes I'll see if I can track it down. I've backup up the MDFS to my Compact which I'll be taking to Bolton, so can work on it when I get there.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
simonm
Posts: 166
Joined: Mon May 09, 2016 2:40 pm
Contact:

Re: DFS - Get sector for a file

Postby simonm » Sat Nov 19, 2016 2:27 pm

Ok I rustled up some code to load data by fetching files one sector at a time (first loads sector to memory, then copies that page to SWR) and its MUCH faster.
Had to do shenanigans to load the catalogue myself and parse it looking for filename matches, get file attributes etc.
Have attached the code for reference in case anyone is interested. It's rough and ready as it was thrown together a bit.
Attachments
disksys.zip
ruff source code for loading file using sector reads
(3.09 KiB) Downloaded 20 times

User avatar
jgharston
Posts: 2756
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: DFS - Get sector for a file

Postby jgharston » Sat Nov 19, 2016 5:33 pm

I can't post the code from where I am, but here's a demo:

http://mdfs.net/temp/MVI_0073.AVI

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
jgharston
Posts: 2756
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: DFS - Get sector for a file

Postby jgharston » Sun Sep 10, 2017 2:35 pm

jgharston wrote:I can't post the code from where I am, but here's a demo:
http://mdfs.net/temp/MVI_0073.AVI

I should have posted the code for that. I'll dig it out.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

crj
Posts: 320
Joined: Thu May 02, 2013 4:58 pm

Re: DFS - Get sector for a file

Postby crj » Sun Sep 10, 2017 9:35 pm

Rather than go grobbling around in the DFS workspace, when I've needed to do this I've fetched the catalogue sectors into my own workspace and picked it apart. The format isn't documented, but I remember it being pretty obvious, and scarcely harder to pick apart than OSGBPB output.

As an added bonus, it allows you to enumerate all files in all directories on a DFS disc. I've never found a documented API call which can do that.


Return to “software: other”

Who is online

Users browsing this forum: No registered users and 4 guests