RISC OS 3.20 ROM's?

Arc/RPCs, peripherals, RISCOS operating system & ARM kit eg GP2x, BeagleBoard
User avatar
danielj
Posts: 5345
Joined: Thu Oct 02, 2008 4:51 pm
Location: Manchester

Re: RISC OS 3.20 ROM's?

Postby danielj » Wed Dec 06, 2017 7:14 pm

flibble wrote:Can you also wave your magic wand and make it not cost 50 quid too? :lol:

That's the barrier to me having a play unfortunately :(

d.

sirbod
Posts: 737
Joined: Mon Apr 09, 2012 8:44 am
Location: Essex
Contact:

Re: RISC OS 3.20 ROM's?

Postby sirbod » Wed Dec 06, 2017 9:55 pm

Phlamethrower wrote:Grab a clean build environment (I used IOMD, although I'd expect almost any to work), make sure it's selected as the current

You lost me at the first step, where does one grab a build environment and select it as current #-o
Phlamethrower wrote:This will extract the modules & resources from the donor ROM and set up an 'Arc' build environment.

I've already coded BASIC programmes to extract both the Modules and Resources from a donor ROM and to reassemble them into a new ROM.

That's the easy part, building ROM versions of CLib and all the C Apps/Modules is the bit we're missing.
Phlamethrower wrote:'HackASWI' is a version of CallASWI that's been hacked to be ROMable (and will only work with RISC OS 3.11). (basically I just *save'd a copy after loading regular CallASWI on a 3.11 machine, then hacked out all the bits of the initialisation which write values to the module body)

I tried doing that with CLib?..have you seen the code in CLib's Init; what on earth is it doing? It branches to a block of code does some very odd things shifting bits around, to simply get the address of a data block. I tried to find it in the OSLib source to see why it was poorly coded, but couldn't find it.
Phlamethrower wrote:(After hacking module command tables to remove the international help flag)

This is why I switched to the RISC OS 3.19 ROM, it's the first to understand that flag and avoids Module hackery.
Phlamethrower wrote:just pointing out that if you're trying to build a ROM, it probably makes sense to use "the" ROM build system :wink:

I'm afraid it makes zero sense to use the ROM build system if it requires DDE, which is why I've written everything from scratch in BASIC. I just need ROM versions of the various Modules listed in the OP and I'm pretty much done, the final piece being fixing up the hardcoded CLib entry branches.

Phlamethrower
Posts: 19
Joined: Fri Nov 24, 2017 1:35 pm

Re: RISC OS 3.20 ROM's?

Postby Phlamethrower » Wed Dec 06, 2017 10:32 pm

sirbod wrote:
Phlamethrower wrote:Grab a clean build environment (I used IOMD, although I'd expect almost any to work), make sure it's selected as the current

You lost me at the first step, where does one grab a build environment and select it as current #-o


https://www.riscosopen.org/wiki/documen ... M%20builds

sirbod wrote:
Phlamethrower wrote:'HackASWI' is a version of CallASWI that's been hacked to be ROMable (and will only work with RISC OS 3.11). (basically I just *save'd a copy after loading regular CallASWI on a 3.11 machine, then hacked out all the bits of the initialisation which write values to the module body)

I tried doing that with CLib?..have you seen the code in CLib's Init; what on earth is it doing? It branches to a block of code does some very odd things shifting bits around, to simply get the address of a data block. I tried to find it in the OSLib source to see why it was poorly coded, but couldn't find it.


Any disassembly to share? There's some funky stuff I added to allow it to pick different versions of routines optimised for different CPU architectures, but that'll be in the SWI handler when clients register. I can't think of much that would go on in the main module init.

sirbod wrote:
Phlamethrower wrote:(After hacking module command tables to remove the international help flag)

This is why I switched to the RISC OS 3.19 ROM, it's the first to understand that flag and avoids Module hackery.


True - but you also said that things broke when you tried moving modules around, so I figured 3.11 might have been a safer bet (plus I'll have a higher chance of understanding any error messages!)

sirbod wrote:
Phlamethrower wrote:just pointing out that if you're trying to build a ROM, it probably makes sense to use "the" ROM build system :wink:

I'm afraid it makes zero sense to use the ROM build system if it requires DDE, which is why I've written everything from scratch in BASIC. I just need ROM versions of the various Modules listed in the OP and I'm pretty much done, the final piece being fixing up the hardcoded CLib entry branches.


You'll essentially have to rewrite the whole DDE linker in BASIC, since ROM builds of C modules are AOF files.

sirbod
Posts: 737
Joined: Mon Apr 09, 2012 8:44 am
Location: Essex
Contact:

Re: RISC OS 3.20 ROM's?

Postby sirbod » Thu Dec 07, 2017 8:11 am

Phlamethrower wrote:
sirbod wrote:have you seen the code in CLib's Init; what on earth is it doing? It branches to a block of code does some very odd things shifting bits around, to simply get the address of a data block


Any disassembly to share?

Not to hand, but it's easy to find. Look at the Init code, a few instructions in it BL's to 17xxx(?), it's that code that's very odd looking.
Phlamethrower wrote:
sirbod wrote:This is why I switched to the RISC OS 3.19 ROM, it's the first to understand that flag and avoids Module hackery.


True - but you also said that things broke when you tried moving modules around, so I figured 3.11 might have been a safer bet (plus I'll have a higher chance of understanding any error messages!)

UtilityModule is staticly linked to the Kernel, so breaks horribly if you mix and match. Once I figured that out, I switched to the 3.19 Kernel and UtilityModule, with 3.11 Resources.
Phlamethrower wrote:You'll essentially have to rewrite the whole DDE linker in BASIC, since ROM builds of C modules are AOF files.

What we need are ROM builds of the Apps/Modules that staticly link to entry points outside of the Module, I can then fix up the B/BL's to the correct addresses when the Modules are inserted into the ROM image.

Alternatively compile/link all the C based Apps/Modules together as one block, with CLib at the head and just insert them all together without any requirement for fixups. We would have to go 4MB for that to work though as they wont fit in a 2MB image, I think we're being forced down this route anyhow due to space constraints and I'd rather not remove apps/Modules to fit alternative ones in.

I'm not certain all Apps/Modules are available to recompile though, so some fixups will probably be inevitable.

I persevered yesterday, despite giving up :roll: and managed to get a 3.20 build that boots to the desktop and appears to be functional. I had fun and games getting *HELP to work, in the end merging bits from RISC OS 3.71 as I couldn't find the relevent help tokens in either 3.11 or 3.19, they must be there somewhere, but I couldn't find them despite using grep.

So...CLib and its associates are the final piece in the puzzle...except for VProtect. R-Comp confirmed they don't have the source, but have put me in touch with someone that possibly does have it. I've emailed them and await a response.

Assuming we can get CLib et al sorted out, the final step will be producing a new !Boot with all the unnecessary rubbish stripped out and additional folders required for a major build update. It's probably going to be pretty bare bones as just about everything can go in a 4MB ROM except for NIC drivers.

Phlamethrower
Posts: 19
Joined: Fri Nov 24, 2017 1:35 pm

Re: RISC OS 3.20 ROM's?

Postby Phlamethrower » Thu Dec 07, 2017 6:43 pm

sirbod wrote:
Phlamethrower wrote:
sirbod wrote:have you seen the code in CLib's Init; what on earth is it doing? It branches to a block of code does some very odd things shifting bits around, to simply get the address of a data block


Any disassembly to share?

Not to hand, but it's easy to find. Look at the Init code, a few instructions in it BL's to 17xxx(?), it's that code that's very odd looking.


Oh, that. That's the relocation code. The C compiler doesn't support generating read-only position-independent code - it only supports read-only position-dependent code. Position-independent code is faked by having the linker generate a relocation function, which will go through the binary and fix up any absolute addresses so that they're correct for the address the binary was loaded to (therefore requiring the binary - or at least the data segment - to be writable). Sometimes you'll see this in assembler modules too (you can put a call to __RelocCode in your sources, and that will cause the linker to generate the relocation code).

For a ROM build of a module, the linker will be told the absolute address of the module, so it won't generate any relocation code.

sirbod wrote:
Phlamethrower wrote:You'll essentially have to rewrite the whole DDE linker in BASIC, since ROM builds of C modules are AOF files.

What we need are ROM builds of the Apps/Modules that staticly link to entry points outside of the Module, I can then fix up the B/BL's to the correct addresses when the Modules are inserted into the ROM image.


References to static data will need fixing up too. In fact, I think that's pretty much all that __RelocCode does. But I don't think it's possible to generate a module that's staticly linked to CLib and contains a (useful) __RelocCode.

If you wanted to, you could probably write a tool to convert softload C modules into ROM versions. Have it search for & interpret the relocation code, then search for the SCL init call and patch up the stubs so that they're already initialised. Chances are other changes to the initialisation would be needed as well (I don't think ROM modules call SCL SWIs at all) - you'd have to check the sources to the stubs to make sure.

sirbod
Posts: 737
Joined: Mon Apr 09, 2012 8:44 am
Location: Essex
Contact:

Re: RISC OS 3.20 ROM's?

Postby sirbod » Mon Dec 11, 2017 2:23 pm

Phlamethrower wrote:That's the relocation code. The C compiler doesn't support generating read-only position-independent code - it only supports read-only position-dependent code. Position-independent code is faked by having the linker generate a relocation function, which will go through the binary and fix up any absolute addresses so that they're correct for the address the binary was loaded to (therefore requiring the binary - or at least the data segment - to be writable)

I've had a look at a selection of C Modules, the "relative code faker" code is consistent so I should be able to write a function that tweaks the code to patch for the correct ROM address at build, call it, and then remove it from the end of the Module. What worries me however is if there's any data being stored or updated within the Module, even the most diligent testing isn't necessarily going to highlight Modules that will only work when softloaded.

CallASWI needs a rewrite to work from ROM as it's storing values within itself. It doesn't look like too much of a change is required, it just needs the SWI entry points and variables moving into the Module's RMA allocation, so they can be accessed by the various SWI routines.

You mentioned FPEmulator needs some tweaks to run in ROM, is that a lot of work?

Phlamethrower
Posts: 19
Joined: Fri Nov 24, 2017 1:35 pm

Re: RISC OS 3.20 ROM's?

Postby Phlamethrower » Mon Dec 11, 2017 3:14 pm

sirbod wrote:What worries me however is if there's any data being stored or updated within the Module, even the most diligent testing isn't necessarily going to highlight Modules that will only work when softloaded.


For well-behaved C modules, I think the only other bit of self-modification that goes on is when it initialises the CLib stubs.

sirbod wrote:CallASWI needs a rewrite to work from ROM as it's storing values within itself. It doesn't look like too much of a change is required, it just needs the SWI entry points and variables moving into the Module's RMA allocation, so they can be accessed by the various SWI routines.


For testing you could probably go with an earlier CallASWI, since the self-modification only appeared in version 0.12. I'm fairly certain CLib only requires it for OS_CallASWI / OS_CallASWIR12, so even the oldest version should do.

sirbod wrote:You mentioned FPEmulator needs some tweaks to run in ROM, is that a lot of work?


Just a couple of tweaks to the makefile.

sirbod
Posts: 737
Joined: Mon Apr 09, 2012 8:44 am
Location: Essex
Contact:

Re: RISC OS 3.20 ROM's?

Postby sirbod » Tue Dec 12, 2017 1:42 pm

Phlamethrower wrote:Just a couple of tweaks to the makefile.

Are you going to build it and eMail to me or PM a link to download it?
sirbod wrote:I've had a look at a selection of C Modules, the "relative code faker" code is consistent so I should be able to write a function that tweaks the code to patch for the correct ROM address at build, call it, and then remove it from the end of the Module.

I coded the "relative code faker" fixup code last night, which has got the latest SharedCLibrary working in ROM, confirmed by play testing Populous. Obviously, I've had to strip out all existing C Apps/Modules, so I'm pondering what to do about them:

  • Filer_Action
  • !Configure
  • !Draw
  • !Paint
  • !Edit
  • DOSFS
To avoid having to code two C entry fixup routines, it would probably be wise to include updated versions in the ROM. !Draw, !Paint and !Edit I've grabbed from my RISC OS 3.71 boot image for the time being and placed in $.Apps, we need to locate the last official Acorn versions. Not sure what to do about Filer_Action, DOSFS and !Configure, did any of these receive softload updates?

I've made a start on building the new !Boot, which I've based on Acorn's Uniboot and stripped out everything now in ROM. I'll update all remaining Modules with ROOL ones, once I know what's going into the ROM.

The next task is figuring out how to patch SharedCLibrary_LibInitAPCS_R and SharedCLibrary_LibInitModule in softloaded C Apps/Modules, so they work from ROM.

Which network driver Modules should we include in the ROM? All bundled with UniBoot, or just ones from ROOL's NicDrivers (if they work below RISC OS 3.5)?

Modules included in UniBoot: Ether1, Ether2, Ether3-8, EtherB, EtherH8, EtherH16, EtherM, EtherO
Modules included in NicDrivers: EtherB, EtherH16, EtherY

Again, we need to find the latest versions of any drivers we include.

Does anyone have a copy of CallASWI v0.11?

User avatar
danielj
Posts: 5345
Joined: Thu Oct 02, 2008 4:51 pm
Location: Manchester

Re: RISC OS 3.20 ROM's?

Postby danielj » Tue Dec 12, 2017 3:29 pm

You need EtherH8 in there for the minipodules in the A30x0/A4000. I'm not sure you should use the latest ROOL EtherH - it's missing certain functionality (EHTest and EHInfo IIRC). The version I used when I reflashed ROMs in some old etherH podules would seem to be the best - it works perfectly well in A5000. Will check which version it was this evening.

d.

Phlamethrower
Posts: 19
Joined: Fri Nov 24, 2017 1:35 pm

Re: RISC OS 3.20 ROM's?

Postby Phlamethrower » Tue Dec 12, 2017 3:51 pm

sirbod wrote:
Phlamethrower wrote:Just a couple of tweaks to the makefile.

Are you going to build it and eMail to me or PM a link to download it?


The changes are now in ROOL's CVS, so keep an eye out for FPEmulator 4.36 in tomorrow's PlingSystem download.

I also had a look at the Freeway/Internet/MimeMap/Net international help issue you spotted. Specifying the international help messages file in the module header is definitely redundant, but removing it is a bit of a pain due to the way CMHG is being used. The only downside to redundantly including the module header entry is a few bytes of wasted space, so I've left it as-is for now.

sirbod
Posts: 737
Joined: Mon Apr 09, 2012 8:44 am
Location: Essex
Contact:

Re: RISC OS 3.20 ROM's?

Postby sirbod » Tue Dec 12, 2017 5:22 pm

Phlamethrower wrote:The changes are now in ROOL's CVS, so keep an eye out for FPEmulator 4.36 in tomorrow's PlingSystem download.

Great, thanks.
Phlamethrower wrote:I also had a look at the Freeway/Internet/MimeMap/Net international help issue you spotted. Specifying the international help messages file in the module header is definitely redundant, but removing it is a bit of a pain due to the way CMHG is being used. The only downside to redundantly including the module header entry is a few bytes of wasted space, so I've left it as-is for now.

So long as the *command help isn't reliant on it, it can be ignored. Thanks for checking.

I looked at ABCLibrary earlier, its has several self-modifying code sections, so we're probably stuck with softloading it.

User avatar
danielj
Posts: 5345
Joined: Thu Oct 02, 2008 4:51 pm
Location: Manchester

Re: RISC OS 3.20 ROM's?

Postby danielj » Tue Dec 12, 2017 8:15 pm

EtherH 4.33

User avatar
flibble
Posts: 591
Joined: Tue Sep 22, 2009 10:29 am
Contact:

Re: RISC OS 3.20 ROM's?

Postby flibble » Tue Dec 12, 2017 8:17 pm

Given this is still a 26bit OS, provided the DCI4 versions of the drivers are in the podule rom would you need to put them in the core OS itself at all?

User avatar
danielj
Posts: 5345
Joined: Thu Oct 02, 2008 4:51 pm
Location: Manchester

Re: RISC OS 3.20 ROM's?

Postby danielj » Tue Dec 12, 2017 8:21 pm

The only advantage is them not being copied across and hogging module space?

d.

User avatar
flibble
Posts: 591
Joined: Tue Sep 22, 2009 10:29 am
Contact:

Re: RISC OS 3.20 ROM's?

Postby flibble » Tue Dec 12, 2017 8:30 pm

danielj wrote:The only advantage is them not being copied across and hogging module space?


At the expense of ROM space, which is tighter still than the RAM :)

User avatar
danielj
Posts: 5345
Joined: Thu Oct 02, 2008 4:51 pm
Location: Manchester

Re: RISC OS 3.20 ROM's?

Postby danielj » Tue Dec 12, 2017 8:44 pm

Although haven't it been decided to go for 4MB now? In which case we can stick Zarch in there too ;) :D

sirbod
Posts: 737
Joined: Mon Apr 09, 2012 8:44 am
Location: Essex
Contact:

Re: RISC OS 3.20 ROM's?

Postby sirbod » Tue Dec 12, 2017 10:18 pm

danielj wrote:EtherH 4.33

Can you PM it too me please.

dp11
Posts: 705
Joined: Sun Aug 12, 2012 8:47 pm

Re: RISC OS 3.20 ROM's?

Postby dp11 » Tue Dec 12, 2017 10:36 pm

Just a thought. If space is really tight I wonder if you can find any common sub expressions across all the modules.

sirbod
Posts: 737
Joined: Mon Apr 09, 2012 8:44 am
Location: Essex
Contact:

Re: RISC OS 3.20 ROM's?

Postby sirbod » Wed Dec 13, 2017 5:55 am

dp11 wrote:Just a thought. If space is really tight I wonder if you can find any common sub expressions across all the modules.

Although on the face of it that sounds like a good idea, in reality it would be a lot of work and introduce issues as ADR's, relative pointers and branches within Modules would all need adjusting.

One possibility is to use ZipFS to store Apps and Resources, but its hard to tell just how much space that would back or if it's even feasible at this stage as its not been coded. It's a waste of time looking at it until we know ZLib can be included.

At the moment the ROM is just below 2MB, with all C based Apps or Modules stripped out. If they can be patched to run from the new ROM, 4MB is the only option as there's over half a MB pending inclusion.

Does anyone fancy taking on the task of coding a command history Module? That would be a very useful addition.

sirbod
Posts: 737
Joined: Mon Apr 09, 2012 8:44 am
Location: Essex
Contact:

Re: RISC OS 3.20 ROM's?

Postby sirbod » Wed Dec 13, 2017 1:52 pm

Phlamethrower wrote:For testing you could probably go with an earlier CallASWI, since the self-modification only appeared in version 0.12. I'm fairly certain CLib only requires it for OS_CallASWI / OS_CallASWIR12, so even the oldest version should do.

I've now looked at 0.11, its not suitable for ROM as it's also storing values within the Module. There's no getting away from modifying the source on ROOL to make it suitable for ROM, it needs the variables and SWI entries moved into its private space.

User avatar
davidb
Posts: 1900
Joined: Sun Nov 11, 2007 10:11 pm
Contact:

Re: RISC OS 3.20 ROM's?

Postby davidb » Wed Dec 13, 2017 4:40 pm

sirbod wrote:Does anyone fancy taking on the task of coding a command history Module? That would be a very useful addition.

Can't you use LineEditor by Olly Betts?


Return to “hardware”

Who is online

Users browsing this forum: No registered users and 4 guests