It is currently Thu Jul 31, 2014 12:26 pm

All times are UTC [ DST ]




 Page 1 of 1 [ 26 posts ] 
Author Message
 Post subject: BeebEm save state UEFs
PostPosted: Thu Aug 26, 2010 9:12 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
Can anyone with a bit of knowledge about such things tell me what the difference is between a UEF cassette image and the UEF files generated by BeebEm's Save State option?

It seems like they're separate functions ... so it seems odd that they share the same file extension - which is a bit confusing ... ?

Sam.


Offline
 Profile  
 
PostPosted: Fri Aug 27, 2010 4:30 pm 
User avatar

Joined: Sun Jan 02, 2005 10:51 pm
Posts: 557
Location: London, UK
Samwise wrote:
Can anyone with a bit of knowledge about such things tell me what the difference is between a UEF cassette image and the UEF files generated by BeebEm's Save State option?

It seems like they're separate functions ... so it seems odd that they share the same file extension - which is a bit confusing ... ?

Sam.
It's exactly the same format. The only difference is the type of 'chunks' inside. Means you only have to write one file format handler for both functions -- kind of like XML, but sane :D

http://en.wikipedia.org/wiki/UEF_(file_format)

Greg


Offline
 Profile  
 
PostPosted: Sun Aug 29, 2010 3:22 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
That's kind've how I thought it works. It still seems odd to me that there's no obvious way for the end-user to determine at a glance the difference between a particular image type, and say a save state - which is a bit of a shame.

Sam.


Offline
 Profile  
 
PostPosted: Sun Aug 29, 2010 5:41 pm 
User avatar

Joined: Mon Mar 31, 2008 10:04 pm
Posts: 3158
Location: Obscurity
This is not 'at a glance' but if you open the file with Notepad or similar, a tape image will be complete garbage whereas the save-state will have the UEF File! header text at the start with BeebEm a few bytes along. (A hex editor is better but text editors are always kicking around for everyone :wink:)


Offline
 Profile  
 
PostPosted: Sun Aug 29, 2010 5:55 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
Oh, I appreciate it's possible to tell them apart if you look at the contents - but I'm talking about from a beginner's point of view. It's not possible to say to a newbie, that if they've downloaded a *.uef file, then it's a cassette tape image ... or a save state .. or whatever. I only noticed because I was writing a beginner's guide to BBC emulation for another site, and paused when I was about to write "if you have a .uef file, then it is a cassette tape image" which, of course, isn't necessarily true.

Comparing with the Spectrum world, for instance, I can see they don't have this confusion.

Sam.


Offline
 Profile  
 
PostPosted: Thu Oct 07, 2010 3:51 pm 

Joined: Tue May 20, 2003 8:21 pm
Posts: 540
I assume then that the state files are not being written compressed. I have a program I call UEF Viewer which displays messages, the version numbers and the number of phase shift changes.


Offline
 Profile  
 
PostPosted: Thu Oct 07, 2010 4:22 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
Yep, Fraser. I appreciate one can tell the two apart either with tools or perhaps as Martin described.

My point was that it's not possible to tell someone new to the world of emulation that a .uef is a cassette image and, for example, a .sta is a save state which is common in other emulation communities.

Instead you have to say, if you've downloaded a .uef it /may/ be a cassette image or it /may/ be a save state. Some emulators may handle these two differently and may need them to be loaded in with Load Tape rather than Load State, so it's potentially confusing to anyone who's not familiar with the use of emulation and/or Acorn machines. Loading in a .uef with Load Tape may fail if it's a save state image.

I was really just surprised when I first noticed that the same extension was in use for two different functions (from the users perspective) when it would be trivial to use two different extensions and make it plain to see e.g. .uef for cassettes and .ues (Uniifed Emulator State) perhaps.

Sam.


Offline
 Profile  
 
PostPosted: Thu Oct 07, 2010 8:51 pm 
User avatar

Joined: Sun Jan 02, 2005 10:51 pm
Posts: 557
Location: London, UK
The merging of purposes was clearly intentional, hence Unified Emulator Format. I imagine it was not foreseen that the two types of file would need to be treated differently - just 'load' or double-click and the emulator detects the content. I agree though that in the present context, organising a .ues extension would be useful.

There's another reason to move state snapshots to another extension - unlike tapes they're non-portable between emulators and, in the case of certain versions of BeebEm, they use chunks 'illegal' to the UEF. :evil:

However, will the loss of double-click behaviour be potentially a greater difficulty? (Just adopt a sub extension, .state.uef) And as un-renamed snapshots will turn up, will we be able to rephrase the instructions anyway?


Offline
 Profile  
 
PostPosted: Thu Oct 07, 2010 8:54 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
Why wouldn't you be able to use the double-click functionality, if there were two extensions?

Sam.


Offline
 Profile  
 
PostPosted: Thu Oct 07, 2010 9:05 pm 
User avatar

Joined: Sun Jan 02, 2005 10:51 pm
Posts: 557
Location: London, UK
Only because emulators don't yet register them both.


Offline
 Profile  
 
PostPosted: Thu Oct 07, 2010 11:44 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
well, I don't think that's a big issue. I'm sure it wouldn't take a huge amount of dev effort to implement support for the change of extension name in B-Em, BeebEm, Elkulator and ElectrEm which is where it would mainly be needed. I don't think MESS supports UEF save states atm so there's no change needed there.

I agree that because there are already save states "out there" which are named .uefs one could never state categorically that all .uefs are cassettes but I honestly don't believe there are that many save states out there. Apart from the sticky save state topic in this forum (and maybe scattered through other topics), I can't think of anywhere else that would host them.

So, just as noone really talks about/supports zips of individual files with associated .inf files, we could just rename any existing save state .uefs we come across to the new extension.

Another reason I think it's better to differentiate the two with extensions is because if you ask ppl to send you a cassette image (e.g. for archiving), you don't want them to send you a save state instead because they don't realise that that is different, and that it's the cassette image version which is really preferable from an archive point of view. I think in the early days some of the other platforms' emulator sites had save states instead of tape images, which they then needed to go back and replace, over time.

I started this topic as a sort of "why is this?" question but now I think about it, I wonder if we could actually get such a change into the spec / tools. I think for the minimal effort it would take, the longer-term benefits would outweigh the hassle.

Looking at the spec page you and Fraser seem to be the two main contributors to Thomas' specification. As you two almost certainly know more about the spec than I do, what would the two of you think if I dropped Thomas a line about it? Do you think this would be a worthwhile change to the spec?

Sam.


Last edited by Samwise on Fri Oct 08, 2010 10:09 am, edited 1 time in total.
Reason: No changes required to B-Em / Elkulator


Offline
 Profile  
 
PostPosted: Fri Oct 08, 2010 1:11 am 
User avatar

Joined: Sun Jan 02, 2005 10:51 pm
Posts: 557
Location: London, UK
By all means, Sam. I agree the distinction is important.

Greg


Offline
 Profile  
 
PostPosted: Fri Oct 08, 2010 10:05 am 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
OK, I've emailed Thomas and pointed him at this topic so I hope he'll stop by.

In the meantime, I've also realised that B-Em and Elkulator don't yet support save states currently so it's only actually BeebEm and ElectrEm which would need updating. All the more reason then to change the spec now - before our emulator maestro Tom gets round to adding support for it. :)

Fraser, if you have any comments on this proposal I'd be interested to hear them?

Sam.


Offline
 Profile  
 
PostPosted: Fri Oct 08, 2010 12:32 pm 

Joined: Fri Jan 14, 2005 4:56 pm
Posts: 807
Elkulator does have save states - see the file menu. B-em v2.0 doesn't but they've been implemented for v2.1. Neither uses the UEF format though.


Offline
 Profile  
 
PostPosted: Fri Oct 08, 2010 1:19 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
My bad, I only checked B-Em 2.0b as I had it to hand, and I knew the codebase was similar.

I see Elkulator uses a .snp extension. Is that your own format?

Any particular reason you chose not to use .uef for save states? Not that I mind, so long as cassettes and save states can be differentiated, I'm just curious?

Sam.


Offline
 Profile  
 
PostPosted: Fri Oct 08, 2010 4:21 pm 

Joined: Fri Jan 14, 2005 4:56 pm
Posts: 807
SNP is my own format. Elkulator saves a reasonable chunk of internal state which wouldn't really be applicable to other emulators or exportable to UEF without more emulator-specific extensions.

Besides, I've never really been a fan of UEF for purposes other than tapes - it's a bit of an 'and the kitchen sink' format.


Offline
 Profile  
 
PostPosted: Fri Oct 08, 2010 4:42 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
Fair enough - I don't know enough to comment on whether one should be preferred over the other or, indeed, whether there should be an attempt to standardise on one format. I think that's best left to cleverer ppl like yourselves.

But from a user point of view, I'm still strongly in favour of not using the same extension for tape images and save states. Haven't yet heard back from Thomas, but hopefully he'll pick up the message soon.

Sam.


Offline
 Profile  
 
PostPosted: Fri Oct 08, 2010 7:38 pm 
User avatar

Joined: Fri Feb 22, 2008 4:44 pm
Posts: 309
I agree about .uef being a tape and a save as being a pain as the emulator can't just run the file by double clicking on it.

This is also a problem with the Archimedes .adf format - sorry guys but the Amiga was already there! (And has about 10,000 more files than the Arch) - I suggest .ach.

I had the same problem with 2600 .bin files - although some emulator writers realized their folly and support .a26 as well.

- PJ


Offline
 Profile  
 
PostPosted: Fri Oct 08, 2010 8:38 pm 

Joined: Tue May 20, 2003 8:21 pm
Posts: 540
The UEF files at the preservation site have a secondary extention hq. So they are name.hq.uef. Something such like state could be a secondary extention for state images. If someone needs custom chunks along with the standard chunks for images then the format probably wants some updating.


Offline
 Profile  
 
PostPosted: Fri Oct 08, 2010 9:45 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
Unfortunately, I don't think a sub extension like name.sta.uef is really going to address the issue.

Windows by default does the horrible thing of hiding extensions out of the box which will confuse the hell out of new users again - they will see .sta files ...

Even when you don't have that behaviour configured, if you name something .sta.uef it is categorised as a .uef file when you look at them Windows Explorer in the List/Details view. So you can't group-by file extensions. :/

Sam.


Offline
 Profile  
 
PostPosted: Sat Oct 09, 2010 10:58 am 

Joined: Tue May 20, 2003 8:21 pm
Posts: 540
The UEF format contains disk images, cassette images, rom images, state images and the content information. I don't see any point in having umpteen formats with countless problems because of some minor ones in organising files. It isn't entirely an Acorn specific format although it heavily supports Acorn things. There are many minor computers that the format can be used for.


Offline
 Profile  
 
PostPosted: Sun Oct 10, 2010 1:21 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
I don't think anyone is denying the flexibility of the format - but in practical usage it isn't generally used for many of the things you've listed.

However, it's all moot, unless we hear back from Thomas.

Sam.


Offline
 Profile  
 
PostPosted: Fri Oct 22, 2010 6:06 pm 

Joined: Sat Dec 23, 2000 6:56 pm
Posts: 49
Apologies for taking forever to get here and comment. History first...

UEF came about because, by pure chance, nobody had seriously addressed Electron emulation before I happened to turn up. The primary software distribution format for Electrons was tape and there were no established tape archive formats. Furthermore, the prevalent file formats for media archival were ad hoc solutions like the manifold faces of the ssd/img/dfs/dsd/adf/adl/adm/etc family, and none of them is capable of representing the source media so as to reproduce the signal that would be encountered by the lowest level bits of logic hardware (which, admittedly, UEF couldn't very well before the work of Fraser, but at least it tried). Those specific ones can also be up to 50% of wasted space if formatted for ADFS since ADFS fills one side of the disk then fills the other, whereas they're all track interleaved. So it's not even like keeping things simple led to an optimal technical solution; doing something beyond being stricter in conventions and supplying better documentation seemed desirable.

There was therefore an extent to which I wanted to ignore and to sweep aside the existing formats on purely technical grounds. Which meant engineering not just a tape file format, but something more generic. UEF is also capable of storing multiple media because some software came as e.g. a ROM and a disc, with the user being expected to use the two together.

The ability to store additional archival material inside a UEF was originally supposed to be a point of emphasis, especially as DOS was still significant amongst the emulator audience in 1999. It would have made what you download more like obtaining the original product rather than like obtaining the copied media, and allowed for much more interesting library navigation.

State saves were included for completeness but I intended to acknowledge the difficulty of doing these in a cross-emulator manner. At the time there were quite a few unanswered questions about the exact inner workings of the Electron ULA. Which, naturally, makes it difficult to determine the minimum amount of information that can record the state. The nature of emulation is also that software can run quite well even if you have an incorrect emulation, provided that it is consistently incorrect. Which sometimes creates problems opening state saves from one version of the same emulator to the next, never mind between emulators. If you look at something like the ZX Spectrum, the state snapshot formats essentially blossom outwards from the popularity of the Multiface and other similar bits of hardware that could do a state save on the real hardware, which created a real backlog of files for conversion to something PC readable. As far as I'm aware, there was nothing like that for either the Electron or BBC, quite probably because the BBC was a much more diverse hardware platform than most of the other machines of its category.

It'll be otherwise forgotten now, but the very early versions of ElectrEm supported UEF only. SSD, DSD, etc could be converted to UEF using an external tool, my feeling being that since there were essentially no Electron archives at the time, it was safe to make the 'legacy' stuff awkward. Obviously I had to relent on that one.

It eventually became clear that tape alone was the main purpose for which others were using UEF. Hence the definition of the UEF-Tape subset, which was essentially an exercise in branding that didn't extend to file extensions.

In retrospect, what has really defined the whole process is tools. It's the flow inward of converted media that has defined the actual real uses of the file format, not the hypothetical limits of the thing. And, inevitably, emulators have responded only to actual user need. MakeUEF and FreeUEF (both the original versions and Fraser's massive improvements on MakeUEF) have defined its ghetto; the absence of a disk reading/writing tool versus the many tools available for SSD/DSD being just as relevant.

So, pragmatically:
  • SSD/DSD aren't going away;
  • state snapshots are rarely distributed since they tend to be a bit flakey;
  • immeasurably small amounts of software require you to use more than one type of media simultaneously.

In my mind, the questions are (i) to what degree is having everything together in one format confusing; and (ii) how difficult would it be to adapt to allow multiple synonyms for a UEF file?

Besides breaking existing software, my primary concerns with splitting the format are that if you start by saying 'ueftape' is for tapes, 'uefdisk' is for disks, etc, then to what degree do you guarantee the type of content? If you try to say that tape chunks aren't allowed in a disk file and so on then you end up adding a burden to the specification. You also either create the arguably more confusing situation where a ueftape can be renamed to a uefdisk and will still work perfectly with the tool, but is actually a tape, or else you push more complexity onto tool authors.

We're coming from however many tens of thousands of files already being out there just as 'uef' with a large amount of the conversion work being done. I don't think is quite the same as any other technology transition would be because in this case it's safe to assume that there's a single body of work of a fixed size from which something like 99% of the UEFs that will ever exist will be drawn (ie, the complete set of all commercial releases, which isn't going to grow by much from here onwards), and that a majority of that work is already available as plain UEFs.

I know that OS X and I believe that Windows now has a feature whereby you can provide a bit of logic to the Finder/Explorer that will quickly examine a file, including its contents, and come up with an appropriate icon. That's normally for providing thumbnails of pictographic content, but if suitable bits were written so that a UEF with primarily tape chunks looked like a tape, one with primarily disk chunks looked like a disk, etc, to what extent would that answer concerns about confusion?

If we were to split, then I'm not completely sure what the correct split would be. Assuming that we prioritise whatever would happen for users of Windows as being the primary concern, does anybody know if it has any special rules for file extensions longer than the traditional DOS/Windows three characters long? Is there any sort of legacy stuff in there that would e.g. first look for an application that knows 'ueftape' but failing that just look for someone that knows the truncated version, 'uef'?


Offline
 Profile  
 
PostPosted: Fri Oct 22, 2010 6:34 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
Hi, Thomas.

I'm just on the way out, so can't reply to your msg in full ... but I just wanted to throw in the ring that there is an alternative that would solve the problem I originally raised without trying to change the spec to mandate that all save saves must be, for example, .ues, all tapes are .uef and all discs are .ued. I can see that would cause problems with existing archives.

Instead, the spec could simply allow the addition of alternative extensions i.e. allowing .ues to represent a .uef save state or even just as a complete alternative to .uef.

If the spec allowed for that and if ElectrEm and BeebEm could default to saving states with a .ues extension then that would practically solve the problem going into the future.

It's not a huge deal right now, as you say, because there are few save states doing the rounds - there is only the sticky topic in these forums that I know about. However, looking ahead at trying to build this site into something which allows members to contribute more, I'd quite like to encourage the contribution of save states for games ... and it'd be so much neater to be able to distinguish between save states and cassette images through an FTP interface ...

Sam.


Offline
 Profile  
 
PostPosted: Sat Oct 30, 2010 12:07 pm 

Joined: Sat Dec 23, 2000 6:56 pm
Posts: 49
Samwise wrote:
... the spec could simply allow the addition of alternative extensions i.e. allowing .ues to represent a .uef save state or even just as a complete alternative to .uef.


Oh, I understand that, I just feel that the knock-on issues are:
  • if .ues is a .uef save state only — i.e. .uef with restrictions — then the restrictions add burden to the spec
  • if .ues is just an alternate name for .uef then you can't actually assume that a .ues is a state save and the situation is arguably more complicated to explain. You go from "a .uef can be this, that or this" to "a .uef can be this, that or this, as can a .ues but people normally use .ues for save states, though they don't have to"

I have had designers who, when I've asked to get something in PNG rather than JPEG have renamed the same file from .jpg to .png and sent it back to me...

That being said, the point I was trying to make with the reference to having caved on SSD/DSD support is that a standard works only to the extent that a community accepts it, and the role of a tool programmer (with an emulator just being another type of tool, really) should always be at least partly reactive to what the wider community wants.


Offline
 Profile  
 
PostPosted: Sat Oct 30, 2010 1:44 pm 
User avatar

Joined: Mon Mar 14, 2005 10:13 pm
Posts: 1743
ThomasHarte wrote:
Samwise wrote:
... the spec could simply allow the addition of alternative extensions i.e. allowing .ues to represent a .uef save state or even just as a complete alternative to .uef.

Oh, I understand that, I just feel that the knock-on issues are:
  • if .ues is a .uef save state only — i.e. .uef with restrictions — then the restrictions add burden to the spec

OK, I don't think it'd a huge burden, but fair enough. That's why I suggested the .ues synonym option instead, which would require a very small optional change to the current spec and wouldn't require any changes at all to existing tools/code (though to make my overall suggestion work, I'd like to see ElectrEm and BeebEm default to using .ues for save states).

ThomasHarte wrote:
  • if .ues is just an alternate name for .uef then you can't actually assume that a .ues is a state save and the situation is arguably more complicated to explain. You go from "a .uef can be this, that or this" to "a .uef can be this, that or this, as can a .ues but people normally use .ues for save states, though they don't have to"

  • The reason I suggested this approach is because it's exactly what the situation already is - the emulator/tool author currently can't assume that a .uef is a cassette image/save state/anything else. Adding .ues as a synonym makes absolutely no difference to this. My point is if you add the synonym into the spec, then tool/emulator authors don't have to do anything - they're automatically still compliant with the spec. However, with that change, if ElectrEm and BeebEm were then also updated to calling their save states by .ues then it's my belief that the convention would simply emerge for save states to be .ues. Yes, programmers need to be aware that .ues is only a synonym for .uef and they can't rely on .ues being a save state - but that's no difference to the current state of play. My proposed change is to make it simpler for users, not the coders.

    ThomasHarte wrote:
    I have had designers who, when I've asked to get something in PNG rather than JPEG have renamed the same file from .jpg to .png and sent it back to me...

    haha. Yeah, that'd be annoying. :)

    But you're not comparing like-with-like - in the case we're talking about, the two file extensions (.uef and .ues) /would/ be interchangable so sending it back renamed would actually work in that case.

    ThomasHarte wrote:
    That being said, the point I was trying to make with the reference to having caved on SSD/DSD support is that a standard works only to the extent that a community accepts it, and the role of a tool programmer (with an emulator just being another type of tool, really) should always be at least partly reactive to what the wider community wants.

    I wouldn't disagree with any of that. ;)

    EDIT: And I would also be suggesting alternative filename extension synonyms to represent disc images/anything else ... except that, practically speaking, my perception is that the community currently only really uses the UEF spec for cassette archiving and, to a much lesser extent, save states.

    Sam.


    Offline
     Profile  
     
    Display posts from previous:  Sort by  
     Page 1 of 1 [ 26 posts ] 

    All times are UTC [ DST ]


    Who is online

    Users browsing this forum: No registered users and 1 guest


    You cannot post new topics in this forum
    You cannot reply to topics in this forum
    You cannot edit your posts in this forum
    You cannot delete your posts in this forum
    You cannot post attachments in this forum

    Search for:
    Jump to:  

    phpBB skin developed by: John Olson
    Powered by phpBB® Forum Software © phpBB Group