[IDEA] Classic Sound Extensions

[FavoritoHJS]

or: an answer to why do we need CEF for any custom sounds

Something I noticed while binging adventure maps in NA2 is that almost every single map recommends CEF for music...and just music.
I wonder whether this can be better served with a protocol extension... I'm sure someone else suggested a similar idea before (probably multiple times, even!), but assuming that this whole idea isn't impossible to begin with, here's my idea.
also sorry if this is in the wrong topic and should have been in Techno Babble instead...

Suggestion guideline:

• Register a Sound.
  • Inputs:
    • short identifier identifying this sound.
    • string containing the URL to a sound effect.
    • Generic metadata relating to the sound, potentially including:
      • int containing size of compressed audio in bytes
      • short indicating codec type
      • float containing the length of audio in seconds
      • whatever other info might be useful for the receiver.
  • Client then uses the identifier to register the sound, such that future packets using said identifier will use the sound.
  • If a sound is already registered to said identifier, the newly-registered sound takes priority, implicitly unregistering the existing sound.
  • Cases of invalid URLs, failure to find/download a sound, and such errors should be printed in chat.
  • Any identifier less than 256 is reserved by ClassiCube, and should not be used for any reason except overriding existing sounds.
  • Additionally, the identifier 0 is special and must not be assigned to.
  • Note: Due to limitations of the string type this can only support URLs up to 64 characters long. This could be a problem with deep path hierarchies, especially with a long domain name.
    This is a problem common to all packets that use URLs, but is probably worse here as you usually want to store the sounds with a descriptive name in a descriptive path.
• Unregister a sound
  • Inputs:
    • short identifier for the sound to be unregistered.
  • Unregisters the sound. Might be unneeded if a blank URL does this as well, but I would prefer a cleaner solution.
• Link a sound
  • Inputs:
    • short containing the sound identifier to be linked.
    • byte containing the "type" of sound to be linked. Think of this as if the sound is registered as music, footsteps, breaking...
    • byte containing the "subtype" to be linked to. Think of this as in a specific category of blocks to have this sound.
      • Some types might not have a meaningful "subtype" assignment. These should have this value set to 0 by the server, and have the client ignore this value.
    • 2 byte containing minimum and maximum sound intensity
    • 2 byte containing minimum and maximum random pitch, in units of 1/128 speed from 0,5 to 2,5.
      • A server must ensure a maximum random value is equal or greater to the corresponding minimum random value.
  • Mostly present to give the server a way to add sounds without manually sending every single sound that would be played, so even with occasional network lag spikes footstep many normal kinds of sounds are played as normal.
• Unlink a Sound.
  • Inputs:
    • short containing the sound identifier to be unlinked. Set to 0 to indicate all links in the type.
    • byte containing the "type" that this identifier will lose its link to.
    • byte containing the "subtype" to be unlinked.
      • Some types might not have a meaningful "subtype" assignment. These should have this value set to 0 by the server, and have the client ignore this value: this means all sounds from this type are removed.
  • This allows for whole types of sounds to lose their links to existing identifiers, to then have custom sound identifiers linked. Ever wanted your metallic blocks to sound extra metallic?
• Play a sound
  • Inputs:
    • short containing the sound identifier to be played.
    • byte containing whether this sound is to be considered special -- Play from Everywhere, Count as Music, the like.
    • byte containing sound intensity, or 0 if it is to be random.
    • byte containing sound pitch, or 0 if it to be random.
      • This does mean a speed of exactly 0,5x is unreachable without having the link specify that as the only valid random value.
    • 3 int (or short if ExtEntityPositions is unset) containing the position of where the sound is played from.
      • Ignored by clients if Play from Everywhere is set.
  • If Count as Music is set, the currently playing music is stopped as soon as another sound with that flag is set, and no music will be played until this sound ends.
    • This includes identifier 0, allowing for early stopping of a very long track.
• Reset Identifiers
  • Resets the identifier table to the initial state at startup (identifiers <256 back to initial values, everything else unregistered), as well as the assigned sounds.

Open questions:

• What to do if a malicious server sends a large amount of huge sound files?
• What to do if a sound can't be played by the client due to codec concerns? Ideally should allow for a fallback sound to be sent.
  • Might be ignored if only some codecs are allowed by protocol. WAV and Opus ought to be enough for everyone, right?
• How to handle implementations that might not support assigning up to 64k sounds? Or those with limits to simultaneous playback?
• How to handle edge cases, such as linking or playing an unassigned identifier

PS: anyone know how to make an inline code block?

[SebadroidX]

It would be nice to have those types of utilities, either to add new sounds, or for plugins (if applicable).

[SPANISH]:Estaria bien tener ese tipo de utilidades, ya sea como para agregar nuevos sonidos, y para plugins (si es el caso).