Sound
Matchstick mixes sound on its own thread, so audio keeps playing when your simulation stalls. You say what should play. The host mixes it, sends it to the speakers and adds it to recorded videos.
You will learn
- How to load a sound and play it
- How to change or stop a sound that is playing
- How to place sounds in 3D
- How to group sounds on buses and save volume settings
- How to stream music and play audio you generate
Get the sound manager
Ask the simulation host for the sound manager once, and keep it. Everything else in this guide starts from it.
MGDF::ComObject<IMGDFSoundManager> sound;
host->GetSound(sound.Assign());
// One list is enough. Submit resets it, so you can reuse it every update
MGDF::ComObject<IMGDFSoundCommandList> commands;
sound->CreateCommandList(commands.Assign());A command list records what you want to happen. Nothing changes until you call Submit, which hands the whole list to the audio thread to apply in order.
NoteYou can load samples, create streams and command lists, submit lists and read state from any thread. Make the other calls from the simulation thread. A command list itself belongs to one thread at a time.
Load and play a sound
A sample set is a WAV or Ogg Vorbis file loaded into memory. Use it for effects that you play often. The file loads on a worker thread, so creating one never blocks.
MGDF::ComObject<IMGDFReadOnlyVirtualFileSystem> vfs;
host->GetVFS(vfs.Assign());
MGDF::ComObject<IMGDFReadOnlyFile> file;
vfs->GetFile(L"sounds/chime.wav", file.Assign());
MGDF::ComObject<IMGDFSoundSamples> chime;
sound->CreateSoundSamples(file, nullptr, chime.Assign());You don't have to wait for the load. Play the sound right away, and the voice starts when the samples are ready.
MGDFVoiceDesc desc{};
desc.Bus = sound->GetBus(MGDF_BUS_SOUND);
desc.Volume = 1;
desc.Pitch = 1;
desc.GainHF = 1;
// Zero means the description was invalid
MGDFVoiceId voice = commands->Play(chime, &desc);
sound->Submit(commands);Each Play makes a new voice, and you can play one sample set on many voices at once. Keep the ID if you want to change the voice later. A one-shot ends by itself, so you can ignore its ID.
PitfallA zeroed MGDFVoiceDesc does not play. Pitch must be greater than zero, zero Volume is silent, and zero GainHF removes the high frequencies. Set all three.
| Field | Set it to |
|---|---|
Bus | The bus the voice plays on. |
Volume | Linear volume. 1 is the recorded level. |
Pitch | Playback rate. 1 is the recorded rate. |
GainHF | High-frequency gain from 0 to 1. Lower it to muffle a sound. |
FadeIn | Fade-in time in seconds. |
StartTime | When to start, on the clock from GetAudioTime. Zero starts as soon as possible. |
Segment | Which part of the file to play. Zero unless you loaded segments. |
To start two sounds in step, read GetAudioTime once and give both the same StartTime a little in the future. The mixer starts each on the exact output frame.
Split one file into segments
One file can hold several sounds, or an intro followed by a loop. Describe each part as a segment when you load the file, then pick one with Segment.
const MGDFSoundSegment segments[]{
{0.0, 1.5, FALSE}, // start, length in seconds, loop
{1.5, 0.0, TRUE}, // a length of zero runs to the end of the file
};
const MGDFSoundSamplesDesc samplesDesc{segments, 2};
sound->CreateSoundSamples(file, &samplesDesc, engine.Assign());A looping segment repeats until you stop the voice.
Change and stop a voice
Record changes against the voice ID, then submit. Most setters take a ramp time in seconds, so a change is smooth without any work each frame.
commands->SetVolume(voice, 0.5f, 0.2f); // reach half volume over 0.2 seconds
commands->SetPitch(voice, 1.2f, 0); // at once
commands->Stop(voice, 0.1f); // fade out, then end
sound->Submit(commands);A command for a voice that has ended does nothing, so you never need to check first. Ask GetVoiceStatus when you do want to know.
| Status | The voice |
|---|---|
MGDF_VOICE_PENDING | Waits for submission, for its samples to load, or for its start time. |
MGDF_VOICE_PLAYING | Plays. |
MGDF_VOICE_VIRTUAL | Keeps time but is silent, because louder voices took every source. |
MGDF_VOICE_PAUSED | Is on a paused bus. |
MGDF_VOICE_ENDED | Finished, was stopped, or failed to load. Unknown IDs report this too. |
Place sounds in 3D
Set Spatial to make a voice come from a point in your world. Tell the mixer where the listener is, and it pans each spatial voice to match.
// Once per update, from your camera
const MGDFSoundListener listener{
cameraPosition, cameraVelocity, cameraForward, cameraUp};
commands->SetListener(&listener);
MGDFVoiceDesc desc{};
desc.Bus = sound->GetBus(MGDF_BUS_SOUND);
desc.Volume = 1;
desc.Pitch = 1;
desc.GainHF = 1;
desc.Spatial = TRUE;
desc.Position = {4, 0, -2};
MGDFVoiceId voice = commands->Play(footstep, &desc);
// Later, as the source moves. Ramp over one update to avoid steps
commands->SetSpatial(voice, &position, &velocity, 0, 1.0f / 60);
sound->Submit(commands);A spatial voice needs a mono file. Stereo files are for 2D voices, such as music beds and menu sounds, which play as recorded.
PitfallThe listener's right is Forward cross Up, as in a right-handed world. If your world has +X right, +Y up and +Z forward, left and right are swapped. Negate Z in every position, velocity and direction that you pass to the sound API.
The mixer pans a spatial voice but does not make it quieter with distance. Work out the falloff you want and set it with SetVolume. Use SetGainHF to muffle a sound that is far away or behind a wall.
Velocities drive the Doppler shift. Tune it with SetSpeedOfSound and SetDopplerFactor on the sound manager, or leave every velocity at zero to turn it off.
Too many sounds at once
The mixer has a fixed number of sources. When more voices play than it has sources, it keeps the ones that matter most: first by Priority, then by how loud each one is right now.
A voice that loses its source becomes virtual. It is silent but keeps its place, and it comes back at the right point when a source is free. A looping engine across the map costs nothing until the player walks over to it.
Raise Priority for sounds that must never drop out, such as dialogue. Call GetStats to see how many voices are real and how many are virtual.
Group sounds on buses
Every voice plays on a bus. A bus has its own volume, mute and pause, and passes them down to its children. Three buses always exist.
| Bus | Use it for |
|---|---|
MGDF_BUS_MASTER | Everything. It is the parent of every other bus. |
MGDF_BUS_SOUND | Sound effects. |
MGDF_BUS_MUSIC | Music. Streams play here by default. |
Create your own buses below these to control a group of sounds together.
const MGDFBusDesc busDesc{
L"Dialogue", sound->GetBus(MGDF_BUS_SOUND), 1.0f, "game.dialogueVolume"};
MGDFBusId dialogue = 0;
sound->CreateBus(&busDesc, &dialogue);
// Pause the game world but keep the menu music
commands->SetBusPaused(sound->GetBus(MGDF_BUS_SOUND), TRUE);
commands->SetBusVolume(sound->GetBus(MGDF_BUS_MUSIC), 0.4f, 0.5f);
sound->Submit(commands);Save volume settings
The last value in MGDFBusDesc is an optional preference name. GetPreferences returns the current volume of every bus that has one, ready to save.
MGDF::ComObject<IMGDFPreferenceSet> preferences;
sound->GetPreferences(preferences.Assign());
MGDF::ComObject<IMGDFGame> game;
host->GetGame(game.Assign());
game->SetPreferences(preferences);
game->SavePreferences();The host restores Master, Sound and Music at startup from host.masterVolume, host.soundVolume and host.musicVolume. For a bus of your own, read its preference and pass the value as Volume when you create the bus.
Stream music
A sound stream plays a WAV or Ogg Vorbis file from disk without loading all of it. Use it for music and long ambience. It keeps playing while your simulation is busy, for example during a level load.
MGDF::ComObject<IMGDFSoundStream> music;
sound->CreateSoundStream(musicFile, music.Assign());
music->SetLoop(12.0, 0); // after the intro, repeat from 12 seconds to the end
music->Play(1.0f); // fade in over one second
// Later
music->Pause(0.5f); // fade out and hold the position
music->Stop(2.0f); // fade out and return to the startA stream takes no command list. Call it directly from one thread. Without SetLoop it plays once, unless an Ogg Vorbis file has LOOPSTART and LOOPLENGTH comments, which set the loop for you.
PitfallReleasing the last reference to a stream stops it. Keep the ComObject for as long as the music should play.
Play audio you generate
A PCM stream is a buffer of samples that you fill yourself. Use it for a video's sound track, voice chat or a synthesizer.
const MGDFSoundPcmDesc pcmDesc{2, 48000, 0}; // stereo, 48 kHz, default buffer
MGDF::ComObject<IMGDFSoundPcmStream> pcm;
sound->CreatePcmStream(&pcmDesc, pcm.Assign());
MGDFVoiceId voice = commands->PlayPcm(pcm, &desc);
sound->Submit(commands);
// Each update: interleaved floats, one per channel in each frame
UINT32 written = pcm->Write(samples, frames);
// When there is no more to play
pcm->End();Write never waits. It returns how many frames fit, so keep the rest for the next update. GetWritableFrames tells you how much room there is. After End, the voice finishes what is queued and then ends.
To keep pictures in step with the sound, call GetPlayPosition. It returns how many of your frames have played and the audio time at which that was true. If the buffer runs dry the voice plays silence and Underruns in the stats goes up.
Output and recording
The host mixes at 48 kHz and plays through the default Windows output device. When the player changes device, the mix moves to the new one and audio time keeps running.
A video from IMGDFRecorder carries the same mix as its sound track, in time with the picture. You do nothing extra to record sound.
NoteSet the host.hrtf preference to 1 for players on headphones. Spatial voices then sound in front, behind, above and below, not only left and right. The host reads the preference at startup.