Is there a way to speed up audio .wav file using some SDL_Mixer's function?

Question

I am making a simple game whose audio speed should increase as the player is approaching the end of the level it is playing. So now I was wondering if there was a way to do this using SDL_Mixer. If SDL_Mixer is not the way to go could you please tell me how could I make this change in the audio file itself to make it faster. I am working with a 8-bit .wav file with 2 channels at the samplerate of 22050.

Answer 1

A few years back I was trying to achieve something very similar and, after a lot of web search, I came up with this solution, involving using Mix_RegisterEffect function, which got close:

#include <SDL2/SDL.h>
#include <SDL2/SDL_mixer.h>
#include <iostream>
#include <cstdlib>
#include <cmath>

/* global vars */
Uint16 audioFormat;  // current audio format constant
int audioFrequency,  // frequency rate of the current audio format
    audioChannelCount,  // number of channels of the current audio format
    audioAllocatedMixChannelsCount;  // number of mix channels allocated

static inline Uint16 formatSampleSize(Uint16 format)
{
    return (format & 0xFF) / 8;
}

// Get chunk time length (in ms) given its size and current audio format
static int computeChunkLengthMillisec(int chunkSize)
{
    /* bytes / samplesize == sample points */
    const Uint32 points = chunkSize / formatSampleSize(audioFormat);

    /* sample points / channels == sample frames */
    const Uint32 frames = (points / audioChannelCount);

    /* (sample frames * 1000) / frequency == play length, in ms */
    return ((frames * 1000) / audioFrequency);
}

// Custom handler object to control which part of the Mix_Chunk's audio data will be played, with which pitch-related modifications.
// This needed to be a template because the actual Mix_Chunk's data format may vary (AUDIO_U8, AUDIO_S16, etc) and the data type varies with it (Uint8, Sint16, etc)
// The AudioFormatType should be the data type that is compatible with the current SDL_mixer-initialized audio format.
template<typename AudioFormatType>
struct PlaybackSpeedEffectHandler
{
    const AudioFormatType* const chunkData;  // pointer to the chunk sample data (as array)
    const float& speedFactor;  // the playback speed factor
    int position;  // current position of the sound, in ms
    const int duration;  // the duration of the sound, in ms
    const int chunkSize;  // the size of the sound, as a number of indexes (or sample points). thinks of this as a array size when using the proper array type (instead of just Uint8*).
    const bool loop;  // flags whether playback should stay looping
    const bool attemptSelfHalting;  // flags whether playback should be halted by this callback when playback is finished
    bool altered;  // true if this playback has been pitched by this handler

    PlaybackSpeedEffectHandler(const Mix_Chunk& chunk, const float& speed, bool loop, bool trySelfHalt)
    : chunkData(reinterpret_cast<AudioFormatType*>(chunk.abuf)), speedFactor(speed),
      position(0), duration(computeChunkLengthMillisec(chunk.alen)),
      chunkSize(chunk.alen / formatSampleSize(audioFormat)),
      loop(loop), attemptSelfHalting(trySelfHalt), altered(false)
    {}

    // processing function to be able to change chunk speed/pitch.
    void modifyStreamPlaybackSpeed(int mixChannel, void* stream, int length)
    {
        AudioFormatType* buffer = static_cast<AudioFormatType*>(stream);
        const int bufferSize = length / sizeof(AudioFormatType);  // buffer size (as array)
        const int bufferDuration = computeChunkLengthMillisec(length);  // buffer time duration
        const float speedFactor = this->speedFactor;  // take a "snapshot" of speed factor

        // if there is still sound to be played
        if(position < duration || loop)
        {
            // if playback is unaltered and pitch is required (for the first time)
            if(!altered && speedFactor != 1.0f)
                altered = true;  // flags playback modification and proceed to the pitch routine.

            if(altered)  // if unaltered, this pitch routine is skipped
            {
                const float delta = 1000.0/audioFrequency,   // normal duration of each sample
                            vdelta = delta*speedFactor; // virtual stretched duration, scaled by 'speedFactor'

                for(int i = 0; i < bufferSize; i += audioChannelCount)
                {
                    const int j = i/audioChannelCount; // j goes from 0 to size/channelCount, incremented 1 by 1
                    const float x = position + j*vdelta;  // get "virtual" index. its corresponding value will be interpolated.
                    const int k = floor(x / delta);  // get left index to interpolate from original chunk data (right index will be this plus 1)
                    const float proportion = (x / delta) - k;  // get the proportion of the right value (left will be 1.0 minus this)

                    // usually just 2 channels: 0 (left) and 1 (right), but who knows...
                    for(int c = 0; c < audioChannelCount; c++)
                    {
                        // check if k will be within bounds
                        if(k*audioChannelCount + audioChannelCount - 1 < chunkSize || loop)
                        {
                            AudioFormatType leftValue =  chunkData[(  k   * audioChannelCount + c) % chunkSize],
                                            rightValue = chunkData[((k+1) * audioChannelCount + c) % chunkSize];

                            // put interpolated value on 'data' (linear interpolation)
                            buffer[i + c] = (1-proportion)*leftValue + proportion*rightValue;
                        }
                        else  // if k will be out of bounds (chunk bounds), it means we already finished; thus, we'll pass silence
                        {
                            buffer[i + c] = 0;
                        }
                    }
                }
            }

            // update position
            position += bufferDuration * speedFactor; // this is not exact since a frame may play less than its duration when finished playing, but its simpler

            // reset position if looping
            if(loop) while(position > duration)
                position -= duration;
        }
        else  // if we already played the whole sound but finished earlier than expected by SDL_mixer (due to faster playback speed)
        {
            // set silence on the buffer since Mix_HaltChannel() poops out some of it for a few ms.
            for(int i = 0; i < bufferSize; i++)
                buffer[i] = 0;

            if(attemptSelfHalting)
                Mix_HaltChannel(mixChannel);  // XXX unsafe call, since it locks audio; but no safer solution was found yet...
        }
    }

    // Mix_EffectFunc_t callback that redirects to handler method (handler passed via userData)
    static void mixEffectFuncCallback(int channel, void* stream, int length, void* userData)
    {
        static_cast<PlaybackSpeedEffectHandler*>(userData)->modifyStreamPlaybackSpeed(channel, stream, length);
    }

    // Mix_EffectDone_t callback that deletes the handler at the end of the effect usage  (handler passed via userData)
    static void mixEffectDoneCallback(int, void *userData)
    {
        delete static_cast<PlaybackSpeedEffectHandler*>(userData);
    }

    // function to register a handler to this channel for the next playback.
    static void registerEffect(int channel, const Mix_Chunk& chunk, const float& speed, bool loop, bool trySelfHalt)
    {
        Mix_RegisterEffect(channel, mixEffectFuncCallback, mixEffectDoneCallback, new PlaybackSpeedEffectHandler(chunk, speed, loop, trySelfHalt));
    }
};

// Register playback speed effect handler according to the current audio format; effect valid for a single playback; if playback is looped, lasts until it's halted
void setupPlaybackSpeedEffect(const Mix_Chunk* const chunk, const float& speed, int channel, bool loop=false, bool trySelfHalt=false)
{
    // select the register function for the current audio format and register the effect using the compatible handlers
    // XXX is it correct to behave the same way to all S16 and U16 formats? Should we create case statements for AUDIO_S16SYS, AUDIO_S16LSB, AUDIO_S16MSB, etc, individually?
    switch(audioFormat)
    {
        case AUDIO_U8:  PlaybackSpeedEffectHandler<Uint8 >::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
        case AUDIO_S8:  PlaybackSpeedEffectHandler<Sint8 >::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
        case AUDIO_U16: PlaybackSpeedEffectHandler<Uint16>::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
        default:
        case AUDIO_S16: PlaybackSpeedEffectHandler<Sint16>::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
        case AUDIO_S32: PlaybackSpeedEffectHandler<Sint32>::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
        case AUDIO_F32: PlaybackSpeedEffectHandler<float >::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
    }
}

// example
// run the executable passing an filename of a sound file that SDL_mixer is able to open (ogg, wav, ...)
int main(int argc, char** argv)
{
    if(argc < 2) { std::cout << "missing argument" << std::endl; return 0; }

    SDL_Init(SDL_INIT_AUDIO);
    Mix_OpenAudio(MIX_DEFAULT_FREQUENCY, MIX_DEFAULT_FORMAT, MIX_DEFAULT_CHANNELS, 4096);
    Mix_QuerySpec(&audioFrequency, &audioFormat, &audioChannelCount);  // query specs
    audioAllocatedMixChannelsCount = Mix_AllocateChannels(MIX_CHANNELS);

    float speed = 1.0;
    Mix_Chunk* chunk = Mix_LoadWAV(argv[1]);

    if(chunk != NULL)
    {
        const int channel = Mix_PlayChannelTimed(-1, chunk, -1, 8000);
        setupPlaybackSpeedEffect(chunk, speed, channel, true);

        // loop for 8 seconds, changing the pitch dynamically
        while(SDL_GetTicks() < 8000)
            speed = 1 + 0.25*sin(0.001*SDL_GetTicks());
    }
    else
        std::cout << "no data" << std::endl;

    Mix_FreeChunk(chunk);
    Mix_CloseAudio();
    Mix_Quit();
    SDL_Quit();
    return EXIT_SUCCESS;
}

While this works, it's not a perfect solution, since the result has some artifacts (crackling) in most cases, which I wasn't able to figure out why.

Github gist I created for this a while ago.

Answer 2

According to this forum here: https://forums.libsdl.org/viewtopic.php?p=44663, you can use a different library called "SoLoud" to change the playback speed of your sounds on the fly. You can get/see more details on SoLoud here: http://sol.gfxile.net/soloud/. From what I can tell, you cannot do this using SDL2, and SoLoud seems easy enough to use, so that would be my suggestion.