Why doesn't my Java application play the sound at each loop?

Question

I'm trying to make a Java application that simulates someone typing on their keyboard. The keystroke sound is played in a loop (Java chose a keystroke sound among others randomly and plays it) at a variable interval (to simulate a real person typing).

It works fine in the beginning, but after around the 95th iteration, it stops playing the sound (while still looping) for less than 4 seconds then plays the sound again. And after the 160th iteration, it plays the sound almost every second (instead of every third to sixth of a second).
After a while, it stops playing the sound for a long time, then forever.

Here is the source for the AudioPlayer.java class:

package entity;

import java.io.File;
import java.io.IOException;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.Clip;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.UnsupportedAudioFileException;

public class AudioPlayer implements Runnable {
    private String audioFilePath;

    public void setAudioFilePath(String audioFilePath) {
         this.audioFilePath = audioFilePath;
    }

    @Override
    public void run() {
        File audioFile = new File(audioFilePath);

        try {
            AudioInputStream audioStream = AudioSystem.getAudioInputStream(audioFile);
            AudioFormat format = audioStream.getFormat();
            DataLine.Info info = new DataLine.Info(Clip.class, format);
            Clip audioClip = (Clip) AudioSystem.getLine(info);
            audioClip.open(audioStream);
            audioClip.start();
            boolean playCompleted = false;
            while (!playCompleted) {
                try {
                    Thread.sleep(500);
                    playCompleted = true;
                }
                catch (InterruptedException ex) {
                    ex.printStackTrace();
                }
            }
            audioClip.close();
        } catch (UnsupportedAudioFileException ex) {
            System.out.println("The specified audio file is not supported.");
            ex.printStackTrace();
        } catch (LineUnavailableException ex) {
            System.out.println("Audio line for playing back is unavailable.");
            ex.printStackTrace();
        } catch (IOException ex) {
            System.out.println("Error playing the audio file.");
            ex.printStackTrace();
        }
    }
}

And here is the Main.java class to test the keystroke simulator:

package sandbox;

import java.util.Random;

import entity.AudioPlayer;

public class Main {
    public static void main(String[] args) {
        Random rnd = new Random();
        AudioPlayer audio;

        for(int i = 0; i < 10000; i++) {
            int delay = rnd.nextInt(200)+75;
            try {
                Thread.sleep(delay);
            }
            catch (InterruptedException ie) {}
            int index = rnd.nextInt(3)+1;
            audio = new AudioPlayer();
            audio.setAudioFilePath("resources/keystroke-0"+index+".wav");
            Thread thread = new Thread(audio);
            thread.start();
            System.out.println("iteration "+i);
        }
    }
}

I used multiple short (less than 200ms) wave files of different sounding keystrokes (3 in total) all in the resources directory.

EDIT

I read your answers and comments. And I'm thinking maybe I misundertood them because the suggested solutions don't work, or maybe I should have made myself clear on what I exactly wanted. Also, I need to note that I don't use threads often (and have no clue what a mutex is).

So I'll first explain what I exactly want the program to do. It should be able to simulate keystroke and so I used a Thread because it allows two keystroke sounds to overlap just like when a real person is typing. Basically the sound clips I am using are keystroke sounds and a keystroke sound is composed of two sounds: the sound of a key being pressed. the sound of a key being released.

If at some point the program allows two keystroke to overlap it will sound as if someone pressed one key then another and then released the first key. That's how really typing sounds like!

Now the issues I encountered using the proposed solutions are: When calling the run() method of the AudioPlayer directly,

public static void main(String[] args)
{
    // Definitions here

    while (running) {
        Date previous = new Date();
        Date delay = new Date(previous.getTime()+rnd.nextInt(300)+75);

        // Setting the audio here

        audio.run();
        Date now = new Date();

        if (now.before(delay)) {
            try { 
                Thread.sleep(delay.getTime()-now.getTime());
            } catch (InterruptedException e) {
            }
        }

        System.out.println("iteration: "+(++i));
    }
}

the sounds play sequentially (one after the other) and at a rate that depends on the sleep duration of the AudioPlayer (or depends on the delay if the delay in the main() method is higher than the sleep duration of the AudioPlayer), which is no good because it won't sound like the average typist (more like someone who is new to typing and still looking for every keys when typing).

When calling the join() method of the AudioPlayer's Thread,

public static void main(String[] args)
{
    //Variable definitions here

    while (running) {
        int delay = rnd.nextInt(200)+75;
        try
        {
            Thread.sleep(delay);
        }
        catch (InterruptedException ie)
        {

        }

        //Setting the AudioPlayer and creating its Thread here

        thread.start();
        try
        {
            thread.join();
        }
        catch(InterruptedException ie)
        {

        }
        System.out.println("iteration "+(++i));
    }
}

the sounds play sequentially as well and at a rate that depends on the sleep duration of the AudioPlayer (or depends on the delay if the delay in the main() method is higher than the sleep duration of the AudioPlayer) which, again, is no good for the same reason as before.

So, to answer one of the commenter's question. Yes! there are other concerns not expressed before which require the threads in the first place.

I found a workaround that "solves" my issue (but that I don't consider as a proper solution since I am, in a way, cheating): What I did is increase the sleep duration of the AudioPlayer to something that is unlikely to be reached before the program is stopped (24 hours) and from what I've seen it doesn't use much resources even after more than an hour.

You can check out what I want, what I get when running the suggested solutions and what I get using my workaround on this youtube videos (Unfortunately StackOverflow doesn't have video uploading feature. so I had to put it on youtube).

EDIT The sound effects can be downloaded here.

Answer 1

The library AudioCue was made for exactly this sort of thing. You might try running the "frog pond" demo, simulating a number of frogs all croaking, all generated from a single frog croak recording.

You can take a single typewriter click and run everything from it, create a cue with, say 10 simultaneous overlaps allowed. Then use an RNG to pick which of the 10 "cursors" to click. The 10 typists can each have their own volume & pan location, and can be pitched slightly differently so that it sounds like the typewriters are different models or the keys are being hit with different weights (as if an old manual typewriter).

One can tweak RNG algorithms for the different typing speeds (using varying sleep times).

For myself, I wrote an event system where the play commands are queued up on an event system using a ConcurrentSkipListSet, where the stored objects include a timing value (milliseconds after a given zero point) that is used for sorting as well as controlling when the play gets executed. That might be overkill if you don't intend to do this sort of thing very often.

Answer 2

How about this single-threaded solution which is a cleaner version of your own, but re-using already opened clips from buffers? To me the typing sounds pretty natural even though there are no two sounds playing at the same time. You can adjust the typing speed by changing the corresponding static constants in the Application class.

package de.scrum_master.stackoverflow.q61159885;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.Clip;
import javax.sound.sampled.DataLine.Info;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.UnsupportedAudioFileException;
import java.io.Closeable;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

import static javax.sound.sampled.AudioSystem.getAudioInputStream;
import static javax.sound.sampled.AudioSystem.getLine;

public class AudioPlayer implements Closeable {
  private final Map<String, Clip> bufferedClips = new HashMap<>();

  public void play(String audioFilePath) throws IOException, UnsupportedAudioFileException, LineUnavailableException {
    Clip clip = bufferedClips.get(audioFilePath);
    if (clip == null) {
      AudioFormat audioFormat = getAudioInputStream(new File(audioFilePath)).getFormat();
      Info lineInfo = new Info(Clip.class, audioFormat);
      clip = (Clip) getLine(lineInfo);
      bufferedClips.put(audioFilePath, clip);
      clip.open(getAudioInputStream(new File(audioFilePath)));
    }
    clip.setMicrosecondPosition(0);
    clip.start();
  }

  @Override
  public void close() {
    bufferedClips.values().forEach(Clip::close);
  }
}

package de.scrum_master.stackoverflow.q61159885;

import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.UnsupportedAudioFileException;
import java.io.IOException;
import java.util.Random;

public class Application {
  private static final Random RANDOM = new Random();

  private static final int ITERATIONS = 10000;
  private static final int MINIMUM_WAIT = 75;
  private static final int MAX_RANDOM_WAIT = 200;

  public static void main(String[] args) throws UnsupportedAudioFileException, IOException, LineUnavailableException {
    try (AudioPlayer audioPlayer = new AudioPlayer()) {
      for (int i = 0; i < ITERATIONS; i++) {
        sleep(MINIMUM_WAIT + RANDOM.nextInt(MAX_RANDOM_WAIT));
        audioPlayer.play(randomAudioFile());
      }
    }
  }

  private static void sleep(int delay) {
    try {
      Thread.sleep(delay);
    } catch (InterruptedException ignored) {}
  }

  private static String randomAudioFile() {
    return "resources/keystroke-0" + (RANDOM.nextInt(3) + 1) + ".wav";
  }
}

You might have noticed that the AudioPlayer is Closeable, i.e. you can use "try with resouces" in the calling application. That way it makes sure that at the end of the program all clips are closed automatically.

The key to replaying the same clip is of course clip.setMicrosecondPosition(0) before you start.

---

Update: If you want to simulate multiple persons, just modify the main class like this. BTW, I don't know anything about audio programming and whether there is a way to better deal with mixers and overlapping sounds. It is just a proof of concept in order to give you an idea. There is one thread per person, but each person types in a serial fashion, not two keys at the same time. But multiple persons can overlap because there is one AudioPlayer per person with its own set of buffered clips.

package de.scrum_master.stackoverflow.q61159885;

import java.util.Random;

public class Application {
  private static final Random RANDOM = new Random();

  private static final int PERSONS = 2;
  private static final int ITERATIONS = 10000;
  private static final int MINIMUM_WAIT = 150;
  private static final int MAX_RANDOM_WAIT = 200;

  public static void main(String[] args) {
    for (int p = 0; p < PERSONS; p++)
      new Thread(() -> {
        try (AudioPlayer audioPlayer = new AudioPlayer()) {
          for (int i = 0; i < ITERATIONS; i++) {
            sleep(MINIMUM_WAIT + RANDOM.nextInt(MAX_RANDOM_WAIT));
            audioPlayer.play(randomAudioFile());
          }
        } catch (Exception ignored) {}
      }).start();
  }

  private static void sleep(int delay) {
    try {
      Thread.sleep(delay);
    } catch (InterruptedException ignored) {}
  }

  private static String randomAudioFile() {
    return "resources/keystroke-0" + (RANDOM.nextInt(3) + 1) + ".wav";
  }
}

Answer 3

Besides the things EdwinBuck said, I believe you are doing way to much work in your AudioPlayer class (and every time). Try pre-create an AudioPlayer instance one time for all of your audio files (I believe it is 4?), and add a separate play() method, so that inside your loop you can do something like audioplayers[index].play().

Also note that in your AudioPlayer class you are waiting 500ms for the sound to finish which is longer than you wait to play the next sound. This will - after a while - lead to you running out of available threads... perhaps there's a callback you could use when the AudioClip is finished, instead of waiting.

Answer 4

With threads, you are talking about independent flows of execution. Your program is designed such that the picking of a delay an the playing of the sound are not independent.

something like

    for (int i = 0; i < 10000; i++) {
        int delay = rnd.nextInt(200)+75;
        try {
            Thread.sleep(delay);
        } catch (InterruptedException ie) {

        }
        int index = rnd.nextInt(3)+1;
        audio = new AudioPlayer();
        audio.setAudioFilePath("resources/keystroke-0"+index+".wav");
        audio.run();
        System.out.println("iteration "+i);
    }

Would express that you "wait" then "run a wav" then "repeat".

Right now you're relying on consistent scheduling of threads of execution on the cores to get the desired result; except that threads aren't intended to be the way to express consistent scheduling of execution. Threads are intended to be the way to express independent scheduling of execution.

Odds are between your current wait and the current "play the wave" a few other things get in between, and with enough time, perhaps even the wav files could be played out-of-order. I'd make the ordering explicit.

If you need good control on the timing within the loop, look to a gaming loop type setup. It's similar to your for loop, but looks like

while (running) {
  SomeTimeType previous = new TimeType();
  SomeTimeOffset delay = new TimeOffset(rnd.nextInt(200)+75);

  ...
  audio.run();
  SomeTimeType now = new TimeType();
  if (now.minus(offset).compareTo(previous) > 0) {
    try { 
      Thread.sleep(now.minus(offset).toMillis())
    } catch (InterruptedException e) {
    }
  }
}

The primary difference here is that your random delays start from the beginning of the wav file's play time to the beginning of the next wav file's play time, and there is no delay between files if the delay is shorter than the wav file's play time.

Also, I'd look into if the AudioPlayer can be reused between wave file playbacks, as that will probably get you even better results.

Now on the off chance you really need to keep the playing in a separate thread, you need to join the thread of the loop to the thread of the AudioPlayer to ensure that the AudioPlayer finishes before the loop thread advances. Even though you are waiting a longer time in the for loop, remember, any process can come off the CPU core at any time, so your wait isn't an assurance that the for loop takes more time per iteration than the AudioPlayer takes per wav file, if the CPU had to handle something else (like a network packet).

    for (int i = 0; i < 10000; i++) {
        int delay = rnd.nextInt(200)+75;
        try {
            Thread.sleep(delay);
        } catch (InterruptedException ie) {

        }
        int index = rnd.nextInt(3)+1;
        audio = new AudioPlayer();
        audio.setAudioFilePath("resources/keystroke-0"+index+".wav");
        Thread thread = new Thread(audio);
        thread.start();
        thread.join();
        System.out.println("iteration "+i);
    }

The thread.join() will force the for loop to go into a sleep state (possibly shifting it off the CPU) until the audio thread completes.