Reputation: 127
I've been looking at making my javascript program sing.
I first looked at the web speech api, but the pitch control seems very limited, so I thought maybe there is a way to send the result to a web audio node, and apply effects from there but it doesn't seems possible.
I found the mespeak.js library: http://www.masswerk.at/mespeak/
It allow to return a audio buffer that I'll treat as a source of my audio nodes, allowing for more control.
My input is a notes sequence, with a frequency and duration. Something like :
var seq = [[440hz,1000ms],[880hz,500ms],...];
I managed to get from this sequence and a series of words to my program saying those words in rhythm with different frequencies
But I'm having a few problems.
If any of you had any experience with that sort of stuff I'd appreciate any input.
Thanks a lot
EDIT: add some code
function sing(text,note,duration){
var buffer = meSpeak.speak(text,{rawdata:'default'});
playSound(buffer,freqToCents(note),duration)
}
function freqToCents(freq){
var root = 440 //no idea what is the base frequency of the speech generator
return 3986*Math.log10(freq/440)
}
function playSound(streamBuffer, cents, duration, callback) {
var source = context.createBufferSource();
source.connect(compressor);
context.decodeAudioData(streamBuffer, function(audioData) {
var duration = audioData.duration;
var delay = (duration)? Math.ceil(duration * 1000) : 1000;
setTimeout(callback, delay);
source.buffer = audioData;
source.detune.value = cents;
source.start(0);
}, function(error) { });
}
My sequencer is working, and at each step, calls the sings function if necessary, for example like this:
sing('test', 440, 1000)
As I was saying, I'd like the duration parameter to impact the result
Upvotes: 3
Views: 1311
Reputation: 25220
Espeak supports SSML mode, you need to use it to modify parameters instead of trying to postprocess results.
You need to play with espeak first and then try to reproduce same results in javascript port. It is not supported yet, but in this part in mespeak.js
'-w', 'wav.wav',
'-a', (typeof args.amplitude !== 'undefined')? String(args.amplitude) : (typeof args.a !== 'undefined')? String(args.a) : '1
'-g', (typeof args.wordgap !== 'undefined')? String(args.wordgap) : (typeof args.g !== 'undefined')? String(args.g) : '0',
'-p', (typeof args.pitch !== 'undefined')? String(args.pitch) : (typeof args.p !== 'undefined')? String(args.p) : '50',
'-s', (typeof args.speed !== 'undefined')? String(args.speed) : (typeof args.s !== 'undefined')? String(args.s) : '175',
You need to add -m
option to enable SSML.
Upvotes: 1