Reputation: 43833
In JavaScript, I have a loop that has many iterations, and in each iteration, I am creating a huge string with many +=
operators. Is there a more efficient way to create a string? I was thinking about creating a dynamic array where I keep adding strings to it and then do a join. Can anyone explain and give an example of the fastest way to do this?
Upvotes: 209
Views: 173921
Reputation: 154
Under the hood, the V8 Javascript engine used in e.g. Google Chrome and Node.js already usually represents the result of a concatenation as a data structure with pointers to the base strings. Therefore, the string concatenation operation itself is basically free, but if you have a string resulting from very many small strings, there might be performance penalties if you perform operations on that string compared to a string represented as a contiguous byte array. As all this depends on the heuristics employed by V8, the only way to know how it performs is testing it in the precise scenario your interested in (and not the toy examples of the other answers).
Upvotes: 0
Reputation: 40533
Seems based on benchmarks at JSPerf that using +=
is the fastest method, though not necessarily in every browser.
For building strings in the DOM, it seems to be better to concatenate the string first and then add to the DOM, rather then iteratively add it to the dom. You should benchmark your own case though.
(Thanks @zAlbee for correction)
Upvotes: 171
Reputation: 106
I did a bit of testing. For small strings -- str.length <= 32
-- .concat
was fastest:
let strAgg = "";
for (...) {
strAgg = strAgg.concat(str);
}
+=
and 'strAgg = `${strAgg}${str}`' are similarly fast.
When dealing with large strings, calling push
on an array, then reduce
also seems to be similar in speed.
Surprisingly, joining arrays with arr.join("")
is very slow when dealing with
large strings -- str.length >= 3072
in my test.
As also shown in @graup's answer, if you already have an array of strings, using
reduces is a fast method to join them. If you're building a string from the ground up, starting with strAgg = ""
, .concat
, +=
and template literals seem to generally be faster.
I'm writing this in June 2024, using V8 version '12.4.254.20-node.13', NodeJS version '22.3.0'. This test is likely to be outdated by future improvements made to V8.
Just like the question, I needed to figure out how to concatenate a large number of strings fast. I tested:
+=
.concat
There is no significant difference in performance between +=
and .concat
although .concat
seems to be slightly faster overall.
I also tried writing to a fix-sized buffer. This was slow overall (I suspect this is due to overhead for UTF-8 encodings in .write(...)
). There may be faster ways to copy from a string to a buffer that I'm unaware of.
Here's the full output of my test:
NodeJS version = 22.3.0
V8 version = 12.4.254.20-node.13
---------------------------------------------------
concatenating strings with 1 characters
no. of strings = 100
no. of reps = 100000
string concat with '+=' : 109.314ms
string concat with '.concat' : 99.817ms
string concat with 'template literals' : 117.137ms
string concat with 'array join' : 156.979ms
string concat with 'array reduce' : 128.196ms
string concat by 'writing to buffer' : 726.769ms
no. of strings = 1000
no. of reps = 10000
string concat with '+=' : 107.4ms
string concat with '.concat' : 82.435ms
string concat with 'template literals' : 109.415ms
string concat with 'array join' : 154.282ms
string concat with 'array reduce' : 119.207ms
string concat by 'writing to buffer' : 741.176ms
no. of strings = 10000
no. of reps = 1000
string concat with '+=' : 95.298ms
string concat with '.concat' : 86.657ms
string concat with 'template literals' : 110.112ms
string concat with 'array join' : 155.329ms
string concat with 'array reduce' : 118.649ms
string concat by 'writing to buffer' : 692.829ms
concatenating strings with 8 characters
no. of strings = 100
no. of reps = 100000
string concat with '+=' : 110.464ms
string concat with '.concat' : 86.682ms
string concat with 'template literals' : 100.87ms
string concat with 'array join' : 153.812ms
string concat with 'array reduce' : 117.609ms
string concat by 'writing to buffer' : 763.109ms
no. of strings = 1000
no. of reps = 10000
string concat with '+=' : 81.168ms
string concat with '.concat' : 81.768ms
string concat with 'template literals' : 108.861ms
string concat with 'array join' : 184.48ms
string concat with 'array reduce' : 122.856ms
string concat by 'writing to buffer' : 755.015ms
no. of strings = 10000
no. of reps = 1000
string concat with '+=' : 83.416ms
string concat with '.concat' : 83.394ms
string concat with 'template literals' : 110.643ms
string concat with 'array join' : 184.926ms
string concat with 'array reduce' : 148.241ms
string concat by 'writing to buffer' : 762.883ms
concatenating strings with 32 characters
no. of strings = 100
no. of reps = 100000
string concat with '+=' : 90.681ms
string concat with '.concat' : 88.55ms
string concat with 'template literals' : 107.382ms
string concat with 'array join' : 189.064ms
string concat with 'array reduce' : 115.724ms
string concat by 'writing to buffer' : 846.54ms
no. of strings = 1000
no. of reps = 10000
string concat with '+=' : 88.77ms
string concat with '.concat' : 89.778ms
string concat with 'template literals' : 104.712ms
string concat with 'array join' : 177.094ms
string concat with 'array reduce' : 123.93ms
string concat by 'writing to buffer' : 800.392ms
no. of strings = 10000
no. of reps = 1000
string concat with '+=' : 91.653ms
string concat with '.concat' : 104.289ms
string concat with 'template literals' : 120.978ms
string concat with 'array join' : 324.626ms
string concat with 'array reduce' : 122.354ms
string concat by 'writing to buffer' : 960.095ms
concatenating strings with 3072 characters
no. of strings = 100
no. of reps = 100000
string concat with '+=' : 123.552ms
string concat with '.concat' : 88.79ms
string concat with 'template literals' : 103.767ms
string concat with 'array join' : 19.090s
string concat with 'array reduce' : 122.827ms
string concat by 'writing to buffer' : 25.593s
no. of strings = 1000
no. of reps = 10000
string concat with '+=' : 91.521ms
string concat with '.concat' : 100.436ms
string concat with 'template literals' : 135.745ms
string concat with 'array join' : 18.129s
string concat with 'array reduce' : 124.824ms
string concat by 'writing to buffer' : 27.104s
no. of strings = 10000
no. of reps = 1000
string concat with '+=' : 227.08ms
string concat with '.concat' : 175.78ms
string concat with 'template literals' : 188.356ms
string concat with 'array join' : 17.376s
string concat with 'array reduce' : 203.14ms
string concat by 'writing to buffer' : 27.164s
concatenating strings with 8192 characters
no. of strings = 100
no. of reps = 100000
string concat with '+=' : 208.383ms
string concat with '.concat' : 93.171ms
string concat with 'template literals' : 109.893ms
string concat with 'array join' : 50.683s
string concat with 'array reduce' : 172.048ms
string concat by 'writing to buffer' : 1:10.489 (m:ss.mmm)
no. of strings = 1000
no. of reps = 10000
string concat with '+=' : 95.325ms
string concat with '.concat' : 90.486ms
string concat with 'template literals' : 107.749ms
string concat with 'array join' : 46.964s
string concat with 'array reduce' : 177.331ms
string concat by 'writing to buffer' : 1:10.699 (m:ss.mmm)
no. of strings = 10000
no. of reps = 1000
string concat with '+=' : 94.509ms
string concat with '.concat' : 92.32ms
string concat with 'template literals' : 109.136ms
string concat with 'array join' : 47.220s
string concat with 'array reduce' : 125.031ms
string concat by 'writing to buffer' : 3:36.064 (m:ss.mmm)
and here's the code:
#!/usr/bin/env node
const { Buffer } = require('node:buffer');
function main() {
console.log(`NodeJS version = ${process.versions.node}`);
console.log(`V8 version = ${process.versions.v8}`);
console.log("---------------------------------------------------")
measureStringConcat();
}
function measureStringConcat() {
for (const strLen of [
1, // Probably 1-2 words in memory
8, // Probably 2 or more words in memory on (on 64-bit)
32, // Probably fits in a single L1 cache line
3072, // <4096 -> Probably fits into a single page
2*4096, // >4096 -> Probably fits into multiple pages
]) {
const RAND_STRS_N = 256;
const RAND_STRS_STRIDE = 13;
const placeholderStr = "\x00".repeat(strLen);
const randStrs = Array(RAND_STRS_N).fill(placeholderStr);
for (let i = 0; i < RAND_STRS_N; i++) {
let randStr = "";
for (let j = 0; j < strLen; j++) {
const randCharCode = Math.floor(Math.random()*127.99999);
randStr += String.fromCharCode(randCharCode);
}
randStrs[i] = randStr;
}
console.log(`concatenating strings with ${strLen} characters`);
let randStrIdx = 0;
for (const [strsN, reps] of [
[ 100, 100_000],
[ 1_000, 10_000],
[10_000, 1000],
]) {
console.log(` no. of strings = ${strsN}`)
console.log(` no. of reps = ${reps}`)
{
const timeLabel = " string concat with '+=' ";
console.time(timeLabel)
for (let i = 0; i < reps; i++) {
let strAgg = "";
for (let j = 0; j < strsN; j++) {
strAgg += randStrs[randStrIdx];
randStrIdx += RAND_STRS_STRIDE;
if (randStrIdx > RAND_STRS_N) randStrIdx -= RAND_STRS_STRIDE;
}
}
console.timeEnd(timeLabel);
}
{
const timeLabel = " string concat with '.concat' ";
console.time(timeLabel)
for (let i = 0; i < reps; i++) {
let strAgg = "";
for (let j = 0; j < strsN; j++) {
strAgg = strAgg.concat(randStrs[randStrIdx]);
randStrIdx += RAND_STRS_STRIDE;
if (randStrIdx > RAND_STRS_N) randStrIdx -= RAND_STRS_STRIDE;
}
}
console.timeEnd(timeLabel);
}
{
const timeLabel = " string concat with 'template literals' ";
console.time(timeLabel)
for (let i = 0; i < reps; i++) {
let strAgg = "";
for (let j = 0; j < strsN; j++) {
strAgg = `${strAgg}${randStrs[randStrIdx]}`;
randStrIdx += RAND_STRS_STRIDE;
if (randStrIdx > RAND_STRS_N) randStrIdx -= RAND_STRS_STRIDE;
}
}
console.timeEnd(timeLabel);
}
{
const timeLabel = " string concat with 'array join' ";
console.time(timeLabel)
for (let i = 0; i < reps; i++) {
const l = [];
for (let j = 0; j < strsN; j++) {
l.push(randStrs[randStrIdx]);
randStrIdx += RAND_STRS_STRIDE;
if (randStrIdx > RAND_STRS_N) randStrIdx -= RAND_STRS_STRIDE;
}
const strAgg = l.join("");
}
console.timeEnd(timeLabel);
}
{
const timeLabel = " string concat with 'array reduce' ";
console.time(timeLabel)
for (let i = 0; i < reps; i++) {
const l = [];
for (let j = 0; j < strsN; j++) {
l.push(randStrs[randStrIdx]);
randStrIdx += RAND_STRS_STRIDE;
if (randStrIdx > RAND_STRS_N) randStrIdx -= RAND_STRS_STRIDE;
}
const strAgg = l.reduce((agg, s) => agg + s, "");
}
console.timeEnd(timeLabel);
}
{
const timeLabel = " string concat by 'writing to buffer' ";
console.time(timeLabel)
for (let i = 0; i < reps; i++) {
const buf = Buffer.allocUnsafe(strLen*strsN);
let bufOffset = 0;
for (let j = 0; j < strsN; j++) {
buf.write(randStrs[randStrIdx], bufOffset);
bufOffset += strLen;
randStrIdx += RAND_STRS_STRIDE;
if (randStrIdx > RAND_STRS_N) randStrIdx -= RAND_STRS_STRIDE;
}
const strAgg = buf.toString();
}
console.timeEnd(timeLabel);
}
}
}
}
main();
It might be easier to curl/wget
this raw gist: https://gist.githubusercontent.com/0scarB/e561bc3819edf909b915ecbdfa406948/raw/2857ccc849bd433deb90ef54f3b487a8d07dafbf/stringConcatBench.js
(Happily do whatever you want with the code. Treat it as public domain / CC0, MIT.)
Really someone needs to look into the V8 source code and explain why the results are what they are. Today, it's not me.
Upvotes: 3
Reputation: 19539
You can also do string concat with template literals. I updated the other posters' JSPerf tests to include it.
for (var res = '', i = 0; i < data.length; i++) {
res = `${res}${data[i]}`;
}
Upvotes: 13
Reputation: 43
I can't comment on other's answers (not enough rep.), so I will say MadBreaks' answer of using template literals is good, but care should be taken if building a site that needs to be compatible with IE (Internet Explorer), because template literals aren't compatible with IE. So, in that case you can just use assignment operators (+, +=).
Upvotes: 0
Reputation: 3396
I did a quick test in both node and chrome and found in both cases +=
is faster:
var profile = func => {
var start = new Date();
for (var i = 0; i < 10000000; i++) func('test');
console.log(new Date() - start);
}
profile(x => "testtesttesttesttest");
profile(x => `${x}${x}${x}${x}${x}`);
profile(x => x + x + x + x + x );
profile(x => { var s = x; s += x; s += x; s += x; s += x; return s; });
profile(x => [x, x, x, x, x].join(""));
profile(x => { var a = [x]; a.push(x); a.push(x); a.push(x); a.push(x); return a.join(""); });
results in node: 7.0.10
results from chrome 86.0.4240.198:
Upvotes: 17
Reputation: 1099
I wonder why String.prototype.concat
is not getting any love. In my tests (assuming you already have an array of strings), it outperforms all other methods.
Test code:
const numStrings = 100;
const strings = [...new Array(numStrings)].map(() => Math.random().toString(36).substring(6));
const concatReduce = (strs) => strs.reduce((a, b) => a + b);
const concatLoop = (strs) => {
let result = ''
for (let i = 0; i < strings.length; i++) {
result += strings[i];
}
return result;
}
// Case 1: 52,570 ops/s
concatLoop(strings);
// Case 2: 96,450 ops/s
concatReduce(strings)
// Case 3: 138,020 ops/s
strings.join('')
// Case 4: 169,520 ops/s
''.concat(...strings)
Upvotes: 7
Reputation: 1287
Three years past since this question was answered but I will provide my answer anyway :)
Actually, accepted answer is not fully correct. Jakub's test uses hardcoded string which allows JS engine to optimize code execution (Google's V8 is really good in this stuff!). But as soon as you use completely random strings (here is JSPerf) then string concatenation will be on a second place.
Upvotes: 20
Reputation: 1189
I have no comment on the concatenation itself, but I'd like to point out that @Jakub Hampl's suggestion:
For building strings in the DOM, in some cases it might be better to iteratively add to the DOM, rather then add a huge string at once.
is wrong, because it's based on a flawed test. That test never actually appends into the DOM.
This fixed test shows that creating the string all at once before rendering it is much, MUCH faster. It's not even a contest.
(Sorry this is a separate answer, but I don't have enough rep to comment on answers yet.)
Upvotes: 99