Does Javac's StringBuilder optimisation do more harm than good?

Question

Let's say we have some code such as the following:

public static void main(String[] args) {
    String s = "";
    for(int i=0 ; i<10000 ; i++) {
        s += "really ";
    }
    s += "long string.";
}

(Yes, I know a far better implementation would use a StringBuilder, but bear with me.)

Trivially, we might expect the bytecode produced to be something akin to the following:

public static void main(java.lang.String[]);
Code:
   0: ldc           #2                  // String 
   2: astore_1      
   3: iconst_0      
   4: istore_2      
   5: iload_2       
   6: sipush        10000
   9: if_icmpge     25
  12: aload_1       
  13: ldc           #3                  // String really 
  15: invokevirtual #4                  // Method java/lang/String.concat:(Ljava/lang/String;)Ljava/lang/String;
  18: astore_1      
  19: iinc          2, 1
  22: goto          5
  25: aload_1       
  26: ldc           #5                  // String long string.
  28: invokevirtual #4                  // Method java/lang/String.concat:(Ljava/lang/String;)Ljava/lang/String;
  31: astore_1      
  32: return

However, instead the compiler tries to be a bit smarter - rather than using the concat method, it has a baked in optimisation to use StringBuilder objects instead, so we get the following:

public static void main(java.lang.String[]);
Code:
   0: ldc           #2                  // String 
   2: astore_1      
   3: iconst_0      
   4: istore_2      
   5: iload_2       
   6: sipush        10000
   9: if_icmpge     38
  12: new           #3                  // class java/lang/StringBuilder
  15: dup           
  16: invokespecial #4                  // Method java/lang/StringBuilder."<init>":()V
  19: aload_1       
  20: invokevirtual #5                  // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
  23: ldc           #6                  // String really 
  25: invokevirtual #5                  // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
  28: invokevirtual #7                  // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
  31: astore_1      
  32: iinc          2, 1
  35: goto          5
  38: new           #3                  // class java/lang/StringBuilder
  41: dup           
  42: invokespecial #4                  // Method java/lang/StringBuilder."<init>":()V
  45: aload_1       
  46: invokevirtual #5                  // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
  49: ldc           #8                  // String long string.
  51: invokevirtual #5                  // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
  54: invokevirtual #7                  // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
  57: astore_1      
  58: return

However, this seems rather counter-productive to me - instead of using one string builder for the entire loop, one is created for each single concatenation operation, making it equivalent to the following:

public static void main(String[] args) {
    String s = "";
    for(int i=0 ; i<10000 ; i++) {
        s = new StringBuilder().append(s).append("really ").toString();
    }
    s = new StringBuilder().append(s).append("long string.").toString();
}

So now instead of the original trivial bad approach of just creating lots of string objects and throwing them away, the compiler has produced an far worse approach of creating lots of String objects, lots of StringBuilder objects, calling more methods, and still throwing them all away to generate the same output as without this optimisation.

So the question has to be - why? I understand that in cases like this:

String s = getString1() + getString2() + getString3();

...the compiler will create just one StringBuilder object for all three strings, so there are cases where the optimisation is useful. However, examing the bytecode reveals that even separating the above case to the following:

String s = getString1();
s += getString2();
s += getString3();

...means that we're back with the case that three StringBuilder objects are individually created. I'd understand if these were odd corner cases, but appending to strings in this way (and in a loop) are really rather common operations.

Surely it would be trivial to determine, at compile time, if a compiler-generated StringBuilder only ever appended one value - and if this was the case, use a simple concat operation instead?

This is all with 8u5 (however, it goes back to at least Java 5, probably before.) FWIW, my benchmarks (unsurprisingly) put the manual concat() approach 2x3 times faster than using += in a loop with 10,000 elements. Of course, using a manual StringBuilder is always the preferable approach, but surely the compiler shouldn't adversely affect the performance of the += approach either?

Answer 1

FWIW, my benchmarks (unsurprisingly) put the manual concat() approach 2x3 times faster than using += in a loop with 10,000 elements.

I'm interested to see your benchmark, because mine (based on the excellent JMH harness) shows that += is slightly faster than String.concat. When we perform three operations per loop iteration (s += "re"; s += "al"; s += "ly ";), += remains nearly as performant, while String.concat takes the obvious factor-of-3 hit.

I ran my benchmark on a Intel Xeon E5-2695 v2 @ 2.40GHz running OpenJDK build 1.8.0_40-ea-b23. There are four implementations:

• implicit, which uses +=
• explicit, which explicitly instantiates a StringBuilder for each concatenation, representing the += desugaring
• concat, which uses String.concat
• smart, which uses one StringBuilder as in Stephen C's answer

There are two versions of each implementation: the normal one and one performing three operations in the loop body.

Here are the numbers. This is throughput, so higher is better. Error is the bounds of a 99.9% confidence interval. (This is JMH's default output.)

Benchmark                      Mode  Cnt     Score     Error  Units
StringBuilderBench.smart      thrpt   30  5438.676 ± 352.088  ops/s
StringBuilderBench.implicit   thrpt   30    10.290 ±   0.878  ops/s
StringBuilderBench.concat     thrpt   30     9.685 ±   0.924  ops/s
StringBuilderBench.explicit   thrpt   30     9.078 ±   0.884  ops/s

StringBuilderBench.smart3     thrpt   30  3335.001 ± 115.600  ops/s
StringBuilderBench.implicit3  thrpt   30     9.303 ±   0.838  ops/s
StringBuilderBench.explicit3  thrpt   30     8.597 ±   0.237  ops/s
StringBuilderBench.concat3    thrpt   30     3.182 ±   0.228  ops/s

The smart implementation using just one StringBuilder is much faster than the others, as expected. Of the remaining implementations, += beats String.concat, which beats explicit StringBuilder instantiation. They're all fairly close, considering the error.

When performing three operations per loop, all the implementations take a small (relative) hit, except for String.concat, whose throughput is reduced by a factor of 3.

These results are not surprising considering HotSpot has specific optimizations for StringBuilder (and StringBuffer) -- see src/share/vm/opto/stringopts.cpp. The commit history for this file shows these optimizations date to late 2009 as part of bug JDK-6892658.

There don't seem to be any changes between 8u5 and the early access build of 8u40 I ran the benchmark with, so that doesn't explain why we got different results. (Of course, changes elsewhere in the compiler could have changed the results too.)

---

Here's the benchmark code, which I ran with java -jar benchmarks.jar -w 5s -wi 10 -r 5s -i 30 -f 1. The code and complete log of the benchmark run are also available as a Gist.

package com.jeffreybosboom.stringbuilderbench;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.State;

@State(Scope.Thread)
public class StringBuilderBench {
    //promote to non-final fields to inhibit constant folding (see JMHSample_10_ConstantFold.java)
    private String really = "really ", long_string = "long string.", re = "re", al = "al", ly = "ly ";
    @Benchmark
    public String implicit() {
        String s = "";
        for (int i = 0; i < 10000; i++)
            s += really;
        s += long_string;
        return s;
    }
    @Benchmark
    public String explicit() {
        String s = "";
        for (int i = 0; i < 10000; i++)
            s = new StringBuilder().append(s).append(really).toString();
        s = new StringBuilder().append(s).append(long_string).toString();
        return s;
    }
    @Benchmark
    public String concat() {
        String s = "";
        for (int i = 0; i < 10000; i++)
            s = s.concat(really);
        s = s.concat(long_string);
        return s;
    }
    @Benchmark
    public String implicit3() {
        String s = "";
        for (int i = 0; i < 10000; i++) {
            s += re;
            s += al;
            s += ly;
        }
        s += long_string;
        return s;
    }
    @Benchmark
    public String explicit3() {
        String s = "";
        for (int i = 0; i < 10000; i++) {
            s = new StringBuilder().append(s).append(re).toString();
            s = new StringBuilder().append(s).append(al).toString();
            s = new StringBuilder().append(s).append(ly).toString();
        }
        s = new StringBuilder().append(s).append(long_string).toString();
        return s;
    }
    @Benchmark
    public String concat3() {
        String s = "";
        for (int i = 0; i < 10000; i++) {
            s = s.concat(re);
            s = s.concat(al);
            s = s.concat(ly);
        }
        s = s.concat(long_string);
        return s;
    }
    @Benchmark
    public String smart() {
        StringBuilder sb = new StringBuilder();
        for (int i = 0; i < 10000; i++)
            sb.append(really);
        sb.append(long_string);
        return sb.toString();
    }
    @Benchmark
    public String smart3() {
        StringBuilder sb = new StringBuilder();
        for (int i = 0; i < 10000; i++) {
            sb.append(re);
            sb.append(al);
            sb.append(ly);
        }
        sb.append(long_string);
        return sb.toString();
    }
}

Answer 2

So the question has to be - why?

It is not clear why they don't optimize this a bit better in the bytecode compiler. You would need to ask the Oracle Java compiler team.

One possible explanation is that there may be code in the HotSpot JIT compiler to optimize the bytecode sequence into something better. (If you were curious, you could modify the code so that it got JIT compiled ... and then capture and examine the native code. However, you might actually find that the JIT compiler optimizes away the method body entirely ...)

Another possible explanation is that the original Java code is so pessimal to start with that they figured that optimizing it would not have a significant effect. Consider that a seasoned Java programmer would write it as:

public static void main(String[] args) {
    StringBuilder sb = new StringBuilder();
    for (int i=0 ; i<10000 ; i++) {
        sb.append("really ");
    }
    sb.append("long string.");
    String s = sb.toString();
}

That is going to run roughly 4 orders of magnitude faster.

---

UPDATE - I used the code link from the linked Q&A to find the actual place in Java bytecode compiler source that generates that code: here.

There are no hints in the source to explain the "dumb"-ness of the code generation strategy.

---

So to your general Question:

Does Javac's StringBuilder optimisation do more harm than good?

No.

My understanding is that the compiler developers did extensive benchmarking to determine that (overall) the StringBuilder optimizations are worthwhile.

You have found an edge case in a badly written program that could be optimized better (it is hypothesized). This is not sufficient to conclude the optimization "does more harm than good" overall.