Development time in various languages

Question

Does anybody know of any research or benchmarks of how long it takes to develop the same application in a variety of languages? Really I'm looking for Java vs. C++ but any comparisons would be useful. I have the feeling there is a section in Code Complete about this but my copy is at work.

Edit:

There are a lot of interesting answers to this question but it seems like there is a lack of really good research. I have made a proposal over at meta about this problem.

Answer 1

Pratt & Whitney, purveyors of jet engines for civilian and military applications, did a study on this many years ago, without actually intending to do the study.

They went on the same metrics kick everyone else went on in the 1990s. They collected a bunch of data about their jet engine controller projects, including timecard data. They crunched it. The poor sap who got to crunch the data noticed something in the results: the military projects uniformly had twice the programmer productivity and one/fourth the defect density as the civilian projects.

This, by itself, is significant. It means you only need half as many programmers, and you aren't going to spend quite as much time fixing bugs. What is even more important is that this was an apples-to-apples comparison. A jet engine controller is a jet engine controller.

He then went looking for candidate explanations. All of the usual candidates: individual experience, team size, toolsets, software processes, requirements stability, everything, were trotted out, and they were ruled out when it was seen that the story on those items was uniformly the same on both sides of the aisle. At the end of the day, only one statistically significant difference showed up.

The civilian projects were written in every language you could think of. The military projects were all written in Ada.

IN EVERY SINGLE CASE, against every other comer, for jet engine controllers at Pratt & Whitney, using Ada gave double the productivity and one/fourth the defect density.

I know what the flying code monkeys are going to say. "You can do good work in any language." In theory, that's true. In practice, however, it appears that, at least at Pratt & Whitney, language made a difference.

Last I heard about this, Pratt & Whitney upper management decreed that ALL jet engine controller projects would be done in Ada.

No, I don't have a citation. No paper was ever written. My source on this story was the poor sap who crunched the numbers. Here's a similar study from 1995:

http://archive.adaic.com/intro/ada-vs-c/cada_art.html

This, incidentally, was BEFORE Boeing did the 777, and BEFORE the 777 brake subcontractor story happened. But that's another story.

Answer 2

Language/framework comparison for web applications

The Plat_Forms project provides some information of this type for web applications.

There are three studies with different tasks (done in 2007, 2011, 2012), all of the following format: Several teams of three professional developers implemented the same application under controlled conditions within two days. It covers Java, Perl, PHP, and Ruby and has multiple teams for each language. The evaluation reports much more than only development time.

Findings of iteration one for instance included

• that experience with the language and framework appeared to be more relevant than what that framework was.
• that Java tended to induce teams to make laborious constructions while Perl induced them to make pragmatic (and quite handy) constructions.

Findings of iteration two included

• that Ruby on Rails was more productive in this type of project (which due to its duration was more rapid prototyping than full-blown development of a mature application)
• and that the one exception to the above rule was the one team using Symfony, a PHP framework that has similar concepts to Ruby on Rails (but still the very different base language underneath it).

Look under http://www.plat-forms.org or search the web for "Plat_Forms". There is plenty more detail in the reports, in particular the thick techreport on iteration 1.

Answer 3

There's a reason why there are no real comparisons in that aspect, except for anecdotal evidence (which can be found in favor of almost any language).

Actually writing code takes relatively small portion of developer's time. Even if language lets you cut coding time in half, it will be barely noticeable by the time project ends. Design, structure of program, development process are all much more important, and then there are libraries, tools and experience with them.

Some languages are better suited for certain development processes than the others, so if you've settled on design and process you can decide which language will be more efficient - but not before.

(didn't notice there's a similar answer already, so feel free to ignore this)

Answer 4

Erlang vs C++/Corba

"... As the Erlang DCC is less than a quarter of the size of a similar C++/CORBA implementation, the product development in Erlang should be fast, and the code maintainable. We conclude that Erlang and associated libraries are suitable for the rapid development of maintainable and highly reliable distributed products."

Paper here

Answer 5

That Ada story might be an embellished version of this: http://www.adaic.com/whyada/ada-vs-c/cada_art.html

Answer 6

This question is a little old fashioned. Focusing on development time solely based on the choice of language is of limited value. There are so many other factors that have equal or more impact than the language itself:

1. The libraries or frameworks available / used.
2. The level of quality required (ie. defect count).
3. The type of application (eg. GUI, server, driver etc...)
4. The level of maintainability required.
5. Developer experience in the language.
6. The platform or OS the application is built on.

As an example, many would say Java is the better choice over C++ to build enterprise (line of business) applications. This is not normally because of the language itself, but instead it is perceived that Java has better (or more mature) web server and database frameworks available to it. This may or may not be true, but that is beside the point.

You may even find that the building an application using the same language on different operating systems or platforms gives greatly differing development time. For example using C++ on Linux to build a GUI application may take longer than a Windows based GUI application using C++ because of less extensive and mature GUI libraries avaialble on Linux (once again this is debatable).

Answer 7

I think most benchmarks and statements on this topic will mean very little.

Benchmarks can always be gamed; see the history of "Pet Store".

A language that's good at solving one kind of problem might not apply as well to another.

What matters most is the skill of your team, its knowledge of a particular technology, and how well you know the domain you're trying to solve.

UPDATE: Control software for jet engines and helicopters is a very specialized subset of computing problems. It's characterized by very rigorous, complete, detailed specs and QA that means the multi-million dollar aircraft cannot crash.

I can second the (very good) citation by John Strohm of Pratt & Whitney control software written in Ada. The control software for Kaman helicopters sold to Australia was also written in Ada.

But this does not lead to the conclusion that if you decided to write your next web site in Ada that you'd have higher productivity and fewer defects than you would if you chose C# or Java or Python or Ruby. All languages are not equally good in all problem domains.

Answer 8

See also

Are there statistical studies that indicates that Python is "more productive"?

for some discussions about this kind of question.

Answer 9

One of the few funded scientific studies that I'm aware of on cross-language productivity, from the early 90s, funded by ARPA and the ONR,

• Haskell vs. Ada Vs. C++ vs Awk vs ... An Experiment in Software Prototyping Productivity, Hudak & Jones, 1994.

We describe the results of an experiment in which several conventional programming languages, together with the functional language Haskell, were used to prototype a Naval Surface Warfare Center (NSWC) requirement for a Geometric Region Server. The resulting programs and development metrics were reviewed by a committee chosen by the Navy. The results indicate that the Haskell prototype took significantly less time to develop and was considerably more concise and easier to understand than the..

Answer 10

According to Norvig, Lutz Prechelt published just such an article in the October 1999 CACM: "Comparing Java vs. C/C++ Efficiency Issues to Interpersonal Issues".

Norvig includes a link to that article. Unfortunately, the ACM, despite having a bitmap graphic proclaiming their goal of "Advancing Computing as a Science & Profession", couldn't figure out how to maintain stable links on their webpage, so it's just a 404 now. Perhaps your local library could help you out.

Answer 11

A couple of anecdotal data points:

On Project Euler, which invites programming solutions to mathematical problems,

• the shortest solutions are almost invariably written in J or K, a relative of APL; there are occasionally MatLab solutions in the same range. It can be argued, though, that these languages specialized in math.
• runners up were Ruby solutions. A lot of algorithm can be wrapped in very little code, and it's much more legible than J / K.
• Python and Haskell solutions also did very well, LOC-wise.

The question asked about "fastest development," not "shortest code." But it's conceivable that shorter solutions are faster to come up with - certainly for slow typists!

---

There's an annual competition among roboticists. Contestants are given some specs for some hardware, a practical problem to solve in software, and limited time to do so. Again very domain specific, of course. Programmers have their choice of tools, including language of course. Every year, the winning team (often a single person) used Forth.

---

This admittedly limited sample suggests that "development speed" and "effect of language on speed" is often very dependent on the problem domain.

Answer 12

This article(a pdf) has some benchmarks (note that it's from 2000) between C, C++, Perl, Java, Perl, Python, Rexx and Tcl.

Some common wisdom I believe holds true (also somewhere within the article):

The number of lines written per hour is independent of the language

Answer 13

Most programs have to interface with some other framework. It tends to be a good idea to pick the language that has libraries specifically for what you are trying to do. For instance are you trying to build a distributed redundant messaging system? If so I would use Erlang. Are you trying to make a quick and dirty data driven website, use Ruby and Rails. You get the idea. Real time DirectX where performance is key, C++/C/Asm.

If you are writing something that is algorithm based I would look to a functional language like Haskell, although it has a very high learning curve.

Answer 14

It would make more sense to benchmark the programmers, not the languages. The time to write a program in any mainstream language depends more on the ability of the programmer in that language than on qualities of that specific language.

Answer 15

Opinion: more important is what is faster for a given developer, for example yourself. What you are used to, will usually be faster. If you are used to 20 years of C++ pitfalls and never skip an uninitialized variable, that will be faster than Java for anybody.

If you remember all parameters of CreateWindowEx() by heart, it will be faster than MFC or winforms.