CODEWITHSUNDEEP
performancelanguage-agnostic
Rune Aamodt
Rune Aamodt

Reputation: 2611

Selecting An Embedded Language

I'm making an application that analyses one or more series of data using several different algorithms (agents). I came to the idea that each of these agents could be implemented as separate Python scripts which I run using either the Python C API or Boost.Python in my app.

I'm a little worried about runtime overhead TBH, as I'm doing some pretty heavy duty data processing and I don't want to have to wait several minutes for each simulation. I will typically be making hundreds of thousands, if not millions, of iterations in which I invoke the external "agents"; am I better of just hardcoding everything in the app, or will the performance drop be tolerable?

Also, are there any other interpreted languages I can use other than Python?

Upvotes: 7

Views: 2607

Answers (7)

Jason Orendorff
Jason Orendorff

Reputation: 45086

I will typically be making hundreds of thousands, if not millions, of iterations in which I invoke the external "agents"

The performance drop will be noticeable, perhaps painful. If you can put the data into arrays and process it in batches using NumPy, it should be much faster.

NumPy makes it super easy to do any kind of arithmetic a million times in a row. For example, squaring every element of an array is like this:

>>> x = numpy.array([1, 2, 3, 4, 5, 6, 7])
>>> x**2
array([1, 4, 9, 16, 25, 36, 49])

Super easy, and the tight inner loop here is actually implemented in C.

Of course NumPy can also do more advanced number-crunching.

Upvotes: 3

Skurmedel
Skurmedel

Reputation: 22149

Yes, tons. Lua and Python seems to be the most popular:

Embedding Lua

Embedding Python

Embedding Tcl

Embedding Ruby

Embed Perl

Embed JavaScript

There are dozens of JavaScript engines around, this is just an example. Some of them are also frighteningly quick.

Upvotes: 10

lhf
lhf

Reputation: 72312

Lua is quite fast as it is. If you need more speed, try LuaJIT, which is excellent.

Upvotes: 9

T.E.D.
T.E.D.

Reputation: 44804

I believe tcl and Rexx were both intended for this purpose.

Upvotes: 2

Bryan Oakley
Bryan Oakley

Reputation: 385970

Tcl was designed from the ground up to be an embedded language.

Upvotes: 5

Colin
Colin

Reputation: 3752

For millions of calls (from I'm assuming c++, because you mentioned boost) into python, yes: you will notice a performance hit. This may or may not be significant - perhaps the speed gain of trying out new 'agents' would be greater than the hit. Python does have fast numerical libraries (such as numpy) that might help, but you'll still incur overhead of marshalling data, calling into python, the gil, etc.

Yes, you can embed many other languages: check out lua. Also, check out swig.org, which can connect to many other languages besides python.

Upvotes: 1

Anycorn
Anycorn

Reputation: 51465

you could probably create an embedded language using C++ templates and operator overloading, see for example ublas or ftensor matrix languages. i do not think python or other interpreted languages of is suitable for having numbercrunching/data processing.

Upvotes: 0

Related Questions