Multimedia framework for video playback in a GUI application?

Question

Some time ago I started a project in which I needed to do the following things:

• Play videos:
  • In any common format (avi, mpeg, xvid, etc.)
  • Be able to pause and reposition the stream programmatically.
  • Be able to set the playback speed (not mandatory, but desirable)
• Let me paint custom graphics (kept in sync with the video) over the playback component.
• Allow access the current frame as a raster image (this is not mandatory, but desirable).

And also be easy to use (e.g. visual component oriented, I don't want to deal a lot with the multimedia streams internals).

As I already had some related code written in Java, I tried with all the Java libraries that I found (JMF, FMJ, JVLC, Jffmpeg, gstreamer-java) but no one satisfied all my hard requirements (for example, those which allowed me to draw over the frame couldn't be paused or rewinded, those which were full-featured were outdated, etc.), so I put the project on hold.

Now I'm on the mood for a comeback and I won't mind to rewrite the code in other language (of a level equal or higher than Java, but please not C or C++, I've become too lazy :-p) that has a good GUI framework or Rich Client Platform freely available too. Being multiplatform is a good point, as I like to develop and test in Linux, but I'd like to reach a larger audience, so being able to target Windows is a must.

So what language and frameworks would you recommend me?

Answer 1

If you're not totally afraid to go windows only, WPF should respond to nearly all, if not all, of your demands.

You can use it with C# and all .net language.

Answer 2

An embedded web browser would handle your stated requirements:

• Video Playback (use the mplayer or vlc plugins)
• High-level language to control playback (javascript)
• Overlay/GUI (HTML + CSS + PNG/SVG)

To raster the current frame would be harder, you may need some custom ffmpeg to dump a frame to disk then read it back in as an < img >

Answer 3

Another framework you might consider is the mlt framework: http://mltframework.org/

It's got a great design, is written in C, and uses SWIG for language bindings (java, perl, php, python, ruby, tcl.) I know people have had it running on most of the major platforms.

To understand what it's capable of, check out the awesome NLE kdenlive, it's most active user.

Answer 4

I would use OpenCV library. There are all the features you need: formats support, any frame fetching, frame modification, etc. Check out this installation guide and some samples.

Answer 5

You say that you tried gstreamer-java. Perhaps it's bindings aren't all that good? I am pretty sure that Gstreamer can do all that you ask. Perhaps you should check out the Python bindings instead. Python is a first class citizen for the Gstreamer developers so the bindings should always be complete and up-to-date.

Python also has multiple cross-platform GUI frameworks available. Personally I like GTK using libglade because it's ridiculously easy to build a GUI using glade and it's XML files (and GTK runs on Windows too, so no problem there). If you prefer something else then there's Qt, wxWidgets and Tk, all of which run on Linux and Windows (and I assume MacOSX as well).

Answer 6

Qt's Phonon. Qt also is available for Java, as Qt Jambi.