CODEWITHSUNDEEP
delphicpuindyindy10idhttp
Schneck
Schneck

Reputation: 73

Indy 10.6 - idHTTP 100% CPU usage when connecting to Network Stream

I was using Indy 10.5 and several older versions for years. In order to support HTTP Digest Authentication I have downloaded the latest version from their SVN.

This is fixing a HTTP Digest Authentication issue I was having...

The problem is the CPU usage with all the versions since 10.6.

I was able to create 20 threads in parallel to retrieve HTTP Streams (Network Camera MJPEG Streams to be precise) and it was using about 1-3% of cpu per task depending on the resolution and frame rate. (including parsing and decoding)

I have cleaned my code which normally uses IOHandler, CookieManager and relies on the InternalWork method until I'm left with the most basic usage of idHttp that could be that is:

procedure TForm1.Button1Click(Sender: TObject);
var
    HTTPClient : TidHTTP;
begin
  HTTPClient := TidHTTP.Create(nil);
  HTTPClient.Get('http://plazacam.studentaffairs.duke.edu/axis-cgi/mjpg/video.cgi?resolution=640x480');
  HTTPClient.Free();
end;

And even the above code without changing any setting is consuming 25% of CPU (so it saturates 1 core of my computer).

It's an infinite stream so it's normal that it's not returning to the client but since the stream is probably < 100kb/s, the cpu usage for retrieving it should be close to zero...

I have read on another forum in German that adding this HTTPOptions may solve the problem of high CPU but it has no effect on CPU in my case: HTTPOptions := [hoNoParseMetaHTTPEquiv]

I have tried with the previous releases from Indy SVN to ensure it's not something specific to the daily build.

The 10.6.0 version that ships with XE4,XE5 and X6 has the same problem but If I rollback to: Indy 10.5.5 - RS2010 RTM

Then everything is fine (apart from HTTP Digest authentication) and it only uses about 2% of cpu...

Did anybody encounter the same and has a way that current version of indy uses reasonable amount of CPU for doing this?

Thank you!

Upvotes: 0

Views: 907

Answers (1)

Remy Lebeau
Remy Lebeau

Reputation: 595320

Think about what you are doing. You are calling the version of TIdHTTP.Get() that returns a String, but you are downloading an "infinite" stream. That means Get() is constantly reading data from the socket and allocating more and more memory to hold that data in anticipation of eventually converting it into a String. So it makes sense that the CPU usage may be high over time, especially if you multiple that work across multiple threads.

As a test, try telling Get() to discard the data it downloads instead of buffering it in memory, and see what the CPU usage looks like:

HTTPClient.Get('http://plazacam.studentaffairs.duke.edu/axis-cgi/mjpg/video.cgi?resolution=640x480', TStream(nil));

TIdHTTP is not really designed to handle "infinite" streams, unless you provide a TStream that is able to receive "infinite" data (like TIdEventStream), or enable the hoNoReadMultipartMIME flag if the server is using MIME to deliver the streaming data, in which case you can read the streaming data yourself as described in this blog article:

New TIdHTTP hoNoReadMultipartMIME flag

Upvotes: 2

Related Questions