CODEWITHSUNDEEP
delphifiledownloaddelphi-2010indy
Rafael Colucci
Rafael Colucci

Reputation: 6078

TidHttp file download trows Out of memory Exception

Consider the follow code that downloads a file from internet using Indy components:

procedure TForm26.Button1Click(Sender: TObject);
var
  vFileStream : TStream;
begin
  DeleteFile('C:\test.exe');
  vFileStream := TFileStream.Create('C:\test.exe', fmCreate);
  IdHTTP1.Get('SomeUrl', vFileStream);
  vFileStream.Free;
end;

I am getting an Out Of memory exception. What is happening is that besides the fact that I am using TFileStream, the bytes that are being written to it do not go directly to the disk, instead, they stay in memory until get terminates.

The file I am trying to download is really, really big.

Does anyone knows how to download a big file without getting an out of memory exception?

Delphi 2010 and latest Indy 10 from Indy's SVN.

EDIT

That is not a FileStream problem. It is a Indy problem. Indy is, somehow, caching the file in memory before it writes to the stream.

Upvotes: 4

Views: 1913

Answers (3)

Rafael Colucci
Rafael Colucci

Reputation: 6078

I found out the issue. I was using, on the server side, the Indys ServeFile function.

This function checks if a Content-Type was specified and if not, it detects automatically the Content-Type. The problem was that I did not change the Content-Type and it by default was text/html. Changing the content-type made the client write directly to the stream.

I think that the serveFile function should always set the correct Content-Type to avoid this kind of problems.

On the client side I found out this code that helped me a lot:

  LParseHTML := IsContentTypeHtml(AResponse) and Assigned(AResponse.ContentStream) and not (hoNoParseMetaHTTPEquiv in FOptions);
  LCreateTmpContent := LParseHTML and not (AResponse.ContentStream is TCustomMemoryStream);

Upvotes: 3

Remy Lebeau
Remy Lebeau

Reputation: 598134

TIdHTTP downloads the entire file into memory if the data is compressed, or if the data is HTML and the TIdHTTP.HTTPOptions property does not contain the hoNoParseMetaHTTPEquiv flag.

Indy does not yet support streaming decompression for HTTP (it does for FTP, though), so TIdHTTP caches the entire compressed data in memory before it can then decompress it to the file.

Parsing HTML is sometimes needed in cases where the HTML overrides HTTP header values with new values via HTML <meta> tags, most importantly the data's Charset value, so TIdHTTP can decode the data using the correct charset when the data is being returned to user's code as a String. Enabling the hoNoParseMetaHTTPEquiv flag disables that parsing, and thus any caching of HTML data (unless compression is also used, that is).

Upvotes: 5

Pol
Pol

Reputation: 5134

Did you tried downloading the file in parts by setting Request.ContentRangeStart and Request.ContentRangeEnd?

Upvotes: 1

Related Questions