CODEWITHSUNDEEP
multithreadingdelphiidhttp
user2864778
user2864778

Reputation: 333

Multiple requests on same IdHTTP inside thread

I have an application that makes thousands of HTTP requests along the day. For best performance, I decided to create only once the IdHTTP object and use this same object for all requests. This is where the problem begins. While creating one IdHTTP for each request, everything went fine. The code is pretty basic:

constructor HTTPThread.Create;
begin
  inherited Create(false);
  httpObject:= TIdHTTP.Create(Nil);
  sslObject:= TIdSSLIOHandlerSocketOpenSSL.Create(Nil);
  sslObject.SSLOptions.Method:= sslvTLSv1_2;
  httpObject.IOHandler:= sslObject;
  httpObject.Request.Accept:= frmHTTPRequests.Edit1.Text;
  httpObject.Request.UserAgent:= frmHTTPRequests.Edit3.Text;
  httpObject.ReadTimeout:= 15000;
  httpObject.HandleRedirects:= true;
  FreeOnTerminate:= true;
  OnTerminate:= TerminateProc;
end;


procedure HTTPThread.DoRequests;
var
    htmlSource: string;
begin
    try
      htmlSource:= httpObject.Get(Link);
      //a bunch of other stuff with HTML source
    except
      on E : Exception do
      Synchronize(procedure
      begin
        errorList.Add(E.Message);
        errorList.SaveToFile('Error.txt');
      end);
    end;
end;

I created this except saving the Error.txt file to watch what is happening... The code runs fine sometimes for the first 1k requests, sometimes for the first 2k, it varies. Suddently, it starts writing on the TXT file the same error:

Connection reset by peer. domain.com - Socket Error # 10054

I tried to disconnect the httpObject, tried httpObject.Request.Clear, nothing seems to work. Are there any chance to make this work?

Upvotes: 0

Views: 784

Answers (1)

Olvin Roght
Olvin Roght

Reputation: 7812

For some reasons Indy doesn't close socket when server respond with Connection reset by peer, so you need to do it manually.

procedure HTTPThread.DoRequests;
const
  MAX_TRIES_COUNT = 5;
var
  htmlSource: string;
  TriesCount: Integer;
begin
  TriesCount := 0;
  repeat
    try
      htmlSource:= httpObject.Get(Link);
      //a bunch of other stuff with HTML source
    except
      on E: Exception do
      begin
        if E.Message.Contains('by peer') then
        begin
          httpObject.Disconnect;
          // Try to solve network connection issues
          Continue;
        end
        else
        begin
          // Some other error handlers
        end;
      end;
      inc(TriesCount);
    end;
  until (httpObject.ResponseCode = 200) or (TriesCount > MAX_TRIES_COUNT);
end;

P.S. I don't recommend you to use Synchronize() for threads synchronization. Try to use TCriticalSection or TMonitor instead.

Upvotes: 1

Related Questions