Solr: missing content stream, Magento: error while reindexing

Question

This one is driving me nuts.

The problem occurs in a two-server-system, where server A runs Magento EE 1.11 and server B runs Solr 3.4.0. Both servers are within the same network (192.168.x.x) and the Magento backend on A is configured to use the Solr daemon on B for catalog searches.

Imo, the Magento backend is configured correctly, as Magentos Test Connection feature always returns Successful! Test again? and I can find the pings caused by the tests in the Solr logs.

But when trying to

System -> Index Management -> Catalog Search Index -> Reindex data

Magento always fails with "There was a problem with reindexing process".

Checking the Solr logs gives missing content stream and returns HTTP 400 (bad request):

Nov 10, 2011 3:50:16 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
Nov 10, 2011 3:50:16 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=2
Nov 10, 2011 3:50:16 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
Nov 10, 2011 3:50:16 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=2
Nov 10, 2011 3:50:16 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: {} 0 0
Nov 10, 2011 3:50:16 PM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: missing content stream
        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:62)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368)
        at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

Nov 10, 2011 3:50:16 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json&<delete fromPending="true"+fromCommitted%3D"true"><query>store_id:2</query></delete>} status=400 QTime=0
Nov 10, 2011 3:50:16 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: {} 0 0
Nov 10, 2011 3:50:16 PM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: missing content stream
        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:62)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368)
        at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

Nov 10, 2011 3:50:16 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={<rollback/>=&wt=json} status=400 QTime=0

The only thing that somehow looks strange to me in this error trace is, that in the params of the /update command only one of the = is url-encoded to %3D, but all others are not.

<delete fromPending="true"+fromCommitted%3D"true"><query>store_id:2</query></delete>

Otoh I'm quite new to Solr and have no idea whether this is just maybe a bug of the Java logger itself.

I also have checked Apache_Solr_Service::deleteByQuery(), the PHP method which is producing the failing Solr query, but couldn't find any bugs in there so far.

The production environment unfortunately is bound to PHP 5.2.16 in this case, so another thing coming to my mind was possible problems with htmlspecialchars, stream_context_set_option or file_get_contents under PHP 5.2.16, but IIRC there weren't such.

So my question is: What is causing the 'missing content stream' error and/or how to fix this?

Answer 1

There's a known bug in early PHP releases with releases built with "--with-curl-wrappers".

By any chance did you compile with curl wrappers? Try re-compiling without and see how you fare.

Answer 2

Magento EE 1.11.0.0 uses combos of stream_content_set_option() and file_get_contents() calls to build and execute the Solr requests, so the problem seems to be related to the stream handling of PHP 5.2.16^*.

As I was running out of time, I've just overridden the Apache_Solr_Service class and modified the methods _sendRawGet() and _sendRawPost() to use cURL requests instead of PHP streams.

This workaround works perfectly fine so far (btw: thanks to @zuloo. I had the same thought to use cURL to workaround, but only wanted to use it as a last resort).

I'm still wondering what is causing this error when using PHP streams (meanwhile my prime suspect is chunked transfer encoding). Maybe someone else drops by and can shed some light, so I'm going to let the question stay unsolved for now.

---

_{* which is somewhat strange, because according to the Magento EE specs only PHP 5.2.13+ should berequired.}

Answer 3

we rewrote the enterprise/search/client/solr.php to use curl.

you would have to alter the _sendRawGet and _sendRawPost Methods:

protected function _sendRawGet($url, $timeout = FALSE)
{
    $curl = curl_init();
    curl_setopt ($curl, CURLOPT_URL,$url);
    curl_setopt ($curl, CURLOPT_RETURNTRANSFER, 1);
    if($timeout && is_numeric($timeout))
    {
        curl_setopt ($curl, CURLOPT_CONNECTTIMEOUT, floatval($timeout));
    }else{
        curl_setopt ($curl, CURLOPT_CONNECTTIMEOUT, floatval(5));
    }
    curl_setopt ($curl, CURLOPT_HEADER, 1);
    curl_setopt($curl, CURLOPT_HTTPHEADER, array("Authorization: Basic " . base64_encode($this->getUserLogin() . ':' . $this->getPassword())));
    $ret = curl_exec($curl);
    $info = curl_getinfo($curl);
    if(curl_errno($curl))
    {
        throw new Exception('"' . curl_errno($curl) . '" Info: ' . print_r($info,true));
    }
    $header = substr($ret, 0, $info['header_size']);
    $body = substr($ret, -$info['download_content_length']);
    $response = new Apache_Solr_Response($body, explode("\r\n",$header), $this->_createDocuments, $this->_collapseSingleValueArrays);
    if ($response->getHttpStatus() != 200 && $response->getHttpStatus() != 100)
    {
        throw new Exception('"' . $response->getHttpStatus() . '" Status: ' . $response->getHttpStatusMessage(), $response->getHttpStatus());
    }
    curl_close($curl);
    return $response;
}


protected function _sendRawPost($url, $rawPost, $timeout = FALSE, $contentType = 'text/xml; charset=UTF-8')
{
    $curl = curl_init();
    curl_setopt ($curl, CURLOPT_URL,$url);
    curl_setopt ($curl, CURLOPT_RETURNTRANSFER, 1);
    if($timeout && is_numeric($timeout))
    {
        curl_setopt ($curl, CURLOPT_CONNECTTIMEOUT, floatval($timeout));
    }else{
        curl_setopt ($curl, CURLOPT_CONNECTTIMEOUT, floatval(5));
    }
    curl_setopt ($curl, CURLOPT_HEADER, 1);
    $sendHeader = array(
        "Authorization: Basic " . base64_encode($this->getUserLogin() . ':' . $this->getPassword()),
        "Content-Type: ".$contentType
    );
    curl_setopt($curl, CURLOPT_POST, 1);
    curl_setopt($curl, CURLOPT_POSTFIELDS, $rawPost);
    curl_setopt($curl, CURLOPT_HTTPHEADER, $sendHeader);
    $ret = curl_exec($curl);
    $info = curl_getinfo($curl);
    if(curl_errno($curl))
    {
         throw new Exception('"' . curl_errno($curl) . '" Info: ' . print_r($info,true));
    }
    $header = substr($ret, 0, $info['header_size']);
    $body = substr($ret, -$info['download_content_length']);
    $response = new Apache_Solr_Response($body, explode("\r\n",$header), $this->_createDocuments, $this->_collapseSingleValueArrays);
    if ($response->getHttpStatus() != 200 && $response->getHttpStatus() != 100)
    {
        throw new Exception('"' . $response->getHttpStatus() . '" Status: ' . $response->getHttpStatusMessage(), $response->getHttpStatus());
    }
    curl_close($curl);
    return $response;
}

maybe this works, we didn't get any problems using this method