CODEWITHSUNDEEP
solrapache-zookeepersolrjtestcontainers
Jeremy
Jeremy

Reputation: 3548

Zookeeper + Solr in TestContainers

I'm using org.testcontainers to perform integration testing with Solr.

[Using SolrJ in my unit tests]

When I start Solr in cloud mode, using an embedded ZooKeeper instance, I'm able to connect to the solr instance from my unit test, but unable to connect to ZooKeeper from my SolrClient.
I think this is because embedded ZooKeeper is bound to IP 127.0.0.1 and inaccessible.

If I start two separate containers [using a shared network], ZooKeeper and Solr, I can connect Solr to ZooKeeper, and I can connect to Zookeeper from my unit tests, BUT when Zookeeper returns the active SOLR node, it return the internal server IP which is not accessible from my unit test [in my SolrJ client].

I'm not sure where to go with this.
Maybe there is a network mode that will do address translation?

Thoughts?

Upvotes: 1

Views: 1830

Answers (3)

dzezzz
dzezzz

Reputation: 1005

The way I used testcontainers with zk and solr is the below

  DockerImageName SOLR_IMAGE = DockerImageName.parse("solr:8.4.1");
    SolrContainer solrContainer = new SolrContainer(SOLR_IMAGE);
    solrContainer.withZookeeper(true);
    solrContainer.start();

    HttpClient httpClient = HttpClientBuilder.create().setProxy(new HttpHost("localhost", solrContainer.getSolrPort())).build();
    CloudSolrClient solrClient = new CloudSolrClient.Builder(Arrays.asList("localhost:" + solrContainer.getZookeeperPort()),
        Optional.empty()).withHttpClient(httpClient).build();
    return solrClient;

Upvotes: 0

Raynigon
Raynigon

Reputation: 701

UPDATE:

There is an official Testcontainers Module: https://www.testcontainers.org/modules/solr/

It seems that this problem can`t be solved that easy.

One way would be to use fixed ports with testcontainer. In this case the ports 9983 and 8983 will be mapped to the same ports on the host. This makes it possible to use the Solr Cloud Client. But this only works if you can ensure that tests will run sequentially, which can be a bit tricky, e.g. on Jenkins with Feature Branches.

A different solution would be to use another client. Since Solrj provides multiple Clients, you can choose which one you want to use. If you only want to search or update you can use the LBHttp2SolrClient which load balances between multiple nodes. If you want to use a specific client for the Integration Tests this example could work:

// Create the solr container.
SolrContainer container = new SolrContainer();

// Start the container. This step might take some time...
container.start();

// Do whatever you want with the client ...
SolrClient client = new Http2SolrClient.Builder("http://localhost:" + container.getSolrPort() + "/solr").build();
SolrPingResponse response = client.ping("dummy");

// Stop the container.
container.stop();

Here is a list of solr client in java: https://lucene.apache.org/solr/guide/8_3/using-solrj.html#types-of-solrclients

Upvotes: 1

Bianca
Bianca

Reputation: 31

I ran into the exact same issue. I solved it using a proxy. In my docker_compose.yml I added:

  squid:
    image: sameersbn/squid:3.5.27-2
    ports:
      - "3128:3128"
    volumes:
      - ./squid.conf:/etc/squid/squid.conf
      - ./cache:/var/spool/squid
    restart: always
    networks:
     - solr

And in the configuration of the SolrClient I added:

[...]

HttpClient httpClient = HttpClientBuilder.create().setProxy(new HttpHost("localhost", 3128)).build()
CloudSolrClient c = new CloudSolrClient.Builder(getZookeeperList(), Optional.empty()).withHttpClient(httpClient).build();

[...]

protected List<String> getZookeeperList() {
  List<String> zookeeperList = new ArrayList<String>();

  for (Zookeepers z : Zookeepers.values()) {
            zookeeperList.add(testcontainer.getServiceHost(z.getServicename(), z.getPort()) + ":"
                    + testcontainer.getServicePort(z.getServicename(), z.getPort()));
  }

  return zookeeperList;
}

But I'd still be interested in the workaround, that Jeremy mentioned in this comment.

Upvotes: 0

Related Questions