CODEWITHSUNDEEP
postgresqldockersolrpsqlsolr9
anvbis
anvbis

Reputation: 33

Setup Solr in Docker and connect to Postgresql db in Docker

For a proof of concept, I need to set up Solr in a Docker container and a Postgresql DB also in a Docker container. Then I should connect both, so Solr can get the data from the Postgresql DB. I'm new to Solr and to Postgresql and don't really know Docker either, which is why I've been struggling to set up the environment. I've read the documentation and did the tutorials from Solr, but still couldn't figure out how I can get it to work.

What I want:

What I use:

For the database I created the following mock data:

CREATE TABLE publications(
    id uuid,
    title varchar(255) NOT NULL,
    subtitle varchar(255),
    content text NOT NULL,
    createdAt timestamp NOT NULL,
    active bool,
    PRIMARY KEY (id)
);

insert into publications (id, title, subtitle, content, createdAt, active) values ('fb566150-344a-49f8-a1df-64efed2af46f', 'Transformers: Dark of the Moon', 'Executive Secretary', 'Retinopathy of prematurity, stage 2, bilateral', '2024-05-30 02:43:09', true;

insert into publications (id, title, subtitle, content, createdAt, active) values ('d8a1dbdf-519a-489d-b667-6d0d5b26a8d0', 'My Dog Tulip', 'Senior Financial Analyst', 'Petaurus breviceps, bilateral', '2023-11-15 20:09:24', false;

insert into publications (id, title, subtitle, content, createdAt, active) values ('7cec9417-52a0-4122-a953-51f5bdbfe440', 'Lulu in Berlin', 'Nurse Practicioner', 'Orcinus orca', '2024-05-02 00:25:31', true;

insert into publications (id, title, subtitle, content, createdAt, active) values ('1d3c7676-48f5-487a-a49b-331e2a39bf61', 'Marianne & Juliane', 'Biostatistician', 'Gabianus pacificus', '2024-08-02 03:43:20', true;

insert into publications (id, title, subtitle, content, createdAt, active) values ('06212410-403b-4ae3-9890-7736d6edd958', 'Emotion', 'Financial Advisor', 'Mechanical ptosis of eyelid', '2024-01-11 06:05:50', false;

First, I tried to create my own docker-compose.yml file, where I start a Solr, a Zookeeper and a Postgresql instance. This all worked really well, but I couldn't figure out how I could get the data from the Postgresql DB in Solr. And I couldn't figure out how I could edit the files in Solr (solrconfig.xml etc). Furthermore I'm not sure if I could work with volumes, but I don't know how these work. I'm sure this would be the cleanest way of all the ways I tried.

Then I tried to run Solr and Postgresql without Docker but also couldn't connect the database. I tried to connect them with a DataImportHandler, but noticed that for Solr 9 the DIH is deprecated, so I passed this option. I tried numerous ways and tutorials, but none were close to what I try to do.

docker-compose.yml

version: '3.7'
services:
  solr:
    image: solr:9.7
    container_name: solr
    ports:
     - "8981:8983"
    environment:
      - ZK_HOST=zoo:2181
    networks:
      - solr
    depends_on:
      - zoo
zoo:
    image: zookeeper:3.9
    container_name: zoo
    hostname: zoo
    ports:
      - 2181:2181
      - 7001:7000
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zoo:2888:3888;2181
      ZOO_4LW_COMMANDS_WHITELIST: mntr, conf, ruok
      ZOO_CFG_EXTRA: "metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider metricsProvider.httpPort=7000 metricsProvider.exportJvmInfo=true"
    networks:
      - solr
postgres:
    image: postgres:16
    container_name: wz-db
    hostname: wz-db
    ports:
      - 5432:5432
    volumes:
      - 
    environment:
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_DB=wz-db
    networks:
      - solr
networks:
  solr:

This is how my Docker container looked docker-container

!!! EDIT

I tried a version with a local Solr (not Cloud) and a Postgresql instance running in Docker, so I could add files, but still can't figure out how to get the data directly from the DB.

New docker-compose.yml:

version: '3.7'
services:
  solr:
    container_name: solr
    build: 
      context: ./solr/var/solr
    ports:
     - "8981:8983"
    user: root # run as root to change the permissions of the solr folder
    # Change permissions of the solr folder, create a default core and start solr as solr user
    command: bash -c "
      chown -R 8983:8983 /var/solr
      && runuser -u solr -- solr-precreate wz-core"
    environment:
      DB_CONNEECTION: "dbname='wz-db' user='postgres' host='wz-db' password='postgres'"
    volumes:
      - ./data/solr:/var/solr 
    depends_on:
      - postgres
  postgres:
    image: postgres:16
    container_name: wz-db
    hostname: wz-db
    ports:
    - 5432:5432
    volumes:
      - ./data/postgres:/var/lib/postgresql/data
    environment:
    - POSTGRES_PASSWORD=postgres
    - POSTGRES_USER=postgres
    - POSTGRES_DB=postgres
volumes:
  data: {}

Upvotes: 0

Views: 87

Answers (0)

Related Questions