CODEWITHSUNDEEP
javapythonpylucene
kolurbo
kolurbo

Reputation: 538

How to build and install pylucene on ubuntu 20.04

I am trying to install Pylucene on my WSL Ubuntu 20.04 clean installation. I tried to follow tutorial on the official page but it looks outdated. So I was wondering if anyone here managed to make it work on Ubuntu 20.04 and python 3.8.2

The commands I run:

sudo apt-get upgrade
sudo apt-get install -y default-jdk ant build-essential python3-dev
mkdir pylucene
cd pylucene
curl https://downloads.apache.org/lucene/pylucene/pylucene-8.3.0-src.tar.gz | tar -xz --strip-components=1
cd jcc
export JCC_JDK=/usr/lib/jvm/default-java
python3 setup.py build

^^^^^ Fails here on:

...
building 'jcc3' extension
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -D_jcc_lib -DJCC_VER="3.7" -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/default-java/include/linux -I_jcc3 -Ijcc3/sources -I/usr/include/python3.8 -c jcc3/sources/jcc.cpp -o build/temp.linux-x86_64-3.8/jcc3/sources/jcc.o -DPYTHON -fno-strict-aliasing -Wno-write-strings
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -D_jcc_lib -DJCC_VER="3.7" -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/default-java/include/linux -I_jcc3 -Ijcc3/sources -I/usr/include/python3.8 -c jcc3/sources/JCCEnv.cpp -o build/temp.linux-x86_64-3.8/jcc3/sources/JCCEnv.o -DPYTHON -fno-strict-aliasing -Wno-write-strings
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.8/jcc3/sources/jcc.o build/temp.linux-x86_64-3.8/jcc3/sources/JCCEnv.o -o build/lib.linux-x86_64-3.8/libjcc3.so -L/usr/lib/jvm/default-java/jre/lib/amd64 -ljava -L/usr/lib/jvm/default-java/jre/lib/amd64/server -ljvm -Wl,-rpath=/usr/lib/jvm/default-java/jre/lib/amd64:/usr/lib/jvm/default-java/jre/lib/amd64/server -Wl,-S
/usr/bin/ld: cannot find -ljava
/usr/bin/ld: cannot find -ljvm
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-g++' failed with exit status 1

Commands I plan to run afterwards:

sudo python3 --preserve-env=JCC_JDK setup.py install
cd ..
make
make test
sudo make install

Upvotes: 3

Views: 3356

Answers (4)

Open Trap
Open Trap

Reputation: 1

This is how I installed Pylucene in my Ubuntu 22.04 system.

conda create -n lucene python=3.12.1
# I have used the latest python to show compatibility feel free to use your preferred version
conda activate lucene
curl https://downloads.apache.org/lucene/pylucene/pylucene-9.4.1-src.tar.gz | tar -xz
cd pylucene-9.4.1
cd jcc
# Please setup your JVM here
# I have used Temurin-17 JDK which is recommended in their installation instructions.
# export JAVA_HOME=$(Your JDK Root Here)
python setup.py build
python setup.py install
cd ..

edit your Makefile in the pylucene directory as follows:

PREFIX_PYTHON=/home/$USER/python/env/  # Make Sure this is your Python Env 
PYTHON=$(PREFIX_PYTHON)/bin/python3
JCC=$(PYTHON) -m jcc --shared
NUM_FILES=16

Once you have done that run this in your pylucene dir

make 
make install

Upvotes: 0

XPLOT1ON
XPLOT1ON

Reputation: 3223

I wrote this dockerfile for PyLucene 8.11.0, Python 3.9, JDK11 (default-jdk) and ubuntu 20.04 (focal). Do extend with your favorite python package manager such as Poetry, Pipenv or Conda etc.

Dockerfile

FROM ubuntu:focal

# Python Version x.y
ARG PYTHON_VERSION=3.9
ARG PYLUCENE_VERSION=8.11.0

# Uncomment to install specific version of poetry
ENV LANG=C.UTF-8

# ADD Python PPA Repository
RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y --no-install-recommends \
        software-properties-common gpg-agent && \
    add-apt-repository ppa:deadsnakes/ppa && \
    apt-get update && \
    apt-get remove -y software-properties-common && \
    apt-get purge --auto-remove -y && \
    apt-get clean

RUN which gpg-agent

# Install Python
RUN apt-get install -y --no-install-recommends \
        "python$PYTHON_VERSION-dev" \
        python3-setuptools \
        python3-pip && \
    apt-get remove -y gpg-agent && \
    apt-get purge --auto-remove -y && \
    apt-get clean

# ======================== START OF ADDITIONAL INSTALLATION ========================

# Install Java
RUN apt-get install -y --no-install-recommends \
    build-essential \
    ant \
    jcc \
    curl \
    git \
    default-jdk 
    
RUN ls /usr/bin/ | grep "python"
RUN ln -s $(which python$PYTHON_VERSION) /usr/bin/python

RUN which python$PYTHON_VERSION && which python && python --version

WORKDIR /usr/lib/jvm/default-java/jre/lib
RUN ln -s ../../lib amd64

# Java 11
RUN java --version && javac --version


# Installing PyLucene
RUN which ant && ant -version

RUN apt-get install -y --no-install-recommends \
    libffi-dev \
    zlib1g-dev

WORKDIR /usr/src/pylucene
RUN curl https://dlcdn.apache.org/lucene/pylucene/pylucene-$PYLUCENE_VERSION-src.tar.gz | tar -xz


ENV PREFIX_PYTHON=/usr \
    JCC_JDK=/usr/lib/jvm/default-java \
    ANT=ant \
    JCC='python -m jcc' \
    NUM_FILES=10 \
    PYTHON=python \
    NO_SHARED=1

RUN cd "pylucene-$PYLUCENE_VERSION/lucene-java-$PYLUCENE_VERSION/lucene" && \
    ant ivy-bootstrap && \
    ant && \
    cd ../../../

RUN cd "pylucene-$PYLUCENE_VERSION/jcc" && \
    ls -la && \
    NO_SHARED=1 JCC_JDK=/usr/lib/jvm/default-java python setup.py build && \
    NO_SHARED=1 JCC_JDK=/usr/lib/jvm/default-java python setup.py install && \
    cd .. && \
    make JCC="python -m jcc" ANT=ant PYTHON=python NUM_FILES=8&& \
    make install JCC="python -m jcc" ANT=ant PYTHON=python NUM_FILES=8 && \
    cd ../../

RUN apt-get remove -y gpg-agent ant jcc build-essential && \
    apt-get purge --auto-remove -y && \
    apt-get clean

WORKDIR /usr/src
RUN rm -rf pylucene

RUN python -c "import lucene; lucene.initVM()"

# ======================== END OF ADDITIONAL INSTALLATION ========================

WORKDIR /app
COPY . .

Upvotes: 1

Code Your Dream
Code Your Dream

Reputation: 387

Here are the steps that lead to the successful installation of pylucene on Ubuntu 18.04 - this may work for you:

  1. Install openjdk-8: apt install openjdk-8-jre openjdk-8-jdk openjdk-8-doc Ensure that you have ant installed, if you don't run apt install ant. Note that if you had a different version of openjdk installed you need to either remove it or run update-alternatives so that version 1.8.0 is used.

  2. Check that Java version is 1.8.0* with java -version

  3. After installing openjdk-8 create a symlink (you'll need it later):

cd /usr/lib/jvm
ln -s java-8-openjdk-amd64 java-8-oracle

  1. Install python-dev: sudo apt install python-dev In my case Python 3 didn't work so I ended up using Python 2. But this might not have been the actual reason of the problem, so you're welcome to try Python 3. If you go with Python 3, use python3 instead of python in the commands below.

  2. Install JCC (in jcc subfolder of your pylucene folder):

python setup.py build
python setup.py install

The symlink you created on step 3 will help here because this path is hardcoded into setup.py - you can check that.

  1. Install pylucene (from the root of your pylucene folder). Edit Makefile, uncomment/edit the variables according to your setup. In my case it was
PREFIX_PYTHON=/usr
ANT=ant
PYTHON=$(PREFIX_PYTHON)/bin/python
JCC=$(PYTHON) -m jcc --shared
NUM_FILES=10

Then run

make
make test
sudo make install
  1. If you see an error related to the shared mode of JCC remove --shared from Makefile.

Upvotes: 1

cronopioelectronico
cronopioelectronico

Reputation: 1

Confirmed that the answer posted by @code-your-dream works also with Python3 (and in particular in a Ubuntu 18.04.1)

For me was important to install jcc that way. I tried installing via conda and there were conflicts in the make of pylucene.

In my case also was needed to modify the setup.py file from jcc changing 'linux': ['-fno-strict-aliasing', '-Wno-write-strings'], by 'linux': ['-fno-strict-aliasing', '-Wno-write-strings','-D__STDC_FORMAT_MACROS'], as mentioned in this other thread Issue with installing PyLucene 6.5.0 on Linux

To confirm which is the version of java needed (in my case also 8, didn't work with 11), you can search in the pylucene folder /jcc/setup.py the block JDK = {...}. In my case with the reference: 'linux': '/usr/lib/jvm/java-8-oracle',

Upvotes: 0

Related Questions