CODEWITHSUNDEEP
pythondockeraws-lambdaamazon-ecr
Mohamed Gamea
Mohamed Gamea

Reputation: 11

AWS lambda stuck importing library

I am trying to deploy my ECR image to aws lambda. The image works fine locally, but on aws, it gets stuck importing this library https://github.com/jianfch/stable-ts.

import json
import boto3
import requests
import numpy
print("All imports ok 1 ...")

from stable_whisper import load_model
print("All imports ok 2 ...")

The first statement is printed but it gets stuck on importing and the second statement never got printed until it timed out.

Docker File:


# Build FFmpeg
FROM public.ecr.aws/lambda/python:3.8 as lambda-base

COPY requirements.txt ./
COPY myfunction.py ./

RUN pip3 install -r requirements.txt

WORKDIR /ffmpeg_sources
RUN yum install autoconf automake bzip2 bzip2-devel cmake libxcb libxcb-devel \
    freetype-devel gcc gcc-c++ git libtool make pkgconfig zlib-devel -y -q

# Compile NASM assembler
RUN curl -OL https://www.nasm.us/pub/nasm/releasebuilds/2.15.05/nasm-2.15.05.tar.bz2
RUN tar xjvf nasm-2.15.05.tar.bz2
RUN cd nasm-2.15.05 && sh autogen.sh && \
    ./configure --prefix="/ffmpeg_sources/ffmpeg_build" \
    --bindir="/ffmpeg_sources/bin" && \
    make && make install

# Compile Yasm assembler
RUN curl -OL https://www.tortall.net/projects/yasm/releases/yasm-1.3.0.tar.gz
RUN tar xzvf yasm-1.3.0.tar.gz
RUN cd yasm-1.3.0 && \
    ./configure --prefix="/ffmpeg_sources/ffmpeg_build" \
    --bindir="/ffmpeg_sources/bin" && \
    make && make install

# Compile FFmpeg
RUN curl -OL https://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2
RUN tar xjvf ffmpeg-snapshot.tar.bz2
RUN cd ffmpeg && \
    export PATH="/ffmpeg_sources/bin:$PATH" && \
    export PKG_CONFIG_PATH="/ffmpeg_sources/ffmpeg_build/lib/pkgconfig" && \
    ./configure \
    --prefix="/ffmpeg_sources/ffmpeg_build" \
    --pkg-config-flags="--static" \
    --extra-cflags="-I/ffmpeg_sources/ffmpeg_build/include" \
    --extra-ldflags="-L/ffmpeg_sources/ffmpeg_build/lib" \
    --extra-libs=-lpthread \
    --extra-libs=-lm \
    --enable-libxcb \
    --bindir="/ffmpeg_sources/bin" && \
    make && \
    make install
# Final image with code and dependencies
FROM lambda-base

COPY myfunction.py /var/task/


CMD ["myfunction.lambda_handler"]

inside the requirements.txt, I tried both stable-ts and git+https://github.com/jianfch/stable-ts.git

I appreciate any help.

Upvotes: 1

Views: 203

Answers (1)

Pawel Kam
Pawel Kam

Reputation: 2134

stable_whisper has a lot of dependencies and some of them contain compiled code (ffmpeg). Python packages that contain compiled code aren't always compatible with Lambda runtimes by default. I don’t know how to build ffmpeg, but I can point you to a useful AWS sample, which utilizes python packages based on this dependency. Maybe it will contribute to solving your problem or maybe others will be able to help you further.

Sample Dockerfile:

# Install dependencies
…

# Build FFmpeg
FROM public.ecr.aws/lambda/python:3.8 as ffmpeg
WORKDIR /ffmpeg_sources
RUN yum install autoconf automake bzip2 bzip2-devel cmake libxcb libxcb-devel \
    freetype-devel gcc gcc-c++ git libtool make pkgconfig zlib-devel -y -q

# Compile NASM assembler
RUN curl -OL https://www.nasm.us/pub/nasm/releasebuilds/2.15.05/nasm-2.15.05.tar.bz2
RUN tar xjvf nasm-2.15.05.tar.bz2
RUN cd nasm-2.15.05 && sh autogen.sh && \
    ./configure --prefix="/ffmpeg_sources/ffmpeg_build" \
    --bindir="/ffmpeg_sources/bin" && \
    make && make install

# Compile Yasm assembler
RUN curl -OL https://www.tortall.net/projects/yasm/releases/yasm-1.3.0.tar.gz
RUN tar xzvf yasm-1.3.0.tar.gz
RUN cd yasm-1.3.0 && \
    ./configure --prefix="/ffmpeg_sources/ffmpeg_build" \
    --bindir="/ffmpeg_sources/bin" && \
    make && make install

# Compile FFmpeg
RUN curl -OL https://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2
RUN tar xjvf ffmpeg-snapshot.tar.bz2
RUN cd ffmpeg && \
    export PATH="/ffmpeg_sources/bin:$PATH" && \
    export PKG_CONFIG_PATH="/ffmpeg_sources/ffmpeg_build/lib/pkgconfig" && \
    ./configure \
    --prefix="/ffmpeg_sources/ffmpeg_build" \
    --pkg-config-flags="--static" \
    --extra-cflags="-I/ffmpeg_sources/ffmpeg_build/include" \
    --extra-ldflags="-L/ffmpeg_sources/ffmpeg_build/lib" \
    --extra-libs=-lpthread \
    --extra-libs=-lm \
    --enable-libxcb \
    --bindir="/ffmpeg_sources/bin" && \
    make && \
    make install

# Final image with code and dependencies
FROM lambda-base 

# Copy FFMpeg binary
COPY --from=ffmpeg /ffmpeg_sources/bin/ffmpeg /usr/bin/

Upvotes: 1

Related Questions