CODEWITHSUNDEEP
google-cloud-vertex-aikubeflowkubeflow-pipelinesvertex-ai-pipeline
filipe
filipe

Reputation: 505

Sudden ImportError: cannot import name 'appengine' from 'requests.packages.urllib3.contrib error on pipeline

My pipelines and schedulers were running smoothly without any problems. After I went out to lunch, I changed the number of epochs a Neural Network would run, save the .yaml file again and leave it in the bucket named "budgetff".

Afterwards, everything stopped working.

No clue

enter image description here

There are the errors and I have 0 clue as to how they are appearing. The code within the components doesn't even seem to start. I've made several different components without any success because they just fail at this step.

If it helps, I installed the kfp --pre and did the imports like this

import kfp.v2.dsl, kfp.v2.compiler
from kfp.v2.dsl import Artifact, Dataset, Input, Metrics, Model, Output

kfp-2.0.0-beta.15 - This is the kfp version running on VertexAi and I'm using Kubeflow with @kfp.v2.dsl.components.

I was trying to just run my pipelines. Forcing a run on the scheduler. When it didn't work, I just tried on the notebook.

Upvotes: 43

Views: 45575

Answers (11)

Mikko Ohtamaa
Mikko Ohtamaa

Reputation: 83656

In my case the root cause was gql package that pinned the old version.

The following version bump solved the issue:

  - Updating requests-toolbelt (0.10.1 -> 1.0.0)
  - Updating gql (3.4.0 -> 3.5.0)

Upvotes: 0

Mary N
Mary N

Reputation: 117

python3 -m pip show urllib3 python3 -m pip show requests

If the paths point to /usr/local/lib, these are overriding system packages. Remove the conflicting Python packages to avoid interference.

sudo pip3 uninstall urllib3 requests requests-toolbelt

Reinstall the certbot package to ensure all its dependencies are intact.

sudo apt update sudo apt install --reinstall certbot python3-certbot

Upvotes: 0

Robin De Schepper
Robin De Schepper

Reputation: 6364

Unrelated to OP's specific issue, but outdated versions of twine are also subject to this same error. pip install --upgrade twine won't fix it, but this will:

pip install --upgrade twine requests-toolbelt

Upvotes: 21

westman379
westman379

Reputation: 723

In my case it has worked when in my dependencies I've updated requests-toolbelt from version 0.10.1 to version 1.0.0.

Upvotes: 4

filipe
filipe

Reputation: 505

Used a base_image="python:3.11 and all is working now. Thank you so much for the valuable feedback and help! Have a great day

Edit: Was having a lot of compatibility issues, decided to go with the first suggestion of adding appengine-python-standard on the packages to install :)

Upvotes: 1

jorgmo02
jorgmo02

Reputation: 41

This happened to me in my GitHub Action and the problem was that I was using an old version of poetry. Using poetry 1.4.2 fixed it.

Source of an issue with the same root problem: https://github.com/ionrock/cachecontrol/issues/292#issuecomment-1536120527

Upvotes: 0

Paul
Paul

Reputation: 2347

The cause is that the latest version of requests does not support urllib3 2.0.0. This is fixed in kfp-2.0.0b16 (see PR with the change), so you can either upgrade to that, or create a new image that downgrades urllib.

Maybe this is triggered by versions of requests-toolbelt and/or urllib3 that were both released in the past few days (May 1 and May 4, 2023, resp).

I have fixed this by building a new container with the following Dockerfile (I use Python 3.9 but use whatever you want):

FROM python:3.9

RUN pip install urllib3==1.26.15 requests-toolbelt==0.10.1

I recommend to build the image using Cloud Build and specify it as the base image for the component.

Upvotes: 42

rayuron
rayuron

Reputation: 31

You should use kfp==1.8.21 ref. https://github.com/kubeflow/pipelines/pull/9323

Upvotes: 2

Abdel M
Abdel M

Reputation: 81

These issues were fixed in kfp==1.8.21 and kfp==2.0.0b16.

Upvotes: 7

mwtmurphy
mwtmurphy

Reputation: 374

I ran into the same issue today and was scratching my head. I found adding appengine-python-standard to the packages_to_install argument in the component decorator solved the issue:

@component(base_image="python:3.7", packages_to_install=["appengine-python-standard",...])

For context, I'm using KFP v1.8.20 through Vertex workbench. Fingers crossed, it works for v2.0.0-beta.15.

Upvotes: 14

Nelson Filipe Rodrigues Nunes
Nelson Filipe Rodrigues Nunes

Reputation: 21

I had the same problem. I updated the pipeline component to use a more recent Python version and now it works:

@component(
    base_image="python:3.11",

Upvotes: 2

Related Questions