CODEWITHSUNDEEP
pythongitlab-cipython-importpythonpath
Guy
Guy

Reputation: 810

Python importing main() problem - module exists, works in PyCharm, Gitlab says not found

Asked a similar question before, but I marked it answered, and I have other information.

Here is the structure:

engine-load-tests\
    |-src\
        |- __init__.py
        |- main.py
            |- on_locust_init()
        |
        |- win_perf_counters
                |- __init__.py
                |- main.py
                    |- main([])
    |- __init__.py
    |- .gitlab-ci.yml

engine-load-tests\src\main.py:

    import src.win_perf_counters.main as wmi_counters

...
    def on_locust_init():
        pid = wmi_counters.main([conf_file, database])
Also tried ...
    from src.win_perf_counters.main import main
...
    pid = main([conf_file, database])

win_perf_counters:
    class WinPerf:
        def x:
            ...
        def y:
            ...

    def main(args):
        // code

    if __name__ == "__main__":
        import sys

        main(sys.argv[1:])

This may not be good form, but it always used to work, and it continues to work when I run in PyCharm.

When I run in Gitlab CI, I get:

  File "C:\GitLab-Runner\builds\eTJPJvW5\0\qa\engine-automation\engine-load-tests\src\main.py", line 14, in <module>
    import src.win_perf_counters.main as wmi_counters
ModuleNotFoundError: No module named 'src.win_perf_counters.main'

I've spent all day reading the PythonDoc, but I am still not seeing what I am missing.

I have a line in the CI YML set PYTHONPATH=%PYTHONPATH%:., but that does not help.

I would think that if there is a fundamental flaw in my imports, it should fail in PyCharm as well.

I wonder if it has to do with the fact that I am trying to include main.

Suggestions for better Python design welcome and appreciated.

Also works on DOS command line.

Gitlab YAML

(I know it would be much faster to make this a Docker image instead of running pip every time.)

test:
  before_script:
    - python -V
    - pip install virtualenv
    - virtualenv venv
    - .\venv\Scripts\activate.ps1
    - refreshenv
    - python -m pip install --upgrade pip
    - python -m pip install locust
    - locust -V
    - python -m pip install multipledispatch
    - python -m pip install pycryptodome
    - python -m pip install pandas
    - python -m pip install wmi
    - python -m pip install pywin32
    - python -m pip install influxdb_client    
  script:
  - set LOAD_TEST_CONF=load_test.conf
  - set PYTHONPATH=%PYTHONPATH%:%CI_PROJECT_DIR%\src
  - echo "**** about to run locust ******"
  - locust -f ./src/main.py --host $TARGET_HOST -u $USERS -t $TEST_DURATION -r $RAMPUP -s 1800 --headless --csv=./LoadTestsData_VPOS --csv-full-history --html=./LoadTestsReport_VPOS.html  --stream-file ./data/stream_jsons/streams_vpos.json --database=csv
  
  - Start-Sleep -s $SLEEP_TIME

  variables:
    LOAD_TEST_CONF: load_test.conf
    PYTHON_VERSION: 3.8.0
    TARGET_HOST: http://x.x.x.x:9000

  tags:
    - win2019
  artifacts:
    paths:
      - ./LoadTests*
      - public
  only:
    - schedules

  after_script:
    - mkdir .public
    - cp -r ./LoadTests* .public 
    - cp metrics.csv .public -ErrorAction SilentlyContinue
    - mv .public/* public

Upvotes: 1

Views: 361

Answers (1)

Hampus
Hampus

Reputation: 346

Since you're doing

set PYTHONPATH=%PYTHONPATH%:%CI_PROJECT_DIR%\src

It will try and execute from /src. You're also doing

src.win_perf_counters.main as wmi_counters

So in turn you're looking for wmi_counters in /src/src/

Python imports are dependant on where the script is ran from. So either remove the \src in PYTHONPATH or in the import

Upvotes: 1

Related Questions