Pycharm: Struggling to properly structure project folders

Question

First of all, the doc related to my problem is here but it's not that helpful

I created a very simple project that I want to package with poetry. It contains:

A library, named mylib, that contains a single package, tools, that contains a single module, lists.py

According to Pycharm doc, the library is put under an src directory, which is marked as a source folder, and has no __init__.py file. When we do that,Pycharm extends the system path with the ./src folder

There's also a test folder, that contains a test file, to test the two functions that exists in the code.

It looks like this:

I build the project with poetry, the problem with that is that Poetry has no idea that the src directory is added in the path (actually, the problem would be the exact same with any wheel or any python interpreter started out of Pycharm), and so, namespaces are not resolvable.

Once build, the entry point is not usable. Pytest too is not able to run the tests from the command line.

Now I could do the opposite, have no src module and simply place the mylib folder at the root, and everything would work python wise, but I would not be able to use Pycharm feature of sources folder.

So my question basically is: how do you structure your packages in order to have both a valid Pycharm folder structure, and a valid python folder structure

Code is added there for reproduction:

list.py:

"""This module contains a collection of list tools"""
from typing import List

def merge_two_lists(target: List, source: List) -> List:
    """Merge two lists

        - Values that do not exist in the target list  will be added to it
        - Values that exist in the target list will not be duplicated

    Args:
        target (List): Merge target, will be returned once the merge is completed
        source (List): Merge source, will not be modified at the end of the merge

    Returns:
        The modified target list.
    """

    if source is None:
        return target

    target = target + [x for x in source if x not in target]
    return target


def substract_two_lists(target: List, source: List) -> List:
    """Substract two lists

        - Values that both exist in target and source will be removed

    Args:
        target (List): Substraction target, will be returned once the merge is completed
        source (List): Substraction source, will not be modified at the end of the merge

    Returns:
        The modified target list.
    """

    if source is None:
        return target

    target = [x for x in target if x not in source]
    return target


if __name__ == "__main__":
    print(merge_two_lists([1, 2, 3, 4], [1, 2, 3, 4, 5]))

test_merges_and_substraction.py:

import pytest

from mylib.tools.lists import merge_two_lists, substract_two_lists


def test_that_distinct_lists_can_be_merged():
    target = ['a', 'b', 'c']
    source = ['d', 'e', 'f']

    target = merge_two_lists(target, source)

    assert target == ['a', 'b', 'c', 'd', 'e', 'f']


def test_that_when_lists_are_merged_common_values_are_not_duplicated():
    target = ['a', 'b', 'c']
    source = ['a', 'd', 'b', 'e', 'f', 'c']

    target = merge_two_lists(target, source)

    assert target == ['a', 'b', 'c', 'd', 'e', 'f']


def test_that_distinct_lists_can_be_substracted():
    target = ['a', 'b', 'c']
    source = ['d', 'e', 'f']

    target = substract_two_lists(target, source)

    assert target == ['a', 'b', 'c']


def test_that_when_lists_are_substracted_common_values_are_removed():
    target = ['a', 'b', 'c']
    source = ['a', 'd', 'b', 'e', 'f', 'c']

    target = substract_two_lists(target, source)

    assert target == []


@pytest.mark.parametrize("source", [None, []])
def test_that_list_merge_is_robust_to_none(source):
    target = ['a', 'b', 'c']

    target = merge_two_lists(target, source)

    assert target == ['a', 'b', 'c']


@pytest.mark.parametrize("source", [None, []])
def test_that_list_substraction_is_robust_to_none(source):
    target = ['a', 'b', 'c']

    target = substract_two_lists(target, source)

    assert target == ['a', 'b', 'c']

pyproject.toml:

[tool.poetry]
name = "toto"
version = "0.1.0"
description = ""
authors = ["Me <me@gmail.com>"]
readme = "README.md"

packages = [
    { include = "src/mylib/tools" },
]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.scripts]
run_merge = 'mylib.tools.lists:main'

[tool.poetry.dependencies]
python = "^3.10"

[tool.poetry.dev-dependencies]
pytest = "*"

Answer 1

poetry defaults to expecting a flat layout (one without src). If you want to use a src layout then you have to tell poetry where to find each top-level package (it will discover subpackages from there on in. For example:

packages = [
    { include = "mylib", from = "src" },
]

See here for a more detailed explnation of flat vs src layouts.

You'll also need to tell pytest where to look for your top-level packages. So add the following section to pyproject.toml

[tool.pytest.ini_options]
pythonpath = [
  "src"
]