Implicit namespace __path__ not merged for multiple VCS/editable dependencies sharing the same namespace (develop=true broken, develop=flase works, pinned works)
kind/bugstatus/triage
### Description
## Summary
When multiple VCS/editable packages share the same **implicit namespace package**
(PEP 420), Poetry installs successfully without any errors. However, at
**application startup**, when Python begins resolving imports, only **one**
directory is registered in the namespace `__path__`. All other packages
contributing to the same namespace become invisible, resulting in
`ModuleNotFoundError` at runtime.
- `poetry install` → **succeeds with no errors**
- Application startup / import resolution → **ModuleNotFoundError**
This is **broken** with `develop = true` (editable/VCS installs).
This **works correctly** with pinned installs.
Affected versions: Poetry 1.8.2 with Python 3.10.9
---
## Background — What Are Implicit Namespace Packages?
PEP 420 allows multiple separate repositories/distributions to contribute to the
same Python namespace without a central `__init__.py`.
Python resolves them by merging all contributing directories into the namespace's `__path__`.
| Package repo | Contributes to |
|---|---|
| `myorg.service.one/myorg/service/one/` | `myorg` namespace |
| `myorg.service.two/myorg/service/two/` | `myorg` namespace |
| `myorg.base.storage/myorg/base/` | `myorg` namespace |
| `myorg.base.common/myorg/base/common/` | `myorg` namespace |
For this to work, `myorg.__path__` must contain **all four** directories.
Poetry only registers **one** when packages are installed as editable.
---
## Minimal Reproducible Example
### Package Structure
```
myorg.service.one/
└── myorg/
└── service/
└── one/
└── __init__.py
myorg.service.two/
└── myorg/
└── service/
└── two/
└── __init__.py
myorg.base.storage/
└── myorg/
└── base/
└── storage/
└── __init__.py ← lives in myorg.base sub-namespace
myorg.base.common/
└── myorg/
└── base/
└── common/
└── __init__.py ← same sub-namespace, different repo
```
### pyproject.toml
```toml
[tool.poetry]
name = "myorg-app"
version = "0.1.0"
description = ""
authors = []
[tool.poetry.dependencies]
python = "^3.10"
myorg-service-one = { git = "https://github.com/org/myorg.service.one.git", develop = true }
myorg-service-two = { git = "https://github.com/org/myorg.service.two.git", develop = true }
myorg-base-storage = { git = "https://github.com/org/myorg.base.storage.git", develop = true }
myorg-base-common = { git = "https://github.com/org/myorg.base.common.git", develop = true }
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
```
### Steps to Reproduce
```bash
# Step 1: Install — succeeds with NO errors
poetry install
# Installing myorg-service-one
# Installing myorg-service-two
# Installing myorg-base-storage
# Installing myorg-base-common
# Step 2: Start the application — fails at import time
python app.py
# OR
gunicorn myorg.service.one.app:create_app
# Step 3: Error appears during import resolution at startup:
# ModuleNotFoundError: No module named 'myorg.base.storage'
# Step 4: Diagnose — check what __path__ actually contains
python -c "import myorg; print(myorg.__path__)"
# Only ONE path instead of four
# Step 5: Switch to pinned — works fine
# myorg-service-one = "1.0.0"
# myorg-service-two = "1.0.0"
# myorg-base-storage = "1.0.0"
# myorg-base-common = "1.0.0"
poetry install
python app.py # starts correctly
```
---
## Error at Application Startup
```
Traceback (most recent call last):
File "app.py", line 3, in <module>
from myorg.base.storage import StorageClient
File ".venv/src/myorg.base.common/myorg/base/__init__.py"
ModuleNotFoundError: No module named 'myorg.base.storage'
```
The error does **not** appear during `poetry install`.
It only appears when Python resolves imports at application startup.
---
## Expected Behaviour
All namespace paths should be merged — same as what pinned installs produce:
```python
# myorg.__path__ should contain ALL contributing directories:
myorg.__path__ = [
'.venv/src/myorg.service.one/myorg',
'.venv/src/myorg.service.two/myorg',
'.venv/src/myorg.base.storage/myorg',
'.venv/src/myorg.base.common/myorg',
]
# Sub-namespace should also be fully merged:
myorg.base.__path__ = [
'.venv/src/myorg.base.common/myorg/base',
'.venv/src/myorg.base.storage/myorg/base',
]
# All imports should succeed:
import myorg.service.one
import myorg.service.two
import myorg.base.storage
import myorg.base.common
```
---
## Actual Behaviour
`poetry install` succeeds, but at runtime only the last package
installed by Poetry wins the namespace:
```python
# myorg.__path__ only has ONE directory:
myorg.__path__ = ['.venv/src/myorg.service.two/myorg'] # only ONE
# Sub-namespace only has ONE directory:
myorg.base.__path__ = ['.venv/src/myorg.base.common/myorg/base'] # missing storage
# Imports fail at application startup:
import myorg.service.one # ModuleNotFoundError
import myorg.base.storage # ModuleNotFoundError
```
---
## Confirmed Behaviour Across Install Methods
| Install method | `poetry install` | App startup imports | `__path__` merged |
|---|---|---|---|
| Pinned (`= "1.0.0"`) | Success | Works | All paths |
| Path editable (`develop = true`) | Success | ModuleNotFoundError | Only one path |
| VCS editable (`git + develop = true`) | Success | ModuleNotFoundError | Only one path |
**Key observation:** `poetry install` always succeeds.
The failure only occurs at Python import resolution time during app startup.
---
## Root Cause Analysis
### Why install succeeds but runtime fails:
```
poetry install (develop = true)
↓
Poetry clones each repo into:
.venv/src/myorg.service.one/
.venv/src/myorg.service.two/
.venv/src/myorg.base.storage/
.venv/src/myorg.base.common/
↓
Each package gets its OWN isolated .pth file:
myorg-service-one.pth → .venv/src/myorg.service.one
myorg-service-two.pth → .venv/src/myorg.service.two
myorg-base-storage.pth → .venv/src/myorg.base.storage
myorg-base-common.pth → .venv/src/myorg.base.common
↓
Install reports success — all packages are on disk
↓
Application starts → Python processes .pth files
↓
Python finds "myorg" namespace in FIRST matching .pth and STOPS
↓
Only ONE contributor registered in myorg.__path__
All others invisible at import time
↓
ModuleNotFoundError at app startup
```
### How pinned installs avoid this (correct):
```
poetry install (pinned)
↓
pip installs into .venv/lib/python3.x/site-packages/
each package has proper .dist-info metadata
↓
pkg_resources / importlib.metadata reads ALL .dist-info at startup
↓
ALL namespace contributors merged into myorg.__path__
↓
All imports succeed at app startup
```
### Why this is a Poetry responsibility:
Poetry knows at install time that multiple packages share the same namespace.
It should generate a **single combined .pth file** that registers all
contributors together — similar to how `zc.buildout` (Plone/Zope) hardcodes
all namespace paths into the generated interpreter script:
```python
# What zc.buildout generates (correct approach):
sys.path[0:0] = [
'/apps/buildout/src/myorg.service.one',
'/apps/buildout/src/myorg.service.two',
'/apps/buildout/src/myorg.base.storage',
'/apps/buildout/src/myorg.base.common',
]
```
Instead, Poetry generates isolated .pth files per package:
```
# What Poetry currently generates (broken at runtime):
myorg-service-one.pth → .venv/src/myorg.service.one
myorg-service-two.pth → .venv/src/myorg.service.two
myorg-base-storage.pth → .venv/src/myorg.base.storage
myorg-base-common.pth → .venv/src/myorg.base.common
# Python picks first match → only one contributor registered
```
---
## Impact
This issue affects any project that:
- Uses implicit namespace packages (PEP 420) — no `__init__.py` in namespace dir
- Has multiple packages contributing to the same namespace
- Installs those packages as editable (`develop = true`) or VCS dependencies
- Common in monorepo setups, plugin architectures, and large enterprise
projects split across many repositories under a shared namespace
---
## Related Issues
- #3636 — Namespace package in development mode throws ModuleNotFoundError (open since 2021)
- #9684 — Virtual environment breaks shared namespaces (open since 2024)
- #10211 — Editable installs fail with nested namespace packages (open since 2025)
- #5513 — Namespace packages + VCS separated packages (open since 2022)
- #1199 — Support implicit namespace PEP 420 (open since 2019)
---
## Proposed Fix
Poetry should detect at install time when multiple editable/VCS packages share
the same namespace and generate a **single combined .pth file** that registers
all contributors:
```
# Proposed: one combined .pth instead of isolated ones
.venv/lib/python3.10/site-packages/myorg-namespace.pth:
.venv/src/myorg.service.one
.venv/src/myorg.service.two
.venv/src/myorg.base.storage
.venv/src/myorg.base.common
```
This would ensure Python sees all namespace contributors at interpreter startup,
before any import occurs — matching the behaviour of pinned installs and
eliminating the silent install-success / runtime-failure gap.
### Workarounds
Manually patch all namespace `__path__`s at application startup
(in entry point):
```python
import os
import importlib
VENV_SRC = os.path.join(os.path.dirname(__file__), ".venv", "src")
def patch_all_namespace_paths(src_root):
namespace_paths = {}
for pkg_dir in sorted(os.listdir(src_root)):
pkg_root = os.path.join(src_root, pkg_dir)
if not os.path.isdir(pkg_root):
continue
for dirpath, dirnames, _ in os.walk(pkg_root):
dirnames[:] = [d for d in dirnames
if not d.startswith(('.', '__pycache__'))]
rel = os.path.relpath(dirpath, pkg_root)
if rel == '.':
continue
namespace_paths.setdefault(
rel.replace(os.sep, '.'), []
).append(dirpath)
def _patch(dotted, dirs):
try:
mod = importlib.import_module(dotted)
if hasattr(mod, '__path__'):
mod.__path__.extend(
d for d in dirs
if os.path.isdir(d) and d not in mod.__path__
)
except Exception:
pass
list(map(lambda item: _patch(*item), namespace_paths.items()))
patch_all_namespace_paths(VENV_SRC)
```
This should **not** be necessary — Poetry should handle namespace merging for
editable installs the same way it does for pinned installs.
Alternatively, switching all dependencies from `develop = true` to pinned
versions resolves the issue but removes the ability to develop locally.
### Poetry Installation Method
install.python-poetry.org
### Operating System
ubuntu 22
### Poetry Version
1.8.2
### Poetry Configuration
```bash session
poetry config --list
cache-dir = "/home/.cache/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs" # /home/.cache/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true
```
### Python Sysconfig
<details>
<summary>sysconfig.log</summary>
<!-- Please leave one blank line below for enabling the code block rendering. -->
```
Paste the output of 'python -m sysconfig', over this line.
```
</details>
### Example pyproject.toml
```TOML
[tool.poetry]
name = "myorg-app"
version = "0.1.0"
description = "Reproduces namespace package merging issue"
authors = ["your name <your@email.com>"]
[tool.poetry.dependencies]
python = "^3.10"
# Multiple packages sharing "myorg" namespace — all editable
myorg-service-one = { git = "https://github.com/org/myorg.service.one.git", develop = true }
myorg-service-two = { git = "https://github.com/org/myorg.service.two.git", develop = true }
myorg-base-storage = { git = "https://github.com/org/myorg.base.storage.git", develop = true }
myorg-base-common = { git = "https://github.com/org/myorg.base.common.git", develop = true }
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
```
### Poetry Runtime Logs
> **Note:** I am unable to share actual company logs due to confidentiality.
> Below is a sanitized reproduction of the error using generic package names.
### Command
```bash
poetry run gunicorn myorg.service.one.wsgi:app -vvv
```
### Error at Application Startup
```
Traceback (most recent call last):
File ".venv/lib/python3.10/site-packages/gunicorn/util.py", line 371, in load_wsgi
mod = import_module(module)
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, anchor)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File ".venv/src/myorg.service.one/myorg/service/one/app/__init__.py", line 3, in <module>
from myorg.base.storage import StorageClient
File ".venv/src/myorg.base.common/myorg/base/__init__.py"
ModuleNotFoundError: No module named 'myorg.base.storage'
```
---
0 条评论