Add support for pyproject.toml configuration

Author: erral

CHANGES.md

@@ -1,5 +1,9 @@
 ## Changes
 
+## 5.2.0 (unreleased)
+
+- Feature: Added support for `pyproject.toml` as a configuration source. mxdev will now automatically look for `[tool.mxdev]` configuration in `pyproject.toml` if `mx.ini` is missing. Users can also explicitly specify it with `-c pyproject.toml`. [erral]
+
 ## 5.1.0
 
 - Feature: Git repositories can now specify multiple push URLs using multiline syntax in the `pushurl` configuration option. This enables pushing to multiple remotes (e.g., GitHub + GitLab mirrors) automatically. Syntax follows the same multiline pattern as `version-overrides` and `ignores`. Example: `pushurl =` followed by indented URLs on separate lines. When `git push` is run in the checked-out repository, it will push to all configured pushurls sequentially, mirroring Git's native multi-pushurl behavior. Backward compatible with single pushurl strings.

README.md

@@ -66,6 +66,35 @@ For more examples see the [example/](https://github.com/mxstack/mxdev/tree/main/
 
 Configuration is done in an INI file (default: `mx.ini`) using [configparser.ExtendedInterpolation](https://docs.python.org/3/library/configparser.html#configparser.ExtendedInterpolation) syntax.
 
+### pyproject.toml support
+
+Starting with version 5.2.0, mxdev supports configuration directly in `pyproject.toml`. This is useful for simple projects that don't need complex features like recursive includes or variable interpolation.
+
+#### Auto-discovery
+
+If no configuration file is explicitly specified via `-c`, mxdev will look for files in this order:
+1. `mx.ini`
+2. `pyproject.toml` (if it contains a `[tool.mxdev]` section)
+
+#### Structure
+
+The TOML configuration mirrors the INI structure but uses nesting:
+
+```toml
+[tool.mxdev.settings]
+requirements-in = "requirements.txt"
+threads = 8
+
+[tool.mxdev.packages.package1]
+url = "https://github.com/org/package1.git"
+branch = "main"
+
+[tool.mxdev.hooks.myhook]
+some-setting = "value"
+```
+
+**Note**: TOML configuration does NOT support variable interpolation (no `${settings:var}`) or the `include` directive. If you need these features, please use `mx.ini`.
+
 ### Settings Section `[settings]`
 
 The **main section** must be called `[settings]`, even if kept empty.

docs/plans/2026-05-20-pyproject-toml-support-design.md

@@ -0,0 +1,73 @@
+# Design: pyproject.toml Support for mxdev
+
+**Date**: 2026-05-20
+**Status**: Approved
+
+## Goal
+Add support for `pyproject.toml` as a configuration source for `mxdev` to align with modern Python tooling standards (PEP 518).
+
+## Brutally Honest Findings & Critique
+
+1.  **The "Standardization" Trap**: By adding TOML support, we are satisfying a superficial desire for "modernity" while losing the features that actually make `mxdev` unique: recursive includes and variable interpolation. Users will likely start in TOML because it's "cleaner," only to hit a wall the moment they need to share configurations across repositories.
+2.  **Architectural Debt**: We are implementing a "translation layer" because the core of `mxdev` is so heavily coupled to `ConfigParser` that a proper refactor to a format-agnostic state model would be a major undertaking. We are essentially faking an INI file from a TOML file.
+3.  **Fragmented Ecosystem**: We are creating two classes of `mxdev` projects: "Simple" (TOML) and "Advanced" (INI). This adds cognitive load for maintainers who now have to troubleshoot two different configuration schemas.
+4.  **Auto-discovery Hazards**: While auto-discovery is convenient, it can lead to "ghost configurations" where `mxdev` behaves unexpectedly because it found settings in a `pyproject.toml` that the user didn't explicitly point it to.
+
+## Design
+
+### 1. Configuration Structure
+Configuration will live in the `[tool.mxdev]` namespace:
+
+```toml
+[tool.mxdev.settings]
+requirements-in = "requirements.txt"
+threads = 8
+
+[tool.mxdev.packages.package1]
+url = "https://github.com/org/package1.git"
+
+[tool.mxdev.hooks.myhook]
+setting = "value"
+```
+
+### 2. Auto-Discovery Logic
+Priority order:
+1.  Explicit `-c / --configuration` flag.
+2.  `mx.ini` (default).
+3.  `pyproject.toml` (if `mx.ini` is missing and `[tool.mxdev]` is present).
+
+### 3. Implementation Details
+- **Parser**: Use `tomllib` (Python 3.11+) or `tomli` (Python 3.10).
+- **Dependency**: Add `toml` extra to `pyproject.toml`.
+- **Normalization**: The TOML loader will return a dictionary structured like `ConfigParser._sections` to ensure compatibility with the existing `Configuration` class.
+
+### 4. Constraints
+- No interpolation (`${var}`).
+- No `include` directive.
+- Static manifests only.
+
+## Testing Strategy
+
+Brutally honest: Testing this logic requires mocking the filesystem and Python version environments, which is always more fragile than it looks. We need to ensure that the "Priority Order" isn't just a suggestion, but a strictly enforced rule.
+
+### 1. Unit Tests (`tests/test_toml.py`)
+- **Structure Parsing**: Verify `tool.mxdev.settings`, `tool.mxdev.packages`, and `tool.mxdev.hooks` correctly map to the internal dictionary format.
+- **Dependency Handling**: Mock `ImportError` for both `tomllib` and `tomli` to verify the helpful error message.
+- **Normalization**: Ensure all values are converted to strings (as `ConfigParser` would do) to prevent type errors in the rest of the pipeline.
+
+### 2. Integration Tests (`tests/test_config_discovery.py`)
+- **Case: Only mx.ini**: Verify it loads `mx.ini`.
+- **Case: Only pyproject.toml**: Verify it auto-discovers and loads `pyproject.toml`.
+- **Case: Both exist**: Verify it picks `mx.ini` and ignores `pyproject.toml`.
+- **Case: Both exist + Explicit flag**: `mxdev -c pyproject.toml` must ignore `mx.ini`.
+- **Case: pyproject.toml without tool.mxdev**: Verify it does NOT load it and errors out if `mx.ini` is also missing.
+- **Case: Missing all**: Verify it retains its original error behavior (looking for `mx.ini`).
+
+### 3. Regression Tests
+- Ensure existing `tests/test_config.py` still passes without modifications, confirming we haven't broken the legacy INI path.
+
+## Success Criteria
+- `mxdev` runs successfully using only a `pyproject.toml` file.
+- `mxdev -c pyproject.toml` works as expected.
+- Clear error message when `tomli` is missing on Python 3.10.
+- Existing `mx.ini` projects are unaffected.

pyproject.toml

@@ -25,6 +25,7 @@ dependencies = ["packaging"]
 
 [project.optional-dependencies]
 mypy = []
+toml = ["tomli; python_version < '3.11'"]
 test = [
     "pytest",
     "pytest-cov",

src/mxdev/config.py

@@ -1,6 +1,7 @@
 from .including import read_with_included
 from .logging import logger
 from packaging.requirements import Requirement
+from pathlib import Path
 
 import os
 import typing
@@ -10,6 +11,7 @@
     from .hooks import Hook
 
 
+
 def to_bool(value):
     if not isinstance(value, str):
         return bool(value)
@@ -36,6 +38,71 @@ def parse_multiline_list(value: str) -> list[str]:
     return [item for item in items if item]
 
 
+class TomlSection:
+    def __init__(self, data: dict):
+        self._data = data
+
+    def items(self):
+        return self._data.items()
+
+    def get(self, key, default=None):
+        return self._data.get(key, default)
+
+    def __getitem__(self, key):
+        return self._data[key]
+
+    def keys(self):
+        return self._data.keys()
+
+
+class TomlWrapper:
+    def __init__(self, data: dict):
+        self._data = {k: TomlSection(v) for k, v in data.items()}
+        self._sections = self._data
+
+    def __getitem__(self, key):
+        return self._data[key]
+
+    def sections(self):
+        return [k for k in self._data if k != "settings"]
+
+    def __contains__(self, key):
+        return key in self._data
+
+
+def read_toml(path: str | Path) -> TomlWrapper:
+    try:
+        import tomllib
+    except ImportError:
+        try:
+            import tomli as tomllib
+        except ImportError:
+            raise ImportError(
+                "TOML support requires Python 3.11+ or the 'tomli' package. "
+                "Install it with 'pip install mxdev[toml]'."
+            )
+
+    with open(path, "rb") as f:
+        data = tomllib.load(f)
+
+    if "tool" not in data or "mxdev" not in data["tool"]:
+        return TomlWrapper({"settings": {}})
+
+    mxdev = data["tool"]["mxdev"]
+    normalized: dict[str, dict[str, str]] = {}
+    normalized["settings"] = {k: str(v) for k, v in mxdev.get("settings", {}).items()}
+
+    # Packages
+    for name, pkg_data in mxdev.get("packages", {}).items():
+        normalized[name] = {k: str(v) for k, v in pkg_data.items()}
+
+    # Hooks
+    for name, hook_data in mxdev.get("hooks", {}).items():
+        normalized[name] = {k: str(v) for k, v in hook_data.items()}
+
+    return TomlWrapper(normalized)
+
+
 class Configuration:
     settings: dict[str, str]
     overrides: dict[str, str]
@@ -50,7 +117,10 @@ def __init__(
         hooks: list["Hook"] = [],
     ) -> None:
         logger.debug("Read configuration")
-        data = read_with_included(mxini)
+        if mxini.endswith(".toml"):
+            data = read_toml(mxini)
+        else:
+            data = read_with_included(mxini)
 
         settings = self.settings = dict(data["settings"].items())
 

src/mxdev/main.py

@@ -18,6 +18,7 @@
 
 import argparse
 import logging
+import os
 import sys
 
 
@@ -29,9 +30,9 @@
 parser.add_argument(
     "-c",
     "--configuration",
-    help="configuration file in INI format",
+    help="configuration file (INI or pyproject.toml)",
     type=str,
-    default="mx.ini",
+    default=None,
 )
 parser.add_argument(
     "-n",
@@ -89,13 +90,24 @@ def main() -> None:
     logger.info("#" * 79)
     hooks = load_hooks()
     logger.info("# Load configuration")
+
+    # Configuration discovery
+    config_file = args.configuration
+    if config_file is None:
+        if os.path.exists("mx.ini"):
+            config_file = "mx.ini"
+        elif os.path.exists("pyproject.toml"):
+            config_file = "pyproject.toml"
+        else:
+            config_file = "mx.ini"
+
     override_args = {}
     if args.offline:
         override_args["offline"] = True
     if args.threads:
         override_args["threads"] = args.threads
     configuration = Configuration(
-        mxini=args.configuration,
+        mxini=config_file,
         override_args=override_args,
         hooks=hooks,
     )

tests/test_config_discovery.py

@@ -0,0 +1,98 @@
+import os
+from unittest.mock import patch
+from mxdev.main import main
+import pytest
+
+def test_discovery_mx_ini_exists(tmp_path, monkeypatch):
+    """If mx.ini exists, it should be preferred over pyproject.toml."""
+    mxini = tmp_path / "mx.ini"
+    mxini.write_text("[settings]\nrequirements-in = mx-reqs.txt", encoding="utf-8")
+    
+    pyproject = tmp_path / "pyproject.toml"
+    pyproject.write_text("[tool.mxdev.settings]\nrequirements-in = toml-reqs.txt", encoding="utf-8")
+    
+    monkeypatch.chdir(tmp_path)
+    
+    import sys
+    main_module = sys.modules["mxdev.main"]
+    
+    with (
+        patch("sys.argv", ["mxdev"]),
+        patch.object(main_module, "load_hooks", return_value=[]),
+        patch.object(main_module, "Configuration") as mock_config,
+        patch.object(main_module, "read"),
+        patch.object(main_module, "write"),
+        patch.object(main_module, "setup_logger"),
+    ):
+        main()
+        # Should pick mx.ini
+        mock_config.assert_called_once()
+        assert mock_config.call_args[1]["mxini"] == "mx.ini"
+
+def test_discovery_pyproject_fallback(tmp_path, monkeypatch):
+    """If mx.ini is missing, it should fallback to pyproject.toml."""
+    pyproject = tmp_path / "pyproject.toml"
+    pyproject.write_text("[tool.mxdev.settings]\nrequirements-in = toml-reqs.txt", encoding="utf-8")
+    
+    monkeypatch.chdir(tmp_path)
+    
+    import sys
+    main_module = sys.modules["mxdev.main"]
+    
+    with (
+        patch("sys.argv", ["mxdev"]),
+        patch.object(main_module, "load_hooks", return_value=[]),
+        patch.object(main_module, "Configuration") as mock_config,
+        patch.object(main_module, "read"),
+        patch.object(main_module, "write"),
+        patch.object(main_module, "setup_logger"),
+    ):
+        main()
+        # Should pick pyproject.toml
+        mock_config.assert_called_once()
+        assert mock_config.call_args[1]["mxini"] == "pyproject.toml"
+
+def test_discovery_explicit_flag(tmp_path, monkeypatch):
+    """Explicit flag should override discovery."""
+    mxini = tmp_path / "mx.ini"
+    mxini.write_text("[settings]", encoding="utf-8")
+    
+    custom = tmp_path / "custom.toml"
+    custom.write_text("[tool.mxdev.settings]", encoding="utf-8")
+    
+    monkeypatch.chdir(tmp_path)
+    
+    import sys
+    main_module = sys.modules["mxdev.main"]
+    
+    with (
+        patch("sys.argv", ["mxdev", "-c", "custom.toml"]),
+        patch.object(main_module, "load_hooks", return_value=[]),
+        patch.object(main_module, "Configuration") as mock_config,
+        patch.object(main_module, "read"),
+        patch.object(main_module, "write"),
+        patch.object(main_module, "setup_logger"),
+    ):
+        main()
+        # Should pick custom.toml
+        mock_config.assert_called_once()
+        assert mock_config.call_args[1]["mxini"] == "custom.toml"
+
+def test_discovery_no_config(tmp_path, monkeypatch):
+    """If no config found, fallback to mx.ini default (to trigger existing error behavior)."""
+    monkeypatch.chdir(tmp_path)
+    
+    import sys
+    main_module = sys.modules["mxdev.main"]
+    
+    with (
+        patch("sys.argv", ["mxdev"]),
+        patch.object(main_module, "load_hooks", return_value=[]),
+        patch.object(main_module, "Configuration") as mock_config,
+        patch.object(main_module, "read"),
+        patch.object(main_module, "write"),
+        patch.object(main_module, "setup_logger"),
+    ):
+        main()
+        mock_config.assert_called_once()
+        assert mock_config.call_args[1]["mxini"] == "mx.ini"

tests/test_main.py

@@ -7,7 +7,7 @@ def test_parser_defaults():
     from mxdev.main import parser
 
     args = parser.parse_args([])
-    assert args.configuration == "mx.ini"
+    assert args.configuration is None
     assert args.no_fetch is False
     assert args.fetch_only is False
     assert args.offline is False