Metadata-Version: 2.4
Name: common-python-tasks
Version: 0.0.3
Summary: Opinionated Poe the Poet tasks for Python package development.
License-Expression: MIT
License-File: LICENSE
Author: Joseph Asbury
Author-email: ci_sourcerer@yahoo.com
Requires-Python: >=3.11,<4.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Build Tools
Requires-Dist: autoflake (>=2.3.1,<3.0.0)
Requires-Dist: black (>=25.11.0,<26.0.0)
Requires-Dist: dunamai (>=1.25.0,<2.0.0)
Requires-Dist: flake8 (>=7.3.0,<8.0.0)
Requires-Dist: git-cliff (>=2.12.0,<3.0.0)
Requires-Dist: isort (>=7.0.0,<8.0.0)
Requires-Dist: jinja2 (>=3.1.6,<4.0.0)
Requires-Dist: poethepoet-tasks (>=0.3.0,<0.4.0)
Requires-Dist: pytest (>=9.0.1,<10.0.0)
Requires-Dist: pytest-cov (>=7.0.0,<8.0.0)
Requires-Dist: tomlkit (>=0.13.3,<0.14.0)
Project-URL: Homepage, http://github.com/ci-sourcerer/common-python-tasks
Project-URL: Issues, http://github.com/ci-sourcerer/common-python-tasks/issues
Project-URL: Source, http://github.com/ci-sourcerer/common-python-tasks.git
Description-Content-Type: text/markdown

# Common Python Tasks

This package is a collection of (very) opinionated [Poe the Poet](https://poethepoet.natn.io/guides/packaged_tasks.html) Python tasks for common Python development workflows.

## Quick start

### Automated setup

You can add `common-python-tasks` to a new project by using the handy automated installation script.

```shell
curl -sSL https://api.github.com/repos/ci-sourcerer/common-python-tasks/contents/scripts/add-common-python-tasks.sh | jq -r '.content' | base64 -d | TAGS_TO_INCLUDE="format lint test" sh
```

This will complete the following steps.

1. Add the latest version of `common-python-tasks` to your `pyproject.toml` dependencies
2. Configure Poe the Poet to include only the tasks with the specified tags
3. Install the package using Poetry

**Always review scripts before running them!** Even though I believe I write good software, it's best practice to verify any script you download from the Internet.

### Manual setup

1. Add `common-python-tasks` to your `pyproject.toml` and configure Poe the Poet to include the desired tasks

    ```toml
    [project]
    name = "my-awesome-project"
    version = "0.0.2"
    dependencies = [
        "common-python-tasks==0.0.2",  # Always pin to a specific version
    ]

    [tool.poe]
    include_script = "common_python_tasks:tasks(include_tags=['format', 'lint', 'test'])"  # Include or exclude tasks by tags
    ```

2. Install the package

    ```shell
    poetry install
    ```

3. Run tasks

    ```shell
    poe format  # Format your code
    poe lint    # Check code quality
    poe test    # Run tests with coverage
    ```

## Available tasks

Internal tasks are used by other tasks and are not meant to be run directly.

<!-- tasks-table -->
| Task | Description | Tags |
| - | - | - |
| `build` | Build the project and its containers (when `containers` tag is included) | packaging, containers |
| `build-image` | Build the container image for this project using the Dockerfile template (and configured extensions) | containers, build |
| `build-package` | Build the package (wheel and sdist) | packaging, build |
| `bump-version` | Bump the project version, defaulting to an inferred semantic bump from git history | packaging |
| `changelog` | Print the changelog for the current version based on git history | packaging, release |
| `clean` | Clean up temporary files and directories | clean |
| `container-shell` | Run the debug image with an interactive shell | containers, debug |
| `db-shell` | Open a psql shell to the database container | web, containers, database |
| `format` | Format code with autoflake, black, and isort | format |
| `lint` | Lint Python code with autoflake, black, isort, and flake8 | lint |
| `publish-package` | Publish the package to the PyPI server | packaging |
| `publish-github-release` | Publish or update a GitHub Release and attach built distribution assets | packaging, release |
| `push-image` | Push the Docker image to the container registry | containers, packaging, release |
| `release` | Run package release flow and publish containers when `containers` tag is included. Supports optional `RELEASE_PRE_SCRIPT` and `RELEASE_POST_SCRIPT` hooks. | packaging, release |
| `reset-db` | Reset the database by deleting the database volume | web, containers, database |
| `run-container` | Run the Docker image as a container | containers |
| `run-db-migrations` | Run database migrations | web, containers, database |
| `stack-down` | Bring down the development stack for the application | web, containers |
| `stack-up` | Bring up the development stack for the application | web, containers |
| `test` | Run the test suite with coverage | test |
<!-- end-tasks-table -->

## Docker Compose Development Stacks

Some tasks of certain tags provide Docker Compose-based development stacks for running your application with supporting services (databases, caches, etc.). Currently supports FastAPI applications with PostgreSQL.

### Configuration

#### `COMPOSE_TYPE`

Specifies the type of application stack. Currently supported:

- `fastapi` - FastAPI application with optional database, Alembic migrations

Set via environment variable:

```toml
[tool.poe.env]
COMPOSE_TYPE = "fastapi"
```

#### `COMPOSE_ADDONS`

Colon-separated list of additional services to include. Available addons:

- `db` - PostgreSQL database with Alembic migration support and Adminer web UI

Example:

```toml
[tool.poe.env]
COMPOSE_ADDONS = "db"
```

For multiple addons (future): `COMPOSE_ADDONS = "db:redis:cache"`

### Compose File Customization

The compose setup follows this precedence.

1. **Environment override** - `COMPOSE_FILE` environment variable with colon-separated paths
2. **Auto-loaded files** - Based on `COMPOSE_TYPE` and `COMPOSE_ADDONS`:
   - `compose-base.yml` - Core application service
   - `compose-{addon}.yml` - For each addon (e.g., `compose-db.yml`)
   - `compose-debug.yml` - When `--debug` flag is used
   - `compose-{addon}-debug.yml` - Debug overlays for addons
3. **Additional overlays** - `COMPOSE_OVERLAY_FILES` with colon-separated paths

You can provide local compose files or let the tasks use bundled templates.

### `fastapi`

The `fastapi` stack includes a service for your FastAPI application. It uses the standard Dockerfile included with this package.

#### Environment variables

- `API_PORT` - Port for the API server (default: `8080`)
- `SECRET_KEY` - Application secret key (auto-generated and stored in `.env` if not set)
- `ENVIRONMENT` - Environment name like `development` or `production` (default: `production`)
- `DEBUG_PORT` - Port for the Python debugger when using `--debug` (default: `5678`)
- `DB_PORT` - PostgreSQL port (default: `5432`)
- `DB_USER` - Database user (default: package name)
- `DB_BASE` - Database name (default: package name)
- `DB_PASS` - Database password (auto-generated and stored in `.env` if not set)
- `POSTGRES_VERSION` - PostgreSQL Docker image version (default: `17`)
- `ADMINER_PORT` - Adminer web UI port (default: `8081`)

## How it works

### Prerequisites

Your project must meet the following requirements.

- Use Poetry for dependency management
- Have a `pyproject.toml` file at the root
- Have a package name (automatically inferred from `project.name` in `pyproject.toml` or set via `PACKAGE_NAME` environment variable)

### Configuration precedence

Tasks that need configuration files (`pytest`, `coverage`, `flake8`, `isort`) follow this order of precedence.

1. **`pyproject.toml` sections** - `[tool.pytest]`, `[tool.coverage]`, `[tool.isort]` take priority
2. **Environment variables** - Override config paths (see [Environment Variables](#environment-variables))
3. **Local config files** - `pytest.ini`, `.coveragerc`, `.flake8`, `.isort.cfg` in project root
4. **Bundled defaults** - Sensible defaults included with this package, found in the [`src/common_python_tasks/data`](src/common_python_tasks/data) directory

You can start with zero configuration and customize as needed.

### Environment variables

#### Configuration files

The following environment variables configure the paths to configuration files.

- `PYTEST_CONFIG` specifies the path to the pytest configuration file
- `COVERAGE_RCFILE` specifies the path to the coverage configuration file  
- `FLAKE8_CONFIG` specifies the path to the flake8 configuration file
- `ISORT_CONFIG` specifies the path to the isort configuration file

#### Package/Container settings

The following environment variables configure package and container behavior.

- `PACKAGE_NAME` overrides the package name (default is from `pyproject.toml`)
- `POETRY_VERSION` overrides the Poetry version for container builds
- `DOCKERHUB_USERNAME` specifies the Docker Hub username for image tagging (default is current local user)
- `CONTAINER_REGISTRY_URL` specifies the registry URL (default is `docker.io/{username}`)
- `CONTAINER_BUILD_ARGS` provides additional Docker build arguments in `KEY=VALUE:OTHER=VALUE` format
- `CONTAINER_EXTENSION_FILES` specifies colon-delimited local extension Dockerfile paths to include in the rendered build.
- `CONTAINER_EXTENSIONS` specifies colon-delimited extension bundle names or parameterized bundle values to include in the rendered build.
- `CONTAINER_ENV` provides colon-delimited `KEY=VALUE` declarations to inject into the builder stage of the rendered Dockerfile.
- `.containerenv` can also supply the same declarations from a file in the project root. It is the fallback source when neither `container_envfile` nor `CONTAINER_ENV` is provided.
- `CONTAINER_PRUNE_KEEP` controls image pruning after builds (`-1` keep all, `0` keep latest only, `N` keep latest + `N` previous)
- `CUSTOM_IMAGE_ENTRYPOINT` specifies a custom entrypoint script name for containers
- `CONTAINER_DEPS_CONTENT` supplies inline Dockerfile instructions for a dependency image that installs artifacts into `/tmp/deps`
- `CONTAINER_DEPS_FILE` points to one or more explicit Dockerfiles to build the dependency image. It may be a colon-delimited list of file paths and is used only when `CONTAINER_DEPS_CONTENT` is unset.
- `CONTAINER_DEPS_MAPPINGS` maps copied dependency names from `/tmp/deps` into destination paths, as whitespace-separated `name:/target/path` entries. This is only used when `CONTAINER_DEPS_MOVE_SCRIPT` or `CONTAINER_DEPS_MOVE_SCRIPT_PATH` is not set.
- `CONTAINER_DEPS_MOVE_SCRIPT` supplies a raw executable script to run after `/tmp/deps` is copied into the image. The script is written to `/tmp/container-deps-move-script` and executed with its own shebang.
- `CONTAINER_DEPS_MOVE_SCRIPT_PATH` supplies a host path to a script file to run after `/tmp/deps` is copied into the image. This path takes precedence over `CONTAINER_DEPS_MOVE_SCRIPT` when both are set.
- `GITHUB_RELEASE_ASSETS` colon-separated list of file paths or glob patterns to attach to the GitHub Release (default: `dist/*`)
- `SKIP_GITHUB_RELEASE` truthy value to skip GitHub Release publication.
- `GITHUB_TOKEN` or `GH_TOKEN` GitHub authentication token used to publish releases and upload assets.
- `GITHUB_REPOSITORY` optional override for the repository slug used when publishing a GitHub Release.
- `GITHUB_API_URL` and `GITHUB_SERVER_URL` configure the GitHub API host for GitHub Enterprise environments.
- `GITHUB_RELEASE_TAG` optional tag name to publish for the GitHub Release.
- `GITHUB_RELEASE_NAME` optional release title to use for the GitHub Release.
- `GITHUB_RELEASE_BODY` optional release body text to use for the GitHub Release.
- `RELEASE_PRE_SCRIPT` optional shell command to run before the release steps.
- `RELEASE_POST_SCRIPT` optional shell command to run after the release completes.
- Hook commands receive the following env vars: `RELEASE_TAG`, `RELEASE_VERSION`, `RELEASE_STAGE`, `RELEASE_COMPONENT`, and `RELEASE_DRY_RUN`.

#### Docker Compose settings

The following environment variables configure Docker Compose stacks (when using the `web` tag).

- `COMPOSE_TYPE` specifies the type of application stack (e.g., `fastapi`)
- `COMPOSE_ADDONS` colon-separated list of services to include (e.g., `db` for database)
- `COMPOSE_FILE` overrides all compose files with colon-separated paths
- `COMPOSE_OVERLAY_FILES` additional compose files to merge (colon-separated paths)
- `API_PORT` port for the API server (default: `8080`)
- `SECRET_KEY` application secret key (auto-generated if not set)
- `ENVIRONMENT` environment name (default: `production`)
- `DEBUG_PORT` port for Python debugger in debug mode (default: `5678`)
- `DB_PORT` PostgreSQL port (default: `5432`)
- `DB_USER` database user (default: package name)
- `DB_BASE` database name (default: package name)
- `DB_PASS` database password (auto-generated if not set)
- `POSTGRES_VERSION` PostgreSQL version (default: `17`)
- `ADMINER_PORT` Adminer web UI port (default: `8081`)

#### Debugging

The following environment variable enables debugging output.

- `COMMON_PYTHON_TASKS_LOG_LEVEL` should be set to `DEBUG` to see detailed configuration resolution

### Usage examples

You can include or exclude tasks by tags in your `pyproject.toml`

#### Minimal setup

```toml
[project]
name = "simple-cli-tool"
version = "0.0.1"
dependencies = ["common-python-tasks==0.0.1"]

[tool.poe]
include_script = "common_python_tasks:tasks(include_tags=['format', 'lint'])"
```

Available tasks: `format`, `lint`.

#### Container-based project

```toml
[project]
name = "containerized-app"
version = "0.0.1"
dependencies = ["common-python-tasks==0.0.1"]

[tool.poe]
include_script = "common_python_tasks:tasks(include_tags=['format', 'lint', 'test', 'containers'])"

[tool.poe.env]
DOCKERHUB_USERNAME = "myusername"
PACKAGE_NAME = "containerized-app"
```

Available tasks: All tasks including `build-image` and `push-image`.

#### Custom pytest configuration

```toml
[project]
name = "custom-test-setup"
dependencies = ["common-python-tasks==0.0.1"]
dynamic = ["version"]

[tool.poe]
include_script = "common_python_tasks:tasks(include_tags=['test'])"

[tool.pytest.ini_options]
testpaths = ["tests", "integration"]
addopts = "-ra"
```

The `test` task will automatically use your `[tool.pytest.ini_options]` configuration.

## Troubleshooting

### Tasks not showing up with `poe --help`

Check your `[tool.poe]` configuration in `pyproject.toml`. Make sure you're using `include_script`, not `includes`.

```toml
# Correct
[tool.poe]
include_script = "common_python_tasks:tasks(exclude_tags=['internal'])"

# Incorrect
[tool.poe]
includes = "common_python_tasks:tasks"
```

### Version bump fails with "no changes since last tag"

This is expected behavior. The `bump-version` task requires commits between the last tag and HEAD. You can resolve this in one of the following ways.

- Make changes and commit them first
- Delete the old tag (for example, `git tag -d v0.0.1`). This is not recommended. Versions should be immutable, and if you need to fix something, you should create a new patch version instead. Rarely do you want to pass off new code as an old version

### Config files not being used

Check the configuration precedence (see [How it works](#how-it-works)). Use debug logging to see which config is selected.

### GitHub Release assets not uploading

If your release task does not attach assets, confirm `dist/` contains the built wheels and sdists. You can override the default asset selection using `GITHUB_RELEASE_ASSETS`, for example:

```shell
GITHUB_RELEASE_ASSETS="dist/*.whl:dist/*.tar.gz" poe publish-github-release
```

```shell
COMMON_PYTHON_TASKS_LOG_LEVEL=DEBUG poe test
```

### Container build fails with "unable to find package"

Make sure your `pyproject.toml` contains the following.

- A correct package name in `[project]`
- A package location defined with this configuration: `[tool.poetry] packages = [{ include = "your_package", from = "src" }]`

### Stack fails to start or services won't connect

If `stack-up` builds successfully but services can't connect:

- Check that required environment variables are set (`COMPOSE_TYPE` at minimum)
- Verify ports aren't already in use (defaults: 8080 for API, 5432 for database, 8081 for Adminer)
- Check Docker daemon is running: `docker info`
- View service logs: `docker-compose logs` in your project directory

### Database migrations fail

If `run-db-migrations` fails:

- Ensure the `db` addon is included: `COMPOSE_ADDONS=db`
- Check that your project has Alembic configured with migrations in the expected location
- Verify database credentials in `.env` match your Alembic configuration
- Manually inspect the database: `poe db-shell`

### Secrets not being generated

If `SECRET_KEY` or `DB_PASS` aren't auto-generated:

- Ensure `.env` file is writable in your project root
- Check file permissions: `ls -la .env`
- Generate manually: `python -c "import secrets; print(secrets.token_hex(32))"`

## Design choices

### Dockerfile (see [src/common_python_tasks/data/Dockerfile](src/common_python_tasks/data/Dockerfile))

The standard Python Dockerfile incorporates several intentional design choices.

- Multi-stage build: The build stage installs Poetry and builds a wheel while the runtime stage installs only the wheel to keep the final image slim and reproducible
- Pip and Poetry cache mounts speed up iterative builds without bloating the final image
- Explicit inputs through build args (`PYTHON_VERSION`, `POETRY_VERSION`, `PACKAGE_NAME`, `AUTHORS`, `GIT_COMMIT`, `CUSTOM_ENTRYPOINT`) make image metadata and behavior predictable and auditable
- Optional debug stage exports and installs the `debug` dependency group only when present without failing otherwise and is not part of the default final image
- Stable package path creates symlinks to the installed package so entrypoints and consumers have a consistent `/pkg` and `/_$PACKAGE_NAME` path regardless of wheel layout, which ensures that the package can be reliably imported and executed from a known location, and allows for the less common use case of reading files directly from the package path
- Safe entrypoint selection means the default entrypoint resolves the console script matching the package name while `CUSTOM_ENTRYPOINT` allows overriding at build time while keeping runtime behavior predictable
- Minimal final image uses the slim Python base by default, cleans wheel artifacts and caches, and sets `runtime` as the explicit final target so the debug stage is opt-in

## Notes

- This project dogfoods itself - it uses `common-python-tasks` for its own development
- `RELEASE_PRE_SCRIPT` and `RELEASE_POST_SCRIPT` hooks may not be necessary for most users, as this package promotes the use of `poetry-dynamic-versioning` and `git-cliff` to automate versioning and changelog generation based on git history. However, one advanced use case for the `RELEASE_PRE_SCRIPT` hook is to edit a file before release, such as a README that references the current stable version number. This allows you to keep the README up to date with the latest version without hardcoding it.
- Contributions welcome! Open an issue/discussion to discuss changes before submitting a PR. I do not claim to have all the answers, and you can help determine the future of low-code solutions for Python. I am very interested in your feedback as I don't want to work in a vacuum
- Alpha status: Expect breaking changes between minor versions until 1.0.0

