Metadata-Version: 2.1
Name: gitlabci-local
Version: 10.0.1
Summary: Launch .gitlab-ci.yml jobs locally
Home-page: https://gitlab.com/RadianDevCore/tools/gcil
Author: Adrian DC
Author-email: radian.dc@gmail.com
License: Apache License 2.0
Project-URL: Bug Reports, https://gitlab.com/RadianDevCore/tools/gcil/-/issues
Project-URL: Changelog, https://gitlab.com/RadianDevCore/tools/gcil/blob/main/CHANGELOG.md
Project-URL: Documentation, https://gitlab.com/RadianDevCore/tools/gcil#gcil
Project-URL: Source, https://gitlab.com/RadianDevCore/tools/gcil
Project-URL: Statistics, https://pypistats.org/packages/gitlabci-local
Keywords: gitlab gitlab-ci local gcil gitlabci-local pipeline
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development
Classifier: Topic :: Utilities
Requires-Python: >=3,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,!=3.7.*
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: colored>=1.4.2
Requires-Dist: docker>=5.0.3
Requires-Dist: prompt-toolkit<4.0,>=2.0
Requires-Dist: python-dotenv>=0.10.5
Requires-Dist: PyYAML>=6.0
Requires-Dist: questionary>=2.0.1
Requires-Dist: requests>=2.26.0
Requires-Dist: setuptools>=45.1.0
Requires-Dist: update_checker>=0.18.0

# gcil

<!-- markdownlint-disable no-inline-html -->

[![Build](https://gitlab.com/RadianDevCore/tools/gcil/badges/main/pipeline.svg)](https://gitlab.com/RadianDevCore/tools/gcil/-/commits/main/)
[![Bugs](https://sonarcloud.io/api/project_badges/measure?project=AdrianDC_gitlabci-local&metric=bugs)](https://sonarcloud.io/dashboard?id=AdrianDC_gitlabci-local)
[![Code Smells](https://sonarcloud.io/api/project_badges/measure?project=AdrianDC_gitlabci-local&metric=code_smells)](https://sonarcloud.io/dashboard?id=AdrianDC_gitlabci-local)
[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=AdrianDC_gitlabci-local&metric=coverage)](https://sonarcloud.io/dashboard?id=AdrianDC_gitlabci-local)
[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=AdrianDC_gitlabci-local&metric=ncloc)](https://sonarcloud.io/dashboard?id=AdrianDC_gitlabci-local)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=AdrianDC_gitlabci-local&metric=alert_status)](https://sonarcloud.io/dashboard?id=AdrianDC_gitlabci-local)
[![Commitizen friendly](https://img.shields.io/badge/commitizen-friendly-brightgreen.svg)](https://commitizen-tools.github.io/commitizen/)

Launch .gitlab-ci.yml jobs locally, wrapped inside the specific images,  
with inplace project volume mounts and adaptive user selections.

---

## Purpose

The main purpose of this project is to unify and enhance reliability  
of builds, tests or releases running on GitLab CI in a similar local context,  
by providing the simplicity of an interactive and automated terminal tool  
and avoiding code duplication (Makefile, Shell scripts, docker run, ...).

Rather than creating yet another standard, the .gitlab-ci.yml specification  
is the common and unique interface between GitLab CI and `gcil`.

---

## Preview

![preview.svg](https://gitlab.com/RadianDevCore/tools/gcil/raw/10.0.1/docs/preview.svg)

---

## Examples

<!-- prettier-ignore-start -->

```bash
gcil                   # Launch the jobs choices interactive menu
gcil -p                # Launch the jobs pipeline automatically
gcil -l                # Launch the job selection interactive menu
gcil 'Dev'             # Launch jobs where the name contains a given string
gcil 'Job 1' --debug   # Hold a finishing specific job for debugging
gcil 'Job 1' --bash    # Prepare a bash environment for a specific job
gitlabci-local         # Shortcut alias to gcil
```

<!-- prettier-ignore-end -->

---

## Usage

<!-- prettier-ignore-start -->
<!-- readme-help-start -->

```yaml
usage: gcil [-h] [--version] [--no-color] [--update-check] [--settings] [--set GROUP KEY VAL] [-p] [-q]
            [-c CONFIGURATION] [-B] [-A] [-C COMMANDS] [-n NETWORK] [-e ENV] [-E ENGINE] [-H] [--notify] [-r] [-S]
            [--ssh [SSH_USER]] [-v VOLUME] [-w WORKDIR] [--bash | --debug] [--display] [--shell SHELL] [--all]
            [--defaults] [-f] [-i] [-m] [--no-console] [--no-git-safeties] [--no-script-fail] [-R] [--no-verbose]
            [--scripts] [-t TAGS] [-d | -s | -l | --pull | --rmi] [--]
            [names ...]

gcil: Launch .gitlab-ci.yml jobs locally (aliases: gitlabci-local)

internal arguments:
  -h, --help           # Show this help message
  --version            # Show the current version
  --no-color           # Disable colors outputs with 'NO_COLOR=1'
                       # (or default settings: [themes] > no_color)
  --update-check       # Check for newer package updates
  --settings           # Show the current settings path and contents
  --set GROUP KEY VAL  # Set settings specific 'VAL' value to [GROUP] > KEY
                       # or unset by using 'UNSET' as 'VAL'

pipeline arguments:
  -p, --pipeline       # Automatically run pipeline stages rather than jobs
  -q, --quiet          # Hide jobs execution context
  -c CONFIGURATION     # Path to the .gitlab-ci.yml configuration file or folder
  -B, --no-before      # Disable before_script executions
  -A, --no-after       # Disable after_script executions
  -C COMMANDS          # Run specific commands instead of "scripts" commands
  -n NETWORK           # Configure the network mode used (or define CI_LOCAL_NETWORK)
                       # Choices: bridge, host, none. Default: bridge
  -e ENV               # Define VARIABLE=value, pass VARIABLE or ENV file
  -E ENGINE            # Force a specific engine (or define CI_LOCAL_ENGINE)
                       # Default list: auto,docker,podman
  -H, --host           # Run all jobs on the host rather than containers
  --notify             # Enable host notifications of pipeline and jobs results
  -r, --real-paths     # Mount real folder paths in the container (Linux / macOS only)
  -S, --sockets        # Mount engine sockets for nested containers
                       # (Enabled by default with services: docker:*dind)
  --ssh [SSH_USER]     # Bind SSH credentials to a container's user
  -v VOLUME            # Mount VOLUME or HOST:TARGET in containers
  -w WORKDIR           # Override the container's working path

debugging arguments:
  --bash               # Prepare runners for manual bash purposes
  --debug              # Keep runners active for debugging purposes
  --display            # Enable host DISPLAY forwarding features
  --shell SHELL        # Configure the default bash/debug shell entrypoint

jobs arguments:
  --all                # Enable all jobs by default in selections
  --defaults           # Use default variables for .local:configurations
  -f, --force          # Force the action (use with --pull)
  -i, --ignore-case    # Ignore case when searching for names
  -m, --manual         # Allow manual jobs to be used
  --no-console         # Disable console launch in bash/debug modes
                       # (or default settings: [runner] > no_console)
  --no-git-safeties    # Disable automated Git safeties configuration
                       # (or default settings: [runner] > no_git_safeties)
  --no-script-fail     # Fail on missing 'script' nodes of jobs
                       # (or default settings: [runner] > no_script_fail)
  -R, --no-regex       # Disable regex search of names
  --no-verbose         # Hide jobs verbose outputs
  --scripts            # Dump parsed jobs entrypoint scripts
  -t TAGS              # Handle listed tags as manual jobs
                       # Default list: deploy,local,publish

features arguments:
  -d, --dump           # Dump parsed .gitlab-ci.yml configuration
  -s, --select         # Force jobs selection from enumerated names
  -l, --list           # Select one job to run (implies --manual)
  --pull               # Pull container images from all jobs
  --rmi                # Delete container images from all jobs

positional arguments:
  --                   # Positional arguments separator (recommended)
  names                # Names of specific jobs (or stages with --pipeline)
                       # Regex names is supported unless --no-regex is used
```

<!-- readme-help-stop -->
<!-- prettier-ignore-end -->

---

## User configurations with '.local:configurations'

`gcil` implements support for specific user configurations  
allowing simple and interactive local pipeline configurations.

Supported user configurations include `boolean`, `choice`, `input`, `yaml` and `json`.

Examples for each of these can be found in the `configurations` unit tests: [tests/configurations](https://gitlab.com/RadianDevCore/tools/gcil/blob/main/tests/configurations/.gitlab-ci.yml)

---

## Additional features in '.local'

`gcil` implements further support of most parameters  
inside a `.local` node to ease default parameters definitions.

Supported local values for a `.local` node are:

<!-- prettier-ignore-start -->

```yaml
.local:
  after: bool            # Enable or disable `after_script` executions (see `-A`)
  all: bool              # Enable all jobs by default in selections (see `--all`)
  bash: bool             # Prepare runners for manual bash purposes (see `--bash`)
  before: bool           # Enable or disable `before_script` executions (see `-B`)
  configurations: dict   # See [User configurations with '.local:configurations'](#user-configurations-with-localconfigurations)
  debug: bool            # Keep runners active for debugging purposes (see `--debug`)
  defaults: bool         # Use default variables for `.local:configurations` (see `--defaults`)
  display: bool          # Enable host `DISPLAY` forwarding features (see `--display`)
  engine: str            # Force a specific engine (see `-E`)
  env: list[str]         # Define `VARIABLE=value`, pass `VARIABLE` or `ENV` file (see `-e`)
  image: dict|str        # Override container image's `name` and/or `entrypoint
  include: dict          # Map `include: project:` names to local paths
  manual: bool           # Allow manual jobs to be used (see `--manual`)
  names: list[str]       # Names of specific jobs (or stages with `--pipeline`) (see `names`)
  network: str           # Configure the network mode used (see `--network`)
  no_console: bool       # Disable console launch in bash/debug modes (see `--no-console`)
  no_regex: bool         # Disable regex search of names (see `--no-regex`)
  no_verbose: bool       # Hide jobs verbose outputs (see `--no-verbose`)
  notify: bool           # Enable host notifications of pipeline and jobs results (see `--notify`)
  pipeline: bool         # Automatically run pipeline stages rather than jobs (see `-p`)
  quiet: bool            # Hide jobs execution context (see `-q`)
  real_paths: bool       # Mount real folder paths in the container (see `-r`)
  shell: str             # Configure the default bash/debug shell entrypoint (see `--shell`)
  sockets: bool          # Mount engine sockets for nested containers (see `-S`)
  ssh: bool|str          # Bind SSH credentials to a container's user (see `--ssh`)
  tags: list[str]        # Handle listed tags as manual jobs (see `--tags`)
  variables: dict[str]   # Define `KEY: VALUE` variables for local jobs
  version: str           # Define a minimum version for `gcil` recommended for this file
  volumes: bool          # Mount `VOLUME` or `HOST:TARGET` in containers (see `-v`)
  workdir: bool          # Override the container's working path (see `-w`)
```

<!-- prettier-ignore-end -->

Examples for each of these can be found in the `local` unit tests: [tests/local](https://gitlab.com/RadianDevCore/tools/gcil/blob/main/tests/local/.gitlab-ci.yml)
and [tests/includes](https://gitlab.com/RadianDevCore/tools/gcil/blob/main/tests/includes/.gitlab-ci.project.yml)

---

## Job execution in native context

`gcil` runs every jobs in the specified container image.

For specific local purposes where the native host context is wished,  
where the host tools, folders or credentials are required,  
`image: local` can be used to run the scripts natively.

For specific purposes, the `image: local:quiet` variant  
can be used to enable the `quiet` option for specific jobs.

The `image: local:silent` variant extends the `quiet` option  
by also disabling the verbose script `set -x` line entry.

An example usage can be found in the local `Changelog` job: [.gitlab-ci.yml](https://gitlab.com/RadianDevCore/tools/gcil/blob/main/.gitlab-ci.yml)

---

## Environment variables

<details>
  <summary>Expand environment variables documentation</summary>
  <div style="padding-left: 30px">
    <br />

`gcil` uses the variables defined in .gitlab-ci.yml,  
parses the simple environment variables file named `.env`  
and the configurations selected through `.local:configurations`.

If specific environment variables are to be used in the job's container:

- `-e VARIABLE`: pass an environment variable
- `-e VARIABLE=value`: set a variable to a specific value
- `-e ENVIRONMENT_FILE`: parse a file as default variables

For example, `-e TERM=ansi` may enable colored terminal outputs.

The variable `CI_LOCAL` is automatically defined to `true` by `gcil`  
to allow specific conditions for local purposes in jobs' scripts.

The variable `CI_LOCAL_HOST` is automatically defined to `true` by `gcil`  
if running the job natively on the host (for example wiht `--host`)

The following variables are also defined by `gcil`:

- `CI_COMMIT_REF_NAME`: The branch or tag name for which project is built (GitLab CI)
- `CI_COMMIT_REF_SLUG`: CI_COMMIT_REF_NAME in lowercase, shortened to 63 bytes,  
  and with everything except 0-9 and a-z replaced with -. No leading / trailing - (GitLab CI)
- `CI_COMMIT_SHA`: The commit revision for which project is built (GitLab CI)
- `CI_COMMIT_SHORT_SHA`: The first eight characters of CI_COMMIT_SHA (GitLab CI)
- `CI_PROJECT_NAME`: The name of the directory for the project (GitLab CI)
- `CI_PROJECT_NAMESPACE`: The project namespace (username or group name) of the job (GitLab CI)
- `CI_LOCAL_USER_HOST_GID`: The host user's group ID value
- `CI_LOCAL_USER_HOST_UID`: The host user's user ID value
- `CI_LOCAL_USER_HOST_USERNAME`: The host user's username value

  </div>

</details>

---

## Supported container engines

`gcil` currently supports these container engines:

- **Docker :** <https://docs.docker.com/get-docker/> (root daemon, as user or sudoer)
- **Podman :** <https://podman.io/getting-started/> (rootless or root CLI)

---

## Supported systems

<details>
  <summary>Supported Linux systems</summary>
  <div style="padding-left: 30px">

|        Engines        | Linux Mint, Ubuntu | CentOS | Others |
| :-------------------: | :----------------: | :----: | :----: |
|    Native (shell)     |       **✓**        | **✓**  | **?**  |
| Docker (as&nbsp;user) |       **✓**        | **✓**  | **?**  |
| Docker (as&nbsp;root) |       **✓**        | **✓**  | **?**  |
| Podman (as&nbsp;user) |       **~**        | **~**  | **?**  |
| Podman (as&nbsp;root) |       **✓**        | **✓**  | **?**  |

  </div>
</details>

<details>
  <summary>Supported macOS systems</summary>
  <div style="padding-left: 30px">

|        Engines        | macOS (10.14, 10.15, 11.0, ...) |
| :-------------------: | :-----------------------------: |
|    Native (shell)     |              **✓**              |
| Docker (as&nbsp;user) |              **?**              |

  </div>
</details>

<details>
  <summary>Supported Windows systems</summary>
  <div style="padding-left: 30px">

|         Engines         | Windows 10 (1909, 2004, 20H2) | Others |
| :---------------------: | :---------------------------: | :----: |
| Native (Command Prompt) |             **~**             | **?**  |
|    Native (Git Bash)    |             **✓**             | **?**  |
| Docker (Hyper&#8209;V)  |             **✓**             | **?**  |
|   Docker (WSL&nbsp;2)   |             **✓**             | **?**  |

  </div>
</details>

<details>
  <summary>Supported Android systems</summary>
  <div style="padding-left: 30px">

|     Engines     | Android (7.0, 7.1, 8.0, 8.1, 9.0, 10, 11, ...) |
| :-------------: | :--------------------------------------------: |
| Native (Termux) |                     **✓**                      |

  </div>
</details>

---

## Userspace available settings

`gcil` creates a `settings.ini` configuration file in a userspace folder.

For example, it allows to change the default engines priority (`[engines] > engine`),  
or to disable the automated updates daily check (`[updates] > enabled`)

The `settings.ini` file location and contents can be shown with the following command:

```bash
gcil --settings
```

---

## Environment available configurations

`gcil` uses `colored` for colors outputs and `questionary` for interactive menus.

If colors of both outputs types do not match the terminal's theme,  
an environment variable `NO_COLOR=1` can be defined to disable colors.

---

## Upgrade issues and hints

<details>
  <summary>Expand upgrade issues and hints documentation</summary>
  <div style="padding-left: 30px">
    <br />

- Versions before `4.6.0` used the now deprecated `PyInquirer` dependency,  
and due to its own old dependency to the `prompt-toolkit` package,  
you shoud uninstall both packages first before updating

  </div>

</details>

---

## Supported .gitlab-ci.yml features

<details>
  <summary>Expand .gitlab-ci.yml template documentation</summary>
  <div style="padding-left: 30px">

```yaml
# Global configurations

include: FILE_PATH
include:
  local: FILE_PATH
include:
  - local: FILE_PATH
include:
  - project: PROJECT_URL
    file: FILE_PATH
include:
  - project: PROJECT_URL
    file:
      - FILE_PATH_1
      - FILE_PATH_2

image: IMAGE_NAME
image:
  name: IMAGE_NAME
  entrypoint: ['COMMANDS']

services:
  - ...docker:dind
  - SERVICE_NAME
  - name: SERVICE_NAME
    alias: SERVICE_ALIAS

stages:
  - STAGE_NAMES

variables:
  - VARIABLES: VALUES
  - !reference [.path, to, YAML, node, variables]

# Global scripts

before_script:
  - COMMANDS
  - !reference [.path, to, YAML, node, before_script]

after_script:
  - COMMANDS
  - !reference [.path, to, YAML, node, after_script]

# Templates nodes

.TEMPLATES: &TEMPLATES
  KEYS: VALUES

# Job nodes

JOB_NAME:

  # Job configurations

  stage: STAGE_NAME

  image: IMAGE_NAME
  image:
    name: IMAGE_NAME
    entrypoint: ['COMMANDS']

  services:
    - ...docker:dind
    - SERVICE_NAME
    - name: SERVICE_NAME
      alias: SERVICE_ALIAS

  variables:
    VARIABLES: VALUES
    VARIABLE_PREFILLED:
      value: VALUE
      description: DESCRIPTION
    VARIABLE_REFERENCED: !reference [.path, to, YAML, node, variables, VARIABLE]
  variables: !reference [.path, to, YAML, node, variables]

  # Job templates

  <<: *TEMPLATES
  extends: TEMPLATE
  extends:
    - TEMPLATES

  # Job scripts

  before_script:
    - COMMANDS
    - !reference [.path, to, YAML, node, after_script]

  script:
    - COMMANDS
    - !reference [.path, to, YAML, node, script]

  after_script:
    - COMMANDS
    - !reference [.path, to, YAML, node, after_script]

  # Job executions

  parallel:
    matrix:
      - VARIABLE1: VALUES
        VARIABLE2: VALUES
      - VARIABLE1: VALUES
        VARIABLE3: VALUES

  retry: RETRY_COUNT
  retry:
    max: RETRY_COUNT

  tags:
    - MANUAL_TAGS

  trigger: SIMPLE_TRIGGER (ignored)
  trigger:
    COMPLEX_TRIGGER (ignored)

  when: on_success / manual / on_failure / always

  allow_failure: true / false
```

  </div>
</details>

---

## Dependencies

- [colored](https://pypi.org/project/colored/): Terminal colors and styles
- [docker](https://pypi.org/project/docker/): Docker Engine API
- [python-dotenv](https://pypi.org/project/python-dotenv/): Support for .env files parsing
- [PyYAML](https://pypi.org/project/PyYAML/): YAML parser and emitter for Python
- [questionary](https://pypi.org/project/questionary/): Interactive terminal user interfaces
- [setuptools](https://pypi.org/project/setuptools/): Build and manage Python packages
- [update-checker](https://pypi.org/project/update-checker/): Check for package updates on PyPI

---

## References

- [.gitlab-ci.yml](https://docs.gitlab.com/ee/ci/yaml/): GitLab CI/CD Pipeline Configuration Reference
- [commitizen](https://pypi.org/project/commitizen/): Simple commit conventions for internet citizens
- [git-cliff](https://github.com/orhun/git-cliff): CHANGELOG generator
- [gitlab-release](https://pypi.org/project/gitlab-release/): Utility for publishing on GitLab
- [OCI](https://opencontainers.org): Open Container Initiative
- [mypy](https://pypi.org/project/mypy/): Optional static typing for Python
- [pexpect-executor](https://pypi.org/project/pexpect-executor/): Automate interactive CLI tools actions
- [pre-commit](https://pre-commit.com/): A framework for managing and maintaining pre-commit hooks
- [PyPI](https://pypi.org/): The Python Package Index
- [termtosvg](https://pypi.org/project/termtosvg/): Record terminal sessions as SVG animations
- [Termux](https://termux.com): Linux terminal emulator on Android
- [twine](https://pypi.org/project/twine/): Utility for publishing on PyPI
- [winpty](https://github.com/rprichard/winpty): Windows PTY interface wrapper
