Metadata-Version: 2.4
Name: bitp
Version: 1.1.14
Summary: Tools for BitBake/Yocto projects
Author-email: Bruce Ashfield <bruce.ashfield@gmail.com>
License: GPL-2.0-only
Project-URL: Homepage, https://gitlab.com/bruce-ashfield/bitbake-project
Project-URL: Repository, https://gitlab.com/bruce-ashfield/bitbake-project
Project-URL: Issues, https://gitlab.com/bruce-ashfield/bitbake-project/-/issues
Keywords: bitbake,yocto,openembedded,layers,git
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU General Public License v2 (GPLv2)
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: Software Development :: Embedded Systems
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: COPYING
Provides-Extra: completion
Requires-Dist: argcomplete>=2.0; extra == "completion"
Provides-Extra: dev
Requires-Dist: argcomplete>=2.0; extra == "dev"
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: pytest-mock>=3.0; extra == "dev"
Dynamic: license-file

# bitp

A CLI tool for managing BitBake/Yocto/OpenEmbedded layer repositories.

The command is `bit` for quick typing.

## Overview

`bit` helps developers working with Yocto/OpenEmbedded projects by providing tools to:

- **Update** git repos backing BitBake layers from a `bblayers.conf`
- **Explore** commits interactively with fzf-based navigation
- **Export** patches with cover letters for upstream submission
- **Manage branches** across multiple layer repositories
- **Search** the OpenEmbedded Layer Index for layers
- **Bootstrap** new projects by cloning core repositories
- **Manage projects** - switch between multiple Yocto builds from anywhere

## Features

- **Automatic layer discovery** - finds layers by searching for `conf/layer.conf` files
- **Multi-layer repo support** - handles repos containing multiple layers (e.g., meta-openembedded)
- **Interactive fzf menus** - fast navigation with preview panes and keyboard shortcuts
- **Tab completion** - bash completion via argcomplete
- **Background refresh** - upstream status checks run in background for fast startup
- **Per-repo configuration** - custom display names, update defaults, push targets

## Quick Start

```bash
# Create a new Yocto project
bit create -b scarthgap --execute
bit setup

# Check status of all layer repos
bit status

# Interactively explore repos and commits
bit explore

# Update all repos (interactive)
bit update

# Update all repos non-interactively (apply saved defaults)
bit update -y

# Search for layers in the OE Layer Index
bit layer-index security
```

## Screenshots

### Project Dashboard

Manage multiple Yocto/OE builds from a single interface. Switch between projects, launch explore/update/config, or open a build shell.

![Dashboard](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/dashboard.png)

### Explore Repos

Browse layer repositories with upstream status, branch info, and dirty/clean indicators. Yocto projects get extra keybindings for recipes, deps, fragments, and build info.

![Explore](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/explore.png)

Expand multi-layer repos to see individual layers:

![Explore Expanded](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/explore-expanded.png)

### Recipe Browser

Search and browse BitBake recipes across layers. Expand recipes to see dependencies inline.

![Recipes](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/recipes.png)

![Recipe Dependencies](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/recipes-deps.png)

### Commit Browser

Drill into a repo to browse commits with preview panes, mark ranges for export, and launch git tools.

![Commits](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/commits.png)

## Commands

| Command | Alias | Description |
|---------|-------|-------------|
| `explore` | `x` | Interactively explore commits in layer repos |
| `update` | `u` | Update git repos referenced by layers |
| `status` | - | Show local commit summary for layer repos |
| `info` | `i` | Show build configuration and layer status |
| `config` | `c` | View and configure repo/layer settings |
| `branch` | `b` | View and switch branches across repos |
| `export` | - | Export patches from layer repos |
| `repos` | - | List layer repos |
| `create` | - | Create a new Yocto/OE project (clone core repos) |
| `setup` | - | Show OE/Yocto build environment setup command |
| `setup shell` | - | Start shell with OE build environment pre-sourced |
| `setup clone` | - | Clone repos from a `.conf.json` config or registry |
| `setup apply` | - | Apply layers and fragments from saved config |
| `setup configs` | - | Browse and manage saved build configurations |
| `setup export` | - | Export current build state as `.conf.json` |
| `setup registry-copy` | - | Copy a `.conf.json` file to the user registry |
| `b4` | - | Mail-based patch management using b4 and lore.kernel.org |
| `patches` | - | Explore and manage patches across layers |
| `deps` | - | Show layer and recipe dependencies |
| `recipes` | - | Search and browse BitBake recipes |
| `fragments` | - | Browse and manage OE configuration fragments |
| `layer-index` | - | Search OpenEmbedded Layer Index for layers |
| `projects` | `p` | Manage and switch between multiple project directories |

Run `bit` with no arguments for an interactive command menu, or `bit --help` for detailed usage.

## Common Workflows

### Starting a New Project

There are several ways to create a new project, depending on whether you have a saved build configuration.

#### Basic clone (no config file)

Clone the three core repos (bitbake, openembedded-core, meta-yocto) and set up the build environment:

```bash
# Clone core repos at a release branch
bit setup clone -b scarthgap --doit

# Show the environment setup command
bit setup

# Or jump straight into a sourced shell
bit setup shell
```

#### Clone from a saved configuration

Build configurations (`.conf.json` files) capture repos, layers, branches, and OE fragments. Clone from one to reproduce a build:

```bash
# Clone from a specific config file
bit setup clone --config poky.conf.json --doit

# Or browse the config registry to pick one
bit setup clone --config --doit

# Apply layers and fragments to the build directory
bit setup apply
```

`setup apply` auto-creates `build/conf/` if missing, adds layers to `bblayers.conf`, and applies OE configuration fragments. One-of fragment categories (e.g., machine, distro) prompt for selection via interactive picker.

#### From the dashboard

The dashboard detects empty or new directories and offers to create a project:

1. Open `bit projects`
2. Press `+` to add a directory (or enter a new path)
3. If the directory is empty, a config picker opens automatically
4. Select a config to clone, or choose "Basic clone" for the 3-repo default
5. After cloning, `setup apply` runs automatically to configure layers and fragments

Projects with unapplied configs show a `⚠ setup` indicator in the dashboard.

### Saving and Sharing Build Configurations

#### Export current build state

```bash
# Export to a .conf.json file in the current directory
bit setup export

# Export with a custom filename
bit setup export my-build.conf.json

# Export and save to the user registry for reuse
bit setup export --registry

# Export a different registered project
bit setup export --project my-other-build
```

The export captures git remotes, branches, layer paths, and enabled OE fragments. The dashboard also supports export via the expand menu on any project.

#### Manage the config registry

```bash
# Copy a config file into the registry
bit setup registry-copy poky.conf.json

# Copy with a custom name
bit setup registry-copy poky.conf.json --as my-custom-build

# Browse and manage all saved configs
bit setup configs
```

The registry browser (`bit setup configs`) provides:
- Preview pane showing config details (repos, layers, fragments)
- Actions: Clone, Apply, Copy, Describe, Edit, Rename, Delete
- Built-in configs are read-only (Clone, Apply, Copy only)
- User configs support all actions including inline rename and describe

### Build Environment Shell

```bash
# Show the source command for oe-init-build-env
bit setup

# Start a shell with the build environment pre-sourced
bit setup shell
```

`setup shell` starts a new shell with the OE build environment already sourced, so `bitbake` and related tools are immediately available.

### Daily Development

```bash
# Check what's changed upstream
bit status

# Update repos (interactive)
bit update

# Update all repos non-interactively (apply saved defaults)
bit update -y

# Or update a specific repo
bit update OE-core
```

### Exploring Commits

```bash
# Interactive two-level browser
bit explore

# Jump directly to a repo
bit x OE-core
```

Keybindings in explore mode (repo list):
- `Enter` - explore commits in selected repo
- `u` - pull --rebase
- `m` - pull (merge)
- `r` - refresh repo (fetch)
- `R` - refresh all repos
- `B` - switch all repos to branch (common branches first, partial dimmed)
- `t` - launch git history viewer (tig/lazygit/gitk)
- `v` - toggle verbose display (HEAD commit per repo/layer)
- `\` - expand/collapse all multi-layer repos
- `c` - open config menu
- `M` - open message log viewer (see [Status Messages](#status-messages))
- `q` - quit

Yocto-specific keybindings (explore mode, Yocto projects only):
- `i` - show build info (same as `bit info`)
- `Ctrl-R` - browse recipes (scoped to selected repo's layers, or single layer if expanded)
- `D` - show dependencies (scoped to selected layer)
- `f` - browse configuration fragments

Keybindings in commit browser:
- `Tab` - mark commit for selection
- `Space` - select range of commits
- `?` - toggle preview pane
- `d` - toggle diff view in preview (stat ↔ full patch)
- `a` - toggle blame view in preview
- `f` - switch to file/tree view (browse files, expand commit history per file)
- `c` - copy commit hash
- `e` - export selected commits
- `i` - interactive rebase (select range, then squash/reword/reorder)
- `t` - launch git history viewer
- `l` - browse commits grouped by layer (multi-layer repos)
- `p` - find patch source on lore.kernel.org
- `PgUp`/`PgDn` - scroll commit list
- `Alt-Up`/`Alt-Down` - scroll preview page
- `Ctrl-U`/`Ctrl-D` - scroll preview half-page
- `←` or `b` - go back
- `q` - quit

Keybindings in file/tree view (`f` from commit browser):
- `Enter`/`→` - expand directory or show file's commit history
- `←` - collapse directory/file or go to parent
- `\` - toggle expand/collapse directory or file commits
- `d` - toggle diff view in preview
- `a` - toggle blame view
- `f` - toggle raw file content view
- `?` - toggle preview pane
- `b`/`Esc` - go back to commit browser
- `q` - quit

### Exporting Patches for Upstream

```bash
# Prepare commits (reorder for upstream)
bit export prep --branch zedd/feature

# Export patches
bit export --target-dir ~/patches

# Export with version number
bit export --target-dir ~/patches -v 2
```

### Searching for Layers

```bash
# Interactive search
bit layer-index

# Search with query
bit layer-index virtualization

# Get layer info for scripting
bit layer-index -i meta-virtualization

# Clone a layer directly
bit layer-index -c meta-security -b scarthgap
```

### Managing Branches

```bash
# Interactive branch management
bit branch

# Switch all repos to a release branch
bit branch --all scarthgap
```

Keybindings in branch management (`bit branch`):
- `Enter`/`→` - expand repo to show branches / switch to selected branch
- `←` - collapse expanded repo
- `B` - switch all repos to branch (common branches first, partial dimmed)
- `q` - quit

### Viewing Build Info

```bash
# Show build configuration (like BitBake summary)
bit info

# Show only layer info with branch:commit
bit info layers

# Show only key variables
bit info vars
```

Output format matches BitBake's build configuration:
```
Build Configuration:
  MACHINE = qemuarm64
  DISTRO  = poky

Layers:
  meta                = "master:01a65e8d5f73..."
  meta-selftest
  meta-poky           = "master:de4abc0a175a..."
```

The `info` entry is also available in the explore repo list menu for interactive browsing.

### Managing Multiple Projects

```bash
# Open project manager (interactive)
bit projects

# Add a project directory
bit projects add /path/to/yocto-build

# List known projects
bit projects list

# Remove a project from tracking
bit projects remove /path/to/old-project
```

Keybindings in projects menu:
- `Space` - activate selected project (all commands will operate on it)
- `Enter`/`→` - expand project; on already-expanded project, runs configurable default command (default: explore)
- `?`/`m` - open commands menu
- `+` - browse for a new directory to add
- `-` - remove selected project from tracking
- `n` - rename project
- `u` - update repos in project
- `x` - explore repos in project
- `c` - open config menu (includes "Default arrow-action" setting to configure Enter/→ on expanded projects)
- `S` - open shell in project
- `M` - open message log viewer (see [Status Messages](#status-messages))
- `q` - quit

Note: When no valid project context exists (no bblayers.conf found), the projects picker is automatically shown. Persona is auto-detected when adding projects (Yocto projects detected by bblayers.conf, layer.conf files, or poky directory; generic projects detected by .git directories without Yocto markers).

## Configuration

Per-repo settings are stored in `.bit.defaults` (JSON):

```json
{
  "/path/to/poky": "rebase",
  "/path/to/meta-oe": "skip",
  "__extra_repos__": ["/path/to/bitbake"],
  "__hidden_repos__": ["/path/to/unwanted-repo"],
  "__push_targets__": {
    "/path/to/oe-core": {
      "push_url": "ssh://git@push.openembedded.org/oe-core-contrib",
      "branch_prefix": "yourname/"
    }
  },
  "fzf_theme": "dark",
  "fzf_text_color": "light-gray",
  "fzf_custom_colors": {"pointer": "green"}
}
```

Configure interactively with `bit config`.

### Theme Customization

The tool supports customizable options via Settings menu (`bit config` -> Settings):

**Colors** submenu:
- **Mode** - Color source: global (`~/.config/bit/colors.json`), per-project (`.bit.defaults`), or built-in defaults
- **Theme** - Base color scheme (default, dark, light, dracula, nord, etc.)
- **Individual** - Per-element color overrides with live preview

New projects default to **global** mode — changing colors in any project updates `~/.config/bit/colors.json`, shared by all projects in global mode. Switch to **custom** for per-project overrides.

A color preview panel shows all themed elements (pointer, header, prompt, etc.) with current colors applied.

**Directory Browser** - Choose preferred file browser for project selection:
- broot (recommended)
- ranger
- nnn
- fzf (fallback)

**Git Viewer** - Choose preferred git history viewer:
- auto (detect first available: tig > lazygit > gitk)
- tig - ncurses git interface
- lazygit - terminal UI for git
- gitk - graphical git browser

**Preview Layout** - Configure commit browser preview pane position:
- Bottom (default) - preview below commit list
- Right (side-by-side) - preview beside commit list
- Top - preview above commit list

**Terminal Colors** - Configure output colors for:
- Upstream indicator (commits to pull)
- Local commit count
- Dirty/clean status
- Repo names (configured, discovered, external)

## Visual Indicators

Repos are color-coded by source:
- **Green** - from bblayers.conf
- **Magenta** with `(?)` - discovered layers (not in bblayers.conf)
- **Cyan** with `(ext)` - external repos (git repos without conf/layer.conf)

Status indicators:
- `5 local` - commits ahead of upstream tracking ref
- `↓ 3` - commits to pull from upstream
- `→ contrib` - tracking a non-origin remote (shown when not tracking origin)
- `[clean]` / `[DIRTY]` - working tree status

### Status Messages

Actions like activating a project, refreshing repos, or running updates produce
status messages (e.g. `✓ Activated: poky`, `Refreshing OE-core... done`). These
are captured silently — they never print to the terminal — and are available in
two ways:

- **Header indicator**: The most recent message appears above the keybinding
  help for one fzf iteration, then disappears on the next redraw.
- **Message log** (`M` key): Opens a scrollable popup showing all messages from
  the current session with timestamps (newest first). Available in both the
  dashboard and explore menus.

### Remote SSH Projects

Remote projects on SSH-accessible build machines can be managed alongside local
ones:

- Add via manage menu (`+` → Remote SSH) with `user@host:/path` URI format
- Dashboard shows remote projects with `⇄ hostname` indicator
- Remote summary (branch, dirty, ahead/behind) gathered via single batched SSH call
- Actions dispatch to remote bit (auto-deployed if not installed) with fallback to raw git
- SSH connection multiplexing (`ControlMaster=auto`, `ControlPersist=600`) avoids repeated auth
- All SSH calls suppress askpass prompts (`SSH_ASKPASS_REQUIRE=never`, `BatchMode=yes`) to
  prevent password dialogs from interfering with the TUI
- Interactive SSH sessions (explore, shell) are preceded by a connection pre-check;
  if auth fails, a clean error message is shown instead of hanging

## Requirements

- Python 3.9+
- Git
- fzf (optional, but recommended for interactive features)
- argcomplete (optional, for tab completion)
- xclip or xsel (optional, for clipboard support)

## Development

### Installation for Development

```bash
# Clone the repository
git clone https://gitlab.com/bruce-ashfield/bitbake-project.git
cd bitbake-project

# Create a virtual environment
python3 -m venv .venv
source .venv/bin/activate

# Install with dev dependencies
pip install -e .[dev]
```

### Running Tests

The project uses pytest with pytest-mock and pytest-cov:

```bash
# Activate virtual environment first
source .venv/bin/activate

# Run all tests
pytest tests/ -v

# Run with coverage report
pytest tests/ --cov=bitbake_project --cov-report=term-missing

# Run only unit tests
pytest tests/unit/ -v

# Run only integration tests
pytest tests/integration/ -v

# Run tests matching a pattern
pytest tests/ -k "test_colors" -v
```

### Test Structure

```
tests/
├── conftest.py                    # Shared fixtures
├── unit/
│   ├── test_colors.py            # Colors class ANSI formatting
│   ├── test_helpers.py           # Pure functions (clean_title, dedupe, sort)
│   ├── test_state.py             # State management (defaults, prep, export)
│   ├── test_gitrepo.py           # GitRepo class
│   ├── test_fzf_menu.py          # FzfMenu class (mocked)
│   ├── test_bblayers_parser.py   # BblayersParser
│   ├── test_cli_parser.py        # CLI argument parsing
│   ├── test_info.py              # Info command parsing and formatting
│   └── test_update.py            # Update command (upstream tracking)
├── integration/
│   ├── test_cli_dispatch.py      # Command dispatch, --help, aliases
│   └── test_projects_command.py  # Projects add/remove/list
└── data/
    └── bblayers/                  # Test bblayers.conf files
```

## Project Structure

The project is organized as a Python package with a commands subpackage:

```
bitbake_project/
├── core.py           # Core abstractions (Colors, GitRepo, FzfMenu)
├── cli.py            # Argument parsing and main()
└── commands/         # Command implementations
    ├── common.py     # Shared utilities (BblayersParser, etc.)
    ├── explore.py    # explore, status commands
    ├── export.py     # export, export prep commands
    ├── config.py     # config command
    ├── branch.py     # branch command
    ├── info.py       # info command (build configuration)
    ├── update.py     # update command (upstream tracking support)
    ├── b4.py         # b4 command (lore, mail-based patch management)
    ├── patches.py    # patches command (patch browser, Upstream-Status)
    ├── search.py     # layer-index command
    ├── setup.py      # create, setup commands
    ├── repos.py      # repos command
    ├── projects.py   # projects command, dashboard, directory browser
    └── ssh_remote.py # SSH transport for remote projects
```

Distribution options:
- **Standalone**: Single-file zipapp (`dist/bit`)
- **Pip install**: `pip install -e .` for development

## License

This project is licensed under the GNU General Public License v2.0 (GPL-2.0-only).

See [COPYING](COPYING) for the full license text.

## See Also

- [INSTALL.md](INSTALL.md) - Installation instructions
- [Yocto Project](https://www.yoctoproject.org/)
- [OpenEmbedded](https://www.openembedded.org/)
- [OpenEmbedded Layer Index](https://layers.openembedded.org/)
