Metadata-Version: 2.4
Name: ddutil
Version: 0.2.1
Summary: A CLI tool for managing DataDog AWS integrations
Project-URL: Homepage, https://github.com/tomburge/datadog-utility
Project-URL: Repository, https://github.com/tomburge/datadog-utility
Project-URL: Bug Tracker, https://github.com/tomburge/datadog-utility/issues
Author-email: Tom Burge <tom@tomburge.org>
License: MIT
License-File: LICENSE
Keywords: aws,cli,datadog,devops,integration,monitoring
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: System :: Systems Administration
Classifier: Topic :: Utilities
Requires-Python: >=3.12
Requires-Dist: boto3>=1.28.0
Requires-Dist: certifi>=2023.7.22
Requires-Dist: click>=8.0.0
Requires-Dist: datadog-api-client>=2.0.0
Requires-Dist: loguru>=0.7.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: python-dotenv>=0.19.0
Requires-Dist: rich>=10.0.0
Description-Content-Type: text/markdown

# DataDog AWS Integration CLI

A command-line tool for managing DataDog AWS integrations with ease.

## Features

- 🚀 Easy setup of DataDog integrations for AWS accounts
- 🔧 Automated IAM role and policy creation/management
- ✅ Comprehensive status checking and validation
  - IAM role and policy verification
  - DataDog account configuration validation
  - Settings comparison (regions, services, metrics, resources)
- 🔄 Update existing integrations (regions, services, metrics, resources)
- 🗑️ Clean deletion of integrations
- 🎨 Beautiful terminal output with tables and colors
- ⚙️ Flexible configuration via .env files or CLI arguments
- 🔒 Dry-run mode to preview changes before applying
- 📝 Verbose logging for debugging
- 📊 JSON output support for automation
- 🔐 Support for multiple AWS partitions (standard, GovCloud, China)
- ⚡ No complex YAML configuration required

## Build Status

[![Test Build](https://github.com/tomburge/datadog-utility/actions/workflows/test-build.yml/badge.svg)](https://github.com/tomburge/datadog-utility/actions/workflows/test-build.yml)

[![Publish to TestPyPI](https://github.com/tomburge/datadog-utility/actions/workflows/publish-test.yml/badge.svg)](https://github.com/tomburge/datadog-utility/actions/workflows/publish-test.yml)

[![Publish to PyPI](https://github.com/tomburge/datadog-utility/actions/workflows/publish.yml/badge.svg)](https://github.com/tomburge/datadog-utility/actions/workflows/publish.yml)

## Installation

### Using UV (Recommended)

[UV](https://github.com/astral-sh/uv) is a blazing-fast Python package manager.

```bash
# Install UV if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh
# or on Windows:
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

# Clone the repository
git clone <repository-url>
cd datadog-utility

# Create a virtual environment and install dependencies
uv venv
uv pip install -e .
```

### Using pip

```bash
# Clone the repository
git clone <repository-url>
cd datadog-utility

# Create and activate virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install the CLI tool
pip install -e .
```

After installation, the `ddutil` command will be available globally.

## Quick Start

### 1. Set Up Environment Variables

Copy the example `.env` file and fill in your credentials:

```bash
cp .env.example .env
```

Edit `.env` with your values:

```bash
# Required
DD_API_KEY=your_datadog_api_key
DD_APP_KEY=your_datadog_app_key
AWS_ACCOUNT_ID=123456789012

# Optional but commonly used
DD_ACCOUNT_ID=your_dd_account_id
AWS_PROFILE=default
DD_IAM_ROLE_NAME=datadog-integration-role
```

### 2. Preview the Setup (Dry-Run)

```bash
ddutil setup --dry-run
```

### 3. Apply the Changes

```bash
ddutil setup
```

That's it! The tool will:

- Create the necessary IAM role and policies
- Configure the DataDog integration
- Update the IAM role with the external ID from DataDog

## Usage

### Available Commands

```bash
ddutil --help              # Show all available commands
ddutil --version           # Show version information
ddutil --license           # Show license information
ddutil -v, --verbose       # Enable verbose output for any command
ddutil -q, --quiet         # Suppress non-error output
```

**Commands:**

- `setup` - Set up DataDog integration for an AWS account
- `status` - Check the status of DataDog integration and validate configuration
- `update` - Update an existing DataDog integration
- `delete` - Delete DataDog integration for an AWS account

### Quick Reference

```bash
# Setup with defaults from .env
ddutil setup

# Preview setup without making changes
ddutil setup --dry-run

# Check configuration status and validation
ddutil status

# Update configuration
ddutil update --regions us-east-1,us-west-2

# Delete integration (with confirmation)
ddutil delete --confirm

# Any command with verbose output
ddutil --verbose <command>
```

### Setup Command

Set up DataDog integration for an AWS account:

```bash
# Using environment variables (.env file)
ddutil setup

# Override with CLI arguments
ddutil setup --account-id 123456789012 --dd-account-id YOUR_DD_ACCOUNT_ID

# Specify AWS profile
ddutil setup --profile aws-prod

# Dry-run mode (preview changes without applying)
ddutil setup --dry-run

# Verbose output (shows debug logs)
ddutil --verbose setup

# Specify regions and services
ddutil setup --regions us-east-1,us-west-2 --services AWS/Lambda,AWS/EC2

# Configure metric and resource collection
ddutil setup --metric-automute true --metric-collect-cloudwatch true \
  --resource-collect-cspm false --resource-collect-extended true

# Specify AWS partition (for GovCloud or China regions)
ddutil setup --partition aws-us-gov
```

### Status Command

Check the status of an existing integration and validate configuration:

```bash
# Basic status check
ddutil status

# Check specific account
ddutil status --account-id 123456789012

# Use specific AWS profile
ddutil status --profile aws-prod

# JSON output for programmatic use
ddutil status --output json

# Verbose output with debug info
ddutil --verbose status
```

The status command validates:

- ✅ IAM role existence and policies
- ✅ DataDog account registration
- ✅ Configuration sync (regions, services, partition)
- ✅ Metric settings (automute, CloudWatch, custom metrics)
- ✅ Resource settings (CSPM, extended collection)
- ✅ Role name and External ID matching

### Update Command

Update an existing integration:

```bash
# Update services monitored
ddutil update --services AWS/Lambda,AWS/EC2,AWS/RDS

# Update regions
ddutil update --regions us-west-2,eu-west-1

# Update metric settings
ddutil update --metric-collect-custom true --metric-automute false

# Update resource collection
ddutil update --resource-collect-cspm true

# Preview changes before applying
ddutil update --regions us-west-2,eu-west-1 --dry-run

# Update with verbose output
ddutil --verbose update --services AWS/Lambda
```

### Delete Command

Remove DataDog integration:

```bash
# With confirmation prompt (interactive)
ddutil delete

# Specify account ID
ddutil delete --account-id 123456789012

# Skip confirmation prompt (for automation)
ddutil delete --confirm

# Delete with specific role name
ddutil delete --role-name custom-datadog-role --confirm

# Use specific AWS profile
ddutil delete --profile aws-prod --confirm
```

## Configuration

### Configuration Priority

Configuration values are resolved in the following order (highest to lowest priority):

1. **CLI arguments** - Values passed directly via command-line flags
2. **Environment variables** - Values from `.env` file or system environment

### Environment Variables

Create a `.env` file in your project directory:

```bash
# Copy the example file
cp .env.example .env

# Edit with your values
nano .env  # or your favorite editor
```

### Required Environment Variables

```bash
# DataDog API Credentials
DD_API_KEY=your_datadog_api_key_here    # DataDog API key (required)
DD_APP_KEY=your_datadog_app_key_here    # DataDog application key (required)

# AWS Configuration
AWS_ACCOUNT_ID=123456789012             # Your AWS account ID (required)
```

### Optional Environment Variables

```bash
# Application Settings
LOG_LEVEL=INFO                         # Logging level (DEBUG, INFO, WARNING, ERROR)

# DataDog Configuration
DD_ACCOUNT_ID=                         # DataDog account ID (obtained from DataDog)
DD_SITE=datadoghq.com                  # DataDog site (datadoghq.com, datadoghq.eu, etc.)
DD_PARTITION=aws                       # AWS partition (aws, aws-cn, aws-us-gov)
DATADOG_VERIFY_SSL=false               # SSL verification for DataDog API

# AWS Configuration
AWS_PROFILE=default                    # AWS CLI profile name

# IAM Configuration
DD_IAM_ROLE_NAME=datadog-integration-role  # IAM role name
DD_MANAGED_POLICIES=                   # Comma-separated managed policy ARNs
                                       # Default: ReadOnlyAccess, SecurityAudit
DD_POLICY_ACTIONS=                     # Comma-separated additional IAM actions
                                       # Default: 32 standard actions (see .env.example)

# Monitoring Configuration
DD_REGIONS=                            # Comma-separated AWS regions (empty = all)
DD_SERVICES=                           # Comma-separated AWS services (empty = default)
DD_TRACES=                             # Comma-separated services for X-Ray tracing

# Metric Settings
DD_METRIC_AUTOMUTE=true                # Auto-mute monitors (true/false)
DD_METRIC_COLLECT_CLOUDWATCH=true      # Collect CloudWatch alarms (true/false)
DD_METRIC_COLLECT_CUSTOM=false         # Collect custom metrics (true/false)
DD_METRIC_COLLECT_METRICS=true         # Enable metric collection (true/false)
DD_METRIC_ENABLE=true                  # Enable metrics globally (true/false)

# Resource Settings
DD_RESOURCE_COLLECT_CSPM=false         # Cloud Security Posture Management (true/false)
DD_RESOURCE_COLLECT_EXTENDED=true      # Extended resource collection (true/false)
```

## Examples

### Basic Setup with .env File

Create `.env`:

```bash
DD_API_KEY=abc123def456...
DD_APP_KEY=xyz789ghi012...
AWS_ACCOUNT_ID=123456789012
AWS_PROFILE=production
```

Run setup:

```bash
# Preview changes
ddutil setup --dry-run

# Apply configuration
ddutil setup

# Check status after setup
ddutil status
```

### Multi-Region Setup

```bash
# Monitor specific regions only
ddutil setup --regions us-east-1,us-west-2,eu-west-1

# Or set in .env
DD_REGIONS=us-east-1,us-west-2,eu-west-1
ddutil setup
```

### Specific Services Only

```bash
# Monitor specific AWS services
ddutil setup --services AWS/Lambda,AWS/EC2,AWS/RDS

# With X-Ray tracing for Lambda
ddutil setup --services AWS/Lambda,AWS/EC2 --traces AWS/Lambda
```

### Metric and Resource Configuration

```bash
# Enable Cloud Security Posture Management
ddutil setup --resource-collect-cspm true

# Enable custom metrics collection
ddutil setup --metric-collect-custom true

# Configure multiple metric settings
ddutil setup \
  --metric-automute true \
  --metric-collect-cloudwatch true \
  --metric-collect-custom false \
  --metric-enable true
```

### Override with CLI Arguments

```bash
# Override environment variables for one-off operations
ddutil setup --account-id 999888777666 --profile dev-account

# Use different AWS partition
ddutil setup --partition aws-us-gov --profile govcloud
```

### Custom IAM Configuration

```bash
# Custom role name and policies
ddutil setup \
  --role-name custom-datadog-role \
  --managed-policies arn:aws:iam::aws:policy/ReadOnlyAccess \
  --policy-actions logs:PutSubscriptionFilter,s3:GetBucketNotification

# Update existing role policies
ddutil update \
  --managed-policies arn:aws:iam::aws:policy/ReadOnlyAccess,arn:aws:iam::aws:policy/SecurityAudit
```

### Validation and Status Checking

```bash
# Check if configuration matches .env settings
ddutil status

# Get detailed JSON output for monitoring
ddutil status --output json > status.json

# Validate specific account configuration
ddutil --verbose status --account-id 123456789012
```

## Development

### Project Structure

```text
datadog-utility/
├── src/
│   └── ddutil/          # Main package
│       ├── __init__.py  # Package initialization
│       ├── cli.py       # CLI command definitions
│       └── common/      # Shared utilities
│           ├── aws/     # AWS-related modules
│           │   ├── auth.py      # AWS authentication
│           │   └── iam.py       # IAM role management
│           ├── datadog/         # DataDog-related modules
│           │   └── aws.py       # DataDog API interactions
│           ├── logs.py          # Logging configuration
│           └── utils.py         # Utility functions
├── pyproject.toml       # Package configuration
├── .env.example         # Example environment variables
├── .python-version      # Python version for UV
├── requirements.txt     # Python dependencies
└── responses/           # Sample API responses
```

## Troubleshooting

### Common Issues

**Missing required environment variables:**

```bash
# Check if all required variables are set with dry-run
ddutil setup --dry-run

# Set missing variables in .env file or export them
export AWS_ACCOUNT_ID=123456789012
export DD_API_KEY=your_api_key
export DD_APP_KEY=your_app_key
```

**AWS authentication errors:**

```bash
# Ensure your AWS profile is configured
aws configure --profile your-profile-name

# Or specify profile explicitly
ddutil setup --profile your-profile-name

# Or use environment variables
export AWS_PROFILE=your-profile-name

# Test AWS connectivity
aws sts get-caller-identity --profile your-profile-name
```

**DataDog API errors:**

```bash
# Verify your API keys are set
echo $DD_API_KEY
echo $DD_APP_KEY

# Make sure they're in your .env file
cat .env | grep DD_

# Test DataDog API connectivity
ddutil --verbose status
```

**Configuration mismatch errors:**

```bash
# Use status command to see what doesn't match
ddutil status

# Status will show:
# - IAM role and policy status
# - DataDog account configuration
# - Mismatches between actual and expected settings
# - Specific issues with regions, services, metrics, resources

# Fix mismatches by updating
ddutil update --regions us-east-1,us-west-2
ddutil update --metric-collect-cloudwatch true
```

**Debug with verbose and dry-run:**

```bash
# Always test with dry-run first
ddutil setup --dry-run

# Use verbose for detailed debugging
ddutil --verbose setup --dry-run

# Check logs for detailed error messages
tail -f ~/.ddutil/logs/ddutil.log  # if logging to file
```

## Roadmap

See [ROADMAP.md](ROADMAP.md) for planned features and improvements.

## License

MIT License - See LICENSE file for details.
