TrueMigration/README.md

# TrueMigration

A Python CLI tool for migrating TrueNAS configuration to a live destination system. Designed for systems integration teams working in pre-production deployment environments.

## What It Does

TrueMigration reads configuration from a source and re-creates it on a destination TrueNAS system via its WebSocket API. It also provides a destination audit mode for inspecting and cleaning up existing configuration before migration.

**Supported source types:**
- **TrueNAS debug archive** — the `.tgz` produced by **System → Save Debug** in the TrueNAS UI (SCALE and CORE)
- **CSV files** — customer-supplied spreadsheets for migrating from non-TrueNAS sources

**Supported migration types:**
- SMB shares
- NFS exports
- iSCSI (extents, initiator groups, portals, targets, target-extent associations)

## Requirements

- Python 3.9+
- No external packages — stdlib only

## Usage

### Interactive Mode (recommended)

Run with no arguments. The wizard will guide you through the full workflow.

```bash
python -m truenas_migrate
```

or

```bash
python deploy.py
```

At startup the wizard presents two top-level options:

```
  1. Migrate configuration to a destination system
  2. Audit destination system  (view and manage existing config)
```

#### Option 1 — Migrate

Walks through:
1. Source selection (archive or CSV)
2. Destination host, port, and API key
3. Migration scope (SMB / NFS / iSCSI, or all)
4. iSCSI portal IP remapping (destination IPs differ from source; MPIO supported)
5. Check for existing iSCSI config — offers to remove it before migration
6. Per-share selection (choose a subset or migrate all)
7. Dry run preview — shows what will be created, flags missing datasets or zvols
8. Optional auto-creation of missing datasets and zvols
9. Final confirmation and live apply

#### Option 2 — Audit

Connects to the destination and displays a full inventory:
- SMB shares (name, path, enabled status)
- NFS exports
- iSCSI configuration (extents, initiators, portals, targets, associations)
- ZFS datasets (with space used)
- ZFS zvols (with allocated size)

After displaying the inventory, offers selective deletion by category. Deletion safeguards:
- SMB shares / NFS exports / iSCSI: standard `[y/N]` confirmation
- Zvols: requires typing `DELETE` — data is permanently destroyed
- Datasets: requires typing `DELETE` — all files and snapshots are permanently destroyed
- A final confirmation gate is shown before any deletions execute

### Command Line Mode — Archive Source

```bash
# Inspect the archive before doing anything
python -m truenas_migrate --debug-tar debug.tgz --list-archive

# Dry run — connects to destination but makes no changes
python -m truenas_migrate \
    --debug-tar  debug.tgz \
    --dest       192.168.1.50 \
    --api-key    "1-xxxxxxxxxxxx" \
    --dry-run

# Live migration (all types)
python -m truenas_migrate \
    --debug-tar  debug.tgz \
    --dest       192.168.1.50 \
    --api-key    "1-xxxxxxxxxxxx"

# Migrate only SMB shares
python -m truenas_migrate \
    --debug-tar  debug.tgz \
    --dest       192.168.1.50 \
    --api-key    "1-xxxxxxxxxxxx" \
    --migrate smb

# Migrate SMB and iSCSI, skip NFS
python -m truenas_migrate \
    --debug-tar  debug.tgz \
    --dest       192.168.1.50 \
    --api-key    "1-xxxxxxxxxxxx" \
    --migrate smb iscsi
```

### Command Line Mode — CSV Source

Fill in the provided template files and pass them on the command line. You can supply one or both.

```bash
# Dry run from CSV files
python -m truenas_migrate \
    --smb-csv    smb_shares.csv \
    --nfs-csv    nfs_shares.csv \
    --dest       192.168.1.50 \
    --api-key    "1-xxxxxxxxxxxx" \
    --dry-run

# Live migration — SMB only from CSV
python -m truenas_migrate \
    --smb-csv    smb_shares.csv \
    --dest       192.168.1.50 \
    --api-key    "1-xxxxxxxxxxxx"
```

### CSV Templates

Copy and fill in the templates included in this repository:

| File | Purpose |
|------|---------|
| `smb_shares_template.csv` | One row per SMB share |
| `nfs_shares_template.csv` | One row per NFS export |

**SMB columns:** `Share Name` *(required)*, `Path` *(required)*, `Description`, `Purpose`, `Read Only`, `Browsable`, `Guest Access`, `Access-Based Enumeration`, `Hosts Allow`, `Hosts Deny`, `Time Machine`, `Enabled`

**NFS columns:** `Path` *(required)*, `Description`, `Read Only`, `Map Root User`, `Map Root Group`, `Map All User`, `Map All Group`, `Security`, `Allowed Hosts`, `Allowed Networks`, `Enabled`

Boolean columns accept `true` or `false`. List columns (`Hosts Allow`, `Hosts Deny`, `Security`, `Allowed Hosts`, `Allowed Networks`) accept space-separated values.

Valid `Purpose` values: `NO_PRESET`, `DEFAULT_SHARE`, `ENHANCED_TIMEMACHINE`, `MULTI_PROTOCOL_NFS`, `PRIVATE_DATASETS`, `WORM_DROPBOX`

Valid `Security` values: `SYS`, `KRB5`, `KRB5I`, `KRB5P`

### Generating an API Key

In the TrueNAS UI: top-right account menu → **API Keys** → **Add**.

## iSCSI Migration Notes

iSCSI configuration involves relational objects with IDs that differ between systems. TrueMigration handles this automatically:

- **Creation order**: extents and initiator groups first (no dependencies), then portals, then targets (which reference portals and initiators), then target-extent associations
- **ID remapping**: old source IDs are mapped to new destination IDs as each object is created; downstream objects are updated accordingly
- **Portal IPs**: the wizard prompts for destination IP addresses for each portal. Enter multiple space-separated IPs for MPIO configurations
- **Zvols**: DISK-type extents reference ZFS zvols. The dry run checks whether the required zvols exist on the destination. If any are missing, the wizard prompts for their size and creates them before the live run
- **Existing config**: if the destination already has iSCSI objects, the wizard detects this and offers to remove them before migration begins

## Conflict Policy

TrueMigration never overwrites or deletes existing configuration on the destination. Conflicts are skipped:

| Type | Conflict detected by |
|------|----------------------|
| SMB share | Share name (case-insensitive) |
| NFS export | Export path (exact match) |
| iSCSI extent | Extent name (case-insensitive) |
| iSCSI initiator group | Comment field (case-insensitive) |
| iSCSI portal | Set of listen IP addresses |
| iSCSI target | Target name (case-insensitive) |
| iSCSI target-extent | Target ID + LUN ID combination |

Always run with `--dry-run` first to preview what will and won't be created.

## Archive Compatibility

| Source version | Archive format | Notes |
|----------------|----------------|-------|
| SCALE 24.04+ | ixdiagnose (lowercase dirs) | Combined JSON plugin files |
| SCALE (older) | ixdiagnose (uppercase dirs) | Per-query JSON files |
| CORE | freenas-debug / fndebug | Plain-text dumps with embedded JSON |
| HA bundles (25.04+) | Outer .tgz + inner .txz per node | Active node archive selected automatically |

## Project Structure

```
deploy.py                    # Entry point shim
smb_shares_template.csv      # SMB CSV template
nfs_shares_template.csv      # NFS CSV template
truenas_migrate/
    __main__.py              # python -m truenas_migrate entry point
    colors.py                # ANSI color helpers and shared logger
    summary.py               # Migration summary dataclass and report
    archive.py               # Debug archive parser (SCALE + CORE)
    csv_source.py            # CSV parser for non-TrueNAS sources
    client.py                # TrueNAS WebSocket API client and utilities
    migrate.py               # SMB, NFS, and iSCSI migration routines
    cli.py                   # Interactive wizard and argument parser
```

## Safety Notes

- **Never destructive by default** — the migration path only creates, never modifies or deletes existing destination config
- **Dry run first** — always preview with `--dry-run` before applying changes
- **Audit deletions require explicit confirmation** — zvol and dataset deletion requires typing `DELETE` and a final confirmation gate
- SSL certificate verification is disabled by default (TrueNAS systems commonly use self-signed certs). Use `--verify-ssl` to enable it
- Targets the TrueNAS 25.04+ WebSocket API endpoint (`wss://<host>/api/current`)
- Exit code `2` is returned if any errors occurred during migration