scott/TrueMigration
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README to reflect iSCSI support, audit wizard, and all recent changes

Browse Source 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
scott
2026-03-05 16:10:20 -05:00
parent 6042dabc8e
commit ecbc636035
1 changed files with 91 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

125
README.md
View File

@@ -1,20 +1,19 @@
# TrueMigration
A Python CLI tool for migrating SMB and NFS share configuration to a live TrueNAS destination system. Designed for systems integration teams working in pre-production deployment environments.
A Python CLI tool for migrating TrueNAS configuration to a live destination system. Designed for systems integration teams working in pre-production deployment environments.
## What It Does
TrueMigration reads share configuration from a source and re-creates it on a destination TrueNAS system via its WebSocket API. Two source types are supported:
TrueMigration reads configuration from a source and re-creates it on a destination TrueNAS system via its WebSocket API. It also provides a destination audit mode for inspecting and cleaning up existing configuration before migration.
- **TrueNAS debug archive** — the `.tgz` produced by **System → Save Debug** in the TrueNAS UI
**Supported source types:**
- **TrueNAS debug archive** — the `.tgz` produced by **System → Save Debug** in the TrueNAS UI (SCALE and CORE)
- **CSV files** — customer-supplied spreadsheets for migrating from non-TrueNAS sources
**Currently supported:**
**Supported migration types:**
- SMB shares
- NFS exports
**Planned:**
- iSCSI (targets, extents, portals, initiator groups)
- iSCSI (extents, initiator groups, portals, targets, target-extent associations)
## Requirements
@@ -25,7 +24,7 @@ TrueMigration reads share configuration from a source and re-creates it on a des
### Interactive Mode (recommended)
Run with no arguments. The wizard will guide you through source selection, destination configuration, per-share filtering, a dry run preview, and final confirmation before making any changes.
Run with no arguments. The wizard will guide you through the full workflow.
```bash
python -m truenas_migrate
@@ -37,6 +36,41 @@ or
python deploy.py
```
At startup the wizard presents two top-level options:
```
1. Migrate configuration to a destination system
2. Audit destination system (view and manage existing config)
```
#### Option 1 — Migrate
Walks through:
1. Source selection (archive or CSV)
2. Destination host, port, and API key
3. Migration scope (SMB / NFS / iSCSI, or all)
4. iSCSI portal IP remapping (destination IPs differ from source; MPIO supported)
5. Check for existing iSCSI config — offers to remove it before migration
6. Per-share selection (choose a subset or migrate all)
7. Dry run preview — shows what will be created, flags missing datasets or zvols
8. Optional auto-creation of missing datasets and zvols
9. Final confirmation and live apply
#### Option 2 — Audit
Connects to the destination and displays a full inventory:
- SMB shares (name, path, enabled status)
- NFS exports
- iSCSI configuration (extents, initiators, portals, targets, associations)
- ZFS datasets (with space used)
- ZFS zvols (with allocated size)
After displaying the inventory, offers selective deletion by category. Deletion safeguards:
- SMB shares / NFS exports / iSCSI: standard `[y/N]` confirmation
- Zvols: requires typing `DELETE` — data is permanently destroyed
- Datasets: requires typing `DELETE` — all files and snapshots are permanently destroyed
- A final confirmation gate is shown before any deletions execute
### Command Line Mode — Archive Source
```bash
@@ -50,7 +84,7 @@ python -m truenas_migrate \
--api-key "1-xxxxxxxxxxxx" \
--dry-run
# Live migration
# Live migration (all types)
python -m truenas_migrate \
--debug-tar debug.tgz \
--dest 192.168.1.50 \
@@ -62,6 +96,13 @@ python -m truenas_migrate \
--dest 192.168.1.50 \
--api-key "1-xxxxxxxxxxxx" \
--migrate smb
# Migrate SMB and iSCSI, skip NFS
python -m truenas_migrate \
--debug-tar debug.tgz \
--dest 192.168.1.50 \
--api-key "1-xxxxxxxxxxxx" \
--migrate smb iscsi
```
### Command Line Mode — CSV Source
@@ -77,7 +118,7 @@ python -m truenas_migrate \
--api-key "1-xxxxxxxxxxxx" \
--dry-run
# Live migration — SMB only
# Live migration — SMB only from CSV
python -m truenas_migrate \
--smb-csv smb_shares.csv \
--dest 192.168.1.50 \
@@ -93,13 +134,11 @@ Copy and fill in the templates included in this repository:
| `smb_shares_template.csv` | One row per SMB share |
| `nfs_shares_template.csv` | One row per NFS export |
Each template includes a header row, annotated comment rows explaining valid values for each column, and one example data row to replace. Lines beginning with `#` are ignored by the parser.
**SMB columns:** `Share Name` *(required)*, `Path` *(required)*, `Description`, `Purpose`, `Read Only`, `Browsable`, `Guest Access`, `Access-Based Enumeration`, `Hosts Allow`, `Hosts Deny`, `Time Machine`, `Enabled`
**NFS columns:** `Path` *(required)*, `Description`, `Read Only`, `Map Root User`, `Map Root Group`, `Map All User`, `Map All Group`, `Security`, `Allowed Hosts`, `Allowed Networks`, `Enabled`
Boolean columns (`Read Only`, `Browsable`, etc.) accept `true` or `false`. List columns (`Hosts Allow`, `Hosts Deny`, `Security`, `Allowed Hosts`, `Allowed Networks`) accept space-separated values.
Boolean columns accept `true` or `false`. List columns (`Hosts Allow`, `Hosts Deny`, `Security`, `Allowed Hosts`, `Allowed Networks`) accept space-separated values.
Valid `Purpose` values: `NO_PRESET`, `DEFAULT_SHARE`, `ENHANCED_TIMEMACHINE`, `MULTI_PROTOCOL_NFS`, `PRIVATE_DATASETS`, `WORM_DROPBOX`
@@ -109,45 +148,63 @@ Valid `Security` values: `SYS`, `KRB5`, `KRB5I`, `KRB5P`
In the TrueNAS UI: top-right account menu → **API Keys** → **Add**.
## iSCSI Migration Notes
iSCSI configuration involves relational objects with IDs that differ between systems. TrueMigration handles this automatically:
- **Creation order**: extents and initiator groups first (no dependencies), then portals, then targets (which reference portals and initiators), then target-extent associations
- **ID remapping**: old source IDs are mapped to new destination IDs as each object is created; downstream objects are updated accordingly
- **Portal IPs**: the wizard prompts for destination IP addresses for each portal. Enter multiple space-separated IPs for MPIO configurations
- **Zvols**: DISK-type extents reference ZFS zvols. The dry run checks whether the required zvols exist on the destination. If any are missing, the wizard prompts for their size and creates them before the live run
- **Existing config**: if the destination already has iSCSI objects, the wizard detects this and offers to remove them before migration begins
## Conflict Policy
TrueMigration never overwrites or deletes existing configuration on the destination. Conflicts are silently skipped:
TrueMigration never overwrites or deletes existing configuration on the destination. Conflicts are skipped:
| Type | Conflict detected by |
|------------|----------------------------------|
| SMB share | Share name (case-insensitive) |
| NFS export | Export path (exact match) |
| Type | Conflict detected by |
|------|----------------------|
| SMB share | Share name (case-insensitive) |
| NFS export | Export path (exact match) |
| iSCSI extent | Extent name (case-insensitive) |
| iSCSI initiator group | Comment field (case-insensitive) |
| iSCSI portal | Set of listen IP addresses |
| iSCSI target | Target name (case-insensitive) |
| iSCSI target-extent | Target ID + LUN ID combination |
Always run with `--dry-run` first to preview what will and won't be created.
## Archive Compatibility
| Source version | Archive format | Notes |
|----------------|-------------------------|---------------------------------------------|
| SCALE 24.04+ | ixdiagnose (lowercase) | Combined JSON plugin files |
| SCALE (older) | ixdiagnose (uppercase) | Per-query JSON files |
| CORE | freenas-debug / fndebug | Plain-text dumps with embedded JSON blocks |
| HA bundles | Outer .tgz + inner .txz | Active node archive selected automatically |
| Source version | Archive format | Notes |
|----------------|----------------|-------|
| SCALE 24.04+ | ixdiagnose (lowercase dirs) | Combined JSON plugin files |
| SCALE (older) | ixdiagnose (uppercase dirs) | Per-query JSON files |
| CORE | freenas-debug / fndebug | Plain-text dumps with embedded JSON |
| HA bundles (25.04+) | Outer .tgz + inner .txz per node | Active node archive selected automatically |
## Project Structure
```
deploy.py # Entry point shim
smb_shares_template.csv # SMB CSV template for customers
nfs_shares_template.csv # NFS CSV template for customers
smb_shares_template.csv # SMB CSV template
nfs_shares_template.csv # NFS CSV template
truenas_migrate/
__main__.py # python -m truenas_migrate
__main__.py # python -m truenas_migrate entry point
colors.py # ANSI color helpers and shared logger
summary.py # Migration summary and report
archive.py # Debug archive parser
summary.py # Migration summary dataclass and report
archive.py # Debug archive parser (SCALE + CORE)
csv_source.py # CSV parser for non-TrueNAS sources
client.py # TrueNAS WebSocket API client
migrate.py # SMB and NFS migration routines
client.py # TrueNAS WebSocket API client and utilities
migrate.py # SMB, NFS, and iSCSI migration routines
cli.py # Interactive wizard and argument parser
```
## Safety Notes
- SSL certificate verification is disabled by default, as TrueNAS systems commonly use self-signed certificates. Use `--verify-ssl` to enable it.
- The tool targets the TrueNAS 25.04+ WebSocket API endpoint (`wss://<host>/api/current`).
- Exit code `2` is returned if any errors occurred during migration.
- **Never destructive by default** — the migration path only creates, never modifies or deletes existing destination config
- **Dry run first** — always preview with `--dry-run` before applying changes
- **Audit deletions require explicit confirmation** — zvol and dataset deletion requires typing `DELETE` and a final confirmation gate
- SSL certificate verification is disabled by default (TrueNAS systems commonly use self-signed certs). Use `--verify-ssl` to enable it
- Targets the TrueNAS 25.04+ WebSocket API endpoint (`wss://<host>/api/current`)
- Exit code `2` is returned if any errors occurred during migration