When a portal fails to create (e.g. IP not on system), its ID is absent
from portal_id_map. Previously this caused every target referencing that
portal to be skipped entirely, which then cascaded to target-extent
failures.
Now each target's groups are filtered individually: groups with unmapped
portal or initiator references are dropped with a warning, and the target
is still created with the remaining valid groups. This keeps the target
and all its extent associations intact while clearly indicating which
groups need manual attention.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TrueNAS 25.x returns path_local in share query results but rejects it
on create with EINVAL. Added to _SMB_SHARE_READONLY so it is stripped
from the payload before submission.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New top-level wizard option (2) lets users inspect and clean up an
existing destination before migration. Queries all SMB shares, NFS
exports, iSCSI objects, datasets, and zvols; displays a structured
inventory report; then offers per-category deletion with escalating
warnings — standard confirm for shares/iSCSI, explicit "DELETE" phrase
required for zvols and datasets to guard against accidental data loss.
Adds to client.py: query_destination_inventory, delete_smb_shares,
delete_nfs_exports, delete_zvols, delete_datasets.
Adds to cli.py: _fmt_bytes, _print_inventory_report, _run_audit_wizard.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
During dry run, "would create" iSCSI objects now populate id_map with
a source-ID placeholder so downstream objects (targets, target-extents)
can remap references without cascading failures.
Adds query_existing_iscsi() and clear_iscsi_config() to migrate.py, and
_prompt_clear_existing_iscsi() to the wizard: if the destination already
has iSCSI config, the user is shown a summary and offered Keep/Remove
before the dry run begins.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When missing zvols are created, the first dry run shows errors
because the zvols don't exist yet. After creation, run a second
dry run so the user sees a clean result before confirming the
live migration.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_ip_set() used (ip, port) tuples for conflict matching. Since port
is no longer present in listen entries, key on IP only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The listen entry display string still referenced l['port'] after
port was stripped from entries. Update to show IP only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The TrueNAS API rejects port inside listen array items
(iscsi_portal_create.listen.0.port: Extra inputs are not permitted).
Port is a global iSCSI setting, not per-listen-IP.
- _iscsi_portal_payload(): strip port from each listen entry,
keeping only {"ip": "..."}
- _prompt_iscsi_portals(): remove port prompt from wizard; show
source IPs without port in the display
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
client.py:
- check_iscsi_zvols(): queries pool.dataset.query for VOLUME type,
returns list of missing zvol names
- create_zvol(): creates a single zvol via pool.dataset.create
- create_missing_zvols(): opens a fresh connection and creates a
batch of zvols from a {name: volsize_bytes} dict
summary.py:
- Add zvols_to_check and missing_zvols list fields
- Report shows a WARNING block listing missing zvols when present
migrate.py:
- _migrate_iscsi_extents() populates summary.zvols_to_check with
the dataset name for each DISK-type extent during dry run
cli.py:
- Add _parse_size() to parse human-friendly size strings
(100G, 500GiB, 1T, etc.) to bytes
- run() calls check_iscsi_zvols() during dry run and stores results
in summary.missing_zvols
- Wizard prompts for size and creates missing zvols after the dry
run report, before asking the user to confirm the live run
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
archive.py:
- Add iscsi_config.json to _CANDIDATES and _KEYWORDS
- parse_archive() now extracts portals, initiators, targets, extents,
targetextents, and global_config into archive["iscsi"]
migrate.py:
- Add payload builders for all five iSCSI object types
(extents, initiators, portals, targets, target-extents)
- Add migrate_iscsi() which creates objects in dependency order
(extents+initiators first, then portals, then targets, then
target-extent associations) and tracks old→new ID mappings at
each step so downstream references are correctly remapped
- Conflict detection: extents/targets by name, portals by IP set,
initiators by comment, target-extents by target+LUN combination
- Skipped objects still populate the ID map so dependent objects
can remap their references correctly
summary.py:
- Add per-sub-type found/created/skipped/failed counters for iSCSI
- iSCSI rows appear in the report only when iSCSI data was processed
cli.py:
- Add _prompt_iscsi_portals() — shows source IPs per portal and
prompts for destination IPs in-place; supports MPIO (space-separated)
- Wizard scope menu gains option 3 (iSCSI); portal prompt fires
automatically after archive parse when iSCSI portals are present
- run() wires in migrate_iscsi()
- argparse --migrate now accepts "iscsi" as a valid choice
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace API field names (guestok, abe, ro, maproot_user, etc.) with
plain-English headers (Guest Access, Access-Based Enumeration, Read Only,
Map Root User, etc.) for customer clarity
- Drop comment rows that rendered poorly in spreadsheet apps
- Use two realistic example rows instead to teach by example
- Update csv_source.py to map friendly header names to API field names
before validation and coercion (raw API names still accepted)
- Update README column reference to match new header names
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add truenas_migrate/csv_source.py: parses SMB and NFS share definitions
from customer-supplied CSV files; returns same dict shape as parse_archive()
so migrate.py is untouched
- Add smb_shares_template.csv and nfs_shares_template.csv with annotated
headers and example rows (# comment rows are skipped by the parser)
- Update cli.py: interactive wizard gains a source-type step (archive vs CSV);
run() resolves CSV source via --smb-csv / --nfs-csv args; --debug-tar is now
optional; argparse validates mutual exclusion of archive and CSV flags
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Split single-file script into truenas_migrate/ package
- Removed SMB global config migration (not needed for deployment use)
- Added compatibility shim so both invocation styles still work
- Added __pycache__ to .gitignore
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the numbered-input share picker with a full TTY checkbox UI:
↑/↓ to move cursor, Space to toggle, A to select/deselect all, Enter to confirm.
Falls back to the original numbered text input when stdin is not a TTY.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After parsing the archive, present a numbered list of SMB and NFS
shares and let the user pick which ones to migrate. Entering nothing
(or 'all') keeps everything; 'n' skips the type entirely; space-
separated numbers select specific shares.
Because archive_data is filtered before the dry run, only selected
shares are processed in both the dry and live runs, and the dataset
existence check covers exactly the chosen share paths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Auto-detected ANSI colors (disabled when stderr is not a TTY)
- Colored log formatter: dim timestamps, level names styled by severity
- Migration status keywords colored: SKIP=yellow, CREATED=bold-green,
FAILED=bold-red, [DRY RUN]=cyan
- Share/export header lines bold with share name in bold-cyan
- Dry-run banner rendered as a bold-yellow framed box
- Summary report: cyan border, colored per-stat counts (green/yellow/red),
errors and warnings highlighted; fixed box width (w=60) with ANSI-aware
padding so columns stay aligned with color codes present
- Interactive wizard: styled header, cyan numbered list items
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Strip CORE-only SMB share field (vuid) and NFS share fields
(paths, alldirs, quiet) that are rejected by the SCALE API.
Convert CORE's NFS paths list to the single path string SCALE expects.
Also include NFS paths in dry-run dataset existence checks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
During dry runs, query pool.dataset.query on the destination to verify that
every share path would have a backing ZFS dataset. Missing paths are collected
in Summary.missing_datasets and surfaced as a WARNING block in the report.
In interactive mode the user is prompted to create any auto-creatable
(/mnt/…) datasets before the live migration proceeds. Non-interactive
--dry-run mode prints the same warning in the summary report.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When run with no arguments the script now guides the user through the
full migration interactively:
1. Lists debug archives found in the current directory and prompts
for selection (auto-selects when only one is present)
2. Prompts for destination host, port, and API key (key input hidden)
3. Prompts for migration scope (SMB shares / NFS shares / SMB config)
4. Runs a dry run and displays the summary
5. Asks for confirmation before applying changes live
The archive is parsed once and reused for both the dry and live runs.
The existing CLI flag interface is unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the third-party `websockets` package with a self-contained
implementation built on asyncio.open_connection, ssl, hashlib, base64,
struct, and os — all Python stdlib modules available on TrueNAS OS.
The script now runs directly on TrueNAS without any pip install.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
