scott/TrueMigration
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

iSCSI support, audit wizard, CSV improvements, bug fixes #1

Merged
scott merged 9 commits from devel into main 2026-03-05 16:08:43 -05:00
Conversation 0 Commits 9 Files Changed 5 +1175 -16
scott commented 2026-03-05 16:08:35 -05:00
Owner
Copy Link

Summary

  • iSCSI migration: Full support for migrating extents, initiator groups, portals, targets, and target-extent associations from SCALE debug archives. Portal IPs are remapped interactively (MPIO supported). Zvols are checked and can be auto-created.
  • Audit wizard: New top-level option (2) to query and inspect all existing configuration on the destination — SMB, NFS, iSCSI, datasets, zvols. Offers selective deletion with escalating warnings (typed DELETE required for data-destructive operations).
  • Pre-migration iSCSI check: Wizard detects existing iSCSI config on the destination and prompts Keep/Remove before the dry run.
  • Dry-run cascade fix: iSCSI sub-routines now populate id_map with placeholder IDs during dry run so downstream objects (targets, target-extents) can remap references without cascading failures.
  • SMB fix: Excluded server-generated path_local field from create payloads (rejected by TrueNAS 25.x with EINVAL).
  • CSV import: Human-readable column headers for SMB/NFS templates with column name mapping in csv_source.py.

Test plan

  • Run interactive wizard → option 2 (Audit) against a live destination; verify inventory display and selective deletion
  • Run iSCSI migration dry run; verify no cascade "cannot remap" errors
  • Run live iSCSI migration end-to-end
  • Run SMB migration; verify no path_local errors
  • Test with both archive source and CSV source

🤖 Generated with Claude Code

## Summary - **iSCSI migration**: Full support for migrating extents, initiator groups, portals, targets, and target-extent associations from SCALE debug archives. Portal IPs are remapped interactively (MPIO supported). Zvols are checked and can be auto-created. - **Audit wizard**: New top-level option (2) to query and inspect all existing configuration on the destination — SMB, NFS, iSCSI, datasets, zvols. Offers selective deletion with escalating warnings (typed `DELETE` required for data-destructive operations). - **Pre-migration iSCSI check**: Wizard detects existing iSCSI config on the destination and prompts Keep/Remove before the dry run. - **Dry-run cascade fix**: iSCSI sub-routines now populate `id_map` with placeholder IDs during dry run so downstream objects (targets, target-extents) can remap references without cascading failures. - **SMB fix**: Excluded server-generated `path_local` field from create payloads (rejected by TrueNAS 25.x with EINVAL). - **CSV import**: Human-readable column headers for SMB/NFS templates with column name mapping in `csv_source.py`. ## Test plan - [ ] Run interactive wizard → option 2 (Audit) against a live destination; verify inventory display and selective deletion - [ ] Run iSCSI migration dry run; verify no cascade "cannot remap" errors - [ ] Run live iSCSI migration end-to-end - [ ] Run SMB migration; verify no `path_local` errors - [ ] Test with both archive source and CSV source 🤖 Generated with [Claude Code](https://claude.com/claude-code)
scott added 9 commits 2026-03-05 16:08:35 -05:00
Add iSCSI migration support (SCALE archive source) e81e3f7fbb 
archive.py:
- Add iscsi_config.json to _CANDIDATES and _KEYWORDS
- parse_archive() now extracts portals, initiators, targets, extents,
  targetextents, and global_config into archive["iscsi"]

migrate.py:
- Add payload builders for all five iSCSI object types
  (extents, initiators, portals, targets, target-extents)
- Add migrate_iscsi() which creates objects in dependency order
  (extents+initiators first, then portals, then targets, then
  target-extent associations) and tracks old→new ID mappings at
  each step so downstream references are correctly remapped
- Conflict detection: extents/targets by name, portals by IP set,
  initiators by comment, target-extents by target+LUN combination
- Skipped objects still populate the ID map so dependent objects
  can remap their references correctly

summary.py:
- Add per-sub-type found/created/skipped/failed counters for iSCSI
- iSCSI rows appear in the report only when iSCSI data was processed

cli.py:
- Add _prompt_iscsi_portals() — shows source IPs per portal and
  prompts for destination IPs in-place; supports MPIO (space-separated)
- Wizard scope menu gains option 3 (iSCSI); portal prompt fires
  automatically after archive parse when iSCSI portals are present
- run() wires in migrate_iscsi()
- argparse --migrate now accepts "iscsi" as a valid choice

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add zvol existence check and creation for iSCSI extents 5886622004 
client.py:
- check_iscsi_zvols(): queries pool.dataset.query for VOLUME type,
  returns list of missing zvol names
- create_zvol(): creates a single zvol via pool.dataset.create
- create_missing_zvols(): opens a fresh connection and creates a
  batch of zvols from a {name: volsize_bytes} dict

summary.py:
- Add zvols_to_check and missing_zvols list fields
- Report shows a WARNING block listing missing zvols when present

migrate.py:
- _migrate_iscsi_extents() populates summary.zvols_to_check with
  the dataset name for each DISK-type extent during dry run

cli.py:
- Add _parse_size() to parse human-friendly size strings
  (100G, 500GiB, 1T, etc.) to bytes
- run() calls check_iscsi_zvols() during dry run and stores results
  in summary.missing_zvols
- Wizard prompts for size and creates missing zvols after the dry
  run report, before asking the user to confirm the live run

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix iSCSI portal create: remove port from listen entries fb4a51b24d 
The TrueNAS API rejects port inside listen array items
(iscsi_portal_create.listen.0.port: Extra inputs are not permitted).
Port is a global iSCSI setting, not per-listen-IP.

- _iscsi_portal_payload(): strip port from each listen entry,
  keeping only {"ip": "..."}
- _prompt_iscsi_portals(): remove port prompt from wizard; show
  source IPs without port in the display

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix KeyError: port in portal log display d2dffacb33 
The listen entry display string still referenced l['port'] after
port was stripped from entries. Update to show IP only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix KeyError: port in portal IP conflict detection 1116f2e17a 
_ip_set() used (ip, port) tuples for conflict matching. Since port
is no longer present in listen entries, key on IP only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-run dry run after zvol creation before live run confirmation 32433d6ac8 
When missing zvols are created, the first dry run shows errors
because the zvols don't exist yet. After creation, run a second
dry run so the user sees a clean result before confirming the
live migration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix dry-run iSCSI ID map cascades; add pre-migration existence check 3fd9e6b6a8 
During dry run, "would create" iSCSI objects now populate id_map with
a source-ID placeholder so downstream objects (targets, target-extents)
can remap references without cascading failures.

Adds query_existing_iscsi() and clear_iscsi_config() to migrate.py, and
_prompt_clear_existing_iscsi() to the wizard: if the destination already
has iSCSI config, the user is shown a summary and offered Keep/Remove
before the dry run begins.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add destination audit wizard with selective deletion c28ce9e3b8 
New top-level wizard option (2) lets users inspect and clean up an
existing destination before migration. Queries all SMB shares, NFS
exports, iSCSI objects, datasets, and zvols; displays a structured
inventory report; then offers per-category deletion with escalating
warnings — standard confirm for shares/iSCSI, explicit "DELETE" phrase
required for zvols and datasets to guard against accidental data loss.

Adds to client.py: query_destination_inventory, delete_smb_shares,
delete_nfs_exports, delete_zvols, delete_datasets.
Adds to cli.py: _fmt_bytes, _print_inventory_report, _run_audit_wizard.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix SMB share create: exclude server-generated path_local field 6042dabc8e 
TrueNAS 25.x returns path_local in share query results but rejects it
on create with EINVAL. Added to _SMB_SHARE_READONLY so it is stripped
from the payload before submission.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
scott merged commit 82bff2d341 into main 2026-03-05 16:08:43 -05:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
scott
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: scott/TrueMigration#1
No description provided.
Delete Branch "devel"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?