CLI Reference#

Usage#

compress-hdf5 <command> [options]

Note

The positional arguments <experiment> [<beamline>] [<session>] apply to the list command only. Other commands operate on a report file via --input.

Global Options (for list)#

  • --root <path> Base directory under which experiments reside (default: /data/visitor).

  • --filter KEY:VAL[,KEY2:VAL2...] Dataset-level attribute substring filters.

  • --output <file> Report file (default: file_list.txt).

Commands#

list Report VDS source files into “TO COMPRESS” and “REMAINING” sections. Usage:

compress-hdf5 list <experiment> [<beamline>] [<session>]
                  [--root <path>] [--filter KEY:VAL[,…]]
                  [--output <file>]

compress Compress files listed in a report. Usage:

compress-hdf5 compress --input <report_file> [--cratio <N>] [--method <meth>] [--layout <mode>]

  • --input, -i: Report file to read (required; must be produced by list).

  • --cratio: Compression ratio for JPEG2000 (default: 10).

  • --method: Compression method (choices: jp2k; default: jp2k).

  • --layout: Output layout for compressed files (choices: sibling, mirror; default: mirror). - sibling: write <basename>_<method>.h5 next to each source file. - mirror: mirror RAW_DATA/... to RAW_DATA_COMPRESSED/..., copy non-compressed folders/files, and write compressed source files with their original names.

check Generate SSIM report for files listed in a report. Usage:

compress-hdf5 check [--input <report_file>] [--method <meth>] [--layout <mode>]

  • --input, -i: Report file to read (default: file_list.txt).

  • --method: Compression method (choices: jp2k; default: jp2k).

  • --layout: Where compressed files are expected (sibling or mirror; default: mirror).

overwrite Swap raw files with their compressed siblings and keep backups; with --final or --undo, perform cleanup/restore only. Usage:

compress-hdf5 overwrite [--input <report_file>] [--final | --undo]

  • --input, -i: Report file to read (default: file_list.txt).

  • --final: Cleanup only. Do not overwrite. Scan for corresponding *.h5.bak files, show their count and total size, and prompt to delete them.

  • --undo: Restore only. Do not overwrite. For each listed path with a <file>.h5.bak, move it back to <file>.h5. The current <file>.h5 is preserved: if <file>_<method>.h5 does not exist, it is created by moving the current <file>.h5 there; otherwise it is left as-is.

Behavior:

  • No flag: For each <file>.h5 that has <file>_<method>.h5, rename <file>.h5 → <file>.h5.bak then rename <file>_<method>.h5 → <file>.h5. Backup files are kept.

  • --final: No file swapping. Only remove existing backups after confirmation.

  • --undo: No file swapping. Only restore from backups as described above.

Examples#

$ compress-hdf5 list ma5567 --root /data/visitor
Report written to file_list.txt

$ compress-hdf5 compress --input file_list.txt --cratio 15 --method jp2k --layout mirror
Compressing 5 file(s) from 'file_list.txt' using 'jp2k' method, ratio 15, layout 'mirror' …
Compression complete.

$ compress-hdf5 check --input file_list.txt --method jp2k --layout mirror
SSIM report written to /full/path/file_list_jp2k_ssim_report.txt

$ compress-hdf5 overwrite --input file_list.txt
Overwriting 5 file(s) …
Overwrite complete (backups kept).

$ compress-hdf5 overwrite --input file_list.txt --undo
Restored 5 file(s) from backups (compressed versions preserved).

$ compress-hdf5 overwrite --input file_list.txt --final
About to remove 5 backup file(s), ~1234.56 MB total.
Proceed? [y/N]: y
Deleted 5 backup file(s).