Skip to content

Latest commit

 

History

History
81 lines (58 loc) · 2.91 KB

data_dir_structure.md

File metadata and controls

81 lines (58 loc) · 2.91 KB

Data Directory Structure

This document describes the layout and naming conventions for the data/ directory used by the program. Organize your data files as follows to ensure seamless operation.

data/
├── backup_failures/
│   └── backup_failures.json
├── nozomi_backups/
│   ├── <STATION_NAME>/
│   │   ├── machine-backup-file.nozomi_backup
│   │   └── ...
├── stations_metadata/
│   ├── general_info.json
│   ├── <station_name>_data.json
│   └── sha256sum.txt
└── stations_secrets/
    ├── encrypted/
    │   └── <station_name>_secrets.json.enc
    └── templates/
        └── <station_name>_secrets.json

backup_failures/

  • Contains a single backup_failures.json file.
  • Records metadata about any failed backup attempts (timestamps, error messagesss, names, IPs).

nozomi_backups/

  • One subfolder per station code (e.g. CENIT, ecopetrol-test).

  • Backups are generated by the n2os-fullbackup script that is present in the Nozomi machines.

  • Backup filename pattern:

    Each backup file follows the pattern

        backup_<hostname>_<YYYYMMDDhhmmss>_<version>.nozomi_backup
    • <hostname>: device hostname (output of hostname command).
    • <YYYYMMDDhhmmss>: timestamp of the backup.
    • <version>: device software version (from n2os-version command).

  • Compression format:

    • The n2os-fullbackup script first creates a tar archive using tar pcf, then compresses it with gzip.
    • The resulting .nozomi_backup file is a gzipped tarball.

  • Decompression:

    • To extract the backup contents, run:

      tar -xzf your-backup-file.nozomi_backup

    • This command will decompress and unpack the archive into the current directory.

stations_metadata/

  • general_info.json: top-level index of all station names and global metadata.
  • <station_name>_data.json: JSON metadata generated from the Excel schema for each station.
  • sha256sum.txt: SHA‑256 checksums for metadata files to verify integrity.

stations_secrets/

  • encrypted/: holds encrypted secret files for each station:

    • <station_name>_secrets.json.enc
    • Decrypt with the program’s key to obtain credentials.

  • templates/: unencrypted JSON templates you can copy and fill in:

    • <station_name>_secrets.json
    • Use these to seed your secret store before encryption.

Tips and Best Practices

  • Prune old backups: periodically remove or archive backups older than your retention policy.
  • Secure secrets: never permanently store unencrypted JSON files from templates/. After running the encryption command on a template file, the plain‑text version is automatically deleted.
  • Consistent station names: folder and/or file names under nozomi_backups/, stations_metadata/, and stations_secrets/ must match Excel sheet names exactly.