Repository navigation
feat: add docker-compose version compatibility and refactor deployment scripts - #1197
Merged
Merged
Conversation
…t scripts Add support for both docker-compose 1.x (with dash) and docker compose 2.x (plugin) versions by creating a shared helper library and refactoring all deployment scripts to use it. Changes: - Add scripts/docker-helper.sh with 30+ utility functions - Refactor backup.sh, restore.sh, deploy.sh, rollback.sh to use helper - Replace all hardcoded 'docker compose' commands with dc_run() wrapper - Eliminate ~441 lines of duplicate code across scripts - Standardize on integer boolean flags (0/1 instead of false/true) - Add explicit dependency checks (jq, flock, curl) - Improve cross-platform compatibility (BSD/macOS support) - Add comprehensive refactoring documentation Key Features: - Auto-detects docker-compose version (1.x with dash vs 2.x plugin) - Centralized validation (backup names, image tags, compose file) - Shared service management (health checks, locking, emergency restart) - Atomic file operations for all version/backup reference writes - Telegram notification integration - Git helper functions Security Improvements: - Input validation prevents injection attacks - Path traversal prevention in backup names - Image tag validation prevents malicious tags - Atomic writes for all critical file operations Compatibility: - Works with docker-compose 1.27.4 (production) and 2.x (modern) - Requires: Bash 3.0+, Git 1.8.5+, jq, flock - Optional: curl (for notifications) Fixes issue where deploy.sh failed on production with docker-compose 1.27.4 due to hardcoded 'docker compose' (without dash) commands.
Add additional helper functions to eliminate duplicate code patterns and fix minor issues identified in code review. New Helpers Added: - format_version_short() - Truncate git SHAs for display - determine_backup_name() - Consolidate backup name determination logic - validate_and_check_backup() - Wrapper for backup validation with error handling - require_services_stopped() - Check and exit if services running - determine_previous_version() - Consolidate version fallback logic - checkout_git_version() - Safe git checkout with validation Constants Added: - HEALTH_CHECK_TIMEOUT_DEPLOY (300s) - HEALTH_CHECK_TIMEOUT_ROLLBACK (120s) - HEALTH_CHECK_TIMEOUT_DEFAULT (300s) Improvements: - backup.sh: Remove eval usage, replace with direct command execution - backup.sh: Add documentation for SERVICES_STOPPED_LOCAL variable - restore.sh: Fix hardcoded "docker compose" in help text - restore.sh: Use new helpers (61 lines → 34 lines, 44% reduction) - rollback.sh: Use new helpers (complex logic simplified) - deploy.sh: Use constants and helpers for consistency Code Reduction: - Eliminated 105 lines of duplicate code - Improved readability and maintainability - All validation and error handling now centralized Security: - Removed eval usage (eliminates potential command injection vector) - All validation centralized in helper functions
Critical fixes for production deployment:
1. **Docker-compose version detection priority**
- Now prefers docker compose 2.x over docker-compose 1.x
- Detects plugin version first, falls back to standalone
2. **JSON format compatibility layer**
- Added supports_json_format() to detect --format json support
- Added version-aware helper functions:
- get_running_services_count() - works with v1.x and v2.x
- get_unhealthy_services() - JSON or text parsing fallback
- get_starting_services() - JSON or text parsing fallback
3. **Backward compatibility for docker-compose 1.27.4**
- Health checks now gracefully skip on v1.27.4 (no JSON support)
- Service counting uses grep fallback for text output
- Warning logs inform users of limited health check capabilities
4. **Removed documentation file**
- Removed scripts/REFACTORING_SUMMARY.md (internal doc)
Impact:
- Scripts now work with docker-compose 1.27.4 (production version)
- Maintains full functionality with docker compose 2.x
- No breaking changes to external interfaces
- Health checks degrade gracefully on older versions
Fixes the critical issue where --format json flag is not supported
in docker-compose v1.27.4 (added in v1.28.0).
This commit addresses 5 critical issues identified in code review: 1. **Fix grep pattern bug in get_running_services_count()** - Changed pattern from " Up " to " Up" (removed trailing space) - Now correctly matches services with health checks on docker-compose 1.x - Fixes: "Up (healthy)", "Up (health: starting)", "Up 5 seconds" - Impact: require_services_stopped() will now work correctly with healthchecks 2. **Add sourcing validation for docker-helper.sh** - All scripts now validate helper library exists before sourcing - Added error handling if sourcing fails with clear error messages - Prevents cryptic bash errors if helper file missing or corrupted - Added shellcheck source directive for static analysis 3. **Prevent trap double-firing in deploy→rollback scenario** - Added mark_services_running() before calling rollback.sh - Disarms emergency restart trap in deploy.sh - Prevents duplicate service restart attempts - Eliminates service state management conflicts 4. **Add atomic write error handling** - Added proper error handling to atomic_write() function - Includes PID in temp filename for uniqueness (*.tmp.$$) - Cleans up temp files on failure - Returns clear error codes with helpful messages - Prevents silent failures on disk full or permission issues 5. **Fix error logging in quiet mode** - Created new error() function never suppressed by QUIET mode - Always outputs to stderr and log file - Updated all validation functions to use error() - Ensures critical errors are always visible BONUS: Made services_running() version-aware - Now uses get_running_services_count() instead of hardcoded JSON - Works with both docker-compose 1.x and 2.x - Simplified implementation using existing version-aware helper These fixes improve robustness, error handling, and version compatibility without breaking any existing functionality.
This commit completes the robustness fixes from commit 1cdc1a1 by addressing two critical issues identified in code review: 1. **Fix grep pattern bug in get_running_services_count()** - Changed from: `grep -c " Up"` - Changed to: `grep -cE " Up\b"` (word boundary) - FIXES: Previous pattern caused false positives on stopped containers with "Up" in command name (e.g., "UpgradeDB", "Uploader") - Now correctly matches only State column, not Command column - Prevents operational issues where backup/restore blocked incorrectly 2. **Complete error() function migration (5 critical errors)** - Updated deploy.sh (3 errors): * "Failed to pull images from GHCR" (line 194) * "Backup failed" (line 213) * "Services unhealthy after timeout" (line 249) - Updated rollback.sh (2 errors): * "Backup restoration failed" (line 172) * "Rollback health check failed" (line 213) - FIXES: Critical operational errors now visible in quiet mode - All deployment/rollback failures now properly logged 3. **Consistency improvements (7 helper functions)** - Updated to use error() instead of direct stderr: * acquire_lock() - flock requirement and lock conflict errors * require_jq() - dependency check * require_curl() - dependency check * validate_compose_file() - file existence check * determine_backup_name() - missing backup error * validate_and_check_backup() - removed duplicate error message * require_services_stopped() - services running error - Ensures consistent error handling and logging across all functions - All errors now include timestamps and log to LOG_FILE **Testing:** - Grep pattern tested against docker-compose output formats: * Simple "Up" (end of line) - ✅ matches * "Up (healthy)" - ✅ matches * "Up 5 seconds" - ✅ matches * "Up ports..." - ✅ matches * "UpgradeDB" in command - ✅ does NOT match (false positive fixed) - Error logging tested in quiet mode - all errors visible **Impact:** - Eliminates grep pattern false positive bug - Ensures critical errors never suppressed in quiet mode - Improves operational visibility and troubleshooting - No breaking changes to functionality These fixes complete the robustness improvements and make the code production-ready for merge.
…ripts Based on multi-aspect code review findings, this commit addresses several medium-priority improvements to enhance error handling, security, and portability across all deployment scripts. ## Changes ### Error Handling Standardization - Use `error()` function consistently instead of `echo "ERROR: ..." >&2` - Consolidate error messages with installation instructions in single calls - Improve error logging in `atomic_write()`, `require_jq()`, `require_curl()` - Add proper error handling for service restart failures in backup.sh ### Security Improvements - Make `DOCKER_COMPOSE_CMD` readonly to prevent modification - Quote `$DOCKER_COMPOSE_CMD` in `dc_run()` to prevent word splitting - Remove unnecessary `export` of secrets (`TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID`) ### Code Quality - Rename `validate_and_check_backup()` to `validate_backup_or_exit()` for clarity about exit behavior - Consolidate duplicate backup error handling in backup.sh - Replace non-portable `grep -P` with `grep -o | sed` for BSD/macOS compatibility ### Files Modified - scripts/docker-helper.sh: Core improvements to helper functions - scripts/backup.sh: Consolidated error handling, portable grep - scripts/restore.sh: Consistent error function usage - scripts/rollback.sh: Updated function name All changes maintain backward compatibility and pass syntax validation. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…ose-compatibility
groupsky
added a commit
that referenced
this pull request
Jan 27, 2026
…ts (#1200) Adds automated testing infrastructure using BATS (Bash Automated Testing System) to ensure deployment scripts reliability, prevent regressions, and enable confident refactoring. This implements the test suite recommended in PR #1197's code quality review. ## Test Coverage - **55 comprehensive tests** (54 passing, 1 skipped with documentation) - docker-helper.sh: 36 tests (version detection, logging, validation, atomic operations, security) - backup.sh: 10 tests (CLI parsing, validation, backup operations, error handling) - restore.sh: 9 tests (CLI parsing, validation, restore operations, service checks) - Fast execution (~30 seconds) - No Docker required (fully mocked) ## Test Infrastructure **BATS Framework (Git Submodules):** - bats-core v1.11.0 - Core test framework - bats-support - Additional test helpers - bats-assert - Rich assertion library **Test Patterns:** - Arrange-Act-Assert structure - Positive and negative test cases - Error message validation - Exit code verification - Isolated test environments with automatic cleanup ## CI/CD Integration **Workflow:** `.github/workflows/deployment-scripts-tests.yml` **Security & Performance:** - Pinned actions to SHA commits (supply chain security) - 10-minute timeout for tests, 5-minute for validation - Concurrency control (cancels outdated PR runs) - Minimal permissions (contents: read) **Jobs:** 1. Run BATS tests on all test suites 2. Validate bash syntax with shellcheck **Triggers:** - Pull requests modifying scripts/ - Push to master branch - Manual workflow dispatch ## Documentation - Comprehensive README in scripts/tests/README.md - Main CLAUDE.md updated with test framework section - Test patterns and best practices documented - Local development instructions - Troubleshooting guide ## Key Improvements **Testing Quality:** - Mock-based approach enables fast, isolated testing - Comprehensive coverage of critical deployment paths - Both happy paths and error scenarios tested - Validates input sanitization and path traversal prevention **Security:** - Tests verify readonly DOCKER_COMPOSE_CMD protection - Validates proper error function usage - Ensures quiet mode doesn't suppress errors - PATH manipulation for test isolation (no global function overrides) **Maintainability:** - Clean test structure following project TDD guidelines - Reusable test helpers and mocking infrastructure - Well-documented test patterns - CI integration prevents regressions ## Related - Implements recommendations from PR #1197 code quality review - Addresses "No test suite" issue identified in multi-aspect review - Provides foundation for future test expansion (deploy.sh, rollback.sh) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for both docker-compose 1.x (with dash) and docker compose 2.x (plugin) versions by creating a shared helper library and refactoring all deployment scripts to use it.
Fixes production deployment issue where scripts failed with docker-compose 1.27.4 due to hardcoded
docker compose(without dash) commands and missing JSON format support.Changes
Commit 1: Initial Refactoring (2558d9d)
New File:
scripts/docker-helper.shRefactored Scripts:
backup.sh: Reduced by ~60 linesrestore.sh: Reduced by ~61 linesdeploy.sh: Reduced by ~113 lines (31% reduction)rollback.sh: Reduced by ~100+ linesKey Improvements:
docker composecommands replaced withdc_run()wrapperCommit 2: Follow-up Improvements (f08494a)
New High-Priority Helpers:
format_version_short()- Truncate git SHAs for displaydetermine_backup_name()- Consolidate backup name determinationvalidate_backup_or_exit()- Wrapper for validation with error handlingrequire_services_stopped()- Check and exit if services runningdetermine_previous_version()- Consolidate version fallback logiccheckout_git_version()- Safe git checkout with validationConstants Added:
HEALTH_CHECK_TIMEOUT_DEPLOY(300s)HEALTH_CHECK_TIMEOUT_ROLLBACK(120s)HEALTH_CHECK_TIMEOUT_DEFAULT(300s)Bug Fixes:
SERVICES_STOPPED_LOCALvariable$DOCKER_COMPOSE_CMDCommit 3: Docker-Compose v1.27.4 Compatibility (986158a)
CRITICAL FIX: JSON Format Fallback
The
--format jsonflag was added in docker-compose v1.28.0 (Nov 2020). Production runs v1.27.4 (Jun 2020) which doesn't support this flag, causing all health checks to fail.Solution:
supports_json_format()to detect JSON supportget_running_services_count()- Uses JSON or grep fallbackget_unhealthy_services()- Uses JSON or skips gracefullyget_starting_services()- Uses JSON or skips gracefullywait_for_health()to skip health checks on v1.27.4 with warningrequire_services_stopped()to use grep fallbackVersion Detection Priority:
docker compose(2.x plugin) overdocker-compose(1.x standalone)Impact:
Commit 4: Critical Robustness Fixes (1cdc1a1)
Addresses code review findings - 5 critical fixes:
Fixed grep pattern bug (
get_running_services_count)" Up "to" Up\b"(added word boundary)Up (healthy),Up (health: starting)Added sourcing validation (all scripts)
docker-helper.shexists before sourcingPrevented trap double-firing (deploy.sh → rollback.sh)
Added atomic write error handling (
atomic_write())*.tmp.$$)Fixed error logging in quiet mode
error()function never suppressed byQUIETmodeerror()BONUS:
services_running()version-aware (now usesget_running_services_count())Commit 5: Complete Error Logging Migration (9f46ca0)
Remaining grep pattern fixes:
get_running_services_count()to use word boundary\bconsistentlyCommit 6: Error Handling and Robustness Improvements (fecffa5)
Based on multi-aspect code review findings:
Error Handling Standardization:
error()function consistently instead ofecho "ERROR: ..." >&2atomic_write(),require_jq(),require_curl()Security Improvements:
DOCKER_COMPOSE_CMDreadonly to prevent modification$DOCKER_COMPOSE_CMDindc_run()to prevent word splittingexportof secrets (TELEGRAM_BOT_TOKEN,TELEGRAM_CHAT_ID)Code Quality:
validate_and_check_backup()tovalidate_backup_or_exit()for claritygrep -Pwithgrep -o | sedfor BSD/macOS compatibilityCode Reduction
Testing
bash -n)DOCKER_COMPOSE_CMDverified as readonlyCompatibility
Minimum Requirements:
Supported Docker Compose Versions:
Platform Support:
Security Improvements
DOCKER_COMPOSE_CMDis readonly and properly quotedBreaking Changes
None. All external interfaces remain unchanged.
Migration Notes
Scripts automatically detect and use the appropriate docker-compose command. No manual configuration needed.
If both docker-compose 1.x and docker compose 2.x are installed, the scripts will prefer the 2.x plugin version.
Related Issues
Fixes production deployment issue where scripts failed on docker-compose 1.27.4 due to:
docker compose(without dash) commands--format jsonflag not supported until v1.28.0