-
-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Fix Docker container race condition during upgrades #7603
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Stop and remove existing Coolify containers before starting new ones to prevent conflicts when project name changes from 'source' to 'coolify'. This resolves volume and container name conflicts during upgrades from older installations. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
|
if you're taking down, make sure you run docker pull beforehand, otherwise any error from the registry (rate limit, token expired) will mean it won't come back up again. |
Thanks. For my changes: Not sure if this is the best option, but also not sure what else we can do tbh. |
Ensures images are available before taking down the system. If pull fails (rate limits, network issues, expired tokens), upgrade aborts safely without leaving Coolify in a broken state. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Show clear progress with numbered steps (1/6 through 6/6) - Display header and footer banners - Show individual image pull progress - Show which containers are being stopped - Display final success message with version and log location - Keep detailed logging to file for debugging 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add log() helper for timestamped entries - Add log_section() for clear section headers - Include upgrade metadata at start (version, registry, etc.) - Log each step with clear descriptions - Add completion timestamp at end - Track container operations individually 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Reorder container stop sequence to stop dependencies first (db, redis, realtime) before stopping the main coolify container. This prevents the upgrade process from being interrupted when triggered from Coolify UI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When upgrade is triggered from Coolify UI, the SSH connection is lost when the coolify container stops. Using nohup ensures the container stop/start sequence continues in the background even after the connection drops. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
perhaps only take down and rm if the project is not coolify # --- Pull helper image ---
docker pull "${REGISTRY_URL:-ghcr.io}/coollabsio/coolify-helper:${LATEST_HELPER_VERSION}" \
>> "$LOGFILE" 2>&1 || { echo "Failed to pull helper image. Aborting upgrade." >> "$LOGFILE"; exit 1; }
echo "Running docker compose via helper..." >> "$LOGFILE"
docker run -v /data/coolify/source:/data/coolify/source \
--use-api-socket \
--rm \
"${REGISTRY_URL:-ghcr.io}/coollabsio/coolify-helper:${LATEST_HELPER_VERSION}" \
bash -c "
set -e
LATEST_IMAGE=${LATEST_IMAGE}
# Build compose file arguments dynamically
COMPOSE_FILES='-f /data/coolify/source/docker-compose.yml -f /data/coolify/source/docker-compose.prod.yml'
if [ -f /data/coolify/source/docker-compose.custom.yml ]; then
COMPOSE_FILES=\"\$COMPOSE_FILES -f /data/coolify/source/docker-compose.custom.yml\"
fi
# Step 1: Pull images first
docker compose --project-name coolify \
--env-file /data/coolify/source/.env \
\$COMPOSE_FILES pull
# Step 2: Check project labels of known containers
WRONG_PROJECT=false
for c in coolify coolify-db coolify-redis coolify-realtime; do
if docker ps -a --format '{{.Names}}' | grep -q \"^\$c\$\"; then
PNAME=\$(docker inspect -f '{{ index .Config.Labels \"com.docker.compose.project\" }}' \"\$c\" 2>/dev/null)
if [ \"\$PNAME\" != \"coolify\" ]; then
echo \"Container \$c has wrong project (\$PNAME). Will run compose down.\" >> /proc/1/fd/1
WRONG_PROJECT=true
break
fi
fi
done
# Step 3: If wrong project detected, run down before up
if [ \"\$WRONG_PROJECT\" = true ]; then
docker compose --project-name coolify \
--env-file /data/coolify/source/.env \
\$COMPOSE_FILES down --remove-orphans
fi
# Step 4: Finally bring services up
docker compose --project-name coolify \
--env-file /data/coolify/source/.env \
\$COMPOSE_FILES up -d --remove-orphans --wait --wait-timeout 60
" >> "$LOGFILE" 2>&1
|
Use double quotes for LATEST_IMAGE variable in log output so it expands correctly inside the nohup subshell. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Changes
scripts/upgrade.shandother/nightly/upgrade.shIssues