Commit Graph

486 Commits

Author SHA1 Message Date
Andras Bacsai
e6ed3130b5 feat(stripe): Add manual subscription sync command with dry-run support
Add cloud:sync-stripe-subscriptions command to manually check all
subscriptions against Stripe. By default it only reports discrepancies
without making changes. Use --fix flag to actually apply corrections.

This addresses race conditions where subscriptions can be cancelled in
Stripe but remain marked as active in Coolify's database.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2025-12-19 09:34:39 +01:00
Andras Bacsai
3962f1a5b9 perf: Remove dead server filtering code from Kernel scheduler
Remove unused server filtering logic in Kernel.php that was querying servers
but never using the results. Simplify Sentinel update checks in ServerManagerJob
by reusing the $isSentinelEnabled variable and removing unnecessary timezone
parameter for hourly cron execution.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-11 10:33:55 +01:00
Andras Bacsai
945cce9587 feat: Add scheduled job to cleanup orphaned PR containers
Add CleanupOrphanedPreviewContainersJob that runs daily to find and remove any PR preview containers that weren't properly cleaned up when their PR was closed.

The job:
- Scans all functional servers for containers with coolify.pullRequestId label
- Checks if the corresponding ApplicationPreview record exists in the database
- Removes containers where the preview record no longer exists (truly orphaned)
- Acts as a safety net for webhook failures or race conditions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-08 17:15:52 +01:00
Andras Bacsai
4002044877 Refactor: Move sentinel update checks to ServerManagerJob and add tests for hourly dispatch 2025-12-04 14:58:18 +01:00
Andras Bacsai
4052d1bd05 Refactor color classes from yellow to warning across the application
- Updated color classes in NotifyDemo.php to use warning colors.
- Added new warning color variables in app.css.
- Changed warning icon colors in callout.blade.php.
- Updated loading spinner and hover states in global-search.blade.php.
- Refactored warning messages and styles in project application views.
- Adjusted log display colors in get-logs.blade.php.
- Updated private key status indicators in index.blade.php.
- Changed hover and text colors for documentation links in cloudflare-tunnel.blade.php.
- Refactored server creation messages in by-hetzner.blade.php.
- Updated proxy warning button colors in proxy.blade.php.
- Changed loading spinner colors in show.blade.php.
- Updated deployment status colors in deployments.blade.php and show.blade.php.
2025-11-28 16:23:32 +01:00
Andras Bacsai
48c3daae88 fix: improve error handling and output capturing during Git operations in SyncBunny command 2025-11-27 10:51:25 +01:00
Andras Bacsai
d27d697b37 fix: log warning on backup failure during name cleanup process 2025-11-27 10:51:19 +01:00
Andras Bacsai
9aea159a24 feat: add functionality to sync releases.json and versions.json to GitHub in one PR 2025-11-26 14:43:25 +01:00
Andras Bacsai
c25272de8d reduce: minimize logging in cleanup commands
Reduce excessive logging in CleanupRedis and CleanupNames commands to output only a single summary line. Remove per-item logs and detailed status messages while keeping the final count of items cleaned up. Detail logs still available in dry-run mode for preview.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-26 14:19:13 +01:00
Andras Bacsai
f6d8ad1775 feat: add support for syncing versions.json to GitHub repository via PR 2025-11-26 10:52:20 +01:00
Andras Bacsai
8c77c63043 feat(proxy): add Traefik version tracking with notifications and dismissible UI warnings
- Add automated Traefik version checking job running weekly on Sundays
- Implement version detection from running containers and comparison with versions.json
- Add notifications across all channels (Email, Discord, Slack, Telegram, Pushover, Webhook) for outdated versions
- Create dismissible callout component with localStorage persistence
- Display cross-branch upgrade warnings (e.g., v3.5 -> v3.6) with changelog links
- Show patch update notifications within same branch
- Add warning icon that appears when callouts are dismissed
- Prevent duplicate notifications during proxy restart by adding restarting parameter
- Fix notification spam with transition-based logic for status changes
- Enable system email settings by default in development mode
- Track last saved/applied proxy settings to detect configuration drift
2025-11-14 11:35:22 +01:00
Andras Bacsai
6202803db2 fix(CleanupRedis): guard against scan() returning false and use lowercase option keys
- Change Redis scan() option keys from uppercase (MATCH, COUNT) to lowercase (match, count) to comply with PhpRedis requirements
- Add guard to handle scan() returning false and display error message
- Add comprehensive test coverage for scan() error handling scenarios
2025-11-11 21:22:29 +01:00
Andras Bacsai
ad69758c56 refactor(CleanupRedis): remove JSON decode error handling from cleanupStuckJobs method 2025-11-11 20:54:25 +01:00
Andras Bacsai
b79aa1b195 refactor(CleanupRedis): optimize key retrieval in cleanupStuckJobs using Redis scan 2025-11-11 15:41:05 +01:00
Andras Bacsai
a95e92f098 feat(CleanupRedis): add error handling for JSON decode failures in cleanupStuckJobs method 2025-11-11 15:40:11 +01:00
Andras Bacsai
eb70fe00ff feat(CleanupRedis): add error handling for JSON decode failures in cleanupStuckJobs method 2025-11-11 15:36:34 +01:00
Andras Bacsai
334892d1ff feat(BackupNotification): include database name in BackupFailed notification for better context 2025-11-11 15:27:57 +01:00
Andras Bacsai
684a08bf75 feat(CleanupRedis): improve stuck job cleanup logic by prioritizing reserved_at timestamp 2025-11-11 15:27:52 +01:00
Andras Bacsai
0cfce06869 feat(Cleanup): implement failure marking for stuck scheduled tasks and database backups during startup 2025-11-11 12:32:52 +01:00
Andras Bacsai
f9ab2a7ca8
Merge branch 'next' into improve-scheduled-tasks 2025-11-11 11:32:15 +01:00
Andras Bacsai
f1d80d6776 fix: enhance error handling in initialization and cleanup process 2025-11-10 15:29:26 +01:00
Andras Bacsai
b22e79caec feat(jobs): improve scheduled tasks with retry logic and queue cleanup
- Add retry configuration to CoolifyTask (3 tries, 600s timeout)
- Add retry configuration to ScheduledTaskJob (3 tries, configurable timeout)
- Add retry configuration to DatabaseBackupJob (2 tries)
- Implement exponential backoff for all jobs (30s, 60s, 120s intervals)
- Add failed() handlers with comprehensive error logging to scheduled-errors channel
- Add execution tracking: started_at, retry_count, duration (decimal), error_details
- Add configurable timeout field to scheduled tasks (60-3600s, default 300s)
- Update UI to include timeout configuration in task creation/editing forms
- Increase ScheduledJobManager lock expiration from 60s to 90s for high-load environments
- Implement safe queue cleanup with restart vs runtime modes
  - Restart mode: aggressive cleanup (marks all processing jobs as failed)
  - Runtime mode: conservative cleanup (only marks jobs >12h as failed, skips deployments)
- Add cleanup:redis --restart flag for system startup
- Integrate cleanup into Dev.php init() for development environment
- Increase scheduled-errors log retention from 7 to 14 days
- Create comprehensive test suite (unit and feature tests)
- Add TESTING_GUIDE.md with manual testing instructions

Fixes issues with jobs failing after single attempt and "attempted too many times" errors
2025-11-10 11:11:18 +01:00
Andras Bacsai
4e734492e0 fix: escape shell arguments in syncBunny command execution 2025-11-06 14:57:53 +01:00
Andras Bacsai
24bcce3f9b
Update app/Console/Commands/SyncBunny.php
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-11-06 14:36:34 +01:00
Andras Bacsai
23a3b47011 refactor: rename sync function and improve error handling
The function previously named syncGitHubReleases has been renamed to syncReleasesToGitHubRepo for clarity, as it now focuses on syncing releases directly to the GitHub repository instead of the CDN. Additionally, error handling has been enhanced to provide more informative messages during the cloning, branching, and committing processes. This refactor aims to improve the maintainability of the code and ensure better feedback in case of failures.
2025-11-05 14:54:13 +01:00
Andras Bacsai
0865ecd3db refactor: move RestoreDatabase command to Cloud namespace
This change organizes the command within the appropriate Cloud namespace, improving code structure and maintainability. By grouping related commands together, it enhances clarity for future developers and helps in locating files more efficiently.
2025-11-05 14:42:12 +01:00
Andras Bacsai
d291d85311 feat: add RestoreDatabase command for PostgreSQL dump restoration 2025-11-03 13:02:14 +01:00
Andras Bacsai
ea649d2a85 Add artisan command to update service Docker image versions
This command queries Docker registries (Docker Hub, GHCR, Quay, Codeberg) to find and update Docker image versions in service template files.

Features:
- Automatically updates 'latest' tags to semantic versions using digest matching
- Supports multiple version formats: semantic (1.2.3), date-based (2025.10.20), RELEASE timestamps
- Prefers shorter version tags (1.8 over 1.8.1) when both available
- In-memory caching to avoid duplicate API queries for same images
- Detects and reports services with available major version updates
- Preserves YAML formatting and comments
- Supports dry-run mode for preview

Usage:
  php artisan services:update-versions [--dry-run] [--service=name]

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 08:29:11 +01:00
Andras Bacsai
9f87d499dd
Update app/Console/Commands/CleanupRedis.php
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-10-24 13:20:24 +02:00
Andras Bacsai
c6a2d1fe0a Fix stale lock issue causing scheduled tasks to stop (#4539)
## Problem
Scheduled tasks, backups, and auto-updates stopped working after 1-2 months
with error: MaxAttemptsExceededException: App\Jobs\ScheduledJobManager has
been attempted too many times.

Root cause: ScheduledJobManager used WithoutOverlapping with only
releaseAfter(60), causing locks without expiration (TTL=-1) that persisted
indefinitely when jobs hung or processes crashed.

## Solution

### Part 1: Prevention (Future Locks)
- Added expireAfter(60) to ScheduledJobManager middleware
- Lock now auto-expires after 60 seconds (matches everyMinute schedule)
- Changed from releaseAfter(60) to expireAfter(60)->dontRelease()
- Follows Laravel best practices and matches other Coolify jobs

### Part 2: Recovery (Existing Locks)
- Enhanced cleanup:redis command with --clear-locks flag
- Scans Redis for stale locks (TTL=-1) and removes them
- Called automatically during app:init on startup/upgrade
- Provides immediate recovery for affected instances

## Changes
- app/Jobs/ScheduledJobManager.php: Added expireAfter(60)->dontRelease()
- app/Console/Commands/CleanupRedis.php: Added cleanupCacheLocks() method
- app/Console/Commands/Init.php: Auto-clear locks on startup
- tests/Unit/ScheduledJobManagerLockTest.php: Test to prevent regression
- STALE_LOCK_FIX.md: Complete documentation

## Testing
- Unit tests pass (2 tests, 8 assertions)
- Code formatted with Pint
- Matches pattern used by CleanupInstanceStuffsJob

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-23 10:07:33 +02:00
Andras Bacsai
2b51363b8c Changes auto-committed by Conductor 2025-10-16 17:23:22 +02:00
Andras Bacsai
975d1b8a6b Changes auto-committed by Conductor 2025-10-16 17:13:47 +02:00
Andras Bacsai
64c4ce210e feat: add artisan command to clear global search cache
Add a new artisan command for manually clearing the global search cache
during development and testing. This is useful when testing new navigation
entries or updates to searchable resources without waiting for the 5-minute
cache TTL.

Command: php artisan search:clear

Usage options:
- search:clear              - Clear cache for current user's team
- search:clear --team=1     - Clear cache for specific team ID
- search:clear --all        - Clear cache for all teams

This helps developers test global search changes immediately, especially
when adding new navigation routes like cloud-init scripts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-11 13:36:14 +02:00
Andras Bacsai
da93a52976 feat(cleanup): add force deletion of stuck servers and orphaned SSL certificates 2025-10-09 16:37:56 +02:00
Andras Bacsai
72f5ae0dc6 feat(user-deletion): implement file locking to prevent concurrent user deletions and enhance error handling 2025-09-29 14:03:49 +02:00
Andras Bacsai
106682b5b8 refactor(cloud-commands): consolidate and enhance subscription management commands
- Deleted obsolete CloudCheckSubscription, CloudCleanupSubscriptions, and CloudDeleteUser commands to streamline the codebase.
- Introduced new CloudDeleteUser and CloudFixSubscription commands with improved functionality for user deletion and subscription management.
- Enhanced subscription handling with options for fixing canceled subscriptions and verifying active subscriptions against Stripe, improving overall command usability and control.
2025-09-23 14:56:58 +02:00
Andras Bacsai
95453bfaaa feat(cloud-check): enhance CloudCheckSubscription command with fix options
- Added options to the CloudCheckSubscription command for fixing canceled subscriptions in the database.
- Implemented a dry-run mode to preview changes without applying them.
- Introduced a flag to limit checks/fixes to the first found subscription, improving command usability and control.
2025-09-23 11:00:24 +02:00
Andras Bacsai
88fa6a4a56 feat(cloud-check): enhance subscription reporting in CloudCheckSubscription command
- Added CSV output for active subscribers, including detailed information on subscription status and invoice status.
- Implemented checks for missing subscription IDs and provided appropriate logging in the CSV for visibility.
- Improved handling of non-active subscriptions with detailed output for better tracking and analysis.
2025-09-23 09:50:44 +02:00
Andras Bacsai
a2a2bfa6c9 feat(user-management): implement user deletion command with phased resource and subscription cancellation, including dry run option 2025-09-13 15:08:30 +02:00
peaklabs-dev
3e9dd6a7bf
chore: remove webhooks table cleanup 2025-09-10 19:56:15 +02:00
Andras Bacsai
a06c79776e feat(dev-command): dispatch CheckHelperImageJob during instance initialization to enhance setup process 2025-09-09 16:46:24 +02:00
Andras Bacsai
2d135071c7 refactor(command): replace forceDelete calls with DeleteResourceJob dispatch for all stuck resources in cleanup process 2025-09-05 19:28:08 +02:00
Andras Bacsai
136ca08305 refactor(command): streamline Init command by removing unnecessary options and enhancing error handling for various operations 2025-09-05 19:27:49 +02:00
Andras Bacsai
28d05f759e refactor(command): remove InitChangelog command as it is no longer needed 2025-09-05 17:45:15 +02:00
Andras Bacsai
49bd0a2a01 refactor(command): improve database collection logic for deletion command by using unique identifiers and enhancing user experience 2025-09-05 17:43:19 +02:00
Andras Bacsai
581b649cd7 fix(command): enhance cleanup process for stuck application previews by adding force delete for trashed records 2025-09-05 17:43:05 +02:00
Andras Bacsai
16447b7391 refactor(command): streamline database deletion process to handle multiple database types and improve user experience 2025-09-05 14:56:30 +02:00
Andras Bacsai
b17c65b224 fix(command): enhance database deletion command to support multiple database types 2025-09-05 14:55:15 +02:00
Andras Bacsai
0bfffb503e feat(command): add option to sync GitHub releases to BunnyCDN and refactor sync logic 2025-09-01 16:14:02 +02:00
Andras Bacsai
d2ed68e605 refactor(jobs): pull github changelogs from cdn instead of github 2025-09-01 16:13:55 +02:00