mirror of
https://github.com/coollabsio/coolify.git
synced 2025-12-28 05:34:50 +00:00
feat(tests): add comprehensive tests for ContainerStatusAggregator and serverStatus accessor
- Introduced tests for ContainerStatusAggregator to validate status aggregation logic across various container states. - Implemented tests to ensure serverStatus accessor correctly checks server infrastructure health without being affected by container status. - Updated ExcludeFromHealthCheckTest to verify excluded status handling in various components. - Removed obsolete PushServerUpdateJobStatusAggregationTest as its functionality is covered elsewhere. - Updated version number for sentinel to 0.0.17 in versions.json.
This commit is contained in:
Andras Bacsai
parent
70fb4c6869
commit
ae6eef3cdb
@ -514,7 +514,7 @@ All status aggregation locations **MUST** follow the same priority:
|
||||
|
||||
### **Excluded Containers**
|
||||
|
||||
When containers have `exclude_from_hc: true` flag:
|
||||
When containers have `exclude_from_hc: true` flag or `restart: no`:
|
||||
|
||||
**Behavior**:
|
||||
- Status is still calculated from container state
|
||||
@ -525,47 +525,80 @@ When containers have `exclude_from_hc: true` flag:
|
||||
**Format**: `{actual-status}:excluded`
|
||||
**Examples**: `running:unknown:excluded`, `degraded:excluded`, `exited:excluded`
|
||||
|
||||
**All-Excluded Scenario**:
|
||||
When ALL containers are excluded from health checks:
|
||||
- All three status update paths (PushServerUpdateJob, GetContainersStatus, ComplexStatusCheck) **MUST** calculate status from excluded containers
|
||||
- Status is returned with `:excluded` suffix (e.g., `running:healthy:excluded`)
|
||||
- **NEVER** skip status updates - always calculate from excluded containers
|
||||
- This ensures consistent status regardless of which update mechanism runs
|
||||
- Shared logic is in `app/Traits/CalculatesExcludedStatus.php`
|
||||
|
||||
### **Important Notes for Developers**
|
||||
|
||||
⚠️ **CRITICAL**: When modifying container status logic:
|
||||
✅ **Container Status Aggregation Service**:
|
||||
|
||||
1. **Update ALL four locations**:
|
||||
- `GetContainersStatus.php` (SSH-based)
|
||||
- `PushServerUpdateJob.php` (Sentinel-based)
|
||||
- `ComplexStatusCheck.php` (multi-server)
|
||||
- `Service.php` (service-level)
|
||||
The container status aggregation logic is centralized in `App\Services\ContainerStatusAggregator`.
|
||||
|
||||
2. **Maintain consistent priority**:
|
||||
- unhealthy > unknown > healthy
|
||||
- Apply same logic across all paths
|
||||
**Status Format Standard**:
|
||||
- **Backend/Storage**: Colon format (`running:healthy`, `degraded:unhealthy`)
|
||||
- **UI/Display**: Transform to human format (`Running (Healthy)`, `Degraded (Unhealthy)`)
|
||||
|
||||
1. **Using the ContainerStatusAggregator Service**:
|
||||
- Import `App\Services\ContainerStatusAggregator` in any class needing status aggregation
|
||||
- Two methods available:
|
||||
- `aggregateFromStrings(Collection $statusStrings, int $maxRestartCount = 0)` - For pre-formatted status strings
|
||||
- `aggregateFromContainers(Collection $containers, int $maxRestartCount = 0)` - For raw Docker container objects
|
||||
- Returns colon format: `running:healthy`, `degraded:unhealthy`, etc.
|
||||
- Automatically handles crash loop detection via `$maxRestartCount` parameter
|
||||
|
||||
2. **State Machine Priority** (handled by service):
|
||||
- Restarting → `degraded:unhealthy` (highest priority)
|
||||
- Crash loop (exited with restarts) → `degraded:unhealthy`
|
||||
- Mixed state (running + exited) → `degraded:unhealthy`
|
||||
- Running → `running:unhealthy` / `running:unknown` / `running:healthy`
|
||||
- Dead/Removing → `degraded:unhealthy`
|
||||
- Paused → `paused:unknown`
|
||||
- Starting/Created → `starting:unknown`
|
||||
- Exited → `exited:unhealthy` (lowest priority)
|
||||
|
||||
3. **Test both update paths**:
|
||||
- Run unit tests: `./vendor/bin/pest tests/Unit/`
|
||||
- Run unit tests: `./vendor/bin/pest tests/Unit/ContainerStatusAggregatorTest.php`
|
||||
- Run integration tests: `./vendor/bin/pest tests/Unit/`
|
||||
- Test SSH updates (manual refresh)
|
||||
- Test Sentinel updates (wait 30 seconds)
|
||||
|
||||
4. **Handle edge cases**:
|
||||
- All containers excluded (`exclude_from_hc: true`)
|
||||
- Mixed excluded/non-excluded containers
|
||||
- Unknown health states
|
||||
- Container crash loops (restart count)
|
||||
4. **Handle excluded containers**:
|
||||
- All containers excluded (`exclude_from_hc: true`) - Use `CalculatesExcludedStatus` trait
|
||||
- Mixed excluded/non-excluded containers - Filter then use `ContainerStatusAggregator`
|
||||
- Containers with `restart: no` - Treated same as `exclude_from_hc: true`
|
||||
|
||||
5. **Use shared trait for excluded containers**:
|
||||
- Import `App\Traits\CalculatesExcludedStatus` in status calculation classes
|
||||
- Use `getExcludedContainersFromDockerCompose()` to parse exclusions
|
||||
- Use `calculateExcludedStatus()` for full Docker inspect objects (ComplexStatusCheck)
|
||||
- Use `calculateExcludedStatusFromStrings()` for status strings (PushServerUpdateJob, GetContainersStatus)
|
||||
|
||||
### **Related Tests**
|
||||
|
||||
- **[tests/Unit/ContainerHealthStatusTest.php](mdc:tests/Unit/ContainerHealthStatusTest.php)**: Health status aggregation
|
||||
- **[tests/Unit/ContainerStatusAggregatorTest.php](mdc:tests/Unit/ContainerStatusAggregatorTest.php)**: Core state machine logic (42 comprehensive tests)
|
||||
- **[tests/Unit/ContainerHealthStatusTest.php](mdc:tests/Unit/ContainerHealthStatusTest.php)**: Health status aggregation integration
|
||||
- **[tests/Unit/PushServerUpdateJobStatusAggregationTest.php](mdc:tests/Unit/PushServerUpdateJobStatusAggregationTest.php)**: Sentinel update logic
|
||||
- **[tests/Unit/ExcludeFromHealthCheckTest.php](mdc:tests/Unit/ExcludeFromHealthCheckTest.php)**: Excluded container handling
|
||||
|
||||
### **Common Bugs to Avoid**
|
||||
|
||||
❌ **Bug**: Forgetting to track `$hasUnknown` flag
|
||||
✅ **Fix**: Initialize and check for "unknown" in all status aggregation
|
||||
✅ **Prevented by ContainerStatusAggregator Service**:
|
||||
- ❌ **Old Bug**: Forgetting to track `$hasUnknown` flag → ✅ Now centralized in service
|
||||
- ❌ **Old Bug**: Inconsistent priority across paths → ✅ Single source of truth
|
||||
- ❌ **Old Bug**: Forgetting to update all 4 locations → ✅ Only one location to update
|
||||
|
||||
❌ **Bug**: Using ternary operator instead of if-elseif-else
|
||||
✅ **Fix**: Use explicit if-elseif-else to handle 3-way priority
|
||||
**Still Relevant**:
|
||||
|
||||
❌ **Bug**: Updating only one path (SSH or Sentinel)
|
||||
✅ **Fix**: Always update all four status calculation locations
|
||||
❌ **Bug**: Forgetting to filter excluded containers before aggregation
|
||||
✅ **Fix**: Always use `CalculatesExcludedStatus` trait to filter before calling `ContainerStatusAggregator`
|
||||
|
||||
❌ **Bug**: Not passing `$maxRestartCount` for crash loop detection
|
||||
✅ **Fix**: Calculate max restart count from containers and pass to `aggregateFromStrings()`/`aggregateFromContainers()`
|
||||
|
||||
❌ **Bug**: Not handling excluded containers with `:excluded` suffix
|
||||
✅ **Fix**: Check for `:excluded` suffix in UI logic and button visibility
|
||||
|
||||
@ -8,6 +8,8 @@ use App\Events\ServiceChecked;
|
||||
use App\Models\ApplicationPreview;
|
||||
use App\Models\Server;
|
||||
use App\Models\ServiceDatabase;
|
||||
use App\Services\ContainerStatusAggregator;
|
||||
use App\Traits\CalculatesExcludedStatus;
|
||||
use Illuminate\Support\Arr;
|
||||
use Illuminate\Support\Collection;
|
||||
use Illuminate\Support\Facades\DB;
|
||||
@ -16,6 +18,7 @@ use Lorisleiva\Actions\Concerns\AsAction;
|
||||
class GetContainersStatus
|
||||
{
|
||||
use AsAction;
|
||||
use CalculatesExcludedStatus;
|
||||
|
||||
public string $jobQueue = 'high';
|
||||
|
||||
@ -103,10 +106,10 @@ class GetContainersStatus
|
||||
$containerHealth = data_get($container, 'State.Health.Status');
|
||||
if ($containerStatus === 'restarting') {
|
||||
$healthSuffix = $containerHealth ?? 'unknown';
|
||||
$containerStatus = "restarting ($healthSuffix)";
|
||||
$containerStatus = "restarting:$healthSuffix";
|
||||
} else {
|
||||
$healthSuffix = $containerHealth ?? 'unknown';
|
||||
$containerStatus = "$containerStatus ($healthSuffix)";
|
||||
$containerStatus = "$containerStatus:$healthSuffix";
|
||||
}
|
||||
$labels = Arr::undot(format_docker_labels_to_json($labels));
|
||||
$applicationId = data_get($labels, 'coolify.applicationId');
|
||||
@ -443,106 +446,22 @@ class GetContainersStatus
|
||||
{
|
||||
// Parse docker compose to check for excluded containers
|
||||
$dockerComposeRaw = data_get($application, 'docker_compose_raw');
|
||||
$excludedContainers = collect();
|
||||
|
||||
if ($dockerComposeRaw) {
|
||||
try {
|
||||
$dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw);
|
||||
$services = data_get($dockerCompose, 'services', []);
|
||||
|
||||
foreach ($services as $serviceName => $serviceConfig) {
|
||||
// Check if container should be excluded
|
||||
$excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false);
|
||||
$restartPolicy = data_get($serviceConfig, 'restart', 'always');
|
||||
|
||||
if ($excludeFromHc || $restartPolicy === 'no') {
|
||||
$excludedContainers->push($serviceName);
|
||||
}
|
||||
}
|
||||
} catch (\Exception $e) {
|
||||
// If we can't parse, treat all containers as included
|
||||
}
|
||||
}
|
||||
$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);
|
||||
|
||||
// Filter out excluded containers
|
||||
$relevantStatuses = $containerStatuses->filter(function ($status, $containerName) use ($excludedContainers) {
|
||||
return ! $excludedContainers->contains($containerName);
|
||||
});
|
||||
|
||||
// If all containers are excluded, don't update status
|
||||
// If all containers are excluded, calculate status from excluded containers
|
||||
if ($relevantStatuses->isEmpty()) {
|
||||
return null;
|
||||
return $this->calculateExcludedStatusFromStrings($containerStatuses);
|
||||
}
|
||||
|
||||
$hasRunning = false;
|
||||
$hasRestarting = false;
|
||||
$hasUnhealthy = false;
|
||||
$hasUnknown = false;
|
||||
$hasExited = false;
|
||||
$hasStarting = false;
|
||||
$hasPaused = false;
|
||||
$hasDead = false;
|
||||
// Use ContainerStatusAggregator service for state machine logic
|
||||
$aggregator = new ContainerStatusAggregator;
|
||||
|
||||
foreach ($relevantStatuses as $status) {
|
||||
if (str($status)->contains('restarting')) {
|
||||
$hasRestarting = true;
|
||||
} elseif (str($status)->contains('running')) {
|
||||
$hasRunning = true;
|
||||
if (str($status)->contains('unhealthy')) {
|
||||
$hasUnhealthy = true;
|
||||
}
|
||||
if (str($status)->contains('unknown')) {
|
||||
$hasUnknown = true;
|
||||
}
|
||||
} elseif (str($status)->contains('exited')) {
|
||||
$hasExited = true;
|
||||
$hasUnhealthy = true;
|
||||
} elseif (str($status)->contains('created') || str($status)->contains('starting')) {
|
||||
$hasStarting = true;
|
||||
} elseif (str($status)->contains('paused')) {
|
||||
$hasPaused = true;
|
||||
} elseif (str($status)->contains('dead') || str($status)->contains('removing')) {
|
||||
$hasDead = true;
|
||||
}
|
||||
}
|
||||
|
||||
if ($hasRestarting) {
|
||||
return 'degraded (unhealthy)';
|
||||
}
|
||||
|
||||
// If container is exited but has restart count > 0, it's in a crash loop
|
||||
if ($hasExited && $maxRestartCount > 0) {
|
||||
return 'degraded (unhealthy)';
|
||||
}
|
||||
|
||||
if ($hasRunning && $hasExited) {
|
||||
return 'degraded (unhealthy)';
|
||||
}
|
||||
|
||||
if ($hasRunning) {
|
||||
if ($hasUnhealthy) {
|
||||
return 'running (unhealthy)';
|
||||
} elseif ($hasUnknown) {
|
||||
return 'running (unknown)';
|
||||
} else {
|
||||
return 'running (healthy)';
|
||||
}
|
||||
}
|
||||
|
||||
if ($hasDead) {
|
||||
return 'degraded (unhealthy)';
|
||||
}
|
||||
|
||||
if ($hasPaused) {
|
||||
return 'paused (unknown)';
|
||||
}
|
||||
|
||||
if ($hasStarting) {
|
||||
return 'starting (unknown)';
|
||||
}
|
||||
|
||||
// All containers are exited with no restart count - truly stopped
|
||||
return 'exited (unhealthy)';
|
||||
return $aggregator->aggregateFromStrings($relevantStatuses, $maxRestartCount);
|
||||
}
|
||||
|
||||
private function aggregateServiceContainerStatuses($services)
|
||||
@ -574,93 +493,31 @@ class GetContainersStatus
|
||||
|
||||
// Parse docker compose from service to check for excluded containers
|
||||
$dockerComposeRaw = data_get($service, 'docker_compose_raw');
|
||||
$excludedContainers = collect();
|
||||
|
||||
if ($dockerComposeRaw) {
|
||||
try {
|
||||
$dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw);
|
||||
$servicesInCompose = data_get($dockerCompose, 'services', []);
|
||||
|
||||
foreach ($servicesInCompose as $serviceName => $serviceConfig) {
|
||||
// Check if container should be excluded
|
||||
$excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false);
|
||||
$restartPolicy = data_get($serviceConfig, 'restart', 'always');
|
||||
|
||||
if ($excludeFromHc || $restartPolicy === 'no') {
|
||||
$excludedContainers->push($serviceName);
|
||||
}
|
||||
}
|
||||
} catch (\Exception $e) {
|
||||
// If we can't parse, treat all containers as included
|
||||
}
|
||||
}
|
||||
$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);
|
||||
|
||||
// Filter out excluded containers
|
||||
$relevantStatuses = $containerStatuses->filter(function ($status, $containerName) use ($excludedContainers) {
|
||||
return ! $excludedContainers->contains($containerName);
|
||||
});
|
||||
|
||||
// If all containers are excluded, don't update status
|
||||
// If all containers are excluded, calculate status from excluded containers
|
||||
if ($relevantStatuses->isEmpty()) {
|
||||
$aggregatedStatus = $this->calculateExcludedStatusFromStrings($containerStatuses);
|
||||
if ($aggregatedStatus) {
|
||||
$statusFromDb = $subResource->status;
|
||||
if ($statusFromDb !== $aggregatedStatus) {
|
||||
$subResource->update(['status' => $aggregatedStatus]);
|
||||
} else {
|
||||
$subResource->update(['last_online_at' => now()]);
|
||||
}
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
// Aggregate status using same logic as applications
|
||||
$hasRunning = false;
|
||||
$hasRestarting = false;
|
||||
$hasUnhealthy = false;
|
||||
$hasUnknown = false;
|
||||
$hasExited = false;
|
||||
$hasStarting = false;
|
||||
$hasPaused = false;
|
||||
$hasDead = false;
|
||||
|
||||
foreach ($relevantStatuses as $status) {
|
||||
if (str($status)->contains('restarting')) {
|
||||
$hasRestarting = true;
|
||||
} elseif (str($status)->contains('running')) {
|
||||
$hasRunning = true;
|
||||
if (str($status)->contains('unhealthy')) {
|
||||
$hasUnhealthy = true;
|
||||
}
|
||||
if (str($status)->contains('unknown')) {
|
||||
$hasUnknown = true;
|
||||
}
|
||||
} elseif (str($status)->contains('exited')) {
|
||||
$hasExited = true;
|
||||
$hasUnhealthy = true;
|
||||
} elseif (str($status)->contains('created') || str($status)->contains('starting')) {
|
||||
$hasStarting = true;
|
||||
} elseif (str($status)->contains('paused')) {
|
||||
$hasPaused = true;
|
||||
} elseif (str($status)->contains('dead') || str($status)->contains('removing')) {
|
||||
$hasDead = true;
|
||||
}
|
||||
}
|
||||
|
||||
$aggregatedStatus = null;
|
||||
if ($hasRestarting) {
|
||||
$aggregatedStatus = 'degraded (unhealthy)';
|
||||
} elseif ($hasRunning && $hasExited) {
|
||||
$aggregatedStatus = 'degraded (unhealthy)';
|
||||
} elseif ($hasRunning) {
|
||||
if ($hasUnhealthy) {
|
||||
$aggregatedStatus = 'running (unhealthy)';
|
||||
} elseif ($hasUnknown) {
|
||||
$aggregatedStatus = 'running (unknown)';
|
||||
} else {
|
||||
$aggregatedStatus = 'running (healthy)';
|
||||
}
|
||||
} elseif ($hasDead) {
|
||||
$aggregatedStatus = 'degraded (unhealthy)';
|
||||
} elseif ($hasPaused) {
|
||||
$aggregatedStatus = 'paused (unknown)';
|
||||
} elseif ($hasStarting) {
|
||||
$aggregatedStatus = 'starting (unknown)';
|
||||
} else {
|
||||
// All containers are exited
|
||||
$aggregatedStatus = 'exited (unhealthy)';
|
||||
}
|
||||
// Use ContainerStatusAggregator service for state machine logic
|
||||
$aggregator = new ContainerStatusAggregator;
|
||||
$aggregatedStatus = $aggregator->aggregateFromStrings($relevantStatuses);
|
||||
|
||||
// Update service sub-resource status with aggregated result
|
||||
if ($aggregatedStatus) {
|
||||
|
||||
@ -3,11 +3,14 @@
|
||||
namespace App\Actions\Shared;
|
||||
|
||||
use App\Models\Application;
|
||||
use App\Services\ContainerStatusAggregator;
|
||||
use App\Traits\CalculatesExcludedStatus;
|
||||
use Lorisleiva\Actions\Concerns\AsAction;
|
||||
|
||||
class ComplexStatusCheck
|
||||
{
|
||||
use AsAction;
|
||||
use CalculatesExcludedStatus;
|
||||
|
||||
public function handle(Application $application)
|
||||
{
|
||||
@ -61,179 +64,25 @@ class ComplexStatusCheck
|
||||
private function aggregateContainerStatuses($application, $containers)
|
||||
{
|
||||
$dockerComposeRaw = data_get($application, 'docker_compose_raw');
|
||||
$excludedContainers = collect();
|
||||
$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);
|
||||
|
||||
if ($dockerComposeRaw) {
|
||||
try {
|
||||
$dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw);
|
||||
$services = data_get($dockerCompose, 'services', []);
|
||||
|
||||
foreach ($services as $serviceName => $serviceConfig) {
|
||||
$excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false);
|
||||
$restartPolicy = data_get($serviceConfig, 'restart', 'always');
|
||||
|
||||
if ($excludeFromHc || $restartPolicy === 'no') {
|
||||
$excludedContainers->push($serviceName);
|
||||
}
|
||||
}
|
||||
} catch (\Exception $e) {
|
||||
// If we can't parse, treat all containers as included
|
||||
}
|
||||
}
|
||||
|
||||
$hasRunning = false;
|
||||
$hasRestarting = false;
|
||||
$hasUnhealthy = false;
|
||||
$hasUnknown = false;
|
||||
$hasExited = false;
|
||||
$hasStarting = false;
|
||||
$hasPaused = false;
|
||||
$hasDead = false;
|
||||
$relevantContainerCount = 0;
|
||||
|
||||
foreach ($containers as $container) {
|
||||
// Filter non-excluded containers
|
||||
$relevantContainers = collect($containers)->filter(function ($container) use ($excludedContainers) {
|
||||
$labels = data_get($container, 'Config.Labels', []);
|
||||
$serviceName = data_get($labels, 'com.docker.compose.service');
|
||||
|
||||
if ($serviceName && $excludedContainers->contains($serviceName)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$relevantContainerCount++;
|
||||
$containerStatus = data_get($container, 'State.Status');
|
||||
$containerHealth = data_get($container, 'State.Health.Status');
|
||||
|
||||
if ($containerStatus === 'restarting') {
|
||||
$hasRestarting = true;
|
||||
$hasUnhealthy = true;
|
||||
} elseif ($containerStatus === 'running') {
|
||||
$hasRunning = true;
|
||||
if ($containerHealth === 'unhealthy') {
|
||||
$hasUnhealthy = true;
|
||||
} elseif ($containerHealth === null) {
|
||||
$hasUnknown = true;
|
||||
}
|
||||
} elseif ($containerStatus === 'exited') {
|
||||
$hasExited = true;
|
||||
$hasUnhealthy = true;
|
||||
} elseif ($containerStatus === 'created' || $containerStatus === 'starting') {
|
||||
$hasStarting = true;
|
||||
} elseif ($containerStatus === 'paused') {
|
||||
$hasPaused = true;
|
||||
} elseif ($containerStatus === 'dead' || $containerStatus === 'removing') {
|
||||
$hasDead = true;
|
||||
}
|
||||
}
|
||||
return ! ($serviceName && $excludedContainers->contains($serviceName));
|
||||
});
|
||||
|
||||
// If all containers are excluded, calculate status from excluded containers
|
||||
// but mark it with :excluded to indicate monitoring is disabled
|
||||
if ($relevantContainerCount === 0) {
|
||||
$excludedHasRunning = false;
|
||||
$excludedHasRestarting = false;
|
||||
$excludedHasUnhealthy = false;
|
||||
$excludedHasUnknown = false;
|
||||
$excludedHasExited = false;
|
||||
$excludedHasStarting = false;
|
||||
$excludedHasPaused = false;
|
||||
$excludedHasDead = false;
|
||||
|
||||
foreach ($containers as $container) {
|
||||
$labels = data_get($container, 'Config.Labels', []);
|
||||
$serviceName = data_get($labels, 'com.docker.compose.service');
|
||||
|
||||
// Only process excluded containers
|
||||
if (! $serviceName || ! $excludedContainers->contains($serviceName)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$containerStatus = data_get($container, 'State.Status');
|
||||
$containerHealth = data_get($container, 'State.Health.Status');
|
||||
|
||||
if ($containerStatus === 'restarting') {
|
||||
$excludedHasRestarting = true;
|
||||
$excludedHasUnhealthy = true;
|
||||
} elseif ($containerStatus === 'running') {
|
||||
$excludedHasRunning = true;
|
||||
if ($containerHealth === 'unhealthy') {
|
||||
$excludedHasUnhealthy = true;
|
||||
} elseif ($containerHealth === null) {
|
||||
$excludedHasUnknown = true;
|
||||
}
|
||||
} elseif ($containerStatus === 'exited') {
|
||||
$excludedHasExited = true;
|
||||
$excludedHasUnhealthy = true;
|
||||
} elseif ($containerStatus === 'created' || $containerStatus === 'starting') {
|
||||
$excludedHasStarting = true;
|
||||
} elseif ($containerStatus === 'paused') {
|
||||
$excludedHasPaused = true;
|
||||
} elseif ($containerStatus === 'dead' || $containerStatus === 'removing') {
|
||||
$excludedHasDead = true;
|
||||
}
|
||||
}
|
||||
|
||||
if ($excludedHasRestarting) {
|
||||
return 'degraded:excluded';
|
||||
}
|
||||
|
||||
if ($excludedHasRunning && $excludedHasExited) {
|
||||
return 'degraded:excluded';
|
||||
}
|
||||
|
||||
if ($excludedHasRunning) {
|
||||
if ($excludedHasUnhealthy) {
|
||||
return 'running:unhealthy:excluded';
|
||||
} elseif ($excludedHasUnknown) {
|
||||
return 'running:unknown:excluded';
|
||||
} else {
|
||||
return 'running:healthy:excluded';
|
||||
}
|
||||
}
|
||||
|
||||
if ($excludedHasDead) {
|
||||
return 'degraded:excluded';
|
||||
}
|
||||
|
||||
if ($excludedHasPaused) {
|
||||
return 'paused:excluded';
|
||||
}
|
||||
|
||||
if ($excludedHasStarting) {
|
||||
return 'starting:excluded';
|
||||
}
|
||||
|
||||
return 'exited:excluded';
|
||||
if ($relevantContainers->isEmpty()) {
|
||||
return $this->calculateExcludedStatus($containers, $excludedContainers);
|
||||
}
|
||||
|
||||
if ($hasRestarting) {
|
||||
return 'degraded:unhealthy';
|
||||
}
|
||||
// Use ContainerStatusAggregator service for state machine logic
|
||||
$aggregator = new ContainerStatusAggregator;
|
||||
|
||||
if ($hasRunning && $hasExited) {
|
||||
return 'degraded:unhealthy';
|
||||
}
|
||||
|
||||
if ($hasRunning) {
|
||||
if ($hasUnhealthy) {
|
||||
return 'running:unhealthy';
|
||||
} elseif ($hasUnknown) {
|
||||
return 'running:unknown';
|
||||
} else {
|
||||
return 'running:healthy';
|
||||
}
|
||||
}
|
||||
|
||||
if ($hasDead) {
|
||||
return 'degraded:unhealthy';
|
||||
}
|
||||
|
||||
if ($hasPaused) {
|
||||
return 'paused:unknown';
|
||||
}
|
||||
|
||||
if ($hasStarting) {
|
||||
return 'starting:unknown';
|
||||
}
|
||||
|
||||
return 'exited:unhealthy';
|
||||
return $aggregator->aggregateFromContainers($relevantContainers);
|
||||
}
|
||||
}
|
||||
|
||||
@ -13,6 +13,8 @@ use App\Models\Server;
|
||||
use App\Models\ServiceApplication;
|
||||
use App\Models\ServiceDatabase;
|
||||
use App\Notifications\Container\ContainerRestarted;
|
||||
use App\Services\ContainerStatusAggregator;
|
||||
use App\Traits\CalculatesExcludedStatus;
|
||||
use Illuminate\Bus\Queueable;
|
||||
use Illuminate\Contracts\Queue\ShouldBeEncrypted;
|
||||
use Illuminate\Contracts\Queue\ShouldQueue;
|
||||
@ -25,6 +27,7 @@ use Laravel\Horizon\Contracts\Silenced;
|
||||
|
||||
class PushServerUpdateJob implements ShouldBeEncrypted, ShouldQueue, Silenced
|
||||
{
|
||||
use CalculatesExcludedStatus;
|
||||
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
|
||||
|
||||
public $tries = 1;
|
||||
@ -145,7 +148,7 @@ class PushServerUpdateJob implements ShouldBeEncrypted, ShouldQueue, Silenced
|
||||
$containerStatus = data_get($container, 'state', 'exited');
|
||||
$rawHealthStatus = data_get($container, 'health_status');
|
||||
$containerHealth = $rawHealthStatus ?? 'unknown';
|
||||
$containerStatus = "$containerStatus ($containerHealth)";
|
||||
$containerStatus = "$containerStatus:$containerHealth";
|
||||
$labels = collect(data_get($container, 'labels'));
|
||||
$coolify_managed = $labels->has('coolify.managed');
|
||||
|
||||
@ -153,81 +156,75 @@ class PushServerUpdateJob implements ShouldBeEncrypted, ShouldQueue, Silenced
|
||||
continue;
|
||||
}
|
||||
|
||||
if ($coolify_managed) {
|
||||
$name = data_get($container, 'name');
|
||||
if ($name === 'coolify-log-drain' && $this->isRunning($containerStatus)) {
|
||||
$this->foundLogDrainContainer = true;
|
||||
}
|
||||
if ($labels->has('coolify.applicationId')) {
|
||||
$applicationId = $labels->get('coolify.applicationId');
|
||||
$pullRequestId = $labels->get('coolify.pullRequestId', '0');
|
||||
try {
|
||||
if ($pullRequestId === '0') {
|
||||
if ($this->allApplicationIds->contains($applicationId) && $this->isRunning($containerStatus)) {
|
||||
$this->foundApplicationIds->push($applicationId);
|
||||
}
|
||||
// Store container status for aggregation
|
||||
if (! $this->applicationContainerStatuses->has($applicationId)) {
|
||||
$this->applicationContainerStatuses->put($applicationId, collect());
|
||||
}
|
||||
$containerName = $labels->get('com.docker.compose.service');
|
||||
if ($containerName) {
|
||||
$this->applicationContainerStatuses->get($applicationId)->put($containerName, $containerStatus);
|
||||
}
|
||||
} else {
|
||||
$previewKey = $applicationId.':'.$pullRequestId;
|
||||
if ($this->allApplicationPreviewsIds->contains($previewKey) && $this->isRunning($containerStatus)) {
|
||||
$this->foundApplicationPreviewsIds->push($previewKey);
|
||||
}
|
||||
$this->updateApplicationPreviewStatus($applicationId, $pullRequestId, $containerStatus);
|
||||
}
|
||||
} catch (\Exception $e) {
|
||||
}
|
||||
} elseif ($labels->has('coolify.serviceId')) {
|
||||
$serviceId = $labels->get('coolify.serviceId');
|
||||
$subType = $labels->get('coolify.service.subType');
|
||||
$subId = $labels->get('coolify.service.subId');
|
||||
if ($subType === 'application') {
|
||||
if ($this->isRunning($containerStatus)) {
|
||||
$this->foundServiceApplicationIds->push($subId);
|
||||
$name = data_get($container, 'name');
|
||||
if ($name === 'coolify-log-drain' && $this->isRunning($containerStatus)) {
|
||||
$this->foundLogDrainContainer = true;
|
||||
}
|
||||
if ($labels->has('coolify.applicationId')) {
|
||||
$applicationId = $labels->get('coolify.applicationId');
|
||||
$pullRequestId = $labels->get('coolify.pullRequestId', '0');
|
||||
try {
|
||||
if ($pullRequestId === '0') {
|
||||
if ($this->allApplicationIds->contains($applicationId)) {
|
||||
$this->foundApplicationIds->push($applicationId);
|
||||
}
|
||||
// Store container status for aggregation
|
||||
$key = $serviceId.':'.$subType.':'.$subId;
|
||||
if (! $this->serviceContainerStatuses->has($key)) {
|
||||
$this->serviceContainerStatuses->put($key, collect());
|
||||
if (! $this->applicationContainerStatuses->has($applicationId)) {
|
||||
$this->applicationContainerStatuses->put($applicationId, collect());
|
||||
}
|
||||
$containerName = $labels->get('com.docker.compose.service');
|
||||
if ($containerName) {
|
||||
$this->serviceContainerStatuses->get($key)->put($containerName, $containerStatus);
|
||||
$this->applicationContainerStatuses->get($applicationId)->put($containerName, $containerStatus);
|
||||
}
|
||||
} elseif ($subType === 'database') {
|
||||
if ($this->isRunning($containerStatus)) {
|
||||
$this->foundServiceDatabaseIds->push($subId);
|
||||
}
|
||||
// Store container status for aggregation
|
||||
$key = $serviceId.':'.$subType.':'.$subId;
|
||||
if (! $this->serviceContainerStatuses->has($key)) {
|
||||
$this->serviceContainerStatuses->put($key, collect());
|
||||
}
|
||||
$containerName = $labels->get('com.docker.compose.service');
|
||||
if ($containerName) {
|
||||
$this->serviceContainerStatuses->get($key)->put($containerName, $containerStatus);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
$uuid = $labels->get('com.docker.compose.service');
|
||||
$type = $labels->get('coolify.type');
|
||||
if ($name === 'coolify-proxy' && $this->isRunning($containerStatus)) {
|
||||
$this->foundProxy = true;
|
||||
} elseif ($type === 'service' && $this->isRunning($containerStatus)) {
|
||||
} else {
|
||||
if ($this->allDatabaseUuids->contains($uuid) && $this->isRunning($containerStatus)) {
|
||||
$this->foundDatabaseUuids->push($uuid);
|
||||
if ($this->allTcpProxyUuids->contains($uuid) && $this->isRunning($containerStatus)) {
|
||||
$this->updateDatabaseStatus($uuid, $containerStatus, tcpProxy: true);
|
||||
} else {
|
||||
$this->updateDatabaseStatus($uuid, $containerStatus, tcpProxy: false);
|
||||
}
|
||||
$previewKey = $applicationId.':'.$pullRequestId;
|
||||
if ($this->allApplicationPreviewsIds->contains($previewKey)) {
|
||||
$this->foundApplicationPreviewsIds->push($previewKey);
|
||||
}
|
||||
$this->updateApplicationPreviewStatus($applicationId, $pullRequestId, $containerStatus);
|
||||
}
|
||||
} catch (\Exception $e) {
|
||||
}
|
||||
} elseif ($labels->has('coolify.serviceId')) {
|
||||
$serviceId = $labels->get('coolify.serviceId');
|
||||
$subType = $labels->get('coolify.service.subType');
|
||||
$subId = $labels->get('coolify.service.subId');
|
||||
if ($subType === 'application') {
|
||||
$this->foundServiceApplicationIds->push($subId);
|
||||
// Store container status for aggregation
|
||||
$key = $serviceId.':'.$subType.':'.$subId;
|
||||
if (! $this->serviceContainerStatuses->has($key)) {
|
||||
$this->serviceContainerStatuses->put($key, collect());
|
||||
}
|
||||
$containerName = $labels->get('com.docker.compose.service');
|
||||
if ($containerName) {
|
||||
$this->serviceContainerStatuses->get($key)->put($containerName, $containerStatus);
|
||||
}
|
||||
} elseif ($subType === 'database') {
|
||||
$this->foundServiceDatabaseIds->push($subId);
|
||||
// Store container status for aggregation
|
||||
$key = $serviceId.':'.$subType.':'.$subId;
|
||||
if (! $this->serviceContainerStatuses->has($key)) {
|
||||
$this->serviceContainerStatuses->put($key, collect());
|
||||
}
|
||||
$containerName = $labels->get('com.docker.compose.service');
|
||||
if ($containerName) {
|
||||
$this->serviceContainerStatuses->get($key)->put($containerName, $containerStatus);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
$uuid = $labels->get('com.docker.compose.service');
|
||||
$type = $labels->get('coolify.type');
|
||||
if ($name === 'coolify-proxy' && $this->isRunning($containerStatus)) {
|
||||
$this->foundProxy = true;
|
||||
} elseif ($type === 'service' && $this->isRunning($containerStatus)) {
|
||||
} else {
|
||||
if ($this->allDatabaseUuids->contains($uuid) && $this->isRunning($containerStatus)) {
|
||||
$this->foundDatabaseUuids->push($uuid);
|
||||
if ($this->allTcpProxyUuids->contains($uuid) && $this->isRunning($containerStatus)) {
|
||||
$this->updateDatabaseStatus($uuid, $containerStatus, tcpProxy: true);
|
||||
} else {
|
||||
$this->updateDatabaseStatus($uuid, $containerStatus, tcpProxy: false);
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -266,71 +263,31 @@ class PushServerUpdateJob implements ShouldBeEncrypted, ShouldQueue, Silenced
|
||||
|
||||
// Parse docker compose to check for excluded containers
|
||||
$dockerComposeRaw = data_get($application, 'docker_compose_raw');
|
||||
$excludedContainers = collect();
|
||||
|
||||
if ($dockerComposeRaw) {
|
||||
try {
|
||||
$dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw);
|
||||
$services = data_get($dockerCompose, 'services', []);
|
||||
|
||||
foreach ($services as $serviceName => $serviceConfig) {
|
||||
// Check if container should be excluded
|
||||
$excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false);
|
||||
$restartPolicy = data_get($serviceConfig, 'restart', 'always');
|
||||
|
||||
if ($excludeFromHc || $restartPolicy === 'no') {
|
||||
$excludedContainers->push($serviceName);
|
||||
}
|
||||
}
|
||||
} catch (\Exception $e) {
|
||||
// If we can't parse, treat all containers as included
|
||||
}
|
||||
}
|
||||
$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);
|
||||
|
||||
// Filter out excluded containers
|
||||
$relevantStatuses = $containerStatuses->filter(function ($status, $containerName) use ($excludedContainers) {
|
||||
return ! $excludedContainers->contains($containerName);
|
||||
});
|
||||
|
||||
// If all containers are excluded, don't update status
|
||||
// If all containers are excluded, calculate status from excluded containers
|
||||
if ($relevantStatuses->isEmpty()) {
|
||||
$aggregatedStatus = $this->calculateExcludedStatusFromStrings($containerStatuses);
|
||||
|
||||
if ($aggregatedStatus && $application->status !== $aggregatedStatus) {
|
||||
$application->status = $aggregatedStatus;
|
||||
$application->save();
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
// Aggregate status: if any container is running, app is running
|
||||
$hasRunning = false;
|
||||
$hasUnhealthy = false;
|
||||
$hasUnknown = false;
|
||||
|
||||
foreach ($relevantStatuses as $status) {
|
||||
if (str($status)->contains('running')) {
|
||||
$hasRunning = true;
|
||||
if (str($status)->contains('unhealthy')) {
|
||||
$hasUnhealthy = true;
|
||||
}
|
||||
if (str($status)->contains('unknown')) {
|
||||
$hasUnknown = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
$aggregatedStatus = null;
|
||||
if ($hasRunning) {
|
||||
if ($hasUnhealthy) {
|
||||
$aggregatedStatus = 'running (unhealthy)';
|
||||
} elseif ($hasUnknown) {
|
||||
$aggregatedStatus = 'running (unknown)';
|
||||
} else {
|
||||
$aggregatedStatus = 'running (healthy)';
|
||||
}
|
||||
} else {
|
||||
// All containers are exited
|
||||
$aggregatedStatus = 'exited (unhealthy)';
|
||||
}
|
||||
// Use ContainerStatusAggregator service for state machine logic
|
||||
$aggregator = new ContainerStatusAggregator;
|
||||
$aggregatedStatus = $aggregator->aggregateFromStrings($relevantStatuses, 0);
|
||||
|
||||
// Update application status with aggregated result
|
||||
if ($aggregatedStatus && $application->status !== $aggregatedStatus) {
|
||||
|
||||
$application->status = $aggregatedStatus;
|
||||
$application->save();
|
||||
}
|
||||
@ -366,67 +323,28 @@ class PushServerUpdateJob implements ShouldBeEncrypted, ShouldQueue, Silenced
|
||||
|
||||
// Parse docker compose from service to check for excluded containers
|
||||
$dockerComposeRaw = data_get($service, 'docker_compose_raw');
|
||||
$excludedContainers = collect();
|
||||
|
||||
if ($dockerComposeRaw) {
|
||||
try {
|
||||
$dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw);
|
||||
$services = data_get($dockerCompose, 'services', []);
|
||||
|
||||
foreach ($services as $serviceName => $serviceConfig) {
|
||||
// Check if container should be excluded
|
||||
$excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false);
|
||||
$restartPolicy = data_get($serviceConfig, 'restart', 'always');
|
||||
|
||||
if ($excludeFromHc || $restartPolicy === 'no') {
|
||||
$excludedContainers->push($serviceName);
|
||||
}
|
||||
}
|
||||
} catch (\Exception $e) {
|
||||
// If we can't parse, treat all containers as included
|
||||
}
|
||||
}
|
||||
$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);
|
||||
|
||||
// Filter out excluded containers
|
||||
$relevantStatuses = $containerStatuses->filter(function ($status, $containerName) use ($excludedContainers) {
|
||||
return ! $excludedContainers->contains($containerName);
|
||||
});
|
||||
|
||||
// If all containers are excluded, don't update status
|
||||
// If all containers are excluded, calculate status from excluded containers
|
||||
if ($relevantStatuses->isEmpty()) {
|
||||
$aggregatedStatus = $this->calculateExcludedStatusFromStrings($containerStatuses);
|
||||
if ($aggregatedStatus && $subResource->status !== $aggregatedStatus) {
|
||||
$subResource->status = $aggregatedStatus;
|
||||
$subResource->save();
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
// Aggregate status: if any container is running, service is running
|
||||
$hasRunning = false;
|
||||
$hasUnhealthy = false;
|
||||
$hasUnknown = false;
|
||||
|
||||
foreach ($relevantStatuses as $status) {
|
||||
if (str($status)->contains('running')) {
|
||||
$hasRunning = true;
|
||||
if (str($status)->contains('unhealthy')) {
|
||||
$hasUnhealthy = true;
|
||||
}
|
||||
if (str($status)->contains('unknown')) {
|
||||
$hasUnknown = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
$aggregatedStatus = null;
|
||||
if ($hasRunning) {
|
||||
if ($hasUnhealthy) {
|
||||
$aggregatedStatus = 'running (unhealthy)';
|
||||
} elseif ($hasUnknown) {
|
||||
$aggregatedStatus = 'running (unknown)';
|
||||
} else {
|
||||
$aggregatedStatus = 'running (healthy)';
|
||||
}
|
||||
} else {
|
||||
// All containers are exited
|
||||
$aggregatedStatus = 'exited (unhealthy)';
|
||||
}
|
||||
// Use ContainerStatusAggregator service for state machine logic
|
||||
// NOTE: Sentinel does NOT provide restart count data, so maxRestartCount is always 0
|
||||
$aggregator = new ContainerStatusAggregator;
|
||||
$aggregatedStatus = $aggregator->aggregateFromStrings($relevantStatuses, 0);
|
||||
|
||||
// Update service sub-resource status with aggregated result
|
||||
if ($aggregatedStatus && $subResource->status !== $aggregatedStatus) {
|
||||
|
||||
@ -12,7 +12,6 @@ use Illuminate\Database\Eloquent\Factories\HasFactory;
|
||||
use Illuminate\Database\Eloquent\Relations\HasMany;
|
||||
use Illuminate\Database\Eloquent\SoftDeletes;
|
||||
use Illuminate\Support\Collection;
|
||||
use Illuminate\Support\Facades\Log;
|
||||
use Illuminate\Support\Facades\Validator;
|
||||
use Illuminate\Support\Str;
|
||||
use OpenApi\Attributes as OA;
|
||||
@ -670,21 +669,23 @@ class Application extends BaseModel
|
||||
{
|
||||
return Attribute::make(
|
||||
get: function () {
|
||||
if (! $this->relationLoaded('additional_servers') || $this->additional_servers->count() === 0) {
|
||||
return $this->destination?->server?->isFunctional() ?? false;
|
||||
// Check main server infrastructure health
|
||||
$main_server_functional = $this->destination?->server?->isFunctional() ?? false;
|
||||
|
||||
if (! $main_server_functional) {
|
||||
return false;
|
||||
}
|
||||
|
||||
$additional_servers_status = $this->additional_servers->pluck('pivot.status');
|
||||
$main_server_status = $this->destination?->server?->isFunctional() ?? false;
|
||||
|
||||
foreach ($additional_servers_status as $status) {
|
||||
$server_status = str($status)->before(':')->value();
|
||||
if ($server_status !== 'running') {
|
||||
return false;
|
||||
// Check additional servers infrastructure health (not container status!)
|
||||
if ($this->relationLoaded('additional_servers') && $this->additional_servers->count() > 0) {
|
||||
foreach ($this->additional_servers as $server) {
|
||||
if (! $server->isFunctional()) {
|
||||
return false; // Real server infrastructure problem
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return $main_server_status;
|
||||
return true;
|
||||
}
|
||||
);
|
||||
}
|
||||
@ -703,13 +704,6 @@ class Application extends BaseModel
|
||||
} else {
|
||||
$status = $value;
|
||||
$health = 'unhealthy';
|
||||
Log::debug('[STATUS-DEBUG] Status set without health - defaulting to unhealthy', [
|
||||
'source' => 'Application model accessor',
|
||||
'app_id' => $this->id,
|
||||
'app_name' => $this->name,
|
||||
'raw_value' => $value,
|
||||
'result' => "$status:$health",
|
||||
]);
|
||||
}
|
||||
|
||||
return "$status:$health";
|
||||
@ -723,13 +717,6 @@ class Application extends BaseModel
|
||||
} else {
|
||||
$status = $value;
|
||||
$health = 'unhealthy';
|
||||
Log::debug('[STATUS-DEBUG] Status set without health (multi-server) - defaulting to unhealthy', [
|
||||
'source' => 'Application model accessor',
|
||||
'app_id' => $this->id,
|
||||
'app_name' => $this->name,
|
||||
'raw_value' => $value,
|
||||
'result' => "$status:$health",
|
||||
]);
|
||||
}
|
||||
|
||||
return "$status:$health";
|
||||
|
||||
222
app/Services/ContainerStatusAggregator.php
Normal file
222
app/Services/ContainerStatusAggregator.php
Normal file
@ -0,0 +1,222 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services;
|
||||
|
||||
use Illuminate\Support\Collection;
|
||||
|
||||
/**
|
||||
* Container Status Aggregator Service
|
||||
*
|
||||
* Centralized service for aggregating container statuses into a single status string.
|
||||
* Uses a priority-based state machine to determine the overall status from multiple containers.
|
||||
*
|
||||
* Output Format: Colon-separated (e.g., "running:healthy", "degraded:unhealthy")
|
||||
* This format is used throughout the backend for consistency and machine-readability.
|
||||
* UI components transform this to human-readable format (e.g., "Running (Healthy)").
|
||||
*
|
||||
* State Priority (highest to lowest):
|
||||
* 1. Restarting → degraded:unhealthy
|
||||
* 2. Crash Loop (exited with restarts) → degraded:unhealthy
|
||||
* 3. Mixed (running + exited) → degraded:unhealthy
|
||||
* 4. Running → running:healthy/unhealthy/unknown
|
||||
* 5. Dead/Removing → degraded:unhealthy
|
||||
* 6. Paused → paused:unknown
|
||||
* 7. Starting/Created → starting:unknown
|
||||
* 8. Exited → exited:unhealthy
|
||||
*/
|
||||
class ContainerStatusAggregator
|
||||
{
|
||||
/**
|
||||
* Aggregate container statuses from status strings into a single status.
|
||||
*
|
||||
* @param Collection $containerStatuses Collection of status strings (e.g., "running (healthy)", "running:healthy")
|
||||
* @param int $maxRestartCount Maximum restart count across containers (for crash loop detection)
|
||||
* @return string Aggregated status in colon format (e.g., "running:healthy")
|
||||
*/
|
||||
public function aggregateFromStrings(Collection $containerStatuses, int $maxRestartCount = 0): string
|
||||
{
|
||||
if ($containerStatuses->isEmpty()) {
|
||||
return 'exited:unhealthy';
|
||||
}
|
||||
|
||||
// Initialize state flags
|
||||
$hasRunning = false;
|
||||
$hasRestarting = false;
|
||||
$hasUnhealthy = false;
|
||||
$hasUnknown = false;
|
||||
$hasExited = false;
|
||||
$hasStarting = false;
|
||||
$hasPaused = false;
|
||||
$hasDead = false;
|
||||
|
||||
// Parse each status string and set flags
|
||||
foreach ($containerStatuses as $status) {
|
||||
if (str($status)->contains('restarting')) {
|
||||
$hasRestarting = true;
|
||||
} elseif (str($status)->contains('running')) {
|
||||
$hasRunning = true;
|
||||
if (str($status)->contains('unhealthy')) {
|
||||
$hasUnhealthy = true;
|
||||
}
|
||||
if (str($status)->contains('unknown')) {
|
||||
$hasUnknown = true;
|
||||
}
|
||||
} elseif (str($status)->contains('exited')) {
|
||||
$hasExited = true;
|
||||
$hasUnhealthy = true;
|
||||
} elseif (str($status)->contains('created') || str($status)->contains('starting')) {
|
||||
$hasStarting = true;
|
||||
} elseif (str($status)->contains('paused')) {
|
||||
$hasPaused = true;
|
||||
} elseif (str($status)->contains('dead') || str($status)->contains('removing')) {
|
||||
$hasDead = true;
|
||||
}
|
||||
}
|
||||
|
||||
// Priority-based status resolution
|
||||
return $this->resolveStatus(
|
||||
$hasRunning,
|
||||
$hasRestarting,
|
||||
$hasUnhealthy,
|
||||
$hasUnknown,
|
||||
$hasExited,
|
||||
$hasStarting,
|
||||
$hasPaused,
|
||||
$hasDead,
|
||||
$maxRestartCount
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Aggregate container statuses from Docker container objects.
|
||||
*
|
||||
* @param Collection $containers Collection of Docker container objects with State property
|
||||
* @param int $maxRestartCount Maximum restart count across containers (for crash loop detection)
|
||||
* @return string Aggregated status in colon format (e.g., "running:healthy")
|
||||
*/
|
||||
public function aggregateFromContainers(Collection $containers, int $maxRestartCount = 0): string
|
||||
{
|
||||
if ($containers->isEmpty()) {
|
||||
return 'exited:unhealthy';
|
||||
}
|
||||
|
||||
// Initialize state flags
|
||||
$hasRunning = false;
|
||||
$hasRestarting = false;
|
||||
$hasUnhealthy = false;
|
||||
$hasUnknown = false;
|
||||
$hasExited = false;
|
||||
$hasStarting = false;
|
||||
$hasPaused = false;
|
||||
$hasDead = false;
|
||||
|
||||
// Parse each container object and set flags
|
||||
foreach ($containers as $container) {
|
||||
$state = data_get($container, 'State.Status', 'exited');
|
||||
$health = data_get($container, 'State.Health.Status');
|
||||
|
||||
if ($state === 'restarting') {
|
||||
$hasRestarting = true;
|
||||
} elseif ($state === 'running') {
|
||||
$hasRunning = true;
|
||||
if ($health === 'unhealthy') {
|
||||
$hasUnhealthy = true;
|
||||
} elseif (is_null($health) || $health === 'starting') {
|
||||
$hasUnknown = true;
|
||||
}
|
||||
} elseif ($state === 'exited') {
|
||||
$hasExited = true;
|
||||
$hasUnhealthy = true;
|
||||
} elseif ($state === 'created' || $state === 'starting') {
|
||||
$hasStarting = true;
|
||||
} elseif ($state === 'paused') {
|
||||
$hasPaused = true;
|
||||
} elseif ($state === 'dead' || $state === 'removing') {
|
||||
$hasDead = true;
|
||||
}
|
||||
}
|
||||
|
||||
// Priority-based status resolution
|
||||
return $this->resolveStatus(
|
||||
$hasRunning,
|
||||
$hasRestarting,
|
||||
$hasUnhealthy,
|
||||
$hasUnknown,
|
||||
$hasExited,
|
||||
$hasStarting,
|
||||
$hasPaused,
|
||||
$hasDead,
|
||||
$maxRestartCount
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the aggregated status based on state flags (priority-based state machine).
|
||||
*
|
||||
* @param bool $hasRunning Has at least one running container
|
||||
* @param bool $hasRestarting Has at least one restarting container
|
||||
* @param bool $hasUnhealthy Has at least one unhealthy container
|
||||
* @param bool $hasUnknown Has at least one container with unknown health
|
||||
* @param bool $hasExited Has at least one exited container
|
||||
* @param bool $hasStarting Has at least one starting/created container
|
||||
* @param bool $hasPaused Has at least one paused container
|
||||
* @param bool $hasDead Has at least one dead/removing container
|
||||
* @param int $maxRestartCount Maximum restart count (for crash loop detection)
|
||||
* @return string Status in colon format (e.g., "running:healthy")
|
||||
*/
|
||||
private function resolveStatus(
|
||||
bool $hasRunning,
|
||||
bool $hasRestarting,
|
||||
bool $hasUnhealthy,
|
||||
bool $hasUnknown,
|
||||
bool $hasExited,
|
||||
bool $hasStarting,
|
||||
bool $hasPaused,
|
||||
bool $hasDead,
|
||||
int $maxRestartCount
|
||||
): string {
|
||||
// Priority 1: Restarting containers (degraded state)
|
||||
if ($hasRestarting) {
|
||||
return 'degraded:unhealthy';
|
||||
}
|
||||
|
||||
// Priority 2: Crash loop detection (exited with restart count > 0)
|
||||
if ($hasExited && $maxRestartCount > 0) {
|
||||
return 'degraded:unhealthy';
|
||||
}
|
||||
|
||||
// Priority 3: Mixed state (some running, some exited = degraded)
|
||||
if ($hasRunning && $hasExited) {
|
||||
return 'degraded:unhealthy';
|
||||
}
|
||||
|
||||
// Priority 4: Running containers (check health status)
|
||||
if ($hasRunning) {
|
||||
if ($hasUnhealthy) {
|
||||
return 'running:unhealthy';
|
||||
} elseif ($hasUnknown) {
|
||||
return 'running:unknown';
|
||||
} else {
|
||||
return 'running:healthy';
|
||||
}
|
||||
}
|
||||
|
||||
// Priority 5: Dead or removing containers
|
||||
if ($hasDead) {
|
||||
return 'degraded:unhealthy';
|
||||
}
|
||||
|
||||
// Priority 6: Paused containers
|
||||
if ($hasPaused) {
|
||||
return 'paused:unknown';
|
||||
}
|
||||
|
||||
// Priority 7: Starting/created containers
|
||||
if ($hasStarting) {
|
||||
return 'starting:unknown';
|
||||
}
|
||||
|
||||
// Priority 8: All containers exited (no restart count = truly stopped)
|
||||
return 'exited:unhealthy';
|
||||
}
|
||||
}
|
||||
130
app/Traits/CalculatesExcludedStatus.php
Normal file
130
app/Traits/CalculatesExcludedStatus.php
Normal file
@ -0,0 +1,130 @@
|
||||
<?php
|
||||
|
||||
namespace App\Traits;
|
||||
|
||||
use App\Services\ContainerStatusAggregator;
|
||||
use Illuminate\Support\Collection;
|
||||
|
||||
trait CalculatesExcludedStatus
|
||||
{
|
||||
/**
|
||||
* Calculate status for containers when all containers are excluded from health checks.
|
||||
*
|
||||
* This method processes excluded containers and returns a status with :excluded suffix
|
||||
* to indicate that monitoring is disabled but still show the actual container state.
|
||||
*
|
||||
* @param Collection $containers Collection of container objects from Docker inspect
|
||||
* @param Collection $excludedContainers Collection of container names that are excluded
|
||||
* @return string Status string with :excluded suffix (e.g., 'running:unhealthy:excluded')
|
||||
*/
|
||||
protected function calculateExcludedStatus(Collection $containers, Collection $excludedContainers): string
|
||||
{
|
||||
// Filter to only excluded containers
|
||||
$excludedOnly = $containers->filter(function ($container) use ($excludedContainers) {
|
||||
$labels = data_get($container, 'Config.Labels', []);
|
||||
$serviceName = data_get($labels, 'com.docker.compose.service');
|
||||
|
||||
return $serviceName && $excludedContainers->contains($serviceName);
|
||||
});
|
||||
|
||||
// Use ContainerStatusAggregator service for state machine logic
|
||||
$aggregator = new ContainerStatusAggregator;
|
||||
$status = $aggregator->aggregateFromContainers($excludedOnly);
|
||||
|
||||
// Append :excluded suffix
|
||||
return $this->appendExcludedSuffix($status);
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate status for containers when all containers are excluded (simplified version).
|
||||
*
|
||||
* This version works with status strings (e.g., "running:healthy") instead of full
|
||||
* container objects, suitable for Sentinel updates that don't have full container data.
|
||||
*
|
||||
* @param Collection $containerStatuses Collection of status strings keyed by container name
|
||||
* @return string Status string with :excluded suffix
|
||||
*/
|
||||
protected function calculateExcludedStatusFromStrings(Collection $containerStatuses): string
|
||||
{
|
||||
// Use ContainerStatusAggregator service for state machine logic
|
||||
$aggregator = new ContainerStatusAggregator;
|
||||
$status = $aggregator->aggregateFromStrings($containerStatuses);
|
||||
|
||||
// Append :excluded suffix
|
||||
$finalStatus = $this->appendExcludedSuffix($status);
|
||||
|
||||
return $finalStatus;
|
||||
}
|
||||
|
||||
/**
|
||||
* Append :excluded suffix to a status string.
|
||||
*
|
||||
* Converts status formats like:
|
||||
* - "running:healthy" → "running:healthy:excluded"
|
||||
* - "degraded:unhealthy" → "degraded:excluded" (simplified)
|
||||
* - "paused:unknown" → "paused:excluded" (simplified)
|
||||
*
|
||||
* @param string $status The base status string
|
||||
* @return string Status with :excluded suffix
|
||||
*/
|
||||
private function appendExcludedSuffix(string $status): string
|
||||
{
|
||||
// For degraded states, simplify to just "degraded:excluded"
|
||||
if (str($status)->startsWith('degraded')) {
|
||||
return 'degraded:excluded';
|
||||
}
|
||||
|
||||
// For paused/starting/exited states, simplify to just "state:excluded"
|
||||
if (str($status)->startsWith('paused')) {
|
||||
return 'paused:excluded';
|
||||
}
|
||||
|
||||
if (str($status)->startsWith('starting')) {
|
||||
return 'starting:excluded';
|
||||
}
|
||||
|
||||
if (str($status)->startsWith('exited')) {
|
||||
return 'exited:excluded';
|
||||
}
|
||||
|
||||
// For running states, keep the health status: "running:healthy:excluded"
|
||||
return "$status:excluded";
|
||||
}
|
||||
|
||||
/**
|
||||
* Get excluded containers from docker-compose YAML.
|
||||
*
|
||||
* Containers are excluded if:
|
||||
* - They have exclude_from_hc: true label
|
||||
* - They have restart: no policy
|
||||
*
|
||||
* @param string|null $dockerComposeRaw The raw docker-compose YAML content
|
||||
* @return Collection Collection of excluded container names
|
||||
*/
|
||||
protected function getExcludedContainersFromDockerCompose(?string $dockerComposeRaw): Collection
|
||||
{
|
||||
$excludedContainers = collect();
|
||||
|
||||
if (! $dockerComposeRaw) {
|
||||
return $excludedContainers;
|
||||
}
|
||||
|
||||
try {
|
||||
$dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw);
|
||||
$services = data_get($dockerCompose, 'services', []);
|
||||
|
||||
foreach ($services as $serviceName => $serviceConfig) {
|
||||
$excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false);
|
||||
$restartPolicy = data_get($serviceConfig, 'restart', 'always');
|
||||
|
||||
if ($excludeFromHc || $restartPolicy === 'no') {
|
||||
$excludedContainers->push($serviceName);
|
||||
}
|
||||
}
|
||||
} catch (\Exception $e) {
|
||||
// If we can't parse, treat all containers as included
|
||||
}
|
||||
|
||||
return $excludedContainers;
|
||||
}
|
||||
}
|
||||
@ -1,15 +1,33 @@
|
||||
@props([
|
||||
'status' => 'Degraded',
|
||||
])
|
||||
@php
|
||||
// Handle both colon format (backend) and parentheses format (from services.blade.php)
|
||||
// degraded:unhealthy → Degraded (unhealthy)
|
||||
// degraded (unhealthy) → degraded (unhealthy) (already formatted, display as-is)
|
||||
|
||||
if (str($status)->contains('(')) {
|
||||
// Already in parentheses format from services.blade.php - use as-is
|
||||
$displayStatus = $status;
|
||||
$healthStatus = str($status)->after('(')->before(')')->trim()->value();
|
||||
} elseif (str($status)->contains(':') && !str($status)->startsWith('Proxy')) {
|
||||
// Colon format from backend - transform it
|
||||
$parts = explode(':', $status);
|
||||
$displayStatus = str($parts[0])->headline();
|
||||
$healthStatus = $parts[1] ?? null;
|
||||
} else {
|
||||
// Simple status without health
|
||||
$displayStatus = str($status)->headline();
|
||||
$healthStatus = null;
|
||||
}
|
||||
@endphp
|
||||
<div class="flex items-center" >
|
||||
<x-loading wire:loading.delay.longer />
|
||||
<span wire:loading.remove.delay.longer class="flex items-center">
|
||||
<div class="badge badge-warning"></div>
|
||||
<div class="pl-2 pr-1 text-xs font-bold dark:text-warning">
|
||||
{{ str($status)->before(':')->headline() }}
|
||||
</div>
|
||||
@if (!str($status)->startsWith('Proxy') && !str($status)->contains('('))
|
||||
<div class="text-xs dark:text-warning">({{ str($status)->after(':') }})</div>
|
||||
<div class="pl-2 pr-1 text-xs font-bold dark:text-warning">{{ $displayStatus }}</div>
|
||||
@if ($healthStatus && !str($displayStatus)->contains('('))
|
||||
<div class="text-xs dark:text-warning">({{ $healthStatus }})</div>
|
||||
@endif
|
||||
</span>
|
||||
</div>
|
||||
|
||||
@ -5,9 +5,9 @@
|
||||
])
|
||||
@if (str($resource->status)->startsWith('running'))
|
||||
<x-status.running :status="$resource->status" :title="$title" :lastDeploymentLink="$lastDeploymentLink" />
|
||||
@elseif(str($resource->status)->startsWith('restarting') ||
|
||||
str($resource->status)->startsWith('starting') ||
|
||||
str($resource->status)->startsWith('degraded'))
|
||||
@elseif(str($resource->status)->startsWith('degraded'))
|
||||
<x-status.degraded :status="$resource->status" :title="$title" :lastDeploymentLink="$lastDeploymentLink" />
|
||||
@elseif(str($resource->status)->startsWith('restarting') || str($resource->status)->startsWith('starting'))
|
||||
<x-status.restarting :status="$resource->status" :title="$title" :lastDeploymentLink="$lastDeploymentLink" />
|
||||
@else
|
||||
<x-status.stopped :status="$resource->status" />
|
||||
|
||||
@ -4,6 +4,26 @@
|
||||
'lastDeploymentLink' => null,
|
||||
'noLoading' => false,
|
||||
])
|
||||
@php
|
||||
// Handle both colon format (backend) and parentheses format (from services.blade.php)
|
||||
// starting:unknown → Starting (unknown)
|
||||
// starting (unknown) → starting (unknown) (already formatted, display as-is)
|
||||
|
||||
if (str($status)->contains('(')) {
|
||||
// Already in parentheses format from services.blade.php - use as-is
|
||||
$displayStatus = $status;
|
||||
$healthStatus = str($status)->after('(')->before(')')->trim()->value();
|
||||
} elseif (str($status)->contains(':') && !str($status)->startsWith('Proxy')) {
|
||||
// Colon format from backend - transform it
|
||||
$parts = explode(':', $status);
|
||||
$displayStatus = str($parts[0])->headline();
|
||||
$healthStatus = $parts[1] ?? null;
|
||||
} else {
|
||||
// Simple status without health
|
||||
$displayStatus = str($status)->headline();
|
||||
$healthStatus = null;
|
||||
}
|
||||
@endphp
|
||||
<div class="flex items-center">
|
||||
@if (!$noLoading)
|
||||
<x-loading wire:loading.delay.longer />
|
||||
@ -13,14 +33,14 @@
|
||||
<div class="pl-2 pr-1 text-xs font-bold dark:text-warning" @if($title) title="{{$title}}" @endif>
|
||||
@if ($lastDeploymentLink)
|
||||
<a href="{{ $lastDeploymentLink }}" target="_blank" class="underline cursor-pointer">
|
||||
{{ str($status)->before(':')->headline() }}
|
||||
{{ $displayStatus }}
|
||||
</a>
|
||||
@else
|
||||
{{ str($status)->before(':')->headline() }}
|
||||
{{ $displayStatus }}
|
||||
@endif
|
||||
</div>
|
||||
@if (!str($status)->startsWith('Proxy') && !str($status)->contains('('))
|
||||
<div class="text-xs dark:text-warning">({{ str($status)->after(':') }})</div>
|
||||
@if ($healthStatus && !str($displayStatus)->contains('('))
|
||||
<div class="text-xs dark:text-warning">({{ $healthStatus }})</div>
|
||||
@endif
|
||||
</span>
|
||||
</div>
|
||||
|
||||
@ -1,21 +1,38 @@
|
||||
@php
|
||||
// Transform colon format to human-readable format for UI display
|
||||
// running:healthy → Running (healthy)
|
||||
// running:unhealthy:excluded → Running (unhealthy, excluded)
|
||||
// exited:excluded → Exited (excluded)
|
||||
$isExcluded = str($complexStatus)->endsWith(':excluded');
|
||||
$displayStatus = $isExcluded ? str($complexStatus)->beforeLast(':excluded') : $complexStatus;
|
||||
$parts = explode(':', $complexStatus);
|
||||
|
||||
if ($isExcluded) {
|
||||
if (count($parts) === 3) {
|
||||
// Has health status: running:unhealthy:excluded → Running (unhealthy, excluded)
|
||||
$displayStatus = str($parts[0])->headline() . ' (' . $parts[1] . ', excluded)';
|
||||
} else {
|
||||
// No health status: exited:excluded → Exited (excluded)
|
||||
$displayStatus = str($parts[0])->headline() . ' (excluded)';
|
||||
}
|
||||
} elseif (count($parts) >= 2 && !str($complexStatus)->startsWith('Proxy')) {
|
||||
// Regular colon format: running:healthy → Running (healthy)
|
||||
$displayStatus = str($parts[0])->headline() . ' (' . $parts[1] . ')';
|
||||
} else {
|
||||
// No transformation needed (simple status or already in parentheses format)
|
||||
$displayStatus = str($complexStatus)->headline();
|
||||
}
|
||||
@endphp
|
||||
@if (str($displayStatus)->contains('running'))
|
||||
@if (str($displayStatus)->lower()->contains('running'))
|
||||
<x-status.running :status="$displayStatus" />
|
||||
@elseif(str($displayStatus)->contains('starting'))
|
||||
@elseif(str($displayStatus)->lower()->contains('starting'))
|
||||
<x-status.restarting :status="$displayStatus" />
|
||||
@elseif(str($displayStatus)->contains('restarting'))
|
||||
@elseif(str($displayStatus)->lower()->contains('restarting'))
|
||||
<x-status.restarting :status="$displayStatus" />
|
||||
@elseif(str($displayStatus)->contains('degraded'))
|
||||
@elseif(str($displayStatus)->lower()->contains('degraded'))
|
||||
<x-status.degraded :status="$displayStatus" />
|
||||
@else
|
||||
<x-status.stopped :status="$displayStatus" />
|
||||
@endif
|
||||
@if ($isExcluded)
|
||||
<span class="text-xs text-neutral-500 dark:text-neutral-400" title="All containers excluded from health monitoring">(Monitoring Disabled)</span>
|
||||
@endif
|
||||
@if (!str($complexStatus)->contains('exited') && $showRefreshButton)
|
||||
<button wire:loading.remove.delay.shortest wire:target="manualCheckStatus" title="Refresh Status" wire:click='manualCheckStatus'
|
||||
class="mx-1 dark:hover:fill-white fill-black dark:fill-warning">
|
||||
|
||||
@ -2,12 +2,35 @@
|
||||
'status' => 'Stopped',
|
||||
'noLoading' => false,
|
||||
])
|
||||
@php
|
||||
// Handle both colon format (backend) and parentheses format (from services.blade.php)
|
||||
// exited:unhealthy → Exited (unhealthy)
|
||||
// exited (unhealthy) → exited (unhealthy) (already formatted, display as-is)
|
||||
|
||||
if (str($status)->contains('(')) {
|
||||
// Already in parentheses format from services.blade.php - use as-is
|
||||
$displayStatus = $status;
|
||||
$healthStatus = str($status)->after('(')->before(')')->trim()->value();
|
||||
} elseif (str($status)->contains(':')) {
|
||||
// Colon format from backend - transform it
|
||||
$parts = explode(':', $status);
|
||||
$displayStatus = str($parts[0])->headline();
|
||||
$healthStatus = $parts[1] ?? null;
|
||||
} else {
|
||||
// Simple status without health
|
||||
$displayStatus = str($status)->headline();
|
||||
$healthStatus = null;
|
||||
}
|
||||
@endphp
|
||||
<div class="flex items-center">
|
||||
@if (!$noLoading)
|
||||
<x-loading wire:loading.delay.longer />
|
||||
@endif
|
||||
<span wire:loading.remove.delay.longer class="flex items-center">
|
||||
<div class="badge badge-error "></div>
|
||||
<div class="pl-2 pr-1 text-xs font-bold text-error">{{ str($status)->before(':')->headline() }}</div>
|
||||
<div class="pl-2 pr-1 text-xs font-bold text-error">{{ $displayStatus }}</div>
|
||||
@if ($healthStatus && !str($displayStatus)->contains('('))
|
||||
<div class="text-xs text-error">({{ $healthStatus }})</div>
|
||||
@endif
|
||||
</span>
|
||||
</div>
|
||||
|
||||
@ -30,15 +30,15 @@
|
||||
@endif
|
||||
<a class="menu-item flex items-center gap-2" wire:current.exact="menu-item-active"
|
||||
href="{{ route('project.application.servers', ['project_uuid' => $project->uuid, 'environment_uuid' => $environment->uuid, 'application_uuid' => $application->uuid]) }}">Servers
|
||||
@if (str($application->status)->contains('degraded'))
|
||||
<span title="Some servers are unavailable">
|
||||
@if ($application->server_status == false)
|
||||
<span title="One or more servers are unreachable or misconfigured.">
|
||||
<svg class="w-4 h-4 text-error" viewBox="0 0 256 256" xmlns="http://www.w3.org/2000/svg">
|
||||
<path fill="currentColor"
|
||||
d="M240.26 186.1L152.81 34.23a28.74 28.74 0 0 0-49.62 0L15.74 186.1a27.45 27.45 0 0 0 0 27.71A28.31 28.31 0 0 0 40.55 228h174.9a28.31 28.31 0 0 0 24.79-14.19a27.45 27.45 0 0 0 .02-27.71m-20.8 15.7a4.46 4.46 0 0 1-4 2.2H40.55a4.46 4.46 0 0 1-4-2.2a3.56 3.56 0 0 1 0-3.73L124 46.2a4.77 4.77 0 0 1 8 0l87.44 151.87a3.56 3.56 0 0 1 .02 3.73M116 136v-32a12 12 0 0 1 24 0v32a12 12 0 0 1-24 0m28 40a16 16 0 1 1-16-16a16 16 0 0 1 16 16" />
|
||||
</svg>
|
||||
</span>
|
||||
@elseif ($application->server_status == false)
|
||||
<span title="The underlying server(s) has problems.">
|
||||
@elseif ($application->additional_servers()->exists() && str($application->status)->contains('degraded'))
|
||||
<span title="Application is in degraded state across multiple servers.">
|
||||
<svg class="w-4 h-4 text-error" viewBox="0 0 256 256" xmlns="http://www.w3.org/2000/svg">
|
||||
<path fill="currentColor"
|
||||
d="M240.26 186.1L152.81 34.23a28.74 28.74 0 0 0-49.62 0L15.74 186.1a27.45 27.45 0 0 0 0 27.71A28.31 28.31 0 0 0 40.55 228h174.9a28.31 28.31 0 0 0 24.79-14.19a27.45 27.45 0 0 0 .02-27.71m-20.8 15.7a4.46 4.46 0 0 1-4 2.2H40.55a4.46 4.46 0 0 1-4-2.2a3.56 3.56 0 0 1 0-3.73L124 46.2a4.77 4.77 0 0 1 8 0l87.44 151.87a3.56 3.56 0 0 1 .02 3.73M116 136v-32a12 12 0 0 1 24 0v32a12 12 0 0 1-24 0m28 40a16 16 0 1 1-16-16a16 16 0 0 1 16 16" />
|
||||
|
||||
@ -118,7 +118,7 @@
|
||||
<div class="max-w-full px-4 truncate box-description" x-text="item.description"></div>
|
||||
<div class="max-w-full px-4 truncate box-description" x-text="item.fqdn"></div>
|
||||
<template x-if="item.server_status == false">
|
||||
<div class="px-4 text-xs font-bold text-error">The underlying server has problems
|
||||
<div class="px-4 text-xs font-bold text-error">Server is unreachable or misconfigured
|
||||
</div>
|
||||
</template>
|
||||
</div>
|
||||
@ -167,7 +167,7 @@
|
||||
<div class="max-w-full px-4 truncate box-description" x-text="item.description"></div>
|
||||
<div class="max-w-full px-4 truncate box-description" x-text="item.fqdn"></div>
|
||||
<template x-if="item.server_status == false">
|
||||
<div class="px-4 text-xs font-bold text-error">The underlying server has problems
|
||||
<div class="px-4 text-xs font-bold text-error">Server is unreachable or misconfigured
|
||||
</div>
|
||||
</template>
|
||||
</div>
|
||||
@ -216,7 +216,7 @@
|
||||
<div class="max-w-full px-4 truncate box-description" x-text="item.description"></div>
|
||||
<div class="max-w-full px-4 truncate box-description" x-text="item.fqdn"></div>
|
||||
<template x-if="item.server_status == false">
|
||||
<div class="px-4 text-xs font-bold text-error">The underlying server has problems
|
||||
<div class="px-4 text-xs font-bold text-error">Server is unreachable or misconfigured
|
||||
</div>
|
||||
</template>
|
||||
</div>
|
||||
|
||||
@ -90,7 +90,31 @@
|
||||
@endcan
|
||||
</span>
|
||||
@endif
|
||||
<div class="pt-2 text-xs">{{ $application->status }}</div>
|
||||
@php
|
||||
// Transform colon format to human-readable format
|
||||
// running:healthy → Running (healthy)
|
||||
// running:unhealthy:excluded → Running (unhealthy, excluded)
|
||||
$appStatus = $application->status;
|
||||
$isExcluded = str($appStatus)->endsWith(':excluded');
|
||||
$parts = explode(':', $appStatus);
|
||||
|
||||
if ($isExcluded) {
|
||||
if (count($parts) === 3) {
|
||||
// Has health status: running:unhealthy:excluded → Running (unhealthy, excluded)
|
||||
$appStatus = str($parts[0])->headline() . ' (' . $parts[1] . ', excluded)';
|
||||
} else {
|
||||
// No health status: exited:excluded → Exited (excluded)
|
||||
$appStatus = str($parts[0])->headline() . ' (excluded)';
|
||||
}
|
||||
} elseif (count($parts) >= 2 && !str($appStatus)->startsWith('Proxy')) {
|
||||
// Regular colon format: running:healthy → Running (healthy)
|
||||
$appStatus = str($parts[0])->headline() . ' (' . $parts[1] . ')';
|
||||
} else {
|
||||
// Simple status or already in parentheses format
|
||||
$appStatus = str($appStatus)->headline();
|
||||
}
|
||||
@endphp
|
||||
<div class="pt-2 text-xs">{{ $appStatus }}</div>
|
||||
</div>
|
||||
<div class="flex items-center px-4">
|
||||
<a class="mx-4 text-xs font-bold hover:underline"
|
||||
@ -139,7 +163,31 @@
|
||||
@if ($database->description)
|
||||
<span class="text-xs">{{ Str::limit($database->description, 60) }}</span>
|
||||
@endif
|
||||
<div class="text-xs">{{ $database->status }}</div>
|
||||
@php
|
||||
// Transform colon format to human-readable format
|
||||
// running:healthy → Running (healthy)
|
||||
// running:unhealthy:excluded → Running (unhealthy, excluded)
|
||||
$dbStatus = $database->status;
|
||||
$isExcluded = str($dbStatus)->endsWith(':excluded');
|
||||
$parts = explode(':', $dbStatus);
|
||||
|
||||
if ($isExcluded) {
|
||||
if (count($parts) === 3) {
|
||||
// Has health status: running:unhealthy:excluded → Running (unhealthy, excluded)
|
||||
$dbStatus = str($parts[0])->headline() . ' (' . $parts[1] . ', excluded)';
|
||||
} else {
|
||||
// No health status: exited:excluded → Exited (excluded)
|
||||
$dbStatus = str($parts[0])->headline() . ' (excluded)';
|
||||
}
|
||||
} elseif (count($parts) >= 2 && !str($dbStatus)->startsWith('Proxy')) {
|
||||
// Regular colon format: running:healthy → Running (healthy)
|
||||
$dbStatus = str($parts[0])->headline() . ' (' . $parts[1] . ')';
|
||||
} else {
|
||||
// Simple status or already in parentheses format
|
||||
$dbStatus = str($dbStatus)->headline();
|
||||
}
|
||||
@endphp
|
||||
<div class="text-xs">{{ $dbStatus }}</div>
|
||||
</div>
|
||||
<div class="flex items-center px-4">
|
||||
@if ($database->isBackupSolutionAvailable() || $database->is_migrated)
|
||||
|
||||
@ -14,7 +14,6 @@ use App\Http\Controllers\Api\TeamController;
|
||||
use App\Http\Middleware\ApiAllowed;
|
||||
use App\Jobs\PushServerUpdateJob;
|
||||
use App\Models\Server;
|
||||
use Illuminate\Support\Facades\Log;
|
||||
use Illuminate\Support\Facades\Route;
|
||||
|
||||
Route::get('/health', [OtherController::class, 'healthcheck']);
|
||||
@ -159,7 +158,6 @@ Route::group([
|
||||
'prefix' => 'v1',
|
||||
], function () {
|
||||
Route::post('/sentinel/push', function () {
|
||||
Log::info('Received Sentinel push request', ['ip' => request()->ip(), 'user_agent' => request()->header('User-Agent')]);
|
||||
$token = request()->header('Authorization');
|
||||
if (! $token) {
|
||||
return response()->json(['message' => 'Unauthorized'], 401);
|
||||
|
||||
223
tests/Unit/AllExcludedContainersConsistencyTest.php
Normal file
223
tests/Unit/AllExcludedContainersConsistencyTest.php
Normal file
@ -0,0 +1,223 @@
|
||||
<?php
|
||||
|
||||
/**
|
||||
* Unit tests to verify consistent handling of all-excluded containers
|
||||
* across PushServerUpdateJob, GetContainersStatus, and ComplexStatusCheck.
|
||||
*
|
||||
* These tests verify the fix for issue where different code paths handled
|
||||
* all-excluded containers inconsistently:
|
||||
* - PushServerUpdateJob (Sentinel, ~30s) previously skipped updates
|
||||
* - GetContainersStatus (SSH, ~1min) previously skipped updates
|
||||
* - ComplexStatusCheck (Multi-server) correctly calculated :excluded status
|
||||
*
|
||||
* After this fix, all three paths now calculate and return :excluded status
|
||||
* consistently, preventing status drift and UI inconsistencies.
|
||||
*/
|
||||
it('ensures CalculatesExcludedStatus trait exists with required methods', function () {
|
||||
$traitFile = file_get_contents(__DIR__.'/../../app/Traits/CalculatesExcludedStatus.php');
|
||||
|
||||
// Verify trait has both status calculation methods
|
||||
expect($traitFile)
|
||||
->toContain('trait CalculatesExcludedStatus')
|
||||
->toContain('protected function calculateExcludedStatus(Collection $containers, Collection $excludedContainers): string')
|
||||
->toContain('protected function calculateExcludedStatusFromStrings(Collection $containerStatuses): string')
|
||||
->toContain('protected function getExcludedContainersFromDockerCompose(?string $dockerComposeRaw): Collection');
|
||||
});
|
||||
|
||||
it('ensures ComplexStatusCheck uses CalculatesExcludedStatus trait', function () {
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
|
||||
// Verify trait is used
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('use App\Traits\CalculatesExcludedStatus;')
|
||||
->toContain('use CalculatesExcludedStatus;');
|
||||
|
||||
// Verify it uses the trait method instead of inline code
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('return $this->calculateExcludedStatus($containers, $excludedContainers);');
|
||||
|
||||
// Verify it uses the trait helper for excluded containers
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);');
|
||||
});
|
||||
|
||||
it('ensures PushServerUpdateJob uses CalculatesExcludedStatus trait', function () {
|
||||
$pushServerUpdateJobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// Verify trait is used
|
||||
expect($pushServerUpdateJobFile)
|
||||
->toContain('use App\Traits\CalculatesExcludedStatus;')
|
||||
->toContain('use CalculatesExcludedStatus;');
|
||||
|
||||
// Verify it calculates excluded status instead of skipping (old behavior: continue)
|
||||
expect($pushServerUpdateJobFile)
|
||||
->toContain('// If all containers are excluded, calculate status from excluded containers')
|
||||
->toContain('$aggregatedStatus = $this->calculateExcludedStatusFromStrings($containerStatuses);');
|
||||
|
||||
// Verify it uses the trait helper for excluded containers
|
||||
expect($pushServerUpdateJobFile)
|
||||
->toContain('$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);');
|
||||
});
|
||||
|
||||
it('ensures PushServerUpdateJob calculates excluded status for applications', function () {
|
||||
$pushServerUpdateJobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// In aggregateMultiContainerStatuses, verify the all-excluded scenario
|
||||
// calculates status and updates the application
|
||||
expect($pushServerUpdateJobFile)
|
||||
->toContain('if ($relevantStatuses->isEmpty()) {')
|
||||
->toContain('$aggregatedStatus = $this->calculateExcludedStatusFromStrings($containerStatuses);')
|
||||
->toContain('if ($aggregatedStatus && $application->status !== $aggregatedStatus) {')
|
||||
->toContain('$application->status = $aggregatedStatus;')
|
||||
->toContain('$application->save();');
|
||||
});
|
||||
|
||||
it('ensures PushServerUpdateJob calculates excluded status for services', function () {
|
||||
$pushServerUpdateJobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// Count occurrences - should appear twice (once for applications, once for services)
|
||||
$calculateExcludedCount = substr_count(
|
||||
$pushServerUpdateJobFile,
|
||||
'$aggregatedStatus = $this->calculateExcludedStatusFromStrings($containerStatuses);'
|
||||
);
|
||||
|
||||
expect($calculateExcludedCount)->toBe(2, 'Should calculate excluded status for both applications and services');
|
||||
});
|
||||
|
||||
it('ensures GetContainersStatus uses CalculatesExcludedStatus trait', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
|
||||
// Verify trait is used
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('use App\Traits\CalculatesExcludedStatus;')
|
||||
->toContain('use CalculatesExcludedStatus;');
|
||||
|
||||
// Verify it calculates excluded status instead of returning null
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('// If all containers are excluded, calculate status from excluded containers')
|
||||
->toContain('return $this->calculateExcludedStatusFromStrings($containerStatuses);');
|
||||
|
||||
// Verify it uses the trait helper for excluded containers
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);');
|
||||
});
|
||||
|
||||
it('ensures GetContainersStatus calculates excluded status for applications', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
|
||||
// In aggregateApplicationStatus, verify the all-excluded scenario returns status
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('if ($relevantStatuses->isEmpty()) {')
|
||||
->toContain('return $this->calculateExcludedStatusFromStrings($containerStatuses);');
|
||||
});
|
||||
|
||||
it('ensures GetContainersStatus calculates excluded status for services', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
|
||||
// In aggregateServiceContainerStatuses, verify the all-excluded scenario updates status
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('$aggregatedStatus = $this->calculateExcludedStatusFromStrings($containerStatuses);')
|
||||
->toContain('if ($aggregatedStatus) {')
|
||||
->toContain('$statusFromDb = $subResource->status;')
|
||||
->toContain("if (\$statusFromDb !== \$aggregatedStatus) {\n \$subResource->update(['status' => \$aggregatedStatus]);");
|
||||
});
|
||||
|
||||
it('ensures excluded status format is consistent across all paths', function () {
|
||||
$traitFile = file_get_contents(__DIR__.'/../../app/Traits/CalculatesExcludedStatus.php');
|
||||
|
||||
// Trait now delegates to ContainerStatusAggregator and uses appendExcludedSuffix helper
|
||||
expect($traitFile)
|
||||
->toContain('use App\\Services\\ContainerStatusAggregator;')
|
||||
->toContain('$aggregator = new ContainerStatusAggregator;')
|
||||
->toContain('private function appendExcludedSuffix(string $status): string');
|
||||
|
||||
// Check that appendExcludedSuffix returns consistent colon format with :excluded suffix
|
||||
expect($traitFile)
|
||||
->toContain("return 'degraded:excluded';")
|
||||
->toContain("return 'paused:excluded';")
|
||||
->toContain("return 'starting:excluded';")
|
||||
->toContain("return 'exited:excluded';")
|
||||
->toContain('return "$status:excluded";'); // For running:healthy:excluded, running:unhealthy:excluded, etc.
|
||||
});
|
||||
|
||||
it('ensures all three paths check for exclude_from_hc flag consistently', function () {
|
||||
// All three should use the trait helper method
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
$pushServerUpdateJobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);');
|
||||
|
||||
expect($pushServerUpdateJobFile)
|
||||
->toContain('$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);');
|
||||
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);');
|
||||
|
||||
// The trait method should check both exclude_from_hc and restart: no
|
||||
$traitFile = file_get_contents(__DIR__.'/../../app/Traits/CalculatesExcludedStatus.php');
|
||||
expect($traitFile)
|
||||
->toContain('$excludeFromHc = data_get($serviceConfig, \'exclude_from_hc\', false);')
|
||||
->toContain('$restartPolicy = data_get($serviceConfig, \'restart\', \'always\');')
|
||||
->toContain('if ($excludeFromHc || $restartPolicy === \'no\') {');
|
||||
});
|
||||
|
||||
it('ensures calculateExcludedStatus uses ContainerStatusAggregator', function () {
|
||||
$traitFile = file_get_contents(__DIR__.'/../../app/Traits/CalculatesExcludedStatus.php');
|
||||
|
||||
// Check that the trait uses ContainerStatusAggregator service instead of duplicating logic
|
||||
expect($traitFile)
|
||||
->toContain('protected function calculateExcludedStatus(Collection $containers, Collection $excludedContainers): string')
|
||||
->toContain('use App\Services\ContainerStatusAggregator;')
|
||||
->toContain('$aggregator = new ContainerStatusAggregator;')
|
||||
->toContain('$aggregator->aggregateFromContainers($excludedOnly)');
|
||||
|
||||
// Check that it has appendExcludedSuffix helper for all states
|
||||
expect($traitFile)
|
||||
->toContain('private function appendExcludedSuffix(string $status): string')
|
||||
->toContain("return 'degraded:excluded';")
|
||||
->toContain("return 'paused:excluded';")
|
||||
->toContain("return 'starting:excluded';")
|
||||
->toContain("return 'exited:excluded';")
|
||||
->toContain('return "$status:excluded";'); // For running:healthy:excluded
|
||||
});
|
||||
|
||||
it('ensures calculateExcludedStatusFromStrings uses ContainerStatusAggregator', function () {
|
||||
$traitFile = file_get_contents(__DIR__.'/../../app/Traits/CalculatesExcludedStatus.php');
|
||||
|
||||
// Check that the trait uses ContainerStatusAggregator service instead of duplicating logic
|
||||
expect($traitFile)
|
||||
->toContain('protected function calculateExcludedStatusFromStrings(Collection $containerStatuses): string')
|
||||
->toContain('use App\Services\ContainerStatusAggregator;')
|
||||
->toContain('$aggregator = new ContainerStatusAggregator;')
|
||||
->toContain('$aggregator->aggregateFromStrings($containerStatuses)');
|
||||
|
||||
// Check that it has appendExcludedSuffix helper for all states
|
||||
expect($traitFile)
|
||||
->toContain('private function appendExcludedSuffix(string $status): string')
|
||||
->toContain("return 'degraded:excluded';")
|
||||
->toContain("return 'paused:excluded';")
|
||||
->toContain("return 'starting:excluded';")
|
||||
->toContain("return 'exited:excluded';")
|
||||
->toContain('return "$status:excluded";'); // For running:healthy:excluded
|
||||
});
|
||||
|
||||
it('verifies no code path skips update when all containers excluded', function () {
|
||||
$pushServerUpdateJobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
|
||||
// These patterns should NOT exist anymore (old behavior that caused drift)
|
||||
expect($pushServerUpdateJobFile)
|
||||
->not->toContain("// If all containers are excluded, don't update status");
|
||||
|
||||
expect($getContainersStatusFile)
|
||||
->not->toContain("// If all containers are excluded, don't update status");
|
||||
|
||||
// Instead, both should calculate excluded status
|
||||
expect($pushServerUpdateJobFile)
|
||||
->toContain('// If all containers are excluded, calculate status from excluded containers');
|
||||
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('// If all containers are excluded, calculate status from excluded containers');
|
||||
});
|
||||
@ -58,30 +58,25 @@ it('does not mark containers as unhealthy when health status is missing', functi
|
||||
|
||||
// We can't easily test the private aggregateContainerStatuses method directly,
|
||||
// but we can verify that the code doesn't default to 'unhealthy'
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify the fix: health status should not default to 'unhealthy'
|
||||
expect($complexStatusCheckFile)
|
||||
expect($aggregatorFile)
|
||||
->not->toContain("data_get(\$container, 'State.Health.Status', 'unhealthy')")
|
||||
->toContain("data_get(\$container, 'State.Health.Status')");
|
||||
|
||||
// Verify the health check logic for non-excluded containers
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('if ($containerHealth === \'unhealthy\') {');
|
||||
// Verify the health check logic
|
||||
expect($aggregatorFile)
|
||||
->toContain('if ($health === \'unhealthy\') {');
|
||||
});
|
||||
|
||||
it('only marks containers as unhealthy when health status explicitly equals unhealthy', function () {
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// For non-excluded containers (line ~108)
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('if ($containerHealth === \'unhealthy\') {')
|
||||
// Verify the service checks for explicit 'unhealthy' status
|
||||
expect($aggregatorFile)
|
||||
->toContain('if ($health === \'unhealthy\') {')
|
||||
->toContain('$hasUnhealthy = true;');
|
||||
|
||||
// For excluded containers (line ~145)
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('if ($containerHealth === \'unhealthy\') {')
|
||||
->toContain('$excludedHasUnhealthy = true;');
|
||||
});
|
||||
|
||||
it('handles missing health status correctly in GetContainersStatus', function () {
|
||||
@ -92,13 +87,14 @@ it('handles missing health status correctly in GetContainersStatus', function ()
|
||||
->not->toContain("data_get(\$container, 'State.Health.Status', 'unhealthy')")
|
||||
->toContain("data_get(\$container, 'State.Health.Status')");
|
||||
|
||||
// Verify it uses 'unknown' when health status is missing
|
||||
// Verify it uses 'unknown' when health status is missing (now using colon format)
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('$healthSuffix = $containerHealth ?? \'unknown\';');
|
||||
->toContain('$healthSuffix = $containerHealth ?? \'unknown\';')
|
||||
->toContain('ContainerStatusAggregator'); // Uses the service
|
||||
});
|
||||
|
||||
it('treats containers with running status and no healthcheck as not unhealthy', function () {
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// The logic should be:
|
||||
// 1. Get health status (may be null)
|
||||
@ -106,67 +102,65 @@ it('treats containers with running status and no healthcheck as not unhealthy',
|
||||
// 3. Don't mark as unhealthy if health status is null/missing
|
||||
|
||||
// Verify the condition explicitly checks for unhealthy
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('if ($containerHealth === \'unhealthy\')');
|
||||
expect($aggregatorFile)
|
||||
->toContain('if ($health === \'unhealthy\')');
|
||||
|
||||
// Verify this check is done for running containers
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('} elseif ($containerStatus === \'running\') {')
|
||||
expect($aggregatorFile)
|
||||
->toContain('} elseif ($state === \'running\') {')
|
||||
->toContain('$hasRunning = true;');
|
||||
});
|
||||
|
||||
it('tracks unknown health state in aggregation', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
// State machine logic now in ContainerStatusAggregator service
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify that $hasUnknown tracking variable exists
|
||||
expect($getContainersStatusFile)
|
||||
// Verify that $hasUnknown tracking variable exists in the service
|
||||
expect($aggregatorFile)
|
||||
->toContain('$hasUnknown = false;');
|
||||
|
||||
// Verify that unknown state is detected in status parsing
|
||||
expect($getContainersStatusFile)
|
||||
->toContain("if (str(\$status)->contains('unknown')) {")
|
||||
expect($aggregatorFile)
|
||||
->toContain("str(\$status)->contains('unknown')")
|
||||
->toContain('$hasUnknown = true;');
|
||||
});
|
||||
|
||||
it('preserves unknown health state in aggregated status with correct priority', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
// State machine logic now in ContainerStatusAggregator service (using colon format)
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify three-way priority in aggregation:
|
||||
// 1. Unhealthy (highest priority)
|
||||
// 2. Unknown (medium priority)
|
||||
// 3. Healthy (only when all explicitly healthy)
|
||||
|
||||
expect($getContainersStatusFile)
|
||||
expect($aggregatorFile)
|
||||
->toContain('if ($hasUnhealthy) {')
|
||||
->toContain("return 'running (unhealthy)';")
|
||||
->toContain("return 'running:unhealthy';")
|
||||
->toContain('} elseif ($hasUnknown) {')
|
||||
->toContain("return 'running (unknown)';")
|
||||
->toContain("return 'running:unknown';")
|
||||
->toContain('} else {')
|
||||
->toContain("return 'running (healthy)';");
|
||||
->toContain("return 'running:healthy';");
|
||||
});
|
||||
|
||||
it('tracks unknown health state in ComplexStatusCheck for multi-server applications', function () {
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
it('tracks unknown health state in ContainerStatusAggregator for all applications', function () {
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify that $hasUnknown tracking variable exists
|
||||
expect($complexStatusCheckFile)
|
||||
expect($aggregatorFile)
|
||||
->toContain('$hasUnknown = false;');
|
||||
|
||||
// Verify that unknown state is detected when containerHealth is null
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('} elseif ($containerHealth === null) {')
|
||||
// Verify that unknown state is detected when health is null or 'starting'
|
||||
expect($aggregatorFile)
|
||||
->toContain('} elseif (is_null($health) || $health === \'starting\') {')
|
||||
->toContain('$hasUnknown = true;');
|
||||
|
||||
// Verify excluded containers also track unknown
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('$excludedHasUnknown = false;');
|
||||
});
|
||||
|
||||
it('preserves unknown health state in ComplexStatusCheck aggregated status', function () {
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
it('preserves unknown health state in ContainerStatusAggregator aggregated status', function () {
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify three-way priority for non-excluded containers
|
||||
expect($complexStatusCheckFile)
|
||||
// Verify three-way priority for running containers in the service
|
||||
expect($aggregatorFile)
|
||||
->toContain('if ($hasUnhealthy) {')
|
||||
->toContain("return 'running:unhealthy';")
|
||||
->toContain('} elseif ($hasUnknown) {')
|
||||
@ -174,114 +168,115 @@ it('preserves unknown health state in ComplexStatusCheck aggregated status', fun
|
||||
->toContain('} else {')
|
||||
->toContain("return 'running:healthy';");
|
||||
|
||||
// Verify three-way priority for excluded containers
|
||||
// Verify ComplexStatusCheck delegates to the service
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('if ($excludedHasUnhealthy) {')
|
||||
->toContain("return 'running:unhealthy:excluded';")
|
||||
->toContain('} elseif ($excludedHasUnknown) {')
|
||||
->toContain("return 'running:unknown:excluded';")
|
||||
->toContain("return 'running:healthy:excluded';");
|
||||
->toContain('use App\\Services\\ContainerStatusAggregator;')
|
||||
->toContain('$aggregator = new ContainerStatusAggregator;')
|
||||
->toContain('$aggregator->aggregateFromContainers($relevantContainers);');
|
||||
});
|
||||
|
||||
it('preserves unknown health state in Service model aggregation', function () {
|
||||
$serviceFile = file_get_contents(__DIR__.'/../../app/Models/Service.php');
|
||||
|
||||
// Verify unknown is handled in non-excluded applications
|
||||
// Verify unknown is handled correctly
|
||||
expect($serviceFile)
|
||||
->toContain("} elseif (\$health->value() === 'unknown') {")
|
||||
->toContain("if (\$complexHealth !== 'unhealthy') {")
|
||||
->toContain("\$complexHealth = 'unknown';");
|
||||
->toContain("if (\$aggregateHealth !== 'unhealthy') {")
|
||||
->toContain("\$aggregateHealth = 'unknown';");
|
||||
|
||||
// The pattern should appear 4 times (non-excluded apps, non-excluded databases,
|
||||
// excluded apps, excluded databases)
|
||||
// The pattern should appear at least once (Service model has different aggregation logic than ContainerStatusAggregator)
|
||||
$unknownCount = substr_count($serviceFile, "} elseif (\$health->value() === 'unknown') {");
|
||||
expect($unknownCount)->toBe(4);
|
||||
expect($unknownCount)->toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('handles starting state (created/starting) in GetContainersStatus', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
// State machine logic now in ContainerStatusAggregator service
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify tracking variable exists
|
||||
expect($getContainersStatusFile)
|
||||
expect($aggregatorFile)
|
||||
->toContain('$hasStarting = false;');
|
||||
|
||||
// Verify detection for created/starting states
|
||||
expect($getContainersStatusFile)
|
||||
->toContain("} elseif (str(\$status)->contains('created') || str(\$status)->contains('starting')) {")
|
||||
expect($aggregatorFile)
|
||||
->toContain("str(\$status)->contains('created') || str(\$status)->contains('starting')")
|
||||
->toContain('$hasStarting = true;');
|
||||
|
||||
// Verify aggregation returns starting status
|
||||
expect($getContainersStatusFile)
|
||||
// Verify aggregation returns starting status (colon format)
|
||||
expect($aggregatorFile)
|
||||
->toContain('if ($hasStarting) {')
|
||||
->toContain("return 'starting (unknown)';");
|
||||
->toContain("return 'starting:unknown';");
|
||||
});
|
||||
|
||||
it('handles paused state in GetContainersStatus', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
// State machine logic now in ContainerStatusAggregator service
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify tracking variable exists
|
||||
expect($getContainersStatusFile)
|
||||
expect($aggregatorFile)
|
||||
->toContain('$hasPaused = false;');
|
||||
|
||||
// Verify detection for paused state
|
||||
expect($getContainersStatusFile)
|
||||
->toContain("} elseif (str(\$status)->contains('paused')) {")
|
||||
expect($aggregatorFile)
|
||||
->toContain("str(\$status)->contains('paused')")
|
||||
->toContain('$hasPaused = true;');
|
||||
|
||||
// Verify aggregation returns paused status
|
||||
expect($getContainersStatusFile)
|
||||
// Verify aggregation returns paused status (colon format)
|
||||
expect($aggregatorFile)
|
||||
->toContain('if ($hasPaused) {')
|
||||
->toContain("return 'paused (unknown)';");
|
||||
->toContain("return 'paused:unknown';");
|
||||
});
|
||||
|
||||
it('handles dead/removing states in GetContainersStatus', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
// State machine logic now in ContainerStatusAggregator service
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify tracking variable exists
|
||||
expect($getContainersStatusFile)
|
||||
expect($aggregatorFile)
|
||||
->toContain('$hasDead = false;');
|
||||
|
||||
// Verify detection for dead/removing states
|
||||
expect($getContainersStatusFile)
|
||||
->toContain("} elseif (str(\$status)->contains('dead') || str(\$status)->contains('removing')) {")
|
||||
expect($aggregatorFile)
|
||||
->toContain("str(\$status)->contains('dead') || str(\$status)->contains('removing')")
|
||||
->toContain('$hasDead = true;');
|
||||
|
||||
// Verify aggregation returns degraded status
|
||||
expect($getContainersStatusFile)
|
||||
// Verify aggregation returns degraded status (colon format)
|
||||
expect($aggregatorFile)
|
||||
->toContain('if ($hasDead) {')
|
||||
->toContain("return 'degraded (unhealthy)';");
|
||||
->toContain("return 'degraded:unhealthy';");
|
||||
});
|
||||
|
||||
it('handles edge case states in ComplexStatusCheck for non-excluded containers', function () {
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
it('handles edge case states in ContainerStatusAggregator for all containers', function () {
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify tracking variables exist
|
||||
expect($complexStatusCheckFile)
|
||||
// Verify tracking variables exist in the service
|
||||
expect($aggregatorFile)
|
||||
->toContain('$hasStarting = false;')
|
||||
->toContain('$hasPaused = false;')
|
||||
->toContain('$hasDead = false;');
|
||||
|
||||
// Verify detection for created/starting
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain("} elseif (\$containerStatus === 'created' || \$containerStatus === 'starting') {")
|
||||
expect($aggregatorFile)
|
||||
->toContain("} elseif (\$state === 'created' || \$state === 'starting') {")
|
||||
->toContain('$hasStarting = true;');
|
||||
|
||||
// Verify detection for paused
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain("} elseif (\$containerStatus === 'paused') {")
|
||||
expect($aggregatorFile)
|
||||
->toContain("} elseif (\$state === 'paused') {")
|
||||
->toContain('$hasPaused = true;');
|
||||
|
||||
// Verify detection for dead/removing
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain("} elseif (\$containerStatus === 'dead' || \$containerStatus === 'removing') {")
|
||||
expect($aggregatorFile)
|
||||
->toContain("} elseif (\$state === 'dead' || \$state === 'removing') {")
|
||||
->toContain('$hasDead = true;');
|
||||
});
|
||||
|
||||
it('handles edge case states in ComplexStatusCheck aggregation', function () {
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
it('handles edge case states in ContainerStatusAggregator aggregation', function () {
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify aggregation logic for edge cases
|
||||
expect($complexStatusCheckFile)
|
||||
// Verify aggregation logic for edge cases in the service
|
||||
expect($aggregatorFile)
|
||||
->toContain('if ($hasDead) {')
|
||||
->toContain("return 'degraded:unhealthy';")
|
||||
->toContain('if ($hasPaused) {')
|
||||
@ -290,51 +285,58 @@ it('handles edge case states in ComplexStatusCheck aggregation', function () {
|
||||
->toContain("return 'starting:unknown';");
|
||||
});
|
||||
|
||||
it('handles edge case states in Service model for all 4 locations', function () {
|
||||
it('handles edge case states in Service model', function () {
|
||||
$serviceFile = file_get_contents(__DIR__.'/../../app/Models/Service.php');
|
||||
|
||||
// Check for created/starting handling pattern
|
||||
$createdStartingCount = substr_count($serviceFile, "\$status->startsWith('created') || \$status->startsWith('starting')");
|
||||
expect($createdStartingCount)->toBe(4, 'created/starting handling should appear in all 4 locations');
|
||||
expect($createdStartingCount)->toBeGreaterThan(0, 'created/starting handling should exist');
|
||||
|
||||
// Check for paused handling pattern
|
||||
$pausedCount = substr_count($serviceFile, "\$status->startsWith('paused')");
|
||||
expect($pausedCount)->toBe(4, 'paused handling should appear in all 4 locations');
|
||||
expect($pausedCount)->toBeGreaterThan(0, 'paused handling should exist');
|
||||
|
||||
// Check for dead/removing handling pattern
|
||||
$deadRemovingCount = substr_count($serviceFile, "\$status->startsWith('dead') || \$status->startsWith('removing')");
|
||||
expect($deadRemovingCount)->toBe(4, 'dead/removing handling should appear in all 4 locations');
|
||||
expect($deadRemovingCount)->toBeGreaterThan(0, 'dead/removing handling should exist');
|
||||
});
|
||||
|
||||
it('appends :excluded suffix to excluded container statuses in GetContainersStatus', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
|
||||
// Verify that we check for exclude_from_hc flag
|
||||
// Verify that we use the trait for calculating excluded status
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('$excludeFromHc = data_get($serviceConfig, \'exclude_from_hc\', false);');
|
||||
->toContain('CalculatesExcludedStatus');
|
||||
|
||||
// Verify that we append :excluded suffix
|
||||
// Verify that we use the trait to calculate excluded status
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('$containerStatus = str_replace(\')\', \':excluded)\', $containerStatus);');
|
||||
->toContain('use CalculatesExcludedStatus;');
|
||||
});
|
||||
|
||||
it('skips containers with :excluded suffix in Service model non-excluded sections', function () {
|
||||
$serviceFile = file_get_contents(__DIR__.'/../../app/Models/Service.php');
|
||||
|
||||
// Verify that we skip :excluded containers in non-excluded sections
|
||||
// This should appear twice (once for applications, once for databases)
|
||||
$skipExcludedCount = substr_count($serviceFile, "if (\$health->contains(':excluded')) {");
|
||||
expect($skipExcludedCount)->toBeGreaterThanOrEqual(2, 'Should skip :excluded containers in non-excluded sections');
|
||||
// Verify that we have exclude_from_status field handling
|
||||
expect($serviceFile)
|
||||
->toContain('exclude_from_status');
|
||||
});
|
||||
|
||||
it('processes containers with :excluded suffix in Service model excluded sections', function () {
|
||||
$serviceFile = file_get_contents(__DIR__.'/../../app/Models/Service.php');
|
||||
|
||||
// Verify that we process :excluded containers in excluded sections
|
||||
$processExcludedCount = substr_count($serviceFile, "if (! \$health->contains(':excluded') && !");
|
||||
expect($processExcludedCount)->toBeGreaterThanOrEqual(2, 'Should process :excluded containers in excluded sections');
|
||||
|
||||
// Verify that we strip :excluded suffix before health comparison
|
||||
$stripExcludedCount = substr_count($serviceFile, "\$health = str(\$health)->replace(':excluded', '');");
|
||||
expect($stripExcludedCount)->toBeGreaterThanOrEqual(2, 'Should strip :excluded suffix in excluded sections');
|
||||
// Verify that we handle excluded status
|
||||
expect($serviceFile)
|
||||
->toContain(':excluded')
|
||||
->toContain('exclude_from_status');
|
||||
});
|
||||
|
||||
it('treats containers with starting health status as unknown in ContainerStatusAggregator', function () {
|
||||
$aggregatorFile = file_get_contents(__DIR__.'/../../app/Services/ContainerStatusAggregator.php');
|
||||
|
||||
// Verify that 'starting' health status is treated the same as null (unknown)
|
||||
// During Docker health check grace period, the health status is 'starting'
|
||||
// This should be treated as 'unknown' rather than 'healthy'
|
||||
expect($aggregatorFile)
|
||||
->toContain('} elseif (is_null($health) || $health === \'starting\') {')
|
||||
->toContain('$hasUnknown = true;');
|
||||
});
|
||||
|
||||
463
tests/Unit/ContainerStatusAggregatorTest.php
Normal file
463
tests/Unit/ContainerStatusAggregatorTest.php
Normal file
@ -0,0 +1,463 @@
|
||||
<?php
|
||||
|
||||
use App\Services\ContainerStatusAggregator;
|
||||
|
||||
beforeEach(function () {
|
||||
$this->aggregator = new ContainerStatusAggregator;
|
||||
});
|
||||
|
||||
describe('aggregateFromStrings', function () {
|
||||
test('returns exited:unhealthy for empty collection', function () {
|
||||
$result = $this->aggregator->aggregateFromStrings(collect());
|
||||
|
||||
expect($result)->toBe('exited:unhealthy');
|
||||
});
|
||||
|
||||
test('returns running:healthy for single healthy running container', function () {
|
||||
$statuses = collect(['running:healthy']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('running:healthy');
|
||||
});
|
||||
|
||||
test('returns running:unhealthy for single unhealthy running container', function () {
|
||||
$statuses = collect(['running:unhealthy']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('running:unhealthy');
|
||||
});
|
||||
|
||||
test('returns running:unknown for single running container with unknown health', function () {
|
||||
$statuses = collect(['running:unknown']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('running:unknown');
|
||||
});
|
||||
|
||||
test('returns degraded:unhealthy for restarting container', function () {
|
||||
$statuses = collect(['restarting']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('returns degraded:unhealthy for mixed running and exited containers', function () {
|
||||
$statuses = collect(['running:healthy', 'exited']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('returns running:unhealthy when one of multiple running containers is unhealthy', function () {
|
||||
$statuses = collect(['running:healthy', 'running:unhealthy', 'running:healthy']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('running:unhealthy');
|
||||
});
|
||||
|
||||
test('returns running:unknown when running containers have unknown health', function () {
|
||||
$statuses = collect(['running:unknown', 'running:healthy']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('running:unknown');
|
||||
});
|
||||
|
||||
test('returns degraded:unhealthy for crash loop (exited with restart count)', function () {
|
||||
$statuses = collect(['exited']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 5);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('returns exited:unhealthy for exited containers without restart count', function () {
|
||||
$statuses = collect(['exited']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 0);
|
||||
|
||||
expect($result)->toBe('exited:unhealthy');
|
||||
});
|
||||
|
||||
test('returns degraded:unhealthy for dead container', function () {
|
||||
$statuses = collect(['dead']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('returns degraded:unhealthy for removing container', function () {
|
||||
$statuses = collect(['removing']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('returns paused:unknown for paused container', function () {
|
||||
$statuses = collect(['paused']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('paused:unknown');
|
||||
});
|
||||
|
||||
test('returns starting:unknown for starting container', function () {
|
||||
$statuses = collect(['starting']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('starting:unknown');
|
||||
});
|
||||
|
||||
test('returns starting:unknown for created container', function () {
|
||||
$statuses = collect(['created']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('starting:unknown');
|
||||
});
|
||||
|
||||
test('handles parentheses format input (backward compatibility)', function () {
|
||||
$statuses = collect(['running (healthy)', 'running (unhealthy)']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('running:unhealthy');
|
||||
});
|
||||
|
||||
test('handles mixed colon and parentheses formats', function () {
|
||||
$statuses = collect(['running:healthy', 'running (unhealthy)', 'running:healthy']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('running:unhealthy');
|
||||
});
|
||||
|
||||
test('prioritizes restarting over all other states', function () {
|
||||
$statuses = collect(['restarting', 'running:healthy', 'paused', 'starting']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('prioritizes crash loop over running containers', function () {
|
||||
$statuses = collect(['exited', 'exited']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 3);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('prioritizes mixed state over healthy running', function () {
|
||||
$statuses = collect(['running:healthy', 'exited']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 0);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('prioritizes running over paused/starting/exited', function () {
|
||||
$statuses = collect(['running:healthy', 'starting', 'paused']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('running:healthy');
|
||||
});
|
||||
|
||||
test('prioritizes dead over paused/starting/exited', function () {
|
||||
$statuses = collect(['dead', 'paused', 'starting']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('prioritizes paused over starting/exited', function () {
|
||||
$statuses = collect(['paused', 'starting', 'exited']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('paused:unknown');
|
||||
});
|
||||
|
||||
test('prioritizes starting over exited', function () {
|
||||
$statuses = collect(['starting', 'exited']);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('starting:unknown');
|
||||
});
|
||||
});
|
||||
|
||||
describe('aggregateFromContainers', function () {
|
||||
test('returns exited:unhealthy for empty collection', function () {
|
||||
$result = $this->aggregator->aggregateFromContainers(collect());
|
||||
|
||||
expect($result)->toBe('exited:unhealthy');
|
||||
});
|
||||
|
||||
test('returns running:healthy for single healthy running container', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'running',
|
||||
'Health' => (object) ['Status' => 'healthy'],
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers);
|
||||
|
||||
expect($result)->toBe('running:healthy');
|
||||
});
|
||||
|
||||
test('returns running:unhealthy for single unhealthy running container', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'running',
|
||||
'Health' => (object) ['Status' => 'unhealthy'],
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers);
|
||||
|
||||
expect($result)->toBe('running:unhealthy');
|
||||
});
|
||||
|
||||
test('returns running:unknown for running container without health check', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'running',
|
||||
'Health' => null,
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers);
|
||||
|
||||
expect($result)->toBe('running:unknown');
|
||||
});
|
||||
|
||||
test('returns degraded:unhealthy for restarting container', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'restarting',
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('returns degraded:unhealthy for mixed running and exited containers', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'running',
|
||||
'Health' => (object) ['Status' => 'healthy'],
|
||||
],
|
||||
],
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'exited',
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('returns degraded:unhealthy for crash loop (exited with restart count)', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'exited',
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers, maxRestartCount: 5);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('returns exited:unhealthy for exited containers without restart count', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'exited',
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers, maxRestartCount: 0);
|
||||
|
||||
expect($result)->toBe('exited:unhealthy');
|
||||
});
|
||||
|
||||
test('returns degraded:unhealthy for dead container', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'dead',
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('returns paused:unknown for paused container', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'paused',
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers);
|
||||
|
||||
expect($result)->toBe('paused:unknown');
|
||||
});
|
||||
|
||||
test('returns starting:unknown for starting container', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'starting',
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers);
|
||||
|
||||
expect($result)->toBe('starting:unknown');
|
||||
});
|
||||
|
||||
test('returns starting:unknown for created container', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'created',
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers);
|
||||
|
||||
expect($result)->toBe('starting:unknown');
|
||||
});
|
||||
|
||||
test('handles multiple containers with various states', function () {
|
||||
$containers = collect([
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'running',
|
||||
'Health' => (object) ['Status' => 'healthy'],
|
||||
],
|
||||
],
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'running',
|
||||
'Health' => (object) ['Status' => 'unhealthy'],
|
||||
],
|
||||
],
|
||||
(object) [
|
||||
'State' => (object) [
|
||||
'Status' => 'running',
|
||||
'Health' => null,
|
||||
],
|
||||
],
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromContainers($containers);
|
||||
|
||||
expect($result)->toBe('running:unhealthy');
|
||||
});
|
||||
});
|
||||
|
||||
describe('state priority enforcement', function () {
|
||||
test('restarting has highest priority', function () {
|
||||
$statuses = collect([
|
||||
'restarting',
|
||||
'running:healthy',
|
||||
'dead',
|
||||
'paused',
|
||||
'starting',
|
||||
'exited',
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('crash loop has second highest priority', function () {
|
||||
$statuses = collect([
|
||||
'exited',
|
||||
'running:healthy',
|
||||
'paused',
|
||||
'starting',
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 1);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('mixed state (running + exited) has third priority', function () {
|
||||
$statuses = collect([
|
||||
'running:healthy',
|
||||
'exited',
|
||||
'paused',
|
||||
'starting',
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 0);
|
||||
|
||||
expect($result)->toBe('degraded:unhealthy');
|
||||
});
|
||||
|
||||
test('running:unhealthy has priority over running:unknown', function () {
|
||||
$statuses = collect([
|
||||
'running:unknown',
|
||||
'running:unhealthy',
|
||||
'running:healthy',
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('running:unhealthy');
|
||||
});
|
||||
|
||||
test('running:unknown has priority over running:healthy', function () {
|
||||
$statuses = collect([
|
||||
'running:unknown',
|
||||
'running:healthy',
|
||||
]);
|
||||
|
||||
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||
|
||||
expect($result)->toBe('running:unknown');
|
||||
});
|
||||
});
|
||||
@ -13,17 +13,23 @@
|
||||
it('ensures ComplexStatusCheck returns excluded status when all containers excluded', function () {
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
|
||||
// Check that when all containers are excluded, the status calculation
|
||||
// processes excluded containers and returns status with :excluded suffix
|
||||
// Check that when all containers are excluded, ComplexStatusCheck uses the trait
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('// If all containers are excluded, calculate status from excluded containers')
|
||||
->toContain('// but mark it with :excluded to indicate monitoring is disabled')
|
||||
->toContain('if ($relevantContainerCount === 0) {')
|
||||
->toContain("return 'running:unhealthy:excluded';")
|
||||
->toContain("return 'running:unknown:excluded';")
|
||||
->toContain("return 'running:healthy:excluded';")
|
||||
->toContain('if ($relevantContainers->isEmpty()) {')
|
||||
->toContain('return $this->calculateExcludedStatus($containers, $excludedContainers);');
|
||||
|
||||
// Check that the trait uses ContainerStatusAggregator and appends :excluded suffix
|
||||
$traitFile = file_get_contents(__DIR__.'/../../app/Traits/CalculatesExcludedStatus.php');
|
||||
expect($traitFile)
|
||||
->toContain('ContainerStatusAggregator')
|
||||
->toContain('appendExcludedSuffix')
|
||||
->toContain('$aggregator->aggregateFromContainers($excludedOnly)')
|
||||
->toContain("return 'degraded:excluded';")
|
||||
->toContain("return 'exited:excluded';");
|
||||
->toContain("return 'paused:excluded';")
|
||||
->toContain("return 'exited:excluded';")
|
||||
->toContain('return "$status:excluded";'); // For running:healthy:excluded
|
||||
});
|
||||
|
||||
it('ensures Service model returns excluded status when all services excluded', function () {
|
||||
@ -32,64 +38,59 @@ it('ensures Service model returns excluded status when all services excluded', f
|
||||
// Check that when all services are excluded from status checks,
|
||||
// the Service model calculates real status and returns it with :excluded suffix
|
||||
expect($serviceModelFile)
|
||||
->toContain('// If all services are excluded from status checks, calculate status from excluded containers')
|
||||
->toContain('// but mark it with :excluded to indicate monitoring is disabled')
|
||||
->toContain('if (! $hasNonExcluded && ($complexStatus === null && $complexHealth === null)) {')
|
||||
->toContain('// Calculate status from excluded containers')
|
||||
->toContain('return "{$excludedStatus}:excluded";')
|
||||
->toContain("return 'exited:excluded';");
|
||||
->toContain('exclude_from_status')
|
||||
->toContain(':excluded')
|
||||
->toContain('CalculatesExcludedStatus');
|
||||
});
|
||||
|
||||
it('ensures Service model returns unknown:excluded when no containers exist', function () {
|
||||
it('ensures Service model returns unknown:unknown:excluded when no containers exist', function () {
|
||||
$serviceModelFile = file_get_contents(__DIR__.'/../../app/Models/Service.php');
|
||||
|
||||
// Check that when a service has no applications or databases at all,
|
||||
// the Service model returns 'unknown:excluded' instead of 'exited:excluded'
|
||||
// the Service model returns 'unknown:unknown:excluded' instead of 'exited:unhealthy:excluded'
|
||||
// This prevents misleading status display when containers don't exist
|
||||
expect($serviceModelFile)
|
||||
->toContain('// If no status was calculated at all (no containers exist), return unknown')
|
||||
->toContain('if ($excludedStatus === null && $excludedHealth === null) {')
|
||||
->toContain("return 'unknown:excluded';");
|
||||
->toContain("return 'unknown:unknown:excluded';");
|
||||
});
|
||||
|
||||
it('ensures GetContainersStatus returns null when all containers excluded', function () {
|
||||
it('ensures GetContainersStatus calculates excluded status when all containers excluded', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
|
||||
// Check that when all containers are excluded, the aggregateApplicationStatus
|
||||
// method returns null to avoid updating status
|
||||
// method calculates and returns status with :excluded suffix
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('// If all containers are excluded, don\'t update status')
|
||||
->toContain("if (\$relevantStatuses->isEmpty()) {\n return null;\n }");
|
||||
->toContain('// If all containers are excluded, calculate status from excluded containers')
|
||||
->toContain('if ($relevantStatuses->isEmpty()) {')
|
||||
->toContain('return $this->calculateExcludedStatusFromStrings($containerStatuses);');
|
||||
});
|
||||
|
||||
it('ensures exclude_from_hc flag is properly checked in ComplexStatusCheck', function () {
|
||||
$complexStatusCheckFile = file_get_contents(__DIR__.'/../../app/Actions/Shared/ComplexStatusCheck.php');
|
||||
|
||||
// Verify that exclude_from_hc is properly parsed from docker-compose
|
||||
// Verify that exclude_from_hc is parsed using trait helper
|
||||
expect($complexStatusCheckFile)
|
||||
->toContain('$excludeFromHc = data_get($serviceConfig, \'exclude_from_hc\', false);')
|
||||
->toContain('if ($excludeFromHc || $restartPolicy === \'no\') {')
|
||||
->toContain('$excludedContainers->push($serviceName);');
|
||||
->toContain('$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);');
|
||||
});
|
||||
|
||||
it('ensures exclude_from_hc flag is properly checked in GetContainersStatus', function () {
|
||||
$getContainersStatusFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
|
||||
// Verify that exclude_from_hc is properly parsed from docker-compose
|
||||
// Verify that exclude_from_hc is parsed using trait helper
|
||||
expect($getContainersStatusFile)
|
||||
->toContain('$excludeFromHc = data_get($serviceConfig, \'exclude_from_hc\', false);')
|
||||
->toContain('if ($excludeFromHc || $restartPolicy === \'no\') {')
|
||||
->toContain('$excludedContainers->push($serviceName);');
|
||||
->toContain('$excludedContainers = $this->getExcludedContainersFromDockerCompose($dockerComposeRaw);');
|
||||
});
|
||||
|
||||
it('ensures UI displays excluded status correctly in status component', function () {
|
||||
$servicesStatusFile = file_get_contents(__DIR__.'/../../resources/views/components/status/services.blade.php');
|
||||
|
||||
// Verify that the status component detects :excluded suffix and shows monitoring disabled message
|
||||
// Verify that the status component transforms :excluded suffix to (excluded) for better display
|
||||
expect($servicesStatusFile)
|
||||
->toContain('$isExcluded = str($complexStatus)->endsWith(\':excluded\');')
|
||||
->toContain('$displayStatus = $isExcluded ? str($complexStatus)->beforeLast(\':excluded\') : $complexStatus;')
|
||||
->toContain('(Monitoring Disabled)');
|
||||
->toContain('$parts = explode(\':\', $complexStatus);')
|
||||
->toContain('// Has health status: running:unhealthy:excluded → Running (unhealthy, excluded)')
|
||||
->toContain('// No health status: exited:excluded → Exited (excluded)');
|
||||
});
|
||||
|
||||
it('ensures UI handles excluded status in service heading buttons', function () {
|
||||
|
||||
@ -1,184 +0,0 @@
|
||||
<?php
|
||||
|
||||
/**
|
||||
* Unit tests for PushServerUpdateJob status aggregation logic.
|
||||
*
|
||||
* These tests verify that the job correctly aggregates container statuses
|
||||
* when processing Sentinel updates, with proper handling of:
|
||||
* - running (healthy) - all containers running and healthy
|
||||
* - running (unhealthy) - some containers unhealthy
|
||||
* - running (unknown) - some containers with unknown health status
|
||||
*
|
||||
* The aggregation follows a priority system: unhealthy > unknown > healthy
|
||||
*
|
||||
* This ensures consistency with GetContainersStatus::aggregateApplicationStatus()
|
||||
* and prevents the bug where "unknown" status was incorrectly converted to "healthy".
|
||||
*/
|
||||
it('aggregates status with unknown health state correctly', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// Verify that hasUnknown tracking variable exists
|
||||
expect($jobFile)
|
||||
->toContain('$hasUnknown = false;')
|
||||
->toContain('if (str($status)->contains(\'unknown\')) {')
|
||||
->toContain('$hasUnknown = true;');
|
||||
|
||||
// Verify 3-way status priority logic (unhealthy > unknown > healthy)
|
||||
expect($jobFile)
|
||||
->toContain('if ($hasUnhealthy) {')
|
||||
->toContain('$aggregatedStatus = \'running (unhealthy)\';')
|
||||
->toContain('} elseif ($hasUnknown) {')
|
||||
->toContain('$aggregatedStatus = \'running (unknown)\';')
|
||||
->toContain('} else {')
|
||||
->toContain('$aggregatedStatus = \'running (healthy)\';');
|
||||
});
|
||||
|
||||
it('checks for unknown status alongside unhealthy status', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// Verify unknown check is placed alongside unhealthy check
|
||||
expect($jobFile)
|
||||
->toContain('if (str($status)->contains(\'unhealthy\')) {')
|
||||
->toContain('if (str($status)->contains(\'unknown\')) {');
|
||||
});
|
||||
|
||||
it('follows same priority as GetContainersStatus', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
$getContainersFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
|
||||
// Both should track hasUnknown
|
||||
expect($jobFile)->toContain('$hasUnknown = false;');
|
||||
expect($getContainersFile)->toContain('$hasUnknown = false;');
|
||||
|
||||
// Both should check for 'unknown' in status strings
|
||||
expect($jobFile)->toContain('if (str($status)->contains(\'unknown\')) {');
|
||||
expect($getContainersFile)->toContain('if (str($status)->contains(\'unknown\')) {');
|
||||
|
||||
// Both should prioritize unhealthy over unknown over healthy
|
||||
expect($jobFile)->toContain('} elseif ($hasUnknown) {');
|
||||
expect($getContainersFile)->toContain('} elseif ($hasUnknown) {');
|
||||
});
|
||||
|
||||
it('does not default unknown to healthy status', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// The old buggy code was:
|
||||
// $aggregatedStatus = $hasUnhealthy ? 'running (unhealthy)' : 'running (healthy)';
|
||||
// This would make unknown -> healthy
|
||||
|
||||
// Verify we're NOT using ternary operator for status assignment
|
||||
expect($jobFile)
|
||||
->not->toContain('$aggregatedStatus = $hasUnhealthy ? \'running (unhealthy)\' : \'running (healthy)\';');
|
||||
|
||||
// Verify we ARE using if-elseif-else with proper unknown handling
|
||||
expect($jobFile)
|
||||
->toContain('if ($hasUnhealthy) {')
|
||||
->toContain('} elseif ($hasUnknown) {')
|
||||
->toContain('$aggregatedStatus = \'running (unknown)\';');
|
||||
});
|
||||
|
||||
it('initializes all required status tracking variables', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// Verify all three tracking variables are initialized together
|
||||
$pattern = '/\$hasRunning\s*=\s*false;\s*\$hasUnhealthy\s*=\s*false;\s*\$hasUnknown\s*=\s*false;/s';
|
||||
|
||||
expect(preg_match($pattern, $jobFile))->toBe(1,
|
||||
'All status tracking variables ($hasRunning, $hasUnhealthy, $hasUnknown) should be initialized together');
|
||||
});
|
||||
|
||||
it('preserves unknown status through sentinel updates', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// The critical path: when a status contains 'running' AND 'unknown',
|
||||
// both flags should be set
|
||||
expect($jobFile)
|
||||
->toContain('if (str($status)->contains(\'running\')) {')
|
||||
->toContain('$hasRunning = true;')
|
||||
->toContain('if (str($status)->contains(\'unhealthy\')) {')
|
||||
->toContain('$hasUnhealthy = true;')
|
||||
->toContain('if (str($status)->contains(\'unknown\')) {')
|
||||
->toContain('$hasUnknown = true;');
|
||||
|
||||
// And then unknown should have priority over healthy in aggregation
|
||||
expect($jobFile)
|
||||
->toContain('} elseif ($hasUnknown) {')
|
||||
->toContain('$aggregatedStatus = \'running (unknown)\';');
|
||||
});
|
||||
|
||||
it('implements service multi-container aggregation', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// Verify service container collection exists
|
||||
expect($jobFile)
|
||||
->toContain('public Collection $serviceContainerStatuses;')
|
||||
->toContain('$this->serviceContainerStatuses = collect();');
|
||||
|
||||
// Verify aggregateServiceContainerStatuses method exists
|
||||
expect($jobFile)
|
||||
->toContain('private function aggregateServiceContainerStatuses()')
|
||||
->toContain('$this->aggregateServiceContainerStatuses();');
|
||||
|
||||
// Verify service aggregation uses same logic as applications
|
||||
expect($jobFile)
|
||||
->toContain('$hasUnknown = false;');
|
||||
});
|
||||
|
||||
it('services use same priority as applications', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// Both aggregation methods should use the same priority logic
|
||||
$applicationAggregation = <<<'PHP'
|
||||
if ($hasUnhealthy) {
|
||||
$aggregatedStatus = 'running (unhealthy)';
|
||||
} elseif ($hasUnknown) {
|
||||
$aggregatedStatus = 'running (unknown)';
|
||||
} else {
|
||||
$aggregatedStatus = 'running (healthy)';
|
||||
}
|
||||
PHP;
|
||||
|
||||
// Count occurrences - should appear twice (once for apps, once for services)
|
||||
$occurrences = substr_count($jobFile, $applicationAggregation);
|
||||
expect($occurrences)->toBeGreaterThanOrEqual(2, 'Priority logic should appear for both applications and services');
|
||||
});
|
||||
|
||||
it('collects service containers before aggregating', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// Verify service containers are collected, not immediately updated
|
||||
expect($jobFile)
|
||||
->toContain('$key = $serviceId.\':\'.$subType.\':\'.$subId;')
|
||||
->toContain('$this->serviceContainerStatuses->get($key)->put($containerName, $containerStatus);');
|
||||
|
||||
// Verify aggregation happens after collection
|
||||
expect($jobFile)
|
||||
->toContain('$this->aggregateMultiContainerStatuses();')
|
||||
->toContain('$this->aggregateServiceContainerStatuses();');
|
||||
});
|
||||
|
||||
it('defaults to unknown when health_status is missing from Sentinel data', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
|
||||
// Verify we use null coalescing to default to 'unknown', not 'unhealthy'
|
||||
// This is critical for containers without healthcheck defined
|
||||
expect($jobFile)
|
||||
->toContain('$rawHealthStatus = data_get($container, \'health_status\');')
|
||||
->toContain('$containerHealth = $rawHealthStatus ?? \'unknown\';')
|
||||
->not->toContain('data_get($container, \'health_status\', \'unhealthy\')');
|
||||
});
|
||||
|
||||
it('matches SSH path default health status behavior', function () {
|
||||
$jobFile = file_get_contents(__DIR__.'/../../app/Jobs/PushServerUpdateJob.php');
|
||||
$getContainersFile = file_get_contents(__DIR__.'/../../app/Actions/Docker/GetContainersStatus.php');
|
||||
|
||||
// Both paths should default to 'unknown' when health status is missing
|
||||
// Sentinel path: health_status field missing -> 'unknown'
|
||||
expect($jobFile)->toContain('?? \'unknown\'');
|
||||
|
||||
// SSH path: State.Health.Status missing -> 'unknown'
|
||||
expect($getContainersFile)->toContain('?? \'unknown\'');
|
||||
|
||||
// Neither should use 'unhealthy' as default for missing health status
|
||||
expect($jobFile)->not->toContain('data_get($container, \'health_status\', \'unhealthy\')');
|
||||
});
|
||||
53
tests/Unit/ServerStatusAccessorTest.php
Normal file
53
tests/Unit/ServerStatusAccessorTest.php
Normal file
@ -0,0 +1,53 @@
|
||||
<?php
|
||||
|
||||
use App\Models\Application;
|
||||
use App\Models\Server;
|
||||
|
||||
/**
|
||||
* Test the Application::serverStatus() accessor
|
||||
*
|
||||
* This accessor determines if the underlying server infrastructure is functional.
|
||||
* It should check Server::isFunctional() for the main server and all additional servers.
|
||||
* It should NOT be affected by container/application health status (e.g., degraded:unhealthy).
|
||||
*
|
||||
* The bug that was fixed: Previously, it checked pivot.status and returned false
|
||||
* when any additional server had status != 'running', including 'degraded:unhealthy'.
|
||||
* This caused false "server has problems" warnings when the server was fine but
|
||||
* containers were unhealthy.
|
||||
*/
|
||||
it('checks server infrastructure health not container status', function () {
|
||||
// This is a documentation test to explain the fix
|
||||
// The serverStatus accessor should:
|
||||
// 1. Check if main server is functional (Server::isFunctional())
|
||||
// 2. Check if each additional server is functional (Server::isFunctional())
|
||||
// 3. NOT check pivot.status (that's application/container status, not server status)
|
||||
//
|
||||
// Before fix: Checked pivot.status !== 'running', causing false positives
|
||||
// After fix: Only checks Server::isFunctional() for infrastructure health
|
||||
|
||||
expect(true)->toBeTrue();
|
||||
})->note('The serverStatus accessor now correctly checks only server infrastructure health, not container status');
|
||||
|
||||
it('has correct logic in serverStatus accessor', function () {
|
||||
// Read the actual code to verify the fix
|
||||
$reflection = new ReflectionClass(Application::class);
|
||||
$source = file_get_contents($reflection->getFileName());
|
||||
|
||||
// Extract just the serverStatus accessor method
|
||||
preg_match('/protected function serverStatus\(\): Attribute\s*\{.*?^\s{4}\}/ms', $source, $matches);
|
||||
$serverStatusCode = $matches[0] ?? '';
|
||||
|
||||
expect($serverStatusCode)->not->toBeEmpty('serverStatus accessor should exist');
|
||||
|
||||
// Check that the new logic exists (checks isFunctional on each server)
|
||||
expect($serverStatusCode)
|
||||
->toContain('$main_server_functional = $this->destination?->server?->isFunctional()')
|
||||
->toContain('foreach ($this->additional_servers as $server)')
|
||||
->toContain('if (! $server->isFunctional())');
|
||||
|
||||
// Check that the old buggy logic is removed from serverStatus accessor
|
||||
expect($serverStatusCode)
|
||||
->not->toContain('pluck(\'pivot.status\')')
|
||||
->not->toContain('str($status)->before(\':\')')
|
||||
->not->toContain('if ($server_status !== \'running\')');
|
||||
})->note('Verifies that the serverStatus accessor uses the correct logic');
|
||||
@ -13,7 +13,7 @@
|
||||
"version": "1.0.10"
|
||||
},
|
||||
"sentinel": {
|
||||
"version": "0.0.16"
|
||||
"version": "0.0.17"
|
||||
}
|
||||
},
|
||||
"traefik": {
|
||||
|
||||
Loading…
Reference in New Issue
Block a user