Skip to content

Installed Versions Cleanup Implementation Plan#

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add bmo query installed and bmo manage uninstall commands that cross-reference agent inventory with CommandCenter stage data, and fix the query versions SQL predicate that ignores Active=0 stages.

Architecture: CommandCenter gains a new /api/versions/cleanup-candidates endpoint (SQL query: version × StageActiveCount). The orchestrator gains an InstalledVersionsResolver (agent fan-out or fleet snapshot), a VersionMatcher (smart --match parser), and two new commands that compose those services with the existing RemoveOnServerAsync helper extracted from RemoveCommand.

Tech Stack: .NET 10, Dapper, Spectre.Console, Moq, FluentAssertions, xUnit

File Map#

File Change
src/BenefitManager.Contracts/Models/ManagementModels.cs Add CleanupCandidate record
src/BenefitManager.CommandCenter.Contracts/CommandCenterContractsJsonContext.cs Register CleanupCandidate + List<CleanupCandidate>
src/BenefitManager.CommandCenter/Services/CommandCenterQueryService.cs Fix GetUnusedVersionsAsync; add GetCleanupCandidatesAsync
src/BenefitManager.CommandCenter/Services/ICommandCenterQueryService.cs Add GetCleanupCandidatesAsync signature
src/BenefitManager.CommandCenter/Endpoints/VersionsEndpoints.cs NEW — hosts /api/versions/unused (moved) + /api/versions/cleanup-candidates
src/BenefitManager.CommandCenter/Endpoints/CustomerEndpoints.cs Remove /api/versions/unused (moved to VersionsEndpoints)
src/BenefitManager.CommandCenter/Program.cs Register MapVersionsEndpoints()
src/BenefitManager.CommandCenter.Client/ICommandCenterClient.cs Add GetCleanupCandidatesAsync
src/BenefitManager.CommandCenter.Client/CommandCenterClient.cs Implement GetCleanupCandidatesAsync
src/BenefitManager.Orchestrator/Services/InstalledVersionsResolver.cs NEW — resolver service + InstalledVersion record + TruthSource enum
src/BenefitManager.Orchestrator/Services/VersionMatcher.cs NEW — pure match parser
src/BenefitManager.Orchestrator/Commands/RemoveCommand.cs Extract RemoveOnServerAsync internal static helper
src/BenefitManager.Orchestrator/Commands/InstalledQueryCommand.cs NEW — bmo query installed
src/BenefitManager.Orchestrator/Commands/UninstallCommand.cs NEW — bmo manage uninstall
src/BenefitManager.Orchestrator/Program.cs Register both new commands
tests/BenefitManager.Orchestrator.Tests/VersionMatcherTests.cs NEW
tests/BenefitManager.Orchestrator.Tests/InstalledVersionsResolverTests.cs NEW
tests/BenefitManager.Orchestrator.Tests/UninstallCommandTests.cs NEW
tests/BenefitManager.Orchestrator.Tests/CommandCenterQueryServiceTests.cs Extend with InstalledQueryCommand tests
readme.md Add query installed and manage uninstall sections

Task 1: Fix GetUnusedVersionsAsync SQL predicate#

Files: - Modify: src/BenefitManager.CommandCenter/Services/CommandCenterQueryService.cs:52-58 - Extend: tests/BenefitManager.Orchestrator.Tests/CommandCenterQueryServiceTests.cs

  • [ ] Step 1: Write a failing test that documents the expected behaviour

Add to tests/BenefitManager.Orchestrator.Tests/CommandCenterQueryServiceTests.cs inside class CommandCenterQueryServiceTests:

[Fact]
public async Task VersionsCommand_IncludesVersion_WhenOnlyBoundToInactiveStage()
{
    // Arrange: CC returns a version that was previously blocked by Active=0 stage
    var version = new ApplicationVersionInfo("25.4.00", false, new DateTime(2025, 1, 1), "vmbenefitsbmp1");
    var client = new Mock<ICommandCenterClient>();
    client.Setup(x => x.GetUnusedVersionsAsync("Productie", It.IsAny<CancellationToken>()))
        .ReturnsAsync([version]); // after the SQL fix, this version IS returned

    // Act
    var result = await new VersionsCommand(client.Object)
        .ExecuteAsync(null!, new VersionsSettings { Environment = "Productie" }, TestContext.Current.CancellationToken);

    // Assert
    result.Should().Be(0);
    client.Verify(x => x.GetUnusedVersionsAsync("Productie", It.IsAny<CancellationToken>()), Times.Once);
}
  • [ ] Step 2: Run the test — it should pass (the command layer already passes through; this confirms the mock contract)
dotnet test tests/BenefitManager.Orchestrator.Tests --filter "VersionsCommand_IncludesVersion_WhenOnlyBoundToInactiveStage" -v n

Expected: Passed!

  • [ ] Step 3: Fix the SQL predicate in CommandCenterQueryService.cs:43-61

Replace the entire GetUnusedVersionsAsync method:

public async Task<List<ApplicationVersionInfo>> GetUnusedVersionsAsync(string environment, CancellationToken ct = default)
{
    using var conn = new SqlConnection(GetConn(environment));
    const string sql = """
        SELECT av.[Identifier]              AS Version,
               av.[Online],
               av.[CreateDate],
               STRING_AGG(s.[Name], ', ')   AS InstalledServers
        FROM [dbo].[TblApplicationVersion] av
        LEFT JOIN [dbo].[TblServerVersion] sv ON sv.VersionKey = av.[Key]
        LEFT JOIN [dbo].[TblServer] s          ON s.[Key] = sv.ServerKey
        WHERE NOT EXISTS (
            SELECT 1 FROM [dbo].[TblCustomerStage] cs
            WHERE cs.VersionKey = av.[Key] AND cs.Active = 1
        )
        GROUP BY av.[Identifier], av.[Online], av.[CreateDate]
        ORDER BY av.[Identifier]
        """;
    return (await conn.QueryAsync<ApplicationVersionInfo>(sql)).ToList();
}

Changes from old: - LEFT JOIN TblServerVersion sv — removed AND sv.Active = 1 so all server records are counted - WHERE cs.Active = 1 added to NOT EXISTS so disabled stages no longer pin a version

  • [ ] Step 4: Build to confirm no errors
dotnet build src/BenefitManager.CommandCenter/BenefitManager.CommandCenter.csproj -v q

Expected: Build succeeded. 0 Warning(s) 0 Error(s)

  • [ ] Step 5: Commit
git add src/BenefitManager.CommandCenter/Services/CommandCenterQueryService.cs `
        tests/BenefitManager.Orchestrator.Tests/CommandCenterQueryServiceTests.cs
git commit -m "fix(query-versions): exclude Active=0 stages from unused-versions predicate"

Task 2: Add CleanupCandidate DTO and register in JSON context#

Files: - Modify: src/BenefitManager.Contracts/Models/ManagementModels.cs - Modify: src/BenefitManager.CommandCenter.Contracts/CommandCenterContractsJsonContext.cs

  • [ ] Step 1: Add CleanupCandidate record to ManagementModels.cs

Append at the end of src/BenefitManager.Contracts/Models/ManagementModels.cs:

public record CleanupCandidate(string Version, string Environment, int StageActiveCount);
  • [ ] Step 2: Register in CommandCenterContractsJsonContext.cs

In src/BenefitManager.CommandCenter.Contracts/CommandCenterContractsJsonContext.cs, add two lines after the List<ApplicationVersionInfo> registration (line 23):

[JsonSerializable(typeof(CleanupCandidate))]
[JsonSerializable(typeof(List<CleanupCandidate>))]
  • [ ] Step 3: Build Contracts + CommandCenter.Contracts
dotnet build src/BenefitManager.Contracts/BenefitManager.Contracts.csproj -v q
dotnet build src/BenefitManager.CommandCenter.Contracts/BenefitManager.CommandCenter.Contracts.csproj -v q

Expected: both Build succeeded. 0 Warning(s) 0 Error(s)

  • [ ] Step 4: Commit
git add src/BenefitManager.Contracts/Models/ManagementModels.cs `
        src/BenefitManager.CommandCenter.Contracts/CommandCenterContractsJsonContext.cs
git commit -m "feat: add CleanupCandidate DTO and register in JSON source-gen context"

Task 3: Add GetCleanupCandidatesAsync to the query service#

Files: - Modify: src/BenefitManager.CommandCenter/Services/ICommandCenterQueryService.cs - Modify: src/BenefitManager.CommandCenter/Services/CommandCenterQueryService.cs

  • [ ] Step 1: Add the interface member

In src/BenefitManager.CommandCenter/Services/ICommandCenterQueryService.cs, add after the existing GetUnusedVersionsAsync line:

Task<List<CleanupCandidate>> GetCleanupCandidatesAsync(string environment, CancellationToken ct = default);

Full file after edit:

using BenefitManager.Contracts.Models;

namespace BenefitManager.CommandCenter.Services;

public interface ICommandCenterQueryService
{
    Task<List<CustomerSummary>> GetCustomersAsync(string environment, CancellationToken ct = default);
    Task<List<StageSummary>> GetStagesAsync(string environment, string? customerName = null, CancellationToken ct = default);
    Task<List<ApplicationVersionInfo>> GetUnusedVersionsAsync(string environment, CancellationToken ct = default);
    Task<List<CleanupCandidate>> GetCleanupCandidatesAsync(string environment, CancellationToken ct = default);
}
  • [ ] Step 2: Implement GetCleanupCandidatesAsync in CommandCenterQueryService.cs

Add after the closing brace of GetUnusedVersionsAsync (before GetConn):

public async Task<List<CleanupCandidate>> GetCleanupCandidatesAsync(string environment, CancellationToken ct = default)
{
    using var conn = new SqlConnection(GetConn(environment));
    const string sql = """
        SELECT av.[Identifier]                                        AS Version,
               @Environment                                           AS Environment,
               COUNT(CASE WHEN cs.Active = 1 THEN 1 END)             AS StageActiveCount
        FROM [dbo].[TblApplicationVersion] av
        LEFT JOIN [dbo].[TblCustomerStage] cs ON cs.VersionKey = av.[Key]
        GROUP BY av.[Identifier]
        ORDER BY av.[Identifier]
        """;
    return (await conn.QueryAsync<CleanupCandidate>(sql, new { Environment = environment })).ToList();
}
  • [ ] Step 3: Build to confirm
dotnet build src/BenefitManager.CommandCenter/BenefitManager.CommandCenter.csproj -v q

Expected: Build succeeded. 0 Warning(s) 0 Error(s)

  • [ ] Step 4: Commit
git add src/BenefitManager.CommandCenter/Services/ICommandCenterQueryService.cs `
        src/BenefitManager.CommandCenter/Services/CommandCenterQueryService.cs
git commit -m "feat(cc): add GetCleanupCandidatesAsync returning version x StageActiveCount"

Task 4: Create VersionsEndpoints.cs and wire it up#

Move /api/versions/unused out of CustomerEndpoints.cs and add the new /api/versions/cleanup-candidates endpoint.

Files: - Create: src/BenefitManager.CommandCenter/Endpoints/VersionsEndpoints.cs - Modify: src/BenefitManager.CommandCenter/Endpoints/CustomerEndpoints.cs:35-42 (remove the /versions/unused block) - Modify: src/BenefitManager.CommandCenter/Program.cs (add MapVersionsEndpoints())

  • [ ] Step 1: Create VersionsEndpoints.cs

Create src/BenefitManager.CommandCenter/Endpoints/VersionsEndpoints.cs:

using BenefitManager.CommandCenter.Services;
using Microsoft.AspNetCore.Mvc;

namespace BenefitManager.CommandCenter.Endpoints;

public static class VersionsEndpoints
{
    public static IEndpointRouteBuilder MapVersionsEndpoints(this IEndpointRouteBuilder app)
    {
        var group = app.MapGroup("/api/versions").RequireAuthorization();

        group.MapGet("/unused", async (
            [FromQuery] string env,
            ICommandCenterQueryService svc,
            CancellationToken ct) =>
        {
            var versions = await svc.GetUnusedVersionsAsync(env, ct);
            return Results.Ok(versions);
        });

        group.MapGet("/cleanup-candidates", async (
            [FromQuery] string env,
            ICommandCenterQueryService svc,
            CancellationToken ct) =>
        {
            var candidates = await svc.GetCleanupCandidatesAsync(env, ct);
            return Results.Ok(candidates);
        });

        return app;
    }
}
  • [ ] Step 2: Remove /versions/unused from CustomerEndpoints.cs

Delete lines 35-42 from src/BenefitManager.CommandCenter/Endpoints/CustomerEndpoints.cs (the group.MapGet("/versions/unused", ...) block). The result should jump from the /stages block directly to group.MapPost("/customers/stages/cleanup", ...).

  • [ ] Step 3: Register MapVersionsEndpoints() in CommandCenter's Program.cs

Search for MapCustomerEndpoints() in src/BenefitManager.CommandCenter/Program.cs and add app.MapVersionsEndpoints(); on the line immediately after it.

  • [ ] Step 4: Build
dotnet build src/BenefitManager.CommandCenter/BenefitManager.CommandCenter.csproj -v q

Expected: Build succeeded. 0 Warning(s) 0 Error(s)

  • [ ] Step 5: Commit
git add src/BenefitManager.CommandCenter/Endpoints/VersionsEndpoints.cs `
        src/BenefitManager.CommandCenter/Endpoints/CustomerEndpoints.cs `
        src/BenefitManager.CommandCenter/Program.cs
git commit -m "feat(cc): add VersionsEndpoints with /versions/unused (moved) and /versions/cleanup-candidates"

Task 5: Add GetCleanupCandidatesAsync to the CommandCenter client#

Files: - Modify: src/BenefitManager.CommandCenter.Client/ICommandCenterClient.cs - Modify: src/BenefitManager.CommandCenter.Client/CommandCenterClient.cs

  • [ ] Step 1: Add to interface

In src/BenefitManager.CommandCenter.Client/ICommandCenterClient.cs, add after GetUnusedVersionsAsync:

Task<List<CleanupCandidate>> GetCleanupCandidatesAsync(string environment, CancellationToken ct = default);
  • [ ] Step 2: Implement in CommandCenterClient.cs

Add after GetUnusedVersionsAsync (after line 89):

public async Task<List<CleanupCandidate>> GetCleanupCandidatesAsync(string environment, CancellationToken ct = default)
{
    var resp = await Client.GetAsync($"/api/versions/cleanup-candidates?env={Uri.EscapeDataString(environment)}", ct);
    resp.EnsureSuccessStatusCode();
    return await resp.Content.ReadFromJsonAsync(CommandCenterContractsJsonContext.Default.ListCleanupCandidate, ct) ?? [];
}
  • [ ] Step 3: Build
dotnet build src/BenefitManager.CommandCenter.Client/BenefitManager.CommandCenter.Client.csproj -v q

Expected: Build succeeded. 0 Warning(s) 0 Error(s)

  • [ ] Step 4: Commit
git add src/BenefitManager.CommandCenter.Client/ICommandCenterClient.cs `
        src/BenefitManager.CommandCenter.Client/CommandCenterClient.cs
git commit -m "feat(cc-client): add GetCleanupCandidatesAsync"

Task 6: Implement VersionMatcher with tests (TDD)#

Files: - Create: tests/BenefitManager.Orchestrator.Tests/VersionMatcherTests.cs - Create: src/BenefitManager.Orchestrator/Services/VersionMatcher.cs

  • [ ] Step 1: Write all failing tests

Create tests/BenefitManager.Orchestrator.Tests/VersionMatcherTests.cs:

using BenefitManager.Orchestrator.Services;
using FluentAssertions;

namespace BenefitManager.Orchestrator.Tests;

public class VersionMatcherTests
{
    // ── Exact ────────────────────────────────────────────────────────────────

    [Fact]
    public void Exact_MatchesIdenticalVersion()
    {
        VersionMatcher.TryParse("25.5.29", out var pred, out _).Should().BeTrue();
        pred!("25.5.29").Should().BeTrue();
        pred("25.5.30").Should().BeFalse();
    }

    [Fact]
    public void Exact_IsCaseSensitive()
    {
        VersionMatcher.TryParse("25.5.29", out var pred, out _).Should().BeTrue();
        pred!("25.5.29").Should().BeTrue();
        pred("25.5.29a").Should().BeFalse();
    }

    // ── Glob ─────────────────────────────────────────────────────────────────

    [Fact]
    public void Glob_Star_MatchesAnyVersion()
    {
        VersionMatcher.TryParse("*", out var pred, out _).Should().BeTrue();
        pred!("25.5.29").Should().BeTrue();
        pred("1.0.0").Should().BeTrue();
    }

    [Fact]
    public void Glob_Prefix_MatchesVersionsInMajorMinor()
    {
        VersionMatcher.TryParse("25.5.*", out var pred, out _).Should().BeTrue();
        pred!("25.5.1").Should().BeTrue();
        pred!("25.5.29").Should().BeTrue();
        pred!("25.6.0").Should().BeFalse();
        pred!("26.5.0").Should().BeFalse();
    }

    [Fact]
    public void Glob_QuestionMark_MatchesSingleChar()
    {
        VersionMatcher.TryParse("25.?.29", out var pred, out _).Should().BeTrue();
        pred!("25.5.29").Should().BeTrue();
        pred!("25.55.29").Should().BeFalse();
    }

    // ── Regex ─────────────────────────────────────────────────────────────────

    [Fact]
    public void Regex_MatchesPattern()
    {
        VersionMatcher.TryParse("/^25\\.[5-7]\\..*/", out var pred, out _).Should().BeTrue();
        pred!("25.5.1").Should().BeTrue();
        pred!("25.7.99").Should().BeTrue();
        pred!("25.8.0").Should().BeFalse();
        pred!("24.5.0").Should().BeFalse();
    }

    [Fact]
    public void Regex_IsCaseInsensitive()
    {
        VersionMatcher.TryParse("/^BETA.*/", out var pred, out _).Should().BeTrue();
        pred!("beta.1").Should().BeTrue();
        pred!("BETA.1").Should().BeTrue();
    }

    [Fact]
    public void Regex_MalformedPattern_ReturnsFalse()
    {
        var ok = VersionMatcher.TryParse("/broken[regex/", out var pred, out var errorToken);
        ok.Should().BeFalse();
        pred.Should().BeNull();
        errorToken.Should().NotBeNullOrEmpty();
    }

    // ── Range ─────────────────────────────────────────────────────────────────

    [Fact]
    public void Range_Inclusive_MatchesBothEndpoints()
    {
        VersionMatcher.TryParse("25.5.10..25.5.29", out var pred, out _).Should().BeTrue();
        pred!("25.5.10").Should().BeTrue();
        pred!("25.5.29").Should().BeTrue();
        pred!("25.5.15").Should().BeTrue();
        pred!("25.5.9").Should().BeFalse();
        pred!("25.5.30").Should().BeFalse();
    }

    [Fact]
    public void Range_OpenEnd_MatchesFromLower()
    {
        VersionMatcher.TryParse("25.5.10..", out var pred, out _).Should().BeTrue();
        pred!("25.5.10").Should().BeTrue();
        pred!("99.99.99").Should().BeTrue();
        pred!("25.5.9").Should().BeFalse();
    }

    [Fact]
    public void Range_OpenStart_MatchesUpToUpper()
    {
        VersionMatcher.TryParse("..25.5.29", out var pred, out _).Should().BeTrue();
        pred!("25.5.29").Should().BeTrue();
        pred!("0.0.1").Should().BeTrue();
        pred!("25.5.30").Should().BeFalse();
    }

    [Fact]
    public void Range_NonParsableVersion_IsExcluded()
    {
        VersionMatcher.TryParse("25.5.0..25.5.29", out var pred, out _).Should().BeTrue();
        pred!("not-a-version").Should().BeFalse();
    }

    [Fact]
    public void Range_MalformedEndpoint_ReturnsFalse()
    {
        var ok = VersionMatcher.TryParse("abc..def", out var pred, out var errorToken);
        ok.Should().BeFalse();
        pred.Should().BeNull();
        errorToken.Should().Contain("abc");
    }
}
  • [ ] Step 2: Run tests — they should all fail (type not found)
dotnet test tests/BenefitManager.Orchestrator.Tests --filter "VersionMatcherTests" -v n 2>&1 | Select-String -Pattern "error|FAILED|Build"

Expected: build error — VersionMatcher does not exist yet.

  • [ ] Step 3: Create src/BenefitManager.Orchestrator/Services/VersionMatcher.cs
using System.Text.RegularExpressions;

namespace BenefitManager.Orchestrator.Services;

public static class VersionMatcher
{
    public static bool TryParse(string input, out Func<string, bool>? predicate, out string? errorToken)
    {
        predicate = null;
        errorToken = null;

        if (input.StartsWith('/') && input.EndsWith('/') && input.Length >= 2)
        {
            var pattern = input[1..^1];
            try
            {
                var regex = new Regex(pattern, RegexOptions.IgnoreCase | RegexOptions.CultureInvariant | RegexOptions.Compiled);
                predicate = v => regex.IsMatch(v);
                return true;
            }
            catch (RegexParseException ex)
            {
                errorToken = ex.Message;
                return false;
            }
        }

        if (input.Contains(".."))
        {
            var parts = input.Split("..", 2);
            var lowerRaw = parts[0];
            var upperRaw = parts[1];

            Version? lower = null;
            Version? upper = null;

            if (!string.IsNullOrEmpty(lowerRaw) && !Version.TryParse(lowerRaw, out lower))
            {
                errorToken = lowerRaw;
                return false;
            }

            if (!string.IsNullOrEmpty(upperRaw) && !Version.TryParse(upperRaw, out upper))
            {
                errorToken = upperRaw;
                return false;
            }

            predicate = v =>
            {
                if (!Version.TryParse(v, out var parsed))
                    return false;
                if (lower is not null && parsed < lower) return false;
                if (upper is not null && parsed > upper) return false;
                return true;
            };
            return true;
        }

        if (input.Contains('*') || input.Contains('?'))
        {
            var escaped = Regex.Escape(input)
                .Replace(@"\*", ".*")
                .Replace(@"\?", ".");
            var regex = new Regex($"^{escaped}$", RegexOptions.IgnoreCase | RegexOptions.CultureInvariant | RegexOptions.Compiled);
            predicate = v => regex.IsMatch(v);
            return true;
        }

        // Exact match
        predicate = v => string.Equals(v, input, StringComparison.Ordinal);
        return true;
    }
}
  • [ ] Step 4: Run tests — they should all pass
dotnet test tests/BenefitManager.Orchestrator.Tests --filter "VersionMatcherTests" -v n

Expected: Passed! Failed: 0, Passed: 14

  • [ ] Step 5: Commit
git add src/BenefitManager.Orchestrator/Services/VersionMatcher.cs `
        tests/BenefitManager.Orchestrator.Tests/VersionMatcherTests.cs
git commit -m "feat(orchestrator): add VersionMatcher with exact/glob/regex/range grammar"

Task 7: Implement InstalledVersionsResolver with tests (TDD)#

Files: - Create: tests/BenefitManager.Orchestrator.Tests/InstalledVersionsResolverTests.cs - Create: src/BenefitManager.Orchestrator/Services/InstalledVersionsResolver.cs

  • [ ] Step 1: Write failing tests

Create tests/BenefitManager.Orchestrator.Tests/InstalledVersionsResolverTests.cs:

using BenefitManager.Agent.Client;
using BenefitManager.CommandCenter.Client;
using BenefitManager.CommandCenter.Contracts;
using BenefitManager.Contracts.Models;
using BenefitManager.Orchestrator.Services;
using FluentAssertions;
using Moq;

namespace BenefitManager.Orchestrator.Tests;

public class InstalledVersionsResolverTests
{
    private static InstallationSummary MakeInstall(string env, string version, DateTimeOffset? registeredAt = null) =>
        new(env, version, $@"F:\{env}\{version}", [], DateTimeOffset.UtcNow, registeredAt);

    private static CleanupCandidate MakeCandidate(string version, string env, int stageCount) =>
        new(version, env, stageCount);

    // ── Agent fan-out ─────────────────────────────────────────────────────────

    [Fact]
    public async Task AgentFanOut_AggregatesVersionsAcrossServers()
    {
        var ts = DateTimeOffset.UtcNow.AddDays(-10);
        var agent1 = new Mock<IAgentClient>();
        agent1.Setup(x => x.GetInstallationsAsync(It.IsAny<CancellationToken>()))
            .ReturnsAsync([MakeInstall("Acceptatie", "25.5.10", ts)]);

        var agent2 = new Mock<IAgentClient>();
        agent2.Setup(x => x.GetInstallationsAsync(It.IsAny<CancellationToken>()))
            .ReturnsAsync([MakeInstall("Acceptatie", "25.5.10", ts.AddHours(1)),
                           MakeInstall("Acceptatie", "25.5.11", ts)]);

        var factory = new Mock<IAgentClientFactory>();
        factory.Setup(x => x.GetServerNames()).Returns(["srv1", "srv2"]);
        factory.Setup(x => x.GetClient("srv1")).Returns(agent1.Object);
        factory.Setup(x => x.GetClient("srv2")).Returns(agent2.Object);

        var ccClient = new Mock<ICommandCenterClient>();
        ccClient.Setup(x => x.GetCleanupCandidatesAsync("Acceptatie", It.IsAny<CancellationToken>()))
            .ReturnsAsync([MakeCandidate("25.5.10", "Acceptatie", 0),
                           MakeCandidate("25.5.11", "Acceptatie", 1)]);

        var resolver = new InstalledVersionsResolver(factory.Object, ccClient.Object);
        var result = await resolver.GetAsync("Acceptatie", ["srv1", "srv2"], TruthSource.Agents);

        result.Should().HaveCount(2);
        var v1 = result.Single(r => r.Version == "25.5.10");
        v1.Servers.Should().BeEquivalentTo(["srv1", "srv2"]);
        v1.StageActiveCount.Should().Be(0);
        v1.RegisteredAt.Should().Be(ts.AddHours(1)); // max across servers

        var v2 = result.Single(r => r.Version == "25.5.11");
        v2.Servers.Should().BeEquivalentTo(["srv2"]);
        v2.StageActiveCount.Should().Be(1);
    }

    [Fact]
    public async Task AgentFanOut_OfflineAgent_IsSkippedSilently()
    {
        var agent1 = new Mock<IAgentClient>();
        agent1.Setup(x => x.GetInstallationsAsync(It.IsAny<CancellationToken>()))
            .ThrowsAsync(new HttpRequestException("offline"));

        var agent2 = new Mock<IAgentClient>();
        agent2.Setup(x => x.GetInstallationsAsync(It.IsAny<CancellationToken>()))
            .ReturnsAsync([MakeInstall("Acceptatie", "25.5.10")]);

        var factory = new Mock<IAgentClientFactory>();
        factory.Setup(x => x.GetServerNames()).Returns(["srv1", "srv2"]);
        factory.Setup(x => x.GetClient("srv1")).Returns(agent1.Object);
        factory.Setup(x => x.GetClient("srv2")).Returns(agent2.Object);

        var ccClient = new Mock<ICommandCenterClient>();
        ccClient.Setup(x => x.GetCleanupCandidatesAsync("Acceptatie", It.IsAny<CancellationToken>()))
            .ReturnsAsync([MakeCandidate("25.5.10", "Acceptatie", 0)]);

        var resolver = new InstalledVersionsResolver(factory.Object, ccClient.Object);
        var result = await resolver.GetAsync("Acceptatie", ["srv1", "srv2"], TruthSource.Agents);

        result.Should().HaveCount(1);
        result[0].Servers.Should().BeEquivalentTo(["srv2"]);
    }

    [Fact]
    public async Task AgentFanOut_ServerFilter_RestrictsToNamedServers()
    {
        var agent1 = new Mock<IAgentClient>();
        agent1.Setup(x => x.GetInstallationsAsync(It.IsAny<CancellationToken>()))
            .ReturnsAsync([MakeInstall("Acceptatie", "25.5.10")]);

        var factory = new Mock<IAgentClientFactory>();
        factory.Setup(x => x.GetServerNames()).Returns(["srv1", "srv2"]);
        factory.Setup(x => x.GetClient("srv1")).Returns(agent1.Object);

        var ccClient = new Mock<ICommandCenterClient>();
        ccClient.Setup(x => x.GetCleanupCandidatesAsync("Acceptatie", It.IsAny<CancellationToken>()))
            .ReturnsAsync([MakeCandidate("25.5.10", "Acceptatie", 0)]);

        var resolver = new InstalledVersionsResolver(factory.Object, ccClient.Object);
        // Only ask for srv1
        var result = await resolver.GetAsync("Acceptatie", ["srv1"], TruthSource.Agents);

        result.Should().HaveCount(1);
        factory.Verify(x => x.GetClient("srv2"), Times.Never);
    }

    // ── Fleet snapshot ───────────────────────────────────────────────────────

    [Fact]
    public async Task FleetSnapshot_AggregatesInstallationsFromFleetState()
    {
        var snapshot = new FleetStateEntry("srv1", "host1", DateTimeOffset.UtcNow, [
            MakeInstall("Acceptatie", "25.5.10")
        ]);

        var factory = new Mock<IAgentClientFactory>();
        var ccClient = new Mock<ICommandCenterClient>();
        ccClient.Setup(x => x.GetFleetStateAsync(It.IsAny<CancellationToken>()))
            .ReturnsAsync([snapshot]);
        ccClient.Setup(x => x.GetCleanupCandidatesAsync("Acceptatie", It.IsAny<CancellationToken>()))
            .ReturnsAsync([MakeCandidate("25.5.10", "Acceptatie", 0)]);

        var resolver = new InstalledVersionsResolver(factory.Object, ccClient.Object);
        var result = await resolver.GetAsync("Acceptatie", [], TruthSource.Fleet);

        result.Should().HaveCount(1);
        result[0].Version.Should().Be("25.5.10");
        result[0].Servers.Should().BeEquivalentTo(["srv1"]);
        factory.Verify(x => x.GetClient(It.IsAny<string>()), Times.Never);
    }

    [Fact]
    public async Task FleetSnapshot_EmptySnapshot_ReturnsEmpty()
    {
        var factory = new Mock<IAgentClientFactory>();
        var ccClient = new Mock<ICommandCenterClient>();
        ccClient.Setup(x => x.GetFleetStateAsync(It.IsAny<CancellationToken>()))
            .ReturnsAsync([]);
        ccClient.Setup(x => x.GetCleanupCandidatesAsync("Acceptatie", It.IsAny<CancellationToken>()))
            .ReturnsAsync([]);

        var resolver = new InstalledVersionsResolver(factory.Object, ccClient.Object);
        var result = await resolver.GetAsync("Acceptatie", [], TruthSource.Fleet);

        result.Should().BeEmpty();
    }
}
  • [ ] Step 2: Run tests — should fail (types not found)
dotnet test tests/BenefitManager.Orchestrator.Tests --filter "InstalledVersionsResolverTests" -v n 2>&1 | Select-String "error|Build"

Expected: build error.

  • [ ] Step 3: Create src/BenefitManager.Orchestrator/Services/InstalledVersionsResolver.cs
using BenefitManager.Agent.Client;
using BenefitManager.CommandCenter.Client;
using BenefitManager.Contracts.Models;
using Microsoft.Extensions.Logging;

namespace BenefitManager.Orchestrator.Services;

public enum TruthSource { Agents, Fleet }

public record InstalledVersion(
    string Version,
    string Environment,
    IReadOnlyList<string> Servers,
    DateTimeOffset? RegisteredAt,
    int StageActiveCount);

public sealed class InstalledVersionsResolver(
    IAgentClientFactory agentClientFactory,
    ICommandCenterClient commandCenterClient,
    ILogger<InstalledVersionsResolver>? logger = null)
{
    public async Task<IReadOnlyList<InstalledVersion>> GetAsync(
        string environment,
        IReadOnlyList<string> serverFilter,
        TruthSource from,
        CancellationToken ct = default)
    {
        // Gather raw (serverName, version, registeredAt) tuples
        var raw = new List<(string Server, string Version, DateTimeOffset? RegisteredAt)>();

        if (from == TruthSource.Fleet)
        {
            var fleet = await commandCenterClient.GetFleetStateAsync(ct);
            foreach (var entry in fleet)
            {
                if (serverFilter.Count > 0 &&
                    !serverFilter.Contains(entry.AgentName, StringComparer.OrdinalIgnoreCase))
                    continue;

                foreach (var inst in entry.Installations)
                {
                    if (!string.Equals(inst.Environment, environment, StringComparison.OrdinalIgnoreCase))
                        continue;
                    raw.Add((entry.AgentName, inst.Version, inst.RegisteredAt));
                }
            }
        }
        else
        {
            var targets = serverFilter.Count > 0
                ? serverFilter
                : agentClientFactory.GetServerNames();

            foreach (var serverName in targets)
            {
                try
                {
                    var client = agentClientFactory.GetClient(serverName);
                    var installs = await client.GetInstallationsAsync(ct);
                    foreach (var inst in installs)
                    {
                        if (!string.Equals(inst.Environment, environment, StringComparison.OrdinalIgnoreCase))
                            continue;
                        raw.Add((serverName, inst.Version, inst.RegisteredAt));
                    }
                }
                catch (Exception ex)
                {
                    logger?.LogWarning("Agent {Server} unreachable, skipping: {Message}", serverName, ex.Message);
                }
            }
        }

        if (raw.Count == 0)
            return [];

        // Aggregate by version: collect servers, pick max RegisteredAt
        var byVersion = raw
            .GroupBy(t => t.Version, StringComparer.OrdinalIgnoreCase)
            .ToDictionary(
                g => g.Key,
                g => (
                    Servers: (IReadOnlyList<string>)g.Select(t => t.Server).Distinct().Order().ToList(),
                    RegisteredAt: g.Select(t => t.RegisteredAt).Where(d => d.HasValue).Max()
                ),
                StringComparer.OrdinalIgnoreCase);

        // Enrich with StageActiveCount from CC
        var candidates = await commandCenterClient.GetCleanupCandidatesAsync(environment, ct);
        var stageMap = candidates.ToDictionary(
            c => c.Version,
            c => c.StageActiveCount,
            StringComparer.OrdinalIgnoreCase);

        return byVersion
            .Select(kv => new InstalledVersion(
                kv.Key,
                environment,
                kv.Value.Servers,
                kv.Value.RegisteredAt,
                stageMap.GetValueOrDefault(kv.Key, 0)))
            .OrderBy(v => v.Version, StringComparer.OrdinalIgnoreCase)
            .ToList();
    }
}
  • [ ] Step 4: Run tests — all should pass
dotnet test tests/BenefitManager.Orchestrator.Tests --filter "InstalledVersionsResolverTests" -v n

Expected: Passed! Failed: 0, Passed: 5

  • [ ] Step 5: Run full suite to check for regressions
dotnet test tests/BenefitManager.Orchestrator.Tests -v n

Expected: all existing tests still pass.

  • [ ] Step 6: Commit
git add src/BenefitManager.Orchestrator/Services/InstalledVersionsResolver.cs `
        tests/BenefitManager.Orchestrator.Tests/InstalledVersionsResolverTests.cs
git commit -m "feat(orchestrator): add InstalledVersionsResolver (agent fan-out + fleet strategies)"

Task 8: Extract RemoveOnServerAsync helper from RemoveCommand#

Files: - Modify: src/BenefitManager.Orchestrator/Commands/RemoveCommand.cs

  • [ ] Step 1: Extract the per-server loop body into an internal static async Task<bool> method

Replace the contents of src/BenefitManager.Orchestrator/Commands/RemoveCommand.cs with:

using BenefitManager.Agent.Client;
using BenefitManager.CommandCenter.Client;
using Microsoft.Extensions.Logging;
using Spectre.Console;
using Spectre.Console.Cli;
using System.ComponentModel;


namespace BenefitManager.Orchestrator.Commands
{
    public sealed class RemoveSettings : ManageSettings
    {
        [CommandArgument(0, "<ENVIRONMENT>")]
        [Description("The target environment (e.g. Acceptatie or Productie)")]
        public string Environment { get; init; } = null!;

        [CommandArgument(1, "<VERSION>")]
        [Description("The version to remove (e.g. 25.5.29)")]
        public string Version { get; init; } = null!;

        [CommandOption("--force")]
        [Description("Force deactivation if the version is associated with a Customer Stage or has logs.")]
        public bool Force { get; init; }

        [CommandOption("--server")]
        [Description("Target specific server(s). Defaults to all configured agents.")]
        public string[]? Servers { get; init; }
    }

    public sealed class RemoveCommand(IAgentClientFactory agentClientFactory, ICommandCenterClient commandCenterClient) : AsyncCommand<RemoveSettings>
    {
        protected override async Task<int> ExecuteAsync(CommandContext context, RemoveSettings settings, CancellationToken cancellationToken)
        {
            IReadOnlyList<string> targets = settings.Servers is { Length: > 0 }
                ? settings.Servers
                : agentClientFactory.GetServerNames();

            var stages = await commandCenterClient.GetAssociatedStagesAsync(settings.Environment, settings.Version, cancellationToken);
            var serverCount = await commandCenterClient.GetServerCountForVersionAsync(settings.Environment, settings.Version, cancellationToken);

            if (stages.Count > 0 && serverCount <= targets.Count && !settings.Force)
            {
                AnsiConsole.MarkupLine("[red]Error:[/] This version is the [yellow]last remaining installation[/] for the following Customer Stages:");
                foreach (var s in stages) AnsiConsole.MarkupLine($" - [cyan]{s.Name.EscapeMarkup()}[/] (DB: {s.Database.EscapeMarkup()})");
                AnsiConsole.MarkupLine("\nRemoval is blocked to prevent breaking these stages. Use [white]--force[/] to deactivate the application instead.");
                return 1;
            }

            var allSucceeded = true;

            await AnsiConsole.Status().StartAsync("Processing removal...", async ctx =>
            {
                foreach (var serverName in targets)
                {
                    ctx.Status($"[{serverName}] Processing {settings.Version}...".EscapeMarkup());
                    var ok = await RemoveOnServerAsync(
                        settings.Environment, settings.Version, serverName, settings.Force,
                        agentClientFactory.GetClient(serverName), commandCenterClient,
                        null, cancellationToken);
                    if (!ok) allSucceeded = false;
                }
            });

            if (allSucceeded)
            {
                AnsiConsole.MarkupLine(settings.Force
                    ? "[yellow]⚠ Application deactivated and set to offline (Force used).[/]"
                    : "[green]✔ Application removed successfully.[/]");
                return 0;
            }

            AnsiConsole.MarkupLine("[red]✘ Removal completed with errors on one or more servers.[/]");
            return 1;
        }

        /// <summary>
        /// Removes a single version from a single server: stops the service, removes IIS app,
        /// deletes binaries, then updates CommandCenter SQL. Returns true on full success.
        /// Idempotent: "not installed" on the agent side is treated as success.
        /// </summary>
        internal static async Task<bool> RemoveOnServerAsync(
            string environment,
            string version,
            string serverName,
            bool force,
            IAgentClient agentClient,
            ICommandCenterClient ccClient,
            ILogger? logger,
            CancellationToken ct)
        {
            try
            {
                await agentClient.RemoveAsync(environment, version, force, ct);
            }
            catch (HttpRequestException ex) when (ex.StatusCode == System.Net.HttpStatusCode.NotFound)
            {
                // Already removed — idempotent
            }
            catch (Exception ex)
            {
                AnsiConsole.MarkupLine($"[red]Error on {serverName.EscapeMarkup()} (Agent):[/] {ex.Message.EscapeMarkup()}");
                logger?.LogError(ex, "Agent removal failed for {Server} {Version}", serverName, version);
                return false;
            }

            try
            {
                await ccClient.RemoveDeployAsync(environment, version, serverName, force, ct);
            }
            catch (Exception ex)
            {
                AnsiConsole.MarkupLine($"[red]Error on {serverName.EscapeMarkup()} (SQL):[/] {ex.Message.EscapeMarkup()}");
                logger?.LogError(ex, "CC RemoveDeploy failed for {Server} {Version}", serverName, version);
                return false;
            }

            return true;
        }
    }
}
  • [ ] Step 2: Build and run existing RemoveCommand tests
dotnet build src/BenefitManager.Orchestrator/BenefitManager.Orchestrator.csproj -v q
dotnet test tests/BenefitManager.Orchestrator.Tests -v n

Expected: build clean, all tests pass.

  • [ ] Step 3: Commit
git add src/BenefitManager.Orchestrator/Commands/RemoveCommand.cs
git commit -m "refactor(orchestrator): extract RemoveOnServerAsync from RemoveCommand for reuse"

Task 9: Implement InstalledQueryCommand (bmo query installed) with tests#

Files: - Create: tests/BenefitManager.Orchestrator.Tests/InstalledQueryCommandTests.cs - Create: src/BenefitManager.Orchestrator/Commands/InstalledQueryCommand.cs

  • [ ] Step 1: Write failing tests

Create tests/BenefitManager.Orchestrator.Tests/InstalledQueryCommandTests.cs:

using BenefitManager.Agent.Client;
using BenefitManager.CommandCenter.Client;
using BenefitManager.Orchestrator.Commands;
using BenefitManager.Orchestrator.Services;
using FluentAssertions;
using Moq;

namespace BenefitManager.Orchestrator.Tests;

public class InstalledQueryCommandTests
{
    private static InstalledVersionsResolver MakeResolver(
        IAgentClientFactory factory, ICommandCenterClient ccClient) =>
        new(factory, ccClient);

    [Fact]
    public async Task ReturnsZero_WhenInstallationsExist()
    {
        var factory = new Mock<IAgentClientFactory>();
        factory.Setup(x => x.GetServerNames()).Returns(["srv1"]);

        var agentClient = new Mock<IAgentClient>();
        agentClient.Setup(x => x.GetInstallationsAsync(It.IsAny<CancellationToken>()))
            .ReturnsAsync([new BenefitManager.Contracts.Models.InstallationSummary(
                "Acceptatie", "25.5.29", @"F:\Acceptatie\25.5.29", [], DateTimeOffset.UtcNow)]);
        factory.Setup(x => x.GetClient("srv1")).Returns(agentClient.Object);

        var ccClient = new Mock<ICommandCenterClient>();
        ccClient.Setup(x => x.GetCleanupCandidatesAsync("Acceptatie", It.IsAny<CancellationToken>()))
            .ReturnsAsync([new BenefitManager.Contracts.Models.CleanupCandidate("25.5.29", "Acceptatie", 1)]);

        var resolver = MakeResolver(factory.Object, ccClient.Object);
        var command = new InstalledQueryCommand(resolver);

        var result = await command.ExecuteAsync(null!, new InstalledQuerySettings
        {
            Environment = "Acceptatie"
        }, TestContext.Current.CancellationToken);

        result.Should().Be(0);
    }

    [Fact]
    public async Task UnusedFlag_FiltersToZeroStageRefs()
    {
        var factory = new Mock<IAgentClientFactory>();
        factory.Setup(x => x.GetServerNames()).Returns(["srv1"]);

        var agentClient = new Mock<IAgentClient>();
        agentClient.Setup(x => x.GetInstallationsAsync(It.IsAny<CancellationToken>()))
            .ReturnsAsync([
                new BenefitManager.Contracts.Models.InstallationSummary("Acceptatie", "25.5.10", @"F:\Acceptatie\25.5.10", [], DateTimeOffset.UtcNow),
                new BenefitManager.Contracts.Models.InstallationSummary("Acceptatie", "25.5.29", @"F:\Acceptatie\25.5.29", [], DateTimeOffset.UtcNow)
            ]);
        factory.Setup(x => x.GetClient("srv1")).Returns(agentClient.Object);

        var ccClient = new Mock<ICommandCenterClient>();
        ccClient.Setup(x => x.GetCleanupCandidatesAsync("Acceptatie", It.IsAny<CancellationToken>()))
            .ReturnsAsync([
                new BenefitManager.Contracts.Models.CleanupCandidate("25.5.10", "Acceptatie", 0),
                new BenefitManager.Contracts.Models.CleanupCandidate("25.5.29", "Acceptatie", 2)
            ]);

        var resolver = MakeResolver(factory.Object, ccClient.Object);
        var command = new InstalledQueryCommand(resolver);

        var result = await command.ExecuteAsync(null!, new InstalledQuerySettings
        {
            Environment = "Acceptatie",
            Unused = true
        }, TestContext.Current.CancellationToken);

        result.Should().Be(0);
        // Output goes to Spectre console — just verify exit code and that ccClient was called
        ccClient.Verify(x => x.GetCleanupCandidatesAsync("Acceptatie", It.IsAny<CancellationToken>()), Times.Once);
    }
}
  • [ ] Step 2: Run tests — should fail (type not found)
dotnet test tests/BenefitManager.Orchestrator.Tests --filter "InstalledQueryCommandTests" -v n 2>&1 | Select-String "error|Build"
  • [ ] Step 3: Create src/BenefitManager.Orchestrator/Commands/InstalledQueryCommand.cs
using BenefitManager.Orchestrator.Infrastructure;
using BenefitManager.Orchestrator.Services;
using Spectre.Console;
using Spectre.Console.Cli;
using System.ComponentModel;
using System.Globalization;

namespace BenefitManager.Orchestrator.Commands
{
    public sealed class InstalledQuerySettings : QuerySettings
    {
        [CommandArgument(0, "<ENVIRONMENT>")]
        [Description("The target environment (e.g. Acceptatie or Productie)")]
        public string Environment { get; init; } = null!;

        [CommandOption("--unused")]
        [Description("Only show versions with no active customer stages.")]
        public bool Unused { get; init; }

        [CommandOption("--server")]
        [Description("Restrict to named agent(s). Repeatable. Default: all configured agents.")]
        public string[]? Servers { get; init; }

        [CommandOption("--from")]
        [Description("Truth source for installed versions: agents (default, live fan-out) or fleet (CC snapshot).")]
        [DefaultValue(TruthSource.Agents)]
        public TruthSource From { get; init; } = TruthSource.Agents;

        [CommandOption("--match")]
        [Description("Version filter pattern (glob 25.5.*, regex /pattern/, range 25.5.10..25.5.29, or exact).")]
        public string? Match { get; init; }
    }

    public sealed class InstalledQueryCommand(InstalledVersionsResolver resolver) : AsyncCommand<InstalledQuerySettings>
    {
        protected override async Task<int> ExecuteAsync(CommandContext context, InstalledQuerySettings settings, CancellationToken cancellationToken)
        {
            Func<string, bool>? matchPred = null;
            if (settings.Match is not null)
            {
                if (!VersionMatcher.TryParse(settings.Match, out matchPred, out var errorToken))
                {
                    AnsiConsole.MarkupLine($"[red]Invalid --match pattern:[/] {errorToken?.EscapeMarkup()}");
                    return 2;
                }
            }

            IReadOnlyList<string> serverFilter = settings.Servers is { Length: > 0 }
                ? settings.Servers
                : [];

            var all = await resolver.GetAsync(settings.Environment, serverFilter, settings.From, cancellationToken);

            var filtered = all
                .Where(v => !settings.Unused || v.StageActiveCount == 0)
                .Where(v => matchPred is null || matchPred(v.Version))
                .ToList();

            var rows = filtered.Select(v => new InstalledVersionRow(
                v.Version,
                v.Environment,
                string.Join(", ", v.Servers),
                v.StageActiveCount,
                v.RegisteredAt?.LocalDateTime.ToString("yyyy-MM-dd HH:mm", CultureInfo.InvariantCulture) ?? ""))
            .ToList();

            OutputFormatter.Render(rows, settings.Output, settings.Query);
            return 0;
        }
    }

    internal record InstalledVersionRow(
        string Version,
        string Environment,
        string Servers,
        int StageRefs,
        string RegisteredAt);
}
  • [ ] Step 4: Run tests — should pass
dotnet test tests/BenefitManager.Orchestrator.Tests --filter "InstalledQueryCommandTests" -v n

Expected: Passed! Failed: 0, Passed: 2

  • [ ] Step 5: Commit
git add src/BenefitManager.Orchestrator/Commands/InstalledQueryCommand.cs `
        tests/BenefitManager.Orchestrator.Tests/InstalledQueryCommandTests.cs
git commit -m "feat(orchestrator): add InstalledQueryCommand (bmo query installed)"

Task 10: Implement UninstallCommand (bmo manage uninstall) with tests#

Files: - Create: tests/BenefitManager.Orchestrator.Tests/UninstallCommandTests.cs - Create: src/BenefitManager.Orchestrator/Commands/UninstallCommand.cs

  • [ ] Step 1: Write failing tests

Create tests/BenefitManager.Orchestrator.Tests/UninstallCommandTests.cs:

using BenefitManager.Agent.Client;
using BenefitManager.CommandCenter.Client;
using BenefitManager.Contracts.Models;
using BenefitManager.Orchestrator.Commands;
using BenefitManager.Orchestrator.Services;
using FluentAssertions;
using Moq;

namespace BenefitManager.Orchestrator.Tests;

public class UninstallCommandTests
{
    private static (Mock<IAgentClientFactory> factory, Mock<IAgentClient> agent, Mock<ICommandCenterClient> cc)
        BuildMocks(string env, string version, string server = "srv1")
    {
        var agent = new Mock<IAgentClient>();
        agent.Setup(x => x.GetInstallationsAsync(It.IsAny<CancellationToken>()))
            .ReturnsAsync([new InstallationSummary(env, version, $@"F:\{env}\{version}", [], DateTimeOffset.UtcNow)]);

        var factory = new Mock<IAgentClientFactory>();
        factory.Setup(x => x.GetServerNames()).Returns([server]);
        factory.Setup(x => x.GetClient(server)).Returns(agent.Object);

        var cc = new Mock<ICommandCenterClient>();
        cc.Setup(x => x.GetCleanupCandidatesAsync(env, It.IsAny<CancellationToken>()))
            .ReturnsAsync([new CleanupCandidate(version, env, 0)]);

        return (factory, agent, cc);
    }

    [Fact]
    public async Task DryRun_DoesNotCallRemove()
    {
        var (factory, agent, cc) = BuildMocks("Acceptatie", "25.5.10");
        var resolver = new InstalledVersionsResolver(factory.Object, cc.Object);
        var command = new UninstallCommand(resolver, factory.Object, cc.Object);

        var result = await command.ExecuteAsync(null!, new UninstallSettings
        {
            Environment = "Acceptatie",
            Match = "25.5.10",
            Apply = false
        }, TestContext.Current.CancellationToken);

        result.Should().Be(0);
        agent.Verify(x => x.RemoveAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<bool>(), It.IsAny<CancellationToken>()), Times.Never);
        cc.Verify(x => x.RemoveDeployAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(), It.IsAny<bool>(), It.IsAny<CancellationToken>()), Times.Never);
    }

    [Fact]
    public async Task Apply_CallsRemoveForEachVersionServer()
    {
        var (factory, agent, cc) = BuildMocks("Acceptatie", "25.5.10");
        var resolver = new InstalledVersionsResolver(factory.Object, cc.Object);
        var command = new UninstallCommand(resolver, factory.Object, cc.Object);

        var result = await command.ExecuteAsync(null!, new UninstallSettings
        {
            Environment = "Acceptatie",
            Match = "25.5.10",
            Apply = true,
            Yes = true
        }, TestContext.Current.CancellationToken);

        result.Should().Be(0);
        agent.Verify(x => x.RemoveAsync("Acceptatie", "25.5.10", false, It.IsAny<CancellationToken>()), Times.Once);
        cc.Verify(x => x.RemoveDeployAsync("Acceptatie", "25.5.10", "srv1", false, It.IsAny<CancellationToken>()), Times.Once);
    }

    [Fact]
    public async Task Apply_OneServerFails_ReturnsExitCode1()
    {
        var agent = new Mock<IAgentClient>();
        agent.Setup(x => x.GetInstallationsAsync(It.IsAny<CancellationToken>()))
            .ReturnsAsync([new InstallationSummary("Acceptatie", "25.5.10", @"F:\path", [], DateTimeOffset.UtcNow)]);
        agent.Setup(x => x.RemoveAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<bool>(), It.IsAny<CancellationToken>()))
            .ThrowsAsync(new HttpRequestException("connection refused"));

        var factory = new Mock<IAgentClientFactory>();
        factory.Setup(x => x.GetServerNames()).Returns(["srv1"]);
        factory.Setup(x => x.GetClient("srv1")).Returns(agent.Object);

        var cc = new Mock<ICommandCenterClient>();
        cc.Setup(x => x.GetCleanupCandidatesAsync("Acceptatie", It.IsAny<CancellationToken>()))
            .ReturnsAsync([new CleanupCandidate("25.5.10", "Acceptatie", 0)]);

        var resolver = new InstalledVersionsResolver(factory.Object, cc.Object);
        var command = new UninstallCommand(resolver, factory.Object, cc.Object);

        var result = await command.ExecuteAsync(null!, new UninstallSettings
        {
            Environment = "Acceptatie",
            Match = "25.5.10",
            Apply = true,
            Yes = true
        }, TestContext.Current.CancellationToken);

        result.Should().Be(1);
        cc.Verify(x => x.RemoveDeployAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(), It.IsAny<bool>(), It.IsAny<CancellationToken>()), Times.Never);
    }

    [Fact]
    public async Task NoMatchNoTty_ReturnsExitCode2()
    {
        var factory = new Mock<IAgentClientFactory>();
        var cc = new Mock<ICommandCenterClient>();
        var resolver = new InstalledVersionsResolver(factory.Object, cc.Object);
        var command = new UninstallCommand(resolver, factory.Object, cc.Object);

        var result = await command.ExecuteAsync(null!, new UninstallSettings
        {
            Environment = "Acceptatie",
            Match = null,
            Apply = true,
            Yes = true,
            ForceNonInteractive = true  // test-only flag to skip TTY check
        }, TestContext.Current.CancellationToken);

        result.Should().Be(2);
    }

    [Fact]
    public async Task IncludeBound_ForwardsForceTrue()
    {
        var (factory, agent, cc) = BuildMocks("Acceptatie", "25.5.10");
        // Override: version has active stages
        cc.Setup(x => x.GetCleanupCandidatesAsync("Acceptatie", It.IsAny<CancellationToken>()))
            .ReturnsAsync([new CleanupCandidate("25.5.10", "Acceptatie", 2)]);

        var resolver = new InstalledVersionsResolver(factory.Object, cc.Object);
        var command = new UninstallCommand(resolver, factory.Object, cc.Object);

        var result = await command.ExecuteAsync(null!, new UninstallSettings
        {
            Environment = "Acceptatie",
            Match = "25.5.10",
            Apply = true,
            Yes = true,
            IncludeBound = true
        }, TestContext.Current.CancellationToken);

        result.Should().Be(0);
        agent.Verify(x => x.RemoveAsync("Acceptatie", "25.5.10", true /* force */, It.IsAny<CancellationToken>()), Times.Once);
    }

    [Fact]
    public async Task InvalidMatchPattern_ReturnsExitCode2()
    {
        var factory = new Mock<IAgentClientFactory>();
        var cc = new Mock<ICommandCenterClient>();
        var resolver = new InstalledVersionsResolver(factory.Object, cc.Object);
        var command = new UninstallCommand(resolver, factory.Object, cc.Object);

        var result = await command.ExecuteAsync(null!, new UninstallSettings
        {
            Environment = "Acceptatie",
            Match = "/broken[regex/"
        }, TestContext.Current.CancellationToken);

        result.Should().Be(2);
    }
}
  • [ ] Step 2: Run tests — should fail (type not found)
dotnet test tests/BenefitManager.Orchestrator.Tests --filter "UninstallCommandTests" -v n 2>&1 | Select-String "error|Build"
  • [ ] Step 3: Create src/BenefitManager.Orchestrator/Commands/UninstallCommand.cs
using BenefitManager.Agent.Client;
using BenefitManager.CommandCenter.Client;
using BenefitManager.Orchestrator.Infrastructure;
using BenefitManager.Orchestrator.Services;
using Spectre.Console;
using Spectre.Console.Cli;
using System.ComponentModel;
using System.Globalization;

namespace BenefitManager.Orchestrator.Commands
{
    public sealed class UninstallSettings : ManageSettings
    {
        [CommandArgument(0, "<ENVIRONMENT>")]
        [Description("The target environment (e.g. Acceptatie or Productie)")]
        public string Environment { get; init; } = null!;

        [CommandOption("--match")]
        [Description("Version filter (glob 25.5.*, regex /pattern/, range 25.5.10..25.5.29, or exact). Required when stdin is not a TTY.")]
        public string? Match { get; init; }

        [CommandOption("--server")]
        [Description("Restrict removal to named agent(s). Repeatable. Default: all configured agents.")]
        public string[]? Servers { get; init; }

        [CommandOption("--from")]
        [Description("Truth source for installed versions: agents (default) or fleet (CC cached snapshot).")]
        [DefaultValue(TruthSource.Agents)]
        public TruthSource From { get; init; } = TruthSource.Agents;

        [CommandOption("--include-bound")]
        [Description("Also uninstall versions that are still bound to active customer stages (implies --force).")]
        public bool IncludeBound { get; init; }

        [CommandOption("--apply")]
        [Description("Execute the removal. Without this flag the command runs in dry-run mode.")]
        public bool Apply { get; init; }

        [CommandOption("--force")]
        [Description("Forward force-deactivation semantics to the remove pipeline.")]
        public bool Force { get; init; }

        [CommandOption("--yes")]
        [Description("Skip the in-TTY confirmation prompt.")]
        public bool Yes { get; init; }

        // Internal: allows tests to bypass TTY detection
        internal bool ForceNonInteractive { get; init; }
    }

    public sealed class UninstallCommand(
        InstalledVersionsResolver resolver,
        IAgentClientFactory agentClientFactory,
        ICommandCenterClient commandCenterClient) : AsyncCommand<UninstallSettings>
    {
        protected override async Task<int> ExecuteAsync(CommandContext context, UninstallSettings settings, CancellationToken cancellationToken)
        {
            // Validate --match
            Func<string, bool>? matchPred = null;
            if (settings.Match is not null)
            {
                if (!VersionMatcher.TryParse(settings.Match, out matchPred, out var errorToken))
                {
                    AnsiConsole.MarkupLine($"[red]Invalid --match pattern:[/] {errorToken?.EscapeMarkup()}");
                    return 2;
                }
            }

            IReadOnlyList<string> serverFilter = settings.Servers is { Length: > 0 }
                ? settings.Servers
                : [];

            bool effectiveForce = settings.Force || settings.IncludeBound;

            // Resolve candidates
            var all = await resolver.GetAsync(settings.Environment, serverFilter, settings.From, cancellationToken);

            var candidates = all
                .Where(v => settings.IncludeBound || v.StageActiveCount == 0)
                .Where(v => matchPred is null || matchPred(v.Version))
                .ToList();

            // No --match and no TTY → refuse
            bool isTty = !settings.ForceNonInteractive && !Console.IsInputRedirected;
            if (settings.Match is null && !isTty)
            {
                AnsiConsole.MarkupLine("[red]Error:[/] No [white]--match[/] provided and stdin is not a TTY. Refusing to uninstall all versions unattended.");
                return 2;
            }

            // No --match and TTY → interactive picker
            if (settings.Match is null && isTty && candidates.Count > 0)
            {
                var prompt = new MultiSelectionPrompt<InstalledVersion>()
                    .Title($"Select versions to uninstall in [yellow]{settings.Environment.EscapeMarkup()}[/]")
                    .NotRequired()
                    .InstructionsText("[grey](space to toggle, enter to confirm, q to quit)[/]")
                    .UseConverter(v => $"{v.Version}  servers: {string.Join(", ", v.Servers)}  stages: {v.StageActiveCount}");

                foreach (var c in candidates)
                    prompt.AddChoice(c);

                candidates = AnsiConsole.Prompt(prompt);
            }

            if (candidates.Count == 0)
            {
                AnsiConsole.MarkupLine("[grey]Nothing to do.[/]");
                return 0;
            }

            // Print candidates table
            var rows = candidates.Select(v => new InstalledVersionRow(
                v.Version, v.Environment,
                string.Join(", ", v.Servers),
                v.StageActiveCount,
                v.RegisteredAt?.LocalDateTime.ToString("yyyy-MM-dd HH:mm", CultureInfo.InvariantCulture) ?? ""))
            .ToList();
            OutputFormatter.Render(rows, OutputFormat.Table);

            if (!settings.Apply)
            {
                AnsiConsole.MarkupLine("[grey]Dry-run. Pass [white]--apply[/] to execute removal.[/]");
                return 0;
            }

            // In-TTY confirmation (unless --yes)
            if (isTty && !settings.Yes)
            {
                int pairCount = candidates.Sum(v => v.Servers.Count);
                var confirm = AnsiConsole.Confirm(
                    $"Remove [yellow]{candidates.Count}[/] version(s) across [yellow]{pairCount}[/] server pair(s)?",
                    defaultValue: false);
                if (!confirm)
                {
                    AnsiConsole.MarkupLine("Aborted.");
                    return 0;
                }
            }

            // Execute removal
            var allSucceeded = true;

            foreach (var v in candidates)
            {
                var servers = serverFilter.Count > 0
                    ? v.Servers.Where(s => serverFilter.Contains(s, StringComparer.OrdinalIgnoreCase)).ToList()
                    : v.Servers.ToList();

                foreach (var serverName in servers)
                {
                    var agentClient = agentClientFactory.GetClient(serverName);
                    var ok = await RemoveCommand.RemoveOnServerAsync(
                        v.Environment, v.Version, serverName, effectiveForce,
                        agentClient, commandCenterClient, null, cancellationToken);

                    var status = ok ? "[green][OK][/]" : "[red][FAIL][/]";
                    AnsiConsole.MarkupLine($"  {status} {serverName.EscapeMarkup()} / {v.Version.EscapeMarkup()}");

                    if (!ok) allSucceeded = false;
                }
            }

            return allSucceeded ? 0 : 1;
        }
    }
}
  • [ ] Step 4: Run tests — all should pass
dotnet test tests/BenefitManager.Orchestrator.Tests --filter "UninstallCommandTests" -v n

Expected: Passed! Failed: 0, Passed: 6

  • [ ] Step 5: Full suite
dotnet test tests/BenefitManager.Orchestrator.Tests -v n

Expected: all pass.

  • [ ] Step 6: Commit
git add src/BenefitManager.Orchestrator/Commands/UninstallCommand.cs `
        tests/BenefitManager.Orchestrator.Tests/UninstallCommandTests.cs
git commit -m "feat(orchestrator): add UninstallCommand (bmo manage uninstall)"

Task 11: Register new commands in Program.cs#

Files: - Modify: src/BenefitManager.Orchestrator/Program.cs

  • [ ] Step 1: Register InstalledVersionsResolver as a singleton service

After the existing services.AddSingleton<BuildContextResolver>(); line (around line 32) in src/BenefitManager.Orchestrator/Program.cs, add:

services.AddSingleton<InstalledVersionsResolver>();
  • [ ] Step 2: Register query installed command

In the query branch (after the status command, around line 152), add:

query.AddCommand<InstalledQueryCommand>("installed")
     .WithDescription("Lists versions installed on agent(s), cross-referenced with CommandCenter stage data.")
     .WithExample(["query", "installed", "Acceptatie", "--unused"])
     .WithExample(["query", "installed", "Acceptatie", "--unused", "--from", "fleet"])
     .WithExample(["query", "installed", "Acceptatie", "--match", "25.5.*", "-o", "json"]);
  • [ ] Step 3: Register manage uninstall command

In the manage branch (after manage remove, around line 194), add:

manage.AddCommand<UninstallCommand>("uninstall")
      .WithDescription("Bulk-uninstalls installed versions that are not bound to any active customer stage.")
      .WithExample(["manage", "uninstall", "Acceptatie", "--match", "25.5.*"])
      .WithExample(["manage", "uninstall", "Acceptatie", "--match", "25.5.10..25.5.29", "--apply"])
      .WithExample(["manage", "uninstall", "Acceptatie", "--apply"]);
  • [ ] Step 4: Build
dotnet build src/BenefitManager.Orchestrator/BenefitManager.Orchestrator.csproj -v q

Expected: Build succeeded. 0 Warning(s) 0 Error(s)

  • [ ] Step 5: Smoke test — check --help output
dotnet run --project src/BenefitManager.Orchestrator -- query --help
dotnet run --project src/BenefitManager.Orchestrator -- manage --help

Expected: both help texts include installed / uninstall respectively.

  • [ ] Step 6: Commit
git add src/BenefitManager.Orchestrator/Program.cs
git commit -m "feat(orchestrator): register query installed and manage uninstall commands"

Task 12: Update documentation#

Files: - Modify: readme.md - Modify: docs/guides/orchestrator-setup-and-operations.md

  • [ ] Step 1: Add query installed to readme.md

In readme.md, inside the ## Query commands section (after ### 4) Agent status), add:

### 5) Installed versions

```powershell
bmo query installed <ENVIRONMENT> [--unused] [--server <NAME>] [--from agents|fleet] [--match <PATTERN>] [-o table|json|md]
```

Shows versions physically installed on agents, cross-referenced with CommandCenter stage data.
`--unused` limits output to versions with no active customer stages.

Examples:

```powershell
bmo query installed Acceptatie --unused
bmo query installed Acceptatie --unused --from fleet
bmo query installed Acceptatie --match 25.5.* -o json
```
  • [ ] Step 2: Add manage uninstall to readme.md

In readme.md, inside ## Manage commands (after ### 2) Lifecycle management), add:

### 3) Bulk uninstall installed versions

```powershell
bmo manage uninstall <ENVIRONMENT>
    [--match <PATTERN>]
    [--server <NAME>]
    [--from agents|fleet]
    [--include-bound]
    [--apply]
    [--force]
    [--yes]
```

`--match` grammar:

| Shape | Interpretation | Example |
|---|---|---|
| `/.../` | Regex (case-insensitive) | `/^25\.[5-7]\..*/` |
| Contains `..` | Semver range (inclusive) | `25.5.10..25.5.29` |
| Contains `*`/`?` | Glob | `25.5.*` |
| Otherwise | Exact | `25.5.29` |

Open-ended ranges: `..25.5.29` (up to), `25.5.10..` (from). Bare `*` matches all.
Without `--match` in a TTY an interactive picker is shown. Default mode is dry-run; add `--apply` to execute.

Examples:

```powershell
bmo manage uninstall Acceptatie --match 25.5.*
bmo manage uninstall Acceptatie --match 25.5.10..25.5.29 --apply
bmo manage uninstall Acceptatie                           # interactive picker
bmo manage uninstall Acceptatie --match * --apply --yes  # all unused versions
```
  • [ ] Step 3: Update docs/guides/orchestrator-setup-and-operations.md

In the Operator checklist or Commands section, add a note about the new cleanup workflow:

### Version cleanup

Use `bmo query installed <ENV> --unused` to see all versions installed on agents that are no longer
assigned to any active customer stage. Then use `bmo manage uninstall <ENV> --match <PATTERN> --apply`
to remove them.

Dry-run first, then add `--apply`:
```powershell
bmo query installed Acceptatie --unused
bmo manage uninstall Acceptatie --match 25.5.* --apply --yes
  • [ ] Step 4: Commit
git add readme.md docs/guides/orchestrator-setup-and-operations.md
git commit -m "docs: add query installed and manage uninstall to readme and operator guide"

Self-Review#

Spec coverage check: - [x] Fix query versions predicate (Active=1) — Task 1 - [x] Fix InstalledServers width (widen STRING_AGG) — Task 1 - [x] CleanupCandidate DTO — Task 2 - [x] GetCleanupCandidatesAsync SQL + interface — Task 3 - [x] VersionsEndpoints.cs with /cleanup-candidates — Task 4 - [x] Client method GetCleanupCandidatesAsync — Task 5 - [x] VersionMatcher all grammar branches — Task 6 - [x] InstalledVersionsResolver (both strategies) — Task 7 - [x] RemoveOnServerAsync extracted — Task 8 - [x] InstalledQueryCommand — Task 9 - [x] UninstallCommand with picker + apply + dry-run — Task 10 - [x] Program.cs wiring — Task 11 - [x] Doc updates — Task 12 - [x] --include-bound forwards force — Task 10, test IncludeBound_ForwardsForceTrue - [x] No --match, no TTY → exit 2 — Task 10, test NoMatchNoTty_ReturnsExitCode2 - [x] Invalid matcher → exit 2 — Task 10, test InvalidMatchPattern_ReturnsExitCode2 - [x] Offline agent silently skipped — Task 7, test AgentFanOut_OfflineAgent_IsSkippedSilently - [x] Interactive picker — Task 10, present in implementation (not unit-tested per spec) - [x] --from fleet path — Task 7, test FleetSnapshot_AggregatesInstallationsFromFleetState

Placeholder scan: No TBD or TODO in the plan. All code steps contain complete, compilable code.

Type consistency check: - InstalledVersion — defined in Task 7, used in Tasks 8, 9, 10 ✓ - TruthSource enum — defined in Task 7, referenced in Tasks 9, 10 ✓ - CleanupCandidate — defined in Task 2, referenced in Tasks 3, 4, 5, 7 ✓ - VersionMatcher.TryParse signature (string, out Func<string,bool>?, out string?) — defined in Task 6, called identically in Tasks 9 and 10 ✓ - RemoveCommand.RemoveOnServerAsync — defined in Task 8, called in Task 10 ✓ - InstalledVersionsResolver.GetAsync(string env, IReadOnlyList<string> serverFilter, TruthSource from, CancellationToken ct) — defined in Task 7, called in Tasks 9 and 10 ✓ - InstalledVersionRow — defined in Task 9, duplicated in Task 10 (same signature, separate usage) ✓