Design: Installed versions list & bulk uninstall#

Date: 2026-05-04 Status: Approved

Problem#

Operators have two cleanup paths today and both fall short:

bmo query versions <ENV> shows versions not bound to any customer stage, but the SQL predicate ignores TblCustomerStage.Active, so versions tied only to disabled stages are hidden. The InstalledServers list only counts active TblServerVersion rows; binaries that persist on disk with a deactivated record are invisible. No agent inventory is consulted.
bmo manage lifecycle <ENV> only considers stale versions (age ≥ --older-than-days). There is no way to list or remove every unused-installed version regardless of age.

Scope#

This spec covers: - Fix query versions accuracy (predicate + InstalledServers width) - New bmo query installed <ENV> — truthful intersection of agent inventory and CommandCenter stage data - New bmo manage uninstall <ENV> — flexible bulk removal with --match grammar

Out of scope: changes to manage lifecycle, manage remove, or any other existing command behavior.

Architecture#

                                  ┌─────────────────────────────┐
                                  │  CommandCenter SQL          │
                                  │  GetCleanupCandidatesAsync  │ (new)
                                  │  GetUnusedVersionsAsync     │ (fixed)
                                  └────────────┬────────────────┘
                                               │ HTTP
                  ┌──── /api/versions/cleanup-candidates ────┐
                  │     /api/versions/unused                  │
                  │                                           │
   bmo query      ▼                  bmo manage               │
   installed  ───┴──► InstalledFilter ──► uninstall ──────────┴──► RemoveOnServerAsync
   (new)             ▲                   (new)                     (extracted from RemoveCommand)
                     │
                  Truth source
             ┌───────┴──────────────────┐
             │ --from agents (default)  │
             │ --from fleet             │
             └──────────────────────────┘

New pieces: - InstalledVersionsResolver — interface + AgentFanOutResolver (default) and FleetSnapshotResolver. Returns IReadOnlyList<InstalledVersion>. - VersionMatcher — parses --match string into a predicate. Pure function, no DI. - InstalledQueryCommand — bmo query installed. - UninstallCommand — bmo manage uninstall.

Existing code changed: - CommandCenterQueryService.GetUnusedVersionsAsync — fix predicate, widen server list. - CommandCenterQueryService — new GetCleanupCandidatesAsync. - RemoveCommand — extract RemoveOnServerAsync shared helper.

Truth sources#

`--from agents` (default)#

Fan-out via IAgentClientFactory.GetServerNames(). Per server: IAgentClient.GetInstallationsAsync() → GET /api/installations. Returns live IStateDiscoveryService snapshot. Offline agents are silently skipped (logged at Warning).

`--from fleet`#

Single call to ICommandCenterClient.GetFleetStateAsync() → GET /api/fleet/state. Returns CC's cached IFleetStateStore entries populated by Service Bus state events. Faster; may lag agent reality during network partitions.

`query installed <ENVIRONMENT>`#

bmo query installed <ENVIRONMENT>
  [--unused]               # filter to StageActiveCount = 0
  [--server <NAME>]        # repeatable; restrict to named agents
  [--from agents|fleet]    # default: agents
  [--match <PATTERN>]      # version filter (see Matcher section)
  [-o table|json|md]

Output columns: Version, Environment, Servers, StageRefs, RegisteredAt.

--unused is a read-only filter with no side effects. Without --unused, the command is also a useful "what's actually installed where" inventory tool.

`manage uninstall <ENVIRONMENT>`#

bmo manage uninstall <ENVIRONMENT>
  [--match <PATTERN>]      # required unless TTY; omit for interactive picker
  [--server <NAME>]        # repeatable; default: all configured agents
  [--from agents|fleet]    # default: agents
  [--include-bound]        # also uninstall versions still bound to active stages (force-deactivates)
  [--apply]                # required to execute; default is dry-run
  [--force]                # forwarded to remove pipeline
  [--yes]                  # skip in-TTY confirmation prompt

Default behaviour: unused versions only (implicit --unused). Lifted by --include-bound.

No --match, TTY: interactive picker (Spectre.Console MultiSelectionPrompt).

No --match, no TTY: exit 2 — refusing to uninstall everything unattended.

Dry-run: prints the candidate table. No agent calls. Exit 0.

--apply: per (version × server) calls RemoveOnServerAsync. Partial failures → continue remaining pairs, exit 1 at end. Same semantics as manage remove.

Matcher grammar#

Single --match flag. Detection order:

Shape	Interpretation	Example
`/.../`	Regex (case-insensitive, compiled)	`/^25\.[5-7]\..*/`
Contains `..`	Semver range (inclusive, open-ended allowed)	`25.5.10..25.5.29`, `..25.5.29`, `25.5.10..`
Contains `*` or `?`	Glob anchored at both ends	`25.5.*`, `25.?.10`
Otherwise	Exact (`Ordinal`)	`25.5.29`

Parse failure → exit 2 with the failing token named. Bare * is valid (match all). Prefix matching requires explicit trailing * — there is no implicit prefix mode. Non-System.Version strings excluded from range comparisons (logged debug); still match exact/glob/regex.

Interactive picker#

Spectre.Console MultiSelectionPrompt<InstalledVersion>. Columns: Version, Servers, StageRefs. After selection → [y/N] confirmation (unless --yes). Without --apply → dry-run table printed, no removal.

`query versions` accuracy fix#

CommandCenterQueryService.GetUnusedVersionsAsync changes: - Predicate: WHERE NOT EXISTS (SELECT 1 FROM TblCustomerStage cs WHERE cs.VersionKey = av.[Key] AND cs.Active = 1) — disabled stages no longer pin a version. - STRING_AGG: join TblServerVersion regardless of Active flag, and include Active as a field in the DTO so callers can distinguish active from inactive deployment records.

Removal pipeline#

Extract from RemoveCommand:

internal static async Task<bool> RemoveOnServerAsync(
    string environment, string version, string serverName, bool force,
    IAgentClient agentClient, ICommandCenterClient ccClient,
    ILogger logger, CancellationToken ct);

Both RemoveCommand and UninstallCommand call this helper. Idempotency: agentClient.RemoveAsync returning "not installed" → treat as success.

Files#

Modify: - src/BenefitManager.CommandCenter/Services/CommandCenterQueryService.cs — fix predicate, add GetCleanupCandidatesAsync - src/BenefitManager.CommandCenter/Services/ICommandCenterQueryService.cs — add interface member - src/BenefitManager.CommandCenter/Endpoints/VersionsEndpoints.cs (new file) — GET /api/versions/cleanup-candidates and GET /api/versions/unused moved here from CustomerEndpoints.cs - src/BenefitManager.CommandCenter.Client/CommandCenterClient.cs — add client method - src/BenefitManager.Contracts/Models/ManagementModels.cs — add CleanupCandidate; extend ApplicationVersionInfo - src/BenefitManager.Orchestrator/Commands/RemoveCommand.cs — extract helper - src/BenefitManager.Orchestrator/Program.cs — register new commands

Create: - src/BenefitManager.Orchestrator/Services/InstalledVersionsResolver.cs - src/BenefitManager.Orchestrator/Services/VersionMatcher.cs - src/BenefitManager.Orchestrator/Commands/InstalledQueryCommand.cs - src/BenefitManager.Orchestrator/Commands/UninstallCommand.cs - tests/BenefitManager.Orchestrator.Tests/VersionMatcherTests.cs - tests/BenefitManager.Orchestrator.Tests/InstalledVersionsResolverTests.cs - tests/BenefitManager.Orchestrator.Tests/UninstallCommandTests.cs

Reuse (no changes): - IAgentClient.GetInstallationsAsync — src/BenefitManager.Agent.Client/AgentClient.cs:68-74 - ICommandCenterClient.GetFleetStateAsync - OutputFormatter

Testing#

File	What it covers
`VersionMatcherTests.cs`	Each grammar branch, malformed inputs, bare `*`, non-`System.Version` under range
`InstalledVersionsResolverTests.cs`	Agent fan-out, duplicate aggregation, offline agent skipped, fleet empty snapshot, `--server` filter
`UninstallCommandTests.cs`	Dry-run paths, `--apply` happy path, one-server failure → exit 1, no-TTY-no-match → exit 2, `--include-bound` forwards force
`CommandCenterQueryServiceTests.cs` (extend)	Fixed predicate excludes `Active=0` stages; `GetCleanupCandidatesAsync` projection

Verification#

dotnet build benefits-mgmt.sln
dotnet test tests/BenefitManager.Orchestrator.Tests
dotnet test tests/BenefitManager.CommandCenter.Tests

# query versions fix
bmo query versions Acceptatie  # versions pinned only by Active=0 stages now appear

# new query
bmo query installed Acceptatie --unused
bmo query installed Acceptatie --unused --from fleet
bmo query installed Acceptatie --match 25.5.* -o json

# dry-run
bmo manage uninstall Acceptatie --match 25.5.10..25.5.29

# interactive picker
bmo manage uninstall Acceptatie  # (in TTY)

# apply
bmo manage uninstall Acceptatie --match <version> --apply --yes

# error modes
bmo manage uninstall Acceptatie --match /broken[regex/  # exit 2
bmo manage uninstall Acceptatie | cat                   # no TTY + no match → exit 2

After --apply: confirm binaries + service + IIS removed on agent; TblServerVersion deactivated in CC. Re-run → idempotent.

Doc updates: extend readme.md and docs/guides/orchestrator-setup-and-operations.md with both new commands.