A Jenkins Mac Agent is offline because disk monitoring fired, but deleting one project workspace barely changes the available space.

Fastest fix: inventory Remote FS, user Library data, temporary files, archives, and caches first. Then remove only rebuildable data, validate the release path, and scale the node if it keeps reaching the stop-work threshold.

Who should use this runbook

You are responsible for Jenkins Mac Agent operations, disk alerts, and node recovery.

You manage iOS build efficiency, cache policy, signing workflows, or release stability.

You need evidence from disk growth and queue behavior before deciding whether to reduce concurrency, isolate signing, or add Mac capacity.

The right answer is rarely “delete the whole workspace.” A Mac build node contains several different asset classes. They have different owners, recovery paths, and business value.

Storage fault domains

Jenkins workspace residue

Jenkins allocates workspaces for Pipeline jobs. Multibranch jobs and concurrent executions can create separate workspace paths, including suffixed directories. The Jenkins Pipeline documentation describes workspace allocation as part of pipeline execution, while the multibranch build discard documentation covers retention controls for obsolete job data.

This creates several common growth patterns:

  • A deleted branch leaves its workspace behind.
  • A custom workspace path bypasses the location you normally inspect.
  • A concurrent build creates another directory instead of reusing the first one.
  • A failed or interrupted build leaves files that no longer have an obvious owner.
  • A cleanup rule runs only after successful builds, so failed builds accumulate residue.

Do not treat every old-looking directory as disposable. First map each path to a job, executor, process, and current build state. A workspace used by an active build can contain source files, generated code, test fixtures, or artifacts that the next stage still needs.

A useful inspection record includes:

  • Absolute path.
  • Jenkins job and branch.
  • Last access or modification evidence.
  • Current process using the path.
  • Approximate size and recent growth.
  • Whether the contents can be recreated from source and dependencies.
  • Whether an external artifact store already contains the required output.

Use a filesystem inventory command at the top level, then descend only into the largest branches:

du -xhd 1 "$JENKINS_HOME" 2>/dev/null
du -xhd 1 "$WORKSPACE" 2>/dev/null

The exact paths depend on your node configuration. Confirm them in Jenkins rather than assuming that the visible project directory is the active workspace.

Xcode build data

Xcode data must be split by recovery cost. Treating every folder under a user Library directory as “cache” is unsafe.

DerivedData is generally rebuildable. Removing it can force a later build to compile again, so the immediate disk gain may become a longer preparation path. Keep it when repeated local reuse is important and the node has adequate capacity. Clean it when the node is blocked, the data is tied to obsolete project state, or the cleanup policy explicitly accepts the rebuild cost.

Simulator device data may include test state, generated files, and information needed to reproduce a failure. A clean simulator can be desirable for isolation, but deleting all device data during an active test run can invalidate the run or make its failure difficult to reproduce.

Simulator Runtimes and platform components are not interchangeable with ordinary build caches. Apple documents how additional Xcode components are downloaded and installed in its Xcode component management documentation. Before removing one, confirm that your node can restore it through the approved network, package, or import path.

Archives and dSYM files carry release and debugging value. Apple’s debugging information documentation explains the role of debug symbols. An archive that appears old may still be needed for crash analysis, release comparison, audit evidence, or a rollback investigation.

Use separate ownership for:

  • Build cache maintenance.
  • Test infrastructure maintenance.
  • Release engineering.
  • Crash analysis.
  • Signing and compliance.

If no owner can approve deletion, the item is not ready for a generic cleanup script.

Dependency and artifact duplication

Dependency storage often exists in more than one place. Check Swift Package Manager data, CocoaPods data, Homebrew downloads, Git LFS objects, project-specific caches, and copied artifacts.

The key question is not “is this a cache?” It is:

  • Who owns the data?
  • Can the pipeline recreate it?
  • How broadly can it be reused?
  • What event should trigger removal?
  • Is another system already storing the same content?
  • How long would recovery take under the current network policy?

A dependency can be rebuildable in theory but operationally expensive in a restricted environment. A package may be available from an internal mirror during normal operations but unavailable during an incident. Record that distinction before setting a short retention policy.

Also compare the Jenkins Controller, Mac Agent, and artifact storage. Deleting the Agent copy does not solve a duplication problem if the same output is copied repeatedly into job records, archives, and external storage. The cleanup owner must see the full path from checkout to publication.

Unsafe deletion boundaries

Several cleanup actions can create production impact:

  • Removing an active workspace while a build is running.
  • Deleting a simulator device while tests are using it.
  • Removing a platform component needed by the selected Xcode toolchain.
  • Deleting an archive before release evidence is copied and verified.
  • Removing dSYM files before crash-analysis ownership confirms retention.
  • Cleaning Keychain items or signing material with a general cache script.
  • Deleting logs needed to explain a failed release.
  • Removing a dependency cache while another executor is restoring it.

Signing credentials must never be included in a general-purpose disk cleanup pattern. A script that searches for “old” files cannot understand certificate ownership, provisioning requirements, access controls, or incident evidence.

The Jenkins node management documentation explains how node disk space and temporary directory conditions can affect node availability. Use that signal as a reason to investigate, not as permission to erase every large directory.

Controlled cleanup decisions

Use these branches before you run a destructive command.

  • If a directory belongs to an active build, keep it. Pause new work or temporarily remove the node from scheduling before maintenance.
  • If the directory is a stale workspace with a confirmed job owner and no active process, use a Jenkins-aware lifecycle policy first. Do not bypass Jenkins state with a global recursive delete.
  • If the data is rebuildable and its recovery source is tested, choose controlled cleanup. Record the path, owner, reason, and expected rebuild effect.
  • If the data is a release archive, dSYM, signing asset, or incident log, retain it until the responsible owner confirms the retention decision.
  • If a platform component or Runtime is required by the production matrix, keep it unless its replacement path is verified on the target node.
  • If cleanup restores space but the node reaches the same condition repeatedly, treat the issue as capacity or workload design.
  • If one node handles ordinary builds and production signing, separate those workloads before adding more automation.
  • If queue time and cleanup frequency increase together, reduce concurrency and evaluate another fixed or on-demand remote Mac node.

For automatic workspace removal, the Workspace Cleanup Plugin provides the cleanWs Pipeline step. Apply it to job classes, not blindly to every job. A development validation job may tolerate aggressive cleanup. A release job may require a longer evidence window and a separate artifact policy.

A minimal post-build example is:

post {
    always {
        cleanWs(
            deleteDirs: true,
            disableDeferredWipeout: true
        )
    }
}

Treat this as a policy component, not a universal answer. Test it with failed builds, retries, parallel stages, and post-build artifact publication. Confirm the plugin version and available parameters in your Jenkins environment before standardizing the snippet.

Cleanup and acceptance evidence

A safe maintenance window has a clear boundary.

First, stop new scheduling for the node or use the Jenkins control that prevents new work from entering it. Allow an active build to finish only if its state is known and the disk condition is not threatening the build.

Next, record the candidate paths and the processes holding them. Keep the inventory with the maintenance ticket. If a deletion causes a later failure, you need to know exactly what changed.

Then remove only the approved category. Start with stale, rebuildable data. Avoid combining workspace removal, Xcode component removal, archive deletion, and dependency cache deletion in one operation. A single large change makes rollback and diagnosis harder.

After cleanup, restore the node to service only after the real pipeline passes:

  1. Source checkout.
  2. Dependency restoration.
  3. Normal compilation.
  4. Test execution.
  5. Archive creation.
  6. Signing.
  7. Artifact publication.
  8. Node restart and reconnection.
  9. Disk monitoring recovery.

The Jenkins Pipeline syntax documentation can help you place cleanup and post-build behavior at the correct pipeline stage. Do not use “space released” as the acceptance criterion. A node is recovered only when the workload and monitoring path both work.

For release validation, use the same signing identity path and artifact destination as production. A debug build that passes without signing does not prove that the release path survived cleanup. Check the archive, dSYM handling, publication record, and logs needed by the incident team. Apple’s distribution documentation provides the relevant release workflow context.

Capacity signals

A disk policy needs more than one alert. Define three operational states:

  • Warning: new builds continue, but the owner receives an alert and records growth.
  • Stop-work: the node stops accepting new jobs until cleanup or capacity action is approved.
  • Intervention: a human reviews workload design, concurrency, retention, and node separation.

Do not copy threshold values from another environment. Derive yours from measured build peaks, required Xcode components, dependency recovery needs, archive retention, and the free-space margin required to complete a failed-build retry. Those inputs vary by project and must be supported by enterprise records or node measurements.

Track these signals together:

  • Daily storage growth by category.
  • Cleanup frequency.
  • Time required to rebuild removed data.
  • Queue time while the node is unavailable.
  • Number of aborted builds caused by storage.
  • Concurrent executors during disk incidents.
  • Frequency of signing jobs sharing the node with ordinary builds.
  • Recovery result after a node restart.

The pattern determines the decision. Rapid growth in stale workspaces points to retention or lifecycle defects. Large peaks during parallel builds point to concurrency or workspace isolation. Stable high usage with clean retention points to insufficient capacity. Repeated cleanup during release windows points to a workload separation problem, not just a housekeeping problem.

FAQ for operations teams

Workspace cleanup

Jenkins Workspace can be removed after a build, but only when the job’s artifact publication and failure-diagnosis requirements are known. Use cleanWs or a Jenkins lifecycle policy instead of deleting directories from the shell while the node is active.

Xcode data

DerivedData is usually the first Xcode category to evaluate because it can be rebuilt. Simulator data, Runtimes, platform components, archives, and dSYM files require separate decisions. Recovery source and release value matter more than folder age.

Repeated disk alerts

A Mac build machine that repeatedly becomes full needs a capacity review when cleanup is frequent, recovery takes longer, or builds queue behind maintenance. Reduce concurrency, isolate signing, and compare the cost of another node with the cost of recurring delivery disruption.

Signing validation

After cleanup, run the production-like checkout, dependency restore, build, test, archive, signing, and publication path. Confirm logs and debug symbols remain available. A successful compilation alone is not sufficient evidence.

Capacity choice near the delivery boundary

The table below separates the main responses. It is a decision aid, not a promise of a universal threshold.

Evidence from the node Preferred response Main risk Required proof
Stale workspaces dominate and active jobs are unaffected Fix retention and add Jenkins-aware cleanup Deleting evidence needed for failed builds Job ownership, lifecycle logs, retry test
Xcode build data dominates but can be restored Clean selected rebuildable data Longer next build preparation Dependency and component recovery test
Archives, dSYM, or signing data dominate Apply owner-approved retention Lost release or crash evidence Release and incident owner approval
Parallel builds create large temporary peaks Reduce concurrency or isolate workspaces Longer queue time Queue and peak-usage records
Clean retention still leaves little recovery margin Add fixed or on-demand Mac capacity Higher operating cost Stable build success on the added node
Ordinary builds compete with production signing Split signing onto a controlled node More scheduling complexity Signing acceptance and failover procedure

If your current setup relies on one purchased Mac, you also carry hardware failure, replacement lead time, local access, and capacity planning risk. If it relies on a shared remote service without workload isolation, you may face scheduling contention, unclear ownership, or insufficient control over the signing boundary. Neither model should be accepted on price alone.

For a team that has measured its workload and needs an additional Mac only during release peaks, MACCOME’s remote Mac options can be evaluated as an on-demand capacity path. If the node must remain dedicated to a stable CI workload, compare it with a dedicated Mac mini setup. For Apple Silicon-specific build validation, review the remote Apple Silicon Mac option only after checking toolchain, signing, network, and data-handling requirements.

The current single-node approach has three common weaknesses: cleanup interrupts delivery, shared signing increases blast radius, and fixed hardware cannot absorb short release peaks without idle capacity at other times. Remote Mac capacity is a better fit when you need a temporary build node, a controlled pilot, or an additional CI lane while you collect real usage evidence. It is not automatically the best choice for permanent heavy workloads, strict physical-interface requirements, or data that cannot leave your approved operating boundary.

Final operating rule

Do not empty the Jenkins Workspace because a Mac Agent is full. Separate workspaces, DerivedData, simulator data, Runtimes, platform components, archives, signing assets, and dependency caches. Clean by ownership and rebuildability, then prove checkout, build, test, signing, publication, and monitoring recovery.

If the same node continues to hit its stop-work state after policy corrections, the evidence points beyond housekeeping. Reduce concurrency, separate production signing, and compare the measured cost of another fixed or on-demand remote Mac node against repeated queue delays and release interruptions.