The release couldn't ship because the phone sync was running

A full-release redeploy that had always been safe to repeat failed production validation with 18 errors before a single test ran: "This schedulable class has jobs pending or in progress". A scheduled RingCentral sync locked its own classes and everything around them, none of which had changed. Why Type.forName in the Schedulable did not help, the Deployment Settings bypass I chose not to use, shipping a smaller manifest with sf project deploy validate and quick deploy, the EmailServicesAddress that is not metadata, and the stop-deploy-reschedule routine for when the scheduled code itself changes.

The new feature was ready, tested and small. The release that should have carried it to production would not even start its tests, and the reason was a different feature entirely: the phone sync that had been running quietly in production for two days.

Nothing about the phone sync had changed. That turned out not to matter.

One manifest, redeployed every time

At Doorvana every production release has gone out the same way: one manifest, manifest/sales-release.xml, that names everything in the sales build, 87 members across Apex, flows, objects, fields, layouts and permission sets. Each release redeploys the whole thing. Components that did not change are redeployed unchanged, which costs a little time and buys a lot of certainty: production always matches the repo, and nobody has to work out which parts of a change depend on which.

“Redeploy the whole release, it’s idempotent” had been true for every release so far. This one was the bid-email intake, an Apex Email Service that turns a forwarded builder email into an Account, Contact and Opportunity. Two new fields, a permission set, four classes, one value set. The validation ran against production with RunLocalTests and failed before a single test ran. Eighteen errors, all variants of these two:

This schedulable class has jobs pending or in progress - CronTrigger IDs (…)
You can bypass this error by allowing deployments with Apex jobs in the Deployment Settings page in Setup

What was holding the lock

Two days earlier the RingCentral call sync had gone live in production. It is a Schedulable that enqueues a Queueable worker, and because a cron expression fires at most once an hour per job, a five-minute cadence is twelve scheduled jobs, RingCentral Sync :00 through :55. There are always twelve CronTrigger rows waiting to fire.

A scheduled class is locked while its jobs exist. What surprised me was how far the lock reached. The eighteen errors landed on every RingCentral and Phone Activity class in the manifest, not just the scheduler. None of them had changed in this release. The full-release habit meant they were in the package anyway, and a scheduled job makes its whole dependency tree undeployable, not just the Schedulable.

The workaround that did not work

This was not the first time the sync had hit this error. When it was built, deploys failed the same way in the sandbox, and the fix at the time was to take the worker out of the scheduler’s compile-time references:

force-app/main/default/classes/RingCentralSyncSchedule.cls

public void execute(SchedulableContext context) {
    Type syncType = Type.forName('RingCentralSyncQueueable');
    if (syncType != null) {
        System.enqueueJob((Queueable) syncType.newInstance());
    }
}

The reasoning was that with no static reference, only a change to the scheduler itself would need the jobs stopped. I wrote that up as the fix in the post on building the sync. This validation is the evidence that it was wrong. Production had been running exactly this scheduler since it went live, and the worker, RingCentralSyncQueueable, was one of the eighteen. That post now carries a correction.

The tempting fix

The error message offers its own way out: a Deployment Settings option that allows deployments while Apex jobs are pending. It would have worked, and it is a production-wide setting changed to get one release through. It also lets a deploy replace classes underneath a job that is about to run, which is exactly the situation the lock exists to prevent.

The release did not need any of the locked classes. So it did not take them.

Ship what changed

The intake went out as its own manifest, built from the commit rather than the whole slice:

manifest/bid-intake-release.xml

<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>BidEmailAddressParser</members>
        <members>BidEmailAddressParserTest</members>
        <members>BidEmailIntakeHandler</members>
        <members>BidEmailIntakeHandlerTest</members>
        <name>ApexClass</name>
    </types>
    <types>
        <members>Lead.Bid_Intake_Key__c</members>
        <members>Opportunity.Bid_Intake_Key__c</members>
        <name>CustomField</name>
    </types>
    <types>
        <members>BidEmailIntake</members>
        <name>EmailServicesFunction</name>
    </types>
    <types>
        <members>Doorvana_Base</members>
        <name>PermissionSet</name>
    </types>
    <types>
        <members>LeadSource</members>
        <name>StandardValueSet</name>
    </types>
    <version>67.0</version>
</Package>

Validate with the full local test run, then promote the validated job instead of deploying a second time:

sf project deploy validate --manifest manifest/bid-intake-release.xml \
  --target-org <prod> --test-level RunLocalTests
sf project deploy quick --job-id <validation job id> --target-org <prod>

The validation passed 69 of 69 local tests, including the RingCentral sync’s own test class, which ran without complaint while the schedule was live. The quick deploy went through, and the sync never stopped.

Two parts of that small manifest still needed care.

The StandardValueSet replaces the whole set. The release added one Lead Source value, Email - Bid. Deploying LeadSource does not add a value, it replaces the value set with whatever the file says. So the production copy was retrieved and compared first. The repo had one value production seemed to lack, Referral - Builder. It turned out to be inactive in both orgs, and a retrieve simply leaves inactive values out. Nothing to fix, but the comparison is the only way to know that.

The email address is not metadata. The EmailServicesFunction, the service and the class it runs, deploys. The address people actually send to is an EmailServicesAddress record, specific to each org, with its own run-as user. It was created in production after the deploy, as a separate step on the release checklist.

When the locked code is the change

The same afternoon brought a change that could not avoid the lock: synced calls from unknown numbers needed a default owner, which meant editing RingCentralSyncQueueable and PhoneActivityMatcher. Both locked. No manifest trick helps when the locked classes are the point of the release.

So it went the deliberate way:

  1. Stop the schedule and confirm it is empty: no CronTrigger rows for the sync, and no worker queued or processing.
  2. Validate the change with RunLocalTests and quick-deploy it.
  3. Schedule the twelve jobs again, as the same user.

manifest/phone-default-owner-release.xml

<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>PhoneActivityMatcher</members>
        <members>PhoneDefaultOwner</members>
        <members>PhoneDefaultOwnerTest</members>
        <members>RingCentralSyncQueueable</members>
        <name>ApexClass</name>
    </types>
    <types>
        <members>Phone_Setting__mdt.Default_Owner_Email__c</members>
        <name>CustomField</name>
    </types>
    <types>
        <members>Phone_Setting.Default</members>
        <name>CustomMetadata</name>
    </types>
    <types>
        <members>Phone_Setting__mdt</members>
        <name>CustomObject</name>
    </types>
    <version>67.0</version>
</Package>

The stop and start are anonymous Apex scripts kept in the repo, so nobody types them from memory under pressure:

scripts/apex/stop-ringcentral-sync.apex

RingCentralSyncSchedule.unschedule();
System.debug('Remaining RingCentral Sync jobs: '
    + [SELECT COUNT() FROM CronTrigger WHERE CronJobDetail.Name LIKE 'RingCentral Sync :%']);

force-app/main/default/classes/RingCentralSyncSchedule.cls

public static void unschedule() {
    String pattern = JOB_PREFIX + '%';
    for (CronTrigger job : [SELECT Id FROM CronTrigger WHERE CronJobDetail.Name LIKE :pattern]) {
        System.abortJob(job.Id);
    }
}

The validation passed 72 of 72. From stopping the schedule to restarting it took about twelve minutes. Before you plan a pause like that, check how the job catches up on what it missed; this one re-reads the call log with an overlap and upserts on an external ID, which is what makes a short stop tolerable.

What changed in how releases ship

The full manifest still exists and still lists everything, the intake included. It is the record of what production should contain. What changed is the rule for using it: a full redeploy now starts by stopping the RingCentral schedule and ends by restarting it, and any release that does not touch the sync ships as its own manifest instead.

Practical summary

  • This schedulable class has jobs pending or in progress fails the deploy before any test runs, and here it hit every class in the scheduled sync’s group, changed or not.
  • Type.forName in the Schedulable did not release the worker from the lock in this org. Do not rely on it.
  • The Deployment Settings bypass works, but it is an org-wide setting that lets a deploy replace code under a job about to run. Prefer not needing it.
  • If the release does not need the locked classes, ship a manifest of what changed, validated with RunLocalTests and promoted with sf project deploy quick.
  • If it does, stop the jobs, deploy, reschedule as the same user, and make sure the job can catch up on what it missed.
  • A StandardValueSet deploy replaces the whole set, and a retrieve omits inactive values. Compare against production first.
  • EmailServicesAddress is per-org data. Create it after the deploy.

Filed under

Disclosure: Doorvana's owner is also a co-owner of Nocona Consulting. This is a real engagement between two separate companies, but not an arm's-length one — and not an independent reference. More on the work page.

FAQ

What does "This schedulable class has jobs pending or in progress" mean?

The deploy includes Apex that a scheduled job depends on while that job still has a CronTrigger waiting to fire. Salesforce will not replace the class under it, and the deploy fails before any test runs. In this build the error listed the CronTrigger IDs and appeared on every class in the scheduled sync's group, not only on the Schedulable class, even though none of those classes had changed.

Does loading the worker with Type.forName stop a scheduled job from locking it?

It did not in this build. The Schedulable resolved its Queueable by name with Type.forName, so it had no compile-time reference to it, and a production validation that included the Queueable still failed with "This schedulable class has jobs pending or in progress" on it and the classes around it. Shipping a change to those classes meant stopping the scheduled jobs first.

How do you deploy Apex that a scheduled job uses?

Either abort the scheduled jobs (System.abortJob on their CronTrigger rows), deploy, and schedule them again, or turn on the Deployment Settings option that allows deployments with Apex jobs. The error itself points at the second: "You can bypass this error by allowing deployments with Apex jobs in the Deployment Settings page in Setup". The first keeps a production setting unchanged and makes the pause explicit.

Is the Email Services Address deployed with the Email Service?

No. The EmailServicesFunction (the service and its Apex class) is metadata and deploys. The address is an EmailServicesAddress record, specific to each org, and has to be created in the target org after the deploy.

Can you redeploy a full release manifest safely every time?

Only while nothing in it is locked. Once any scheduled Apex exists in the target org, a manifest that includes the scheduled code or anything it depends on fails validation, whether or not those components changed. Ship a manifest of what changed, or stop the schedule around the full redeploy.

Working on something like this?

I can tell you what it takes.

If you run a home services business and this looks like a problem you have, book a call and we will talk through it.