Automate Litmus Spam Testing in CI

A template change that quietly pushes your SpamAssassin score above the threshold, or a misconfigured sender that breaks DKIM alignment, will not fail any unit test — it will just land your mail in spam after you ship. This guide makes the spam-filter and authentication verdict a build gate: submit the email to Litmus from CI, poll for the score and auth results, and fail the job before the regression reaches production.

The need: catch score and auth regressions before sending

Two classes of regression silently destroy deliverability. The first is content-driven: spammy phrasing, a bad text-to-image ratio, a broken unsubscribe link, or a newly added tracking domain that lands on a blocklist — each nudges the SpamAssassin score upward. The second is authentication-driven: a sending change that breaks SPF alignment, an unsigned DKIM body, or a DMARC policy your From domain no longer satisfies. Both are invisible to local rendering and to a DOM diff. Litmus runs the email through real spam filters and reports a numeric score plus per-mechanism authentication results, so you can assert on them mechanically.

The plan: build the email, submit it to Litmus, poll until the spam test completes, read the SpamAssassin score and the SPF/DKIM/DMARC verdicts, and exit non-zero if the score exceeds your threshold or any auth check fails. This is the gating counterpart to the rendering checks covered across Litmus and Email on Acid workflows.

The deploy is gated on a single sequence: build, submit to Litmus, poll for the verdict, then enforce the threshold.

Submitting and polling: a CI step

The flow has three API touchpoints: create a spam test (which returns the seed addresses and a test id), send the rendered email to those seed addresses over your real signing path, then poll the test until results are ready. Sending through the real path matters — that is what exercises SPF, DKIM, and DMARC; authentication outcomes tie directly to the records described in the SPF, DKIM, and DMARC guide, and a test that bypasses signing tells you nothing about alignment.

// scripts/litmus-spam-gate.mjs — run as a CI step; exits non-zero to fail the build.
const API = 'https://<account>.litmus.com/api/v1';
const AUTH = 'Basic ' + Buffer.from(`${process.env.LITMUS_USER}:${process.env.LITMUS_PASS}`).toString('base64');
const MAX_SCORE = Number(process.env.MAX_SPAM_SCORE ?? '3.0'); // SpamAssassin fail threshold

const headers = { Authorization: AUTH, 'Content-Type': 'application/json', Accept: 'application/json' };

// 1) Create a spam test; the response carries the seed list to send the email to.
async function createSpamTest() {
  const res = await fetch(`${API}/spam_tests`, { method: 'POST', headers });
  if (!res.ok) throw new Error(`create failed: ${res.status}`);
  return res.json(); // { id, spam_seed_email_addresses: [...] }
}

// 2) Poll the test until its state is "complete"; back off between attempts.
async function pollUntilComplete(id, { tries = 30, delayMs = 10000 } = {}) {
  for (let i = 0; i < tries; i++) {
    const res = await fetch(`${API}/spam_tests/${id}`, { headers });
    const test = await res.json();
    if (test.state === 'complete') return test;
    await new Promise(r => setTimeout(r, delayMs)); // async polling, ~5 min ceiling
  }
  throw new Error('Litmus spam test did not complete in time');
}

const { id, spam_seed_email_addresses } = await createSpamTest();
// Send the BUILT email to every seed via your real signing relay so SPF/DKIM/DMARC
// are actually exercised (see your ESP/transport send code — omitted for brevity).
await sendBuiltEmailTo(spam_seed_email_addresses);

const test = await pollUntilComplete(id);

// 3) Apply the gate: SpamAssassin score AND each authentication mechanism.
const score = test.results.spamassassin.score;             // numeric, higher = spammier
const { spf, dkim, dmarc } = test.results.authentication;  // each: "pass" | "fail" | "neutral"

const failures = [];
if (score > MAX_SCORE) failures.push(`SpamAssassin score ${score} > ${MAX_SCORE}`);
for (const [name, verdict] of Object.entries({ spf, dkim, dmarc })) {
  if (verdict !== 'pass') failures.push(`${name.toUpperCase()} = ${verdict}`);
}

if (failures.length) {
  console.error('Spam/auth gate FAILED:\n - ' + failures.join('\n - '));
  process.exit(1); // non-zero -> the deploy is blocked
}
console.log(`Spam/auth gate PASSED (score ${score}, SPF/DKIM/DMARC all pass).`);

Variant: Email on Acid, and webhook over polling

Email on Acid offers an equivalent flow — create a test, send to its seed list, fetch results — through its own API. The gate logic is identical; only the endpoints, auth header, and the JSON field names for the SpamAssassin score and authentication verdicts change. Keep the threshold-and-fail decision in one shared function and swap the provider client behind it.

Webhook instead of polling. Polling is simple and self-contained, which is why it suits CI, but it holds the runner open for minutes. If your pipeline is event-driven, register a results webhook so the provider posts back when the test completes, have the webhook handler write the verdict to a status store, and let a deploy gate read that status. Polling keeps everything in one job; webhooks decouple submission from the gate at the cost of an extra moving part. For a synchronous CI job, the async-polling approach above is usually the right trade.

Pipeline integration

Wrap the script in a GitHub Actions job that runs before the deploy job depends on it.

# .github/workflows/spam-gate.yml
name: Spam & Auth Gate
on:
  pull_request:
  push:
    branches: [main]

jobs:
  spam-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '22'
          cache: 'npm'
      - run: npm ci
      - run: npm run build:emails        # produce the exact bytes to submit
      - name: Litmus spam + auth gate
        env:
          LITMUS_USER: ${{ secrets.LITMUS_USER }}
          LITMUS_PASS: ${{ secrets.LITMUS_PASS }}
          MAX_SPAM_SCORE: '3.0'          # tune to your provider's headroom
        run: node scripts/litmus-spam-gate.mjs

Make the deploy job declare needs: [spam-gate] so a failing score or a broken auth verdict blocks the release. Keep credentials in repository secrets, never in the workflow file. For tighter integration with the rendering side, the companion guide on integrating the Litmus API into GitHub Actions shows how to run rendering captures in the same workflow so one job covers both how the email looks and whether it will be delivered.

Setting the SpamAssassin score threshold

SpamAssassin sums per-rule points; higher means spammier, and most mailbox providers treat a score at or above 5.0 as spam. That default is the wrong gate for CI, because you want to fail before you reach the inbox-provider cutoff, not at it. Set your build threshold low enough to leave headroom for the rules you can't control from a template — recipient-side reputation, time-of-day, and per-domain heuristics all add points after the email leaves your pipeline.

Threshold (`MAX_SPAM_SCORE`)	Posture	When to use
`2.0`	Strict	High-volume transactional senders with thin reputation headroom
`3.0`	Balanced	Default for most teams; blocks regressions, tolerates minor content rules
`5.0`	Lenient	Matches the provider cutoff; only catches egregious content, leaves no margin

The score is only actionable if you read the rule breakdown, not just the total. Litmus returns the contributing rules; log them on failure so the author sees exactly which point was added:

// On failure, surface the rules that pushed the score up, not just the number.
const rules = test.results.spamassassin.rules ?? []; // [{ name, score, description }]
for (const r of rules.filter(r => r.score > 0)) {
  // e.g. HTML_IMAGE_ONLY_28, MISSING_MID, URIBL_BLOCKED — name the rule, show its points.
  console.error(`  +${r.score}  ${r.name} — ${r.description}`);
}

Common rules a template change can trigger: HTML_IMAGE_ONLY_* (image-heavy, too little text), MISSING_MID (no Message-ID header — a sending-path bug), HTML_MESSAGE baseline points, and URIBL_BLOCKED/URIBL_DBL_SPAM when a link domain is on a URI blocklist. Documenting the threshold alongside the breakdown turns a red build into a one-line fix instead of a hunt.

Blocklist and authentication checks

Beyond the aggregate score, the spam test exposes two signals worth gating on independently: blocklist membership and per-mechanism authentication. A clean SpamAssassin score with a blocklisted sending IP is still undeliverable, so assert on both.

// Blocklist + auth, evaluated separately from the SpamAssassin total.
const { spf, dkim, dmarc } = test.results.authentication;   // "pass" | "fail" | "neutral"
const blocklists = test.results.blocklists ?? [];            // [{ name, listed: bool }]

const listed = blocklists.filter(b => b.listed).map(b => b.name); // e.g. Spamhaus ZEN, Barracuda
if (listed.length) failures.push(`Blocklisted on: ${listed.join(', ')}`);

// DMARC only passes if SPF OR DKIM aligns with the From domain; a "neutral" SPF means the
// seed send did not traverse the real signing relay — treat that as a misconfigured test.
if (dmarc !== 'pass') failures.push(`DMARC=${dmarc} (check SPF/DKIM alignment to From domain)`);

If spf reports neutral, the test email almost certainly bypassed your real transport — the fix is the test harness, not the records. The records themselves are covered in the SPF, DKIM, and DMARC guide; the gate here only verifies they are satisfied at send time.

Variant: Email on Acid

Email on Acid exposes an equivalent deliverability test — create a test, send the built email to its seed list, then fetch results — with the same gate semantics. Only the surface differs: a different base URL, an API-key auth header rather than Litmus Basic auth, and different JSON paths for the score, blocklists, and auth verdicts.

// Email on Acid adapter — same gate, different field names. Swap behind one interface.
const r = await fetch(`${EOA_API}/v5/spamtest/results/${id}`, { headers: eoaAuth });
const data = await r.json();
const score = data.spam_score;                 // Email on Acid: flat numeric field
const auth = { spf: data.spf_status, dkim: data.dkim_status, dmarc: data.dmarc_status };
// Feed `score` and `auth` into the SAME threshold-and-fail function used for Litmus.

Keep the threshold-and-fail decision in one shared function (evaluateGate({ score, auth, blocklists })) and let each provider's adapter normalize into that shape. Swapping platforms, or running both during a migration, then never touches the gating logic.

Validation checklist

Integrating the Litmus API into GitHub Actions — run rendering captures in the same workflow as the spam gate
Litmus and Email on Acid workflows — the broader cross-client and deliverability testing setup
Email authentication: SPF, DKIM, DMARC — the records the authentication checks in this gate verify

← Back to Litmus & Email on Acid Workflows