Sorting sales pitches out of our contact form with Clef on Workers AI

How we sort contact form inquiries into valid, sales, recruiting, and spam with Clef on Workers AI: what we send, how we apply results without overriding staff, and how rewording the criteria changed the probabilities.

Verification

Environment
Cloudflare Workers production, and Workers AI called directly from the cf CLI
Versions
@cf/cloudflare/clef, cf 1.0.0-beta.12
Result
11 fictional messages were classified with the old and new criteria; the new criteria classified all 11 correctly. On production, the old criteria classified a sales message as spam twice
Limitations
Only 11 messages, all fictional. Each probability was measured once; variance was not checked
Table of contents

Our website's contact form receives sales pitches as well as real inquiries. When we cleaned up the list in November 2026, 7 of the 8 remaining inquiries were sales messages. Staff opened each one, read it, and closed it once it turned out not to be a consultation. If a model does this sorting, how much can it be trusted with, and what happens when it gets one wrong? This article describes how we built the sorting with Clef on Cloudflare Workers AI, with the numbers we measured.

Overall architecture. The intake Worker stores the submission in D1 and returns a receipt number; the triage job started with waitUntil applies Clef's probabilities to D1, and a Cron job every minute retries failed runs

Why Clef

Clef is a classification model that Cloudflare offers on Workers AI. It returns a probability for each of the options we define. The input has the material to judge (state) and typed questions (questions); with a choice question, the response contains the selected option (choice), a probability for each option (probabilities), and a confidence value (confidence).

Input costs USD 0.24 per million tokens, and output is not billed (checked against the official description in November 2026). The same description lists Clef as a Zero data retention model, which does not keep what is sent to it.

We also considered asking a text generation model to "answer with one of these four in JSON". That approach needs handling for words outside the options and for responses that are not valid JSON, and recording how uncertain the model was takes extra work. Clef has a fixed response shape, and we can store the uncertainty as probabilities.

Limiting what the model receives

Inquiry bodies are stored encrypted in D1. The triage job decrypts the body, extracts only the fields used for the judgment, and sends them to Clef. It sends the service category, the message, the company name, the domain part of the email address, the desired timing, the budget, the locale, and the path the visitor came from. It does not send the name, the part of the email address before the @, or the phone number.

Fields sent to Clef and fields kept back. Only the 8 fields used for judgment are sent; the name, the part of the email address before the @, and the phone number are not
{
  "model": "clef",
  "state": {
    "service": "ai",
    "message": "現場データの集計を自動化したい",
    "company": "例示工業",
    "email_domain": "example.jp",
    "desired_timing": "within_3_months",
    "budget": "consult",
    "locale": "ja",
    "source_path": "/services/ai-consulting/"
  },
  "questions": {
    "classification": {
      "type": "choice",
      "instructions": "ソフトウェア開発会社のポノテク株式会社の問い合わせフォームに届いた連絡です。送り主の目的として最も当てはまるものはどれか",
      "criteria": {
        "valid": "送り主が自分の組織の課題について、ポノテクに開発、導入、支援を依頼したい、または相談したい",
        "sales": "送り主が自分の会社の商品、サービス、広告掲載、営業代行、人材紹介、開発の協力などをポノテクに売り込みたい。会社名を名乗り、面談や資料送付を求めることが多い",
        "recruit": "送り主がポノテクで働きたい、またはポノテクの求人に応募したい",
        "spam": "目的が読み取れない文字の羅列、当選や送金をうたう詐欺、SEO や被リンクの一斉送信など、ポノテクの事業と無関係な迷惑送信"
      }
    }
  }
}

Most of our inquiries are in Japanese, so the question and criteria are written in Japanese too. In English, the instruction reads "This message arrived through the contact form of Ponotech, a software development company. Which best describes the sender's purpose?", and the four criteria describe a sender who wants to ask Ponotech for development or support (valid), a sender who wants to sell their own products or services to Ponotech (sales), a sender who wants to work at Ponotech (recruit), and unrelated bulk messages such as gibberish, prize scams, and SEO backlink offers (spam).

The Worker calls the model through its binding with env.AI.run("@cf/cloudflare/clef", input). Before using the response, the job checks that choice is one of the four classes and that each of the four probabilities is a number between 0 and 1; any other shape is treated as a failure.

Applying the result without overriding staff

Triage runs in waitUntil after the submission response has been returned. Meanwhile, a staff member may open the inquiry and change its classification or status by hand. To keep the model's result from overwriting what staff did, the result is applied with a conditional UPDATE.

UPDATE lead_inquiries
SET classification = ?,
    state = CASE WHEN ? IN ('sales', 'spam') THEN 'not_required' ELSE state END,
    row_version = row_version + 1,
    updated_at = ?
WHERE id = ?
  AND classification = 'unclassified'
  AND state = 'new'
  AND row_version = ?;

The row changes only while the classification is unclassified, the status is new, and the row version (row_version) is the same as when triage started. If staff changed the inquiry first, the statement updates no rows.

Order of events when staff change the class during triage. While the job reads the row at version 3 and calls Clef, staff change the class and the version becomes 4, so the UPDATE conditioned on version 3 changes no rows

The triage result is saved in a separate table, together with whether it was applied. Applying the result, saving it, writing the audit record, and holding the auto-reply for spam run together in one D1 batch. Each statement decides whether the result was applied by checking that the row version and update time match this triage run, so the records stay consistent even if staff change the inquiry midway. The admin detail view also shows results that were not applied, with a note that staff had already changed the inquiry.

Failures and waiting for the auto-reply

The binding's run call has a 20-second limit set with Promise.race. We could not find documentation that the binding supports AbortSignal, so the timeout does not rely on cancellation.

Failed runs stay pending, and a Cron job picks up to 10 of them every minute and tries again. After three failures for one inquiry, triage ends as failed and staff classify it. The submission response and its success do not depend on the model.

Staff notifications are sent without waiting for triage. Only the auto-reply to the sender waits, for up to five minutes, until triage finishes. This keeps auto-replies from going to senders judged as spam; after five minutes the auto-reply is sent even if triage is still running.

Triage states. Triage starts pending and ends as done when classified, failed after three failures, or skipped when AI triage is disabled; failures below three are retried by Cron

Wording of the criteria changed the probabilities

In a test on production, a pitch for outsourced sales services was classified as spam twice. The sales probability was 0.43 and 0.42, and the spam probability was 0.54 and 0.56. Both classes set the status to "no action needed", but the wrong class stays in the records, and a sender treated as spam does not get an auto-reply either.

At that time, the question and criteria were as follows.

{
  "instructions": "この問い合わせの分類はどれか",
  "criteria": {
    "valid": "自社の課題や開発、導入について具体的な相談である",
    "sales": "商品、サービス、広告、協業などの営業連絡である",
    "recruit": "採用、応募、就職に関する連絡である",
    "spam": "無関係、詐欺的、または自動送信と判断できる迷惑送信である"
  }
}

In English, the instruction asked "Which class does this inquiry belong to?", and the criteria read "a specific consultation about our own issues, development, or adoption", "a sales message about products, services, advertising, or partnerships", "a message about hiring, applications, or jobs", and "spam that is unrelated, fraudulent, or automated".

We prepared 11 fictional messages (3 valid consultations, 4 sales, 1 job application, 3 spam) and compared the criteria by calling Clef directly with cf ai run. The old criteria classified all 11 correctly, but for the sales messages the sales probability was 0.59 to 0.72 and the spam probability was 0.12 to 0.40. We could not reproduce the production misclassification locally. Still, with a gap this small between sales and spam, a slightly different message could flip the result.

In the old criteria, "our own" (自社の) did not say whether it meant the sender's company or the recipient's, and "unrelated" did not say what the message was unrelated to. We therefore named the recipient company in the question and wrote every criterion as what the sender wants to do. The sales criterion also lists traits common in sales messages: the sender gives a company name and asks for a meeting or permission to send materials. The new criteria are the ones in the JSON in the previous section.

MessageExpectedOld criteriaNew criteria
Using AI to draft quotesValid0.9720.975
Systematizing orders and inventoryValid0.9760.976
Small request from an individual farmerValid0.9190.961
Outsourced sales servicesSales0.6600.907
Listing on a comparison siteSales0.5850.822
Offshore development partnershipSales0.7190.920
Recruitment agency pitchSales0.6500.948
Job applicationRecruit0.9720.946
English SEO backlink offerSpam0.9600.848
Prize scamSpam0.9820.893
Meaningless stringSpam0.9620.870

The numbers are the probabilities assigned to the expected class. With the new criteria, the sales probability for the sales messages rose to 0.82 to 0.95, and their spam probability fell to 0.01 to 0.12. On the other hand, the spam probability for the spam messages fell from 0.96 to 0.98 to 0.85 to 0.89, and most of the difference moved to sales. Widening the sales criterion also shifts the confidence on the spam side.

Probability of the expected class under the old and new criteria. It rose for the 4 sales messages and fell for the 3 spam messages

With only 11 messages, each measured once, the table shows a tendency rather than a benchmark.

Input grew from 255 to 375 tokens per inquiry to 358 to 478 tokens.

Cost and latency

At 500 tokens per inquiry and USD 0.24 per million input tokens, one inquiry costs USD 0.00012. At 150 yen per dollar that is about 0.02 yen per inquiry, which stays under 2 yen even at 100 inquiries a month.

For the three inquiries triaged on production, the time from creating the triage row to saving the result was 75 to 143 milliseconds. This includes decrypting the body and calling Clef.

Open issues

Results are applied automatically, with no threshold that skips low-probability results. Treating an inquiry as spam only holds the auto-reply; staff are still notified and can fix the classification and status from the list. Because the impact of a wrong result stays within that range, we will set a threshold after more inquiries have come in.

We have not yet checked how far the probabilities can be trusted. Each run stores all four probabilities, so we plan to compare them with the inquiries whose classification staff corrected.

When we change the criteria, we measure the same 11 messages again and update the numbers in the design document. A unit test checks that the JSON example in the design document and the wording in the code match.

Sources

Still figuring out what to build?

Whether you want to reduce manual work or bring a service idea to life, we’ll help clarify your goals and explore a practical path forward.

Contact us