Salesforce Vector Knowledge (Hybrid Semantic Search for Lightning Knowledge)

Role: Solo, end-to-end. Architecture, Apex backend, the LWC front end, the data model, the OpenAI integration, and the seed/deploy tooling.
Tech Stack: Salesforce Apex (API v62), Lightning Web Components, SOQL/SOSL, OpenAI (gpt-4o-mini + text-embedding-3-small), Named Credential auth.
Repository: Private. Runs in a Salesforce scratch org via a one-command bootstrap script.

Overview

A support rep opens a Case, clicks Search using Case, and within about two seconds sees a predicted topic, the 3-5 most relevant Knowledge articles with one-sentence summaries, and a running per-search cost. A synthesized answer with [1] [2] citations fills in a few seconds later.

I built this independently, in a personal scratch org against seeded demo data, when I learned I’d be testing an Agentforce knowledge-summary agent at work. Before evaluating the commercial product I wanted to know, from the inside, what plain Apex and a direct LLM API could do on the same problem. Everything here is my own build on my own time.

The reason this exists is cost. Salesforce sells this exact “synthesized answer over your Knowledge base” experience through Agentforce, which bills a $0.10 minimum per action. For a small internal knowledge base, a direct LLM API call does the same job for roughly $0.0004 to $0.0009 per search, over 100x cheaper, with no Einstein and no Agentforce license. This is a proof of concept that measures that gap honestly: about 2,300 lines of Apex and LWC source plus about 1,200 lines of tests, with every search’s token count and cost logged to a custom object so the ROI claim is auditable rather than asserted.

Agentforce charges you per action for the answer. This computes the same answer at raw token cost and writes the receipt to a log object.

System Architecture

Two pipelines share one data model. An ingestion pipeline (Knowledge trigger to async queueable) chunks, classifies, and embeds each article once at publish time. A search pipeline (an @AuraEnabled Apex controller) runs hybrid retrieval on every query. Retrieval ranks at chunk granularity through Reciprocal Rank Fusion, then rolls chunks up to their parent articles for display.

graph TD
    subgraph Ingestion["Ingestion (on publish)"]
        Pub[Knowledge article published] --> Trig[KnowledgeArticleTrigger]
        Trig --> Q[ChunkAndEmbedQueueable]
        Q --> Hash{Source hash<br/>changed?}
        Hash -->|no| Skip[Skip, zero callouts]
        Hash -->|yes| Chunk[Chunk body ~400 tok<br/>50 overlap]
        Chunk --> Cls[Classify + summarize<br/>1 gpt-4o-mini call]
        Chunk --> Emb[Batch embed chunks<br/>1 callout]
        Cls --> Meta[(Knowledge_Article_Meta__c<br/>topic, tags, summary, hash)]
        Emb --> Chk[(Knowledge_Article_Chunk__c<br/>float16 base64 vectors)]
    end

    subgraph Search["Search (on click)"]
        LWC[kbHybridSearch LWC] --> Ctrl[KbSearchController.search]
        Ctrl --> C1[Classify query -> topic + confidence]
        Ctrl --> C2[Embed query @ 256 dims]
        C1 --> PF[Pre-filter SOQL by topic<br/>skipped if confidence < 0.6]
        PF --> SOSL[SOSL keyword: top 50 chunks]
        PF --> COS[Cosine over float16 vectors: top 50]
        SOSL --> RRF[RRF fusion]
        COS --> RRF
        RRF --> Roll[Roll up to top 5 articles]
        Roll --> Sum[Overall synthesis<br/>conditional on Summary_Mode]
        Sum --> Log[(KB_Search_Event__c<br/>cost + latency + phases)]
        Log --> LWC
    end

    Chk -.read.-> PF
    Meta -.read.-> PF
    Meta -.stored summaries.-> Roll

Components

Engineering Details

1. Storing float vectors in Apex, which has no half-float and no hex literals

Problem. Salesforce has no vector column and no array-of-float storage. Embeddings had to live in a Long Text field, and a naive JSON-of-doubles representation was both large and slow to parse back at query time. Apex also lacks any 16-bit float type, and it rejects 0x... hex integer literals, so there was no library path to lean on.

Approach. I wrote an IEEE-754 binary16 encoder from scratch: pack each double into 16 bits (sign, 5-bit exponent, 10-bit mantissa), including the subnormal and overflow paths, then hex-encode and base64 the byte stream for storage. Decoding reverses it. Because embeddings are L2-normalized to unit length before storage, cosine similarity at query time is a plain dot product over the decoded vectors.

Result. Storage runs about 700 bytes of base64 per chunk at the 256-dimension default, versus about 2KB at 768 dimensions. At 5,000 chunks that is roughly 3.5MB resident, comfortably inside the Apex async heap. The encoder supports 256 / 512 / 768 / 1536 dimensions so the precision-vs-speed trade is one config field.

2. Taxonomy sizing turned a 10-second query into a sub-second one

Problem. The topic pre-filter narrows the candidate chunk set before the expensive in-Apex cosine pass. The first taxonomy had 32 fine-grained topics, and it never actually narrowed anything. Cosine was taking about 10 seconds per search.

Approach. The cause was distribution, not the algorithm. With 266 articles spread across 32 topics, the busiest support topics landed on labels holding 3 or 4 articles each. The candidate loader has a soft fallback: if a topic matches fewer than 5 articles it expands to the full corpus to protect recall. That fallback was firing on nearly every query, so cosine ran over all ~800 chunks. I collapsed the taxonomy to 10 broad buckets sized so every topic holds at least 10-15 articles, leaving headroom for classifier near-misses.

Result. For the same query on the same corpus, candidates considered dropped from ~800 to 12, and the cosine phase dropped from ~10,000ms to ~330ms. The design notes carry the rule of thumb: size every topic to at least 2-3x the fallback cutoff.

3. Pre-computing summaries and hash-gating ingestion

Problem. The obvious way to build result cards is to summarize each returned article at search time. That is a per-article LLM call on every search, and it dominates both latency and cost.

Approach. The one-sentence per-article summary is generated once, during ingestion, in the same gpt-4o-mini call that classifies the article, and stored on the meta object. Search reads it back for free. Ingestion itself is idempotent behind two SHA-256 gates: an article-body hash skips the whole job if nothing changed, and a per-chunk hash re-embeds only the chunks whose text actually moved.

Result. Publishing an article twice makes zero callouts the second time; a small edit re-embeds only the changed chunks. At search time the article cards cost nothing beyond storage reads, so the only search-time LLM spend is the query classify, the query embed, and the optional overall synthesis.

4. Hybrid retrieval with RRF and max-based article scoring

Problem. Pure keyword search misses paraphrases; pure vector search misses exact identifiers and rare terms. And once chunk matches roll up to articles, summing a chunk’s scores per article gives long articles an unfair edge.

Approach. SOSL and cosine each produce a ranked list of chunk IDs. Reciprocal Rank Fusion combines them with score = Σ 1/(60 + rank), where k=60 is the standard IR default, rewarding chunks that do well in either modality without needing exact agreement. On rollup, each article inherits the score of its single best chunk (the max), not the sum, which removes the length bias.

Two smaller retrieval-quality decisions feed both modalities. Every chunk is stored title-prefixed (title, blank line, body) so SOSL and the embedding model both see the article title’s topical signal, with the LWC stripping the prefix back off for display. And the classifier emits a rewritten query that drives both retrieval passes, while the user’s original wording is kept for the synthesis step.

Result. The two modalities cover each other’s blind spots, and article ranking matches a human’s “best match” intuition rather than favoring the longest document.

5. Four summary modes, because synthesis is the slow, expensive phase

Problem. The synthesized answer is the one search phase that costs real latency and real tokens, and different orgs weigh that trade differently. Hardcoding one behavior would be wrong for somebody.

Approach. Summary_Mode__c selects one of four modes: inline (block until the answer is ready), deferred (paint cards, then fetch the answer), parallel (the default: fire the per-article and overall synthesis callouts overlapping, progressively painting the UI as each returns), and none (retrieval only, zero synthesis spend).

Result. The default paints article cards in ~2 seconds with the answer filling in a few seconds later, and an org that just wants cheap fast retrieval flips one config field to none. The async modes are why telemetry does additive updates under a FOR UPDATE lock: the cost row gets finished by whichever callout lands last.

Code Snippet: packing a double into an IEEE-754 half-float

From the encoder that lets Apex, which has no 16-bit float, store embedding vectors compactly. Bit masks are spelled in decimal because Apex rejects hex literals.

// Float16Encoder.cls
private static Integer toHalfBits(Double v) {
    if (v == 0.0) return 0;
    Boolean negative = v < 0;
    Double abs = Math.abs(v);
    if (abs >= 65504.0) {                      // clamp to max normal
        return (negative ? SIGN_BIT : 0) | MAX_NORMAL;
    }
    Integer expVal = (Integer) Math.floor(Math.log(abs) / Math.log(2));
    Double normalized = abs / pow2(expVal);
    if (normalized >= 2.0) { normalized = normalized / 2.0; expVal += 1; }
    else if (normalized < 1.0) { normalized = normalized * 2.0; expVal -= 1; }

    Integer halfExp = expVal + 15;             // half-float exponent bias
    Integer mantissa = (Integer) Math.round((normalized - 1.0) * 1024.0);
    if (mantissa >= 1024) { mantissa = 0; halfExp += 1; }
    return (negative ? SIGN_BIT : 0) | (halfExp << 10) | mantissa;
}

Sample Output

Every search writes a KB_Search_Event__c row. Results_JSON__c holds the full retrieval response: per-phase timings, the three-state pre-filter status, and the ranked articles with their stored summaries. This is the audit trail behind the cost claim.

{
  "topic": "Billing_Payments",
  "confidence": 0.92,
  "prefilterStatus": "on",
  "chunksConsidered": 12,
  "mode": "parallel",
  "estimatedCostUsd": 0.00058,
  "timingsMs": {
    "classify": 380,
    "embed": 70,
    "prefilter": 40,
    "sosl": 120,
    "cosine": 330,
    "rrf": 6,
    "hydrate": 60,
    "total": 1006
  },
  "articles": [
    {
      "id": "ka0...AAM",
      "title": "How to update a saved payment method",
      "urlName": "update-saved-payment-method",
      "score": 0.0322,
      "summary": "Explains how customers replace or remove a stored card on their account.",
      "chunks": ["kc1...AAA", "kc1...AAB"]
    },
    {
      "id": "ka0...AAN",
      "title": "Why a charge shows as pending",
      "urlName": "pending-charge-explained",
      "score": 0.0180,
      "summary": "Describes authorization holds and when a pending charge settles or drops off."
    }
  ],
  "overallSummary": "Customers can update a stored card under Account > Payment Methods [1]. A pending charge is an authorization hold that settles within a few business days [2]."
}

Tech Stack

Where It Goes Next

The design notes map the generalization: pluggable LLM clients beyond OpenAI, and a propose-approve-execute gate so an agent can suggest record changes that a human approves before anything writes. Spec’d and documented, not yet built, and I’d rather say that plainly than let a roadmap read as a feature list.