We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
From the blog
Elixir Embeddings: Classification, Deduplication and Recommendations in Phoenix
By Liam Killingback ·
Elixir Embeddings: Classification, Deduplication and Recommendations in Phoenix
Elixir embeddings usually show up as step one of a RAG tutorial: embed the chunks, embed the question, find the nearest chunks, hand them to a model. That is a good use, and we covered it in RAG in Elixir with pgvector and Postgres. It is also the smallest part of what embeddings can do for a Phoenix app.
An embedding is a cheap, deterministic way to turn “these two pieces of text mean roughly the same thing” into arithmetic. Once you have that, a surprising number of features stop needing a chat model at all:
- Related items: “more like this” under an article, product or ticket.
- Classification: route a support message to billing, bugs or sales without a prompt.
- Deduplication: catch the fourth copy of the same feature request before it is saved.
All three cost a fraction of a cent, answer in milliseconds, and return the same result every time. This guide builds each one in Elixir with Req, Ecto and pgvector, and is honest about where they fall down.
What an embedding actually is
An embedding model takes text and returns a fixed-length list of floats. OpenAI’s text-embedding-3-small returns 1,536 of them. The numbers are not meaningful one at a time. What matters is geometry: texts with similar meaning land close together, and texts about different things land far apart.
“Close” is measured with cosine similarity, the cosine of the angle between two vectors. It runs from -1 to 1, and in practice most text pairs land between 0 and 1. Cosine distance is 1 - similarity, which is what pgvector’s <=> operator returns, so smaller means closer.
Two properties are worth remembering:
- Scores are relative, not absolute. A similarity of 0.45 can be a strong match with one model and noise with another. Never copy a threshold from a blog post (including this one) without checking it against your own data.
- Vectors from different models are not comparable. Mixing them in one column produces confident nonsense.
An embeddings client with Req
You do not need an SDK. The endpoint takes a list of inputs and returns a list of vectors, so batching is built in.
defmodule MyApp.AI.Embeddings do
@url "https://api.openai.com/v1/embeddings"
@model "text-embedding-3-small"
@batch_size 100
def model, do: @model
def embed(text) when is_binary(text) do
with {:ok, [vector]} <- embed_many([text]), do: {:ok, vector}
end
def embed_many(texts) when is_list(texts) do
texts
|> Enum.chunk_every(@batch_size)
|> Enum.reduce_while({:ok, []}, fn batch, {:ok, acc} ->
case request(batch) do
{:ok, vectors} -> {:cont, {:ok, acc ++ vectors}}
{:error, _reason} = error -> {:halt, error}
end
end)
end
defp request(batch) do
response =
Req.post(@url,
auth: {:bearer, System.fetch_env!("OPENAI_API_KEY")},
json: %{model: @model, input: batch},
retry: :transient,
receive_timeout: 30_000
)
case response do
{:ok, %{status: 200, body: %{"data" => data}}} ->
{:ok, data |> Enum.sort_by(& &1["index"]) |> Enum.map(& &1["embedding"])}
{:ok, %{status: status, body: body}} ->
{:error, {status, body}}
{:error, reason} ->
{:error, reason}
end
end
end
Three details that save you later:
-
Sort by
index. The response carries each vector’s position in the input. Sorting makes the pairing explicit instead of assumed. - Batch. One request with 100 inputs is far faster than 100 requests, and it uses one slot of your requests-per-minute limit instead of 100.
-
retry: :transientretries 429s and 5xx responses with backoff. For anything heavier, our guide to OpenAI rate limits in Elixir covers the full pattern.
Each input has a token ceiling (roughly 8,000 tokens for this model). For a title plus a paragraph that never matters. For whole documents it does, and the right fix is chunking, not truncation.
Similarity in plain Elixir
For a handful of vectors you do not need a database. OpenAI’s vectors are already normalised to length 1, so the dot product is the cosine similarity. A general version that does not assume that:
defmodule MyApp.AI.Similarity do
def cosine(a, b) do
{dot, norm_a, norm_b} =
Enum.zip_reduce(a, b, {0.0, 0.0, 0.0}, fn x, y, {dot, na, nb} ->
{dot + x * y, na + x * x, nb + y * y}
end)
dot / (:math.sqrt(norm_a) * :math.sqrt(norm_b))
end
end
This is fine for comparing one vector against ten labels. It is the wrong tool for comparing one vector against a hundred thousand rows. That is what pgvector is for.
Storing embeddings in Postgres
Add the pgvector package, register its Postgrex types, and enable the extension:
# lib/my_app/postgrex_types.ex
Postgrex.Types.define(
MyApp.PostgrexTypes,
Pgvector.extensions() ++ Ecto.Adapters.Postgres.extensions(),
[]
)
# config/config.exs
config :my_app, MyApp.Repo, types: MyApp.PostgrexTypes
defmodule MyApp.Repo.Migrations.AddEmbeddingToArticles do
use Ecto.Migration
def change do
execute "CREATE EXTENSION IF NOT EXISTS vector", "DROP EXTENSION vector"
alter table(:articles) do
add :embedding, :vector, size: 1536
add :embedding_model, :string
add :embedding_hash, :string
end
create index(:articles, ["embedding vector_cosine_ops"], using: :hnsw)
end
end
The two extra columns are not decoration:
-
embedding_modelrecords which model produced the vector, so a model change is a query (where embedding_model != ^new_model) and not an archaeology project. -
embedding_hashis a hash of the exact text that was embedded. If the text has not changed, you skip the API call.
In the schema, the field type is Pgvector.Ecto.Vector:
schema "articles" do
field :title, :string
field :body, :string
field :embedding, Pgvector.Ecto.Vector
field :embedding_model, :string
field :embedding_hash, :string
timestamps()
end
Embed on write, in the background
Never call the embeddings API inside the request that saves the record. The save should succeed even when OpenAI is slow. Enqueue an Oban job instead, and let the hash decide whether there is work to do:
defmodule MyApp.Workers.EmbedArticle do
use Oban.Worker, queue: :embeddings, max_attempts: 5, unique: [period: 60]
alias MyApp.{Repo, Content.Article, AI.Embeddings}
@impl true
def perform(%Oban.Job{args: %{"id" => id}}) do
article = Repo.get!(Article, id)
text = article.title <> "\n\n" <> article.body
hash = :crypto.hash(:sha256, text) |> Base.encode16(case: :lower)
if article.embedding_hash == hash and article.embedding_model == Embeddings.model() do
:ok
else
with {:ok, vector} <- Embeddings.embed(text) do
article
|> Ecto.Changeset.change(%{
embedding: Pgvector.new(vector),
embedding_model: Embeddings.model(),
embedding_hash: hash
})
|> Repo.update!()
:ok
end
end
end
end
Returning the {:error, reason} tuple from the with lets Oban retry with backoff. New to Oban? Start with our practical guide to Oban background jobs.
To backfill an existing table, enqueue one job per row with Oban.insert_all/1 and let the queue’s concurrency limit pace the API calls. Because the worker is idempotent (the hash check), running the backfill twice costs nothing.
Use 1: related items
“More like this” is one query. The article already has a vector, so there is no API call at read time:
defmodule MyApp.Content do
import Ecto.Query
import Pgvector.Ecto.Query
alias MyApp.{Repo, Content.Article}
def related(%Article{embedding: nil}, _limit), do: []
def related(%Article{id: id, embedding: embedding}, limit \\ 5) do
from(a in Article,
where: a.id != ^id and not is_nil(a.embedding),
order_by: cosine_distance(a.embedding, ^embedding),
limit: ^limit,
select: %{article: a, distance: cosine_distance(a.embedding, ^embedding)}
)
|> Repo.all()
end
end
Two things make this production-ready:
-
Filter first. Add your tenant, visibility and language conditions to the
where. With an HNSW index, a very selective filter can return fewer rows than the limit, because the index walks nearest neighbours first and filters afterwards. For small per-tenant sets, skipping the index and scanning is often both faster and exact. - Cut off the tail. Five neighbours always exist, even when none are relevant. Drop results past a distance you have checked by eye, so an unrelated article is not presented as “related”.
Render it in LiveView with assign_async/3 so the page does not wait on it, and cache the result per article if the catalogue changes slowly.
Use 2: classification without a prompt
Say you want to route incoming support messages. The chat-model approach is a prompt, a JSON schema and a second or two of latency. The embedding approach is: describe each label in a sentence, embed those once, then compare.
defmodule MyApp.Support.Router do
alias MyApp.AI.{Embeddings, Similarity}
@labels %{
billing: "Invoices, charges, refunds, payment methods, receipts and pricing",
bug: "Something is broken, errors, crashes, unexpected behaviour",
account: "Login problems, passwords, email changes, two-factor, deleting an account",
sales: "Questions before buying, plan comparison, discounts, enterprise terms"
}
@min_score 0.30
@min_margin 0.03
def classify(message) do
with {:ok, vector} <- Embeddings.embed(message) do
[{best, best_score}, {_second, second_score} | _] =
label_vectors()
|> Enum.map(fn {label, v} -> {label, Similarity.cosine(vector, v)} end)
|> Enum.sort_by(&elem(&1, 1), :desc)
if best_score >= @min_score and best_score - second_score >= @min_margin do
{:ok, best, best_score}
else
{:ok, :unknown, best_score}
end
end
end
defp label_vectors do
case :persistent_term.get({__MODULE__, :labels}, nil) do
nil ->
{names, descriptions} = Enum.unzip(@labels)
{:ok, vectors} = Embeddings.embed_many(descriptions)
labels = Enum.zip(names, vectors)
:persistent_term.put({__MODULE__, :labels}, labels)
labels
labels ->
labels
end
end
end
The important part is the :unknown branch. A classifier that always answers is worse than one that admits doubt. Two checks do the work: the best score has to clear a floor, and it has to beat the runner-up by a margin. The numbers above are placeholders. Run fifty real messages through it, print the scores, and set them from what you see.
When you have labelled examples, use them
Label descriptions are a cold start. Once your team has hand-routed a few hundred messages, those examples are a better classifier than any sentence you can write. Store each one with its vector and label, then let the nearest neighbours vote:
def classify_by_examples(vector, k \\ 7) do
query_vector = Pgvector.new(vector)
from(e in MyApp.Support.Example,
order_by: cosine_distance(e.embedding, ^query_vector),
limit: ^k,
select: {e.label, 1 - cosine_distance(e.embedding, ^query_vector)}
)
|> Repo.all()
|> Enum.group_by(&elem(&1, 0), &elem(&1, 1))
|> Enum.map(fn {label, scores} -> {label, Enum.sum(scores)} end)
|> Enum.max_by(&elem(&1, 1), fn -> nil end)
end
Each neighbour votes with its similarity, so a close match counts for more than a distant one. Every correction a human makes becomes a new row, and the classifier improves without a deploy or a retrain. Measure it the same way you would any model: hold back a labelled set and compute accuracy, as in our post on LLM evals with ExUnit.
Use 3: duplicate detection
Exact duplicates are a hash comparison. Near duplicates (“Add dark mode” and “Please support a dark theme”) are an embedding lookup at insert time:
def find_duplicate(board_id, text, max_distance \\ 0.15) do
with {:ok, vector} <- Embeddings.embed(text) do
query_vector = Pgvector.new(vector)
match =
from(r in FeatureRequest,
where: r.board_id == ^board_id and not is_nil(r.embedding),
order_by: cosine_distance(r.embedding, ^query_vector),
limit: 1,
select: %{request: r, distance: cosine_distance(r.embedding, ^query_vector)}
)
|> Repo.one()
case match do
%{distance: distance} = hit when distance <= max_distance -> {:duplicate, hit, vector}
_ -> {:unique, vector}
end
end
end
Return the vector in both branches so the caller can save it with the new row and not pay to embed the same text twice.
The product decision matters more than the code here. Do not silently reject or merge. Embeddings measure topic similarity, and “Add dark mode” sits very close to “Remove dark mode”. Show the match and let the person decide: “This looks similar to an existing request. Add your vote there?” That turns a fuzzy signal into a useful one, and a false positive costs a click.
Smaller vectors, lower bills
The text-embedding-3 models accept a dimensions parameter that shortens the vector at the source:
json: %{model: @model, input: batch, dimensions: 512}
A 512-float vector takes a third of the storage and makes index builds and scans faster, with a modest loss in quality for most classification and similarity work. If you change it, change the column size to match (size: 512), and treat it as a model change: every stored vector has to be regenerated. This is why the embedding_model column should record the dimensions too, for example "text-embedding-3-small:512".
Cost is rarely the constraint. Embedding models are priced per million input tokens at a small fraction of chat-model rates (check the provider’s pricing page for the current figure), so embedding an entire back catalogue is typically a matter of cents. The costs that do bite are operational: re-embedding after a model change, and index memory once you are into millions of rows.
If you would prefer no API at all, Bumblebee runs embedding models locally inside your Phoenix node with Nx.Serving. Everything in this post works the same way: only the client module and the vector size change.
Where embeddings mislead you
Honest limits, because each of these has caught real projects:
- Negation and direction are weak. “Cancel my plan” and “do not cancel my plan” embed close together. Do not let an embedding alone trigger a destructive action.
-
Short text is noisy. A two-word message carries little signal. Add context before embedding (the page it was sent from, the product area) or fall back to
:unknown. - Length mismatches skew scores. Comparing a one-line query with a ten-page document works poorly. Embed things of similar size, or chunk the long side.
- Thresholds drift with the model. A new model version means recalibrating every threshold, not only re-embedding.
- Vectors are personal data when the text is. An embedding of a private message should be scoped, retained and deleted with the message it came from.
When a task needs reasoning (“is this refund request within policy?”), use a chat model. When it needs “which of these is this most like?”, an embedding is faster, cheaper and repeatable.
Skip the setup
Everything above is a client module, a migration, a worker and three queries. It is also the plumbing every AI feature in a Phoenix app sits on, and wiring it into auth, uploads, background jobs and a UI is where the days go.
pdfai ships that pipeline already working: document upload, text extraction, embeddings in pgvector and a LiveView chat over the results, so you can start from the feature and not the plumbing. If you want the whole catalogue of Phoenix starters for one price, the Builder Pass includes every template, including the AI ones.
Summary
- Elixir embeddings are a general similarity tool, not only the first step of RAG.
-
One Req module handles it: batch inputs, sort by
index, retry transient errors. - Store the vector with the model name and a hash of the embedded text, and generate it in an Oban job.
-
Related items are an
order_by: cosine_distance(...)query with a distance cut-off. -
Classification starts with embedded label descriptions and gets better with nearest-neighbour voting over real labelled examples. Always keep an
:unknownanswer. - Deduplication should suggest, never decide: similar topic is not the same as same meaning.
- Calibrate every threshold on your own data, and recalibrate whenever the model changes.