Hybrid Search in Elixir: Combining pgvector and Postgres Full-Text Search

Semantic search feels like magic right up until a user types an order number.

You built a pgvector pipeline, embeddings are flowing, and “how do I cancel my plan” happily finds the paragraph titled “Ending your subscription” even though the two share almost no words. Then someone searches for ERR_TOKEN_4021 and the top result is a friendly page about password resets. The embedding model never saw that error code during training, so it has no idea what it means. It returns something vaguely technical and shrugs.

Keyword search has the opposite failure. It nails ERR_TOKEN_4021 and completely misses “cancel my plan” because the document says “terminate” instead of “cancel”.

Hybrid search in Elixir fixes both by running the two searches side by side and merging the results. This post shows how to build it on plain Postgres with pgvector and tsvector, fuse the two ranked lists with Reciprocal Rank Fusion, and wire the whole thing into a LiveView. No search cluster, no extra service, no Python sidecar. If you have not set up pgvector yet, start with our guide to RAG in Elixir with pgvector and Postgres, then come back here to make it actually usable.

Why hybrid search beats either half

The two retrieval methods fail in different, complementary places.

Dense vector search (pgvector) encodes meaning. It handles paraphrase, synonyms and questions phrased nothing like the source text. It is weak on rare tokens: product SKUs, error codes, person names, version numbers, internal acronyms. Those get averaged into a 1536 dimensional soup where nothing distinguishes 4021 from 4022.

Sparse keyword search (Postgres full-text) encodes exact lexical overlap. It is unbeatable on rare tokens and worthless when the query and the document use different vocabulary.

Real user queries are a mix. Someone searching your docs types “invoice PDF not generating for org 8812” in one session and “why is billing broken” in the next. A single retriever gets one of those right. Hybrid gets both, which matters enormously if these results feed an LLM: bad retrieval is the single largest source of wrong answers in a RAG app, and no amount of prompt engineering fixes a context window full of irrelevant chunks.

The honest caveat up front: hybrid search costs you two index scans per query instead of one, plus fusion work. On a corpus of a few million chunks that is still single-digit milliseconds of Postgres time. It is not free, and if your corpus is pure prose with no identifiers in it, plain vector search may genuinely be enough. Measure before you add complexity.

The schema: one table, two indexes

Everything lives in one table. Postgres stores the embedding and the lexical index side by side, which is the whole reason this stays simple.

defmodule MyApp.Repo.Migrations.CreateDocumentChunks do
  use Ecto.Migration

  def up do
    execute "CREATE EXTENSION IF NOT EXISTS vector"

    create table(:document_chunks) do
      add :document_id, references(:documents, on_delete: :delete_all), null: false
      add :content, :text, null: false
      add :embedding, :vector, size: 1536
      timestamps(type: :utc_datetime)
    end

    create index(:document_chunks, [:document_id])

    # A generated column means Postgres keeps the lexical index in sync for us.
    # No triggers, no application code, no chance of drift.
    execute """
    ALTER TABLE document_chunks
    ADD COLUMN search_vector tsvector
    GENERATED ALWAYS AS (to_tsvector('english', content)) STORED
    """

    execute "CREATE INDEX document_chunks_search_vector_idx ON document_chunks USING GIN (search_vector)"

    execute """
    CREATE INDEX document_chunks_embedding_idx
    ON document_chunks
    USING hnsw (embedding vector_cosine_ops)
    """
  end

  def down do
    drop table(:document_chunks)
    execute "DROP EXTENSION IF EXISTS vector"
  end
end

Two details worth pausing on.

The generated tsvector column is the part people usually get wrong. The alternative is a trigger or an after_insert hook in your context, and both eventually drift when someone updates a row in a migration or a console session. GENERATED ALWAYS AS ... STORED makes drift structurally impossible.

The hnsw index uses vector_cosine_ops because OpenAI embeddings are normalised and cosine distance is the right metric for them. If you switch to a model that is not normalised, or you use inner product, change the operator class to match. An index built for the wrong operator is silently ignored by the planner, and your query stays correct but crawls.

The Ecto schema is unremarkable:

defmodule MyApp.Search.DocumentChunk do
  use Ecto.Schema

  schema "document_chunks" do
    field :content, :string
    field :embedding, Pgvector.Ecto.Vector

    belongs_to :document, MyApp.Documents.Document

    timestamps(type: :utc_datetime)
  end
end

Add {:pgvector, "~> 0.3"} to your deps, and add Postgrex.Types setup as described in the pgvector-elixir readme so the vector type round-trips properly.

Half one: keyword candidates

Postgres full-text search is genuinely good, and most Phoenix developers only ever use ilike because nobody told them otherwise.

defmodule MyApp.Search do
  import Ecto.Query
  import Pgvector.Ecto.Query

  alias MyApp.Repo
  alias MyApp.Search.DocumentChunk

  def keyword_candidates(query_text, limit \\ 40) do
    from(c in DocumentChunk,
      where: fragment("search_vector @@ websearch_to_tsquery('english', ?)", ^query_text),
      order_by: [
        desc:
          fragment(
            "ts_rank_cd(search_vector, websearch_to_tsquery('english', ?))",
            ^query_text
          )
      ],
      limit: ^limit,
      select: %{id: c.id, content: c.content, document_id: c.document_id}
    )
    |> Repo.all()
  end
end

Use websearch_to_tsquery rather than plainto_tsquery or to_tsquery. It accepts the syntax users already know from Google (quoted phrases, or, a leading - to exclude) and, critically, it never raises on malformed input. to_tsquery will happily crash your LiveView when someone types a stray &.

ts_rank_cd is the cover-density ranker: it rewards documents where the query terms appear close together, which is usually what you want for chunk-level retrieval.

Half two: semantic candidates

The vector side needs an embedding of the query, which means an API call on the hot path.

defmodule MyApp.Search.Embeddings do
  @model "text-embedding-3-small"

  def embed(text) do
    Req.post("https://api.openai.com/v1/embeddings",
      auth: {:bearer, System.fetch_env!("OPENAI_API_KEY")},
      json: %{model: @model, input: text},
      receive_timeout: 20_000,
      retry: :transient
    )
    |> case do
      {:ok, %Req.Response{status: 200, body: %{"data" => [%{"embedding" => embedding} | _]}}} ->
        {:ok, embedding}

      {:ok, %Req.Response{status: status, body: body}} ->
        {:error, {:api_error, status, body}}

      {:error, reason} ->
        {:error, reason}
    end
  end
end

Then the query itself:

def semantic_candidates(embedding, limit \\ 40) do
  vector = Pgvector.new(embedding)

  from(c in DocumentChunk,
    order_by: cosine_distance(c.embedding, ^vector),
    limit: ^limit,
    select: %{id: c.id, content: c.content, document_id: c.document_id}
  )
  |> Repo.all()
end

cosine_distance/2 comes from Pgvector.Ecto.Query and compiles down to the <=> operator, so the HNSW index is used.

One practical note: embedding the query costs roughly 20 to 60 milliseconds and real money at scale. Cache it. A tiny ETS cache keyed on the normalised query string removes the API call for repeat searches, which on a typical support-docs corpus is a large share of traffic. If you are already metering AI calls per customer, the same hot path is a natural place to count them.

Fusing the two lists with Reciprocal Rank Fusion

Now you have two ranked lists and a problem: their scores are not comparable. ts_rank_cd returns something like 0.08, cosine distance returns something like 0.31, and normalising them against each other requires knowing the score distribution of your corpus, which changes every time you reindex.

Reciprocal Rank Fusion sidesteps this entirely by throwing the scores away and using only the ranks. Each document scores weight / (k + rank), summed across every list it appears in. The constant k (60 is the value from the original paper and a fine default) damps the influence of the very top positions so that a document ranked second in both lists can beat a document ranked first in one and absent from the other.

defmodule MyApp.Search.RRF do
  @default_k 60

  @doc """
  Fuses ranked lists into a single ordering.

  Takes `{rows, weight}` tuples where each `rows` list is already sorted best-first
  and every row has an `:id`.
  """
  def fuse(weighted_lists, opts \\ []) do
    k = Keyword.get(opts, :k, @default_k)

    weighted_lists
    |> Enum.flat_map(fn {rows, weight} ->
      rows
      |> Enum.with_index(1)
      |> Enum.map(fn {row, rank} -> {row.id, weight / (k + rank), row} end)
    end)
    |> Enum.group_by(fn {id, _score, _row} -> id end)
    |> Enum.map(fn {_id, entries} ->
      score = entries |> Enum.map(fn {_id, score, _row} -> score end) |> Enum.sum()
      {_id, _score, row} = hd(entries)
      Map.put(row, :score, score)
    end)
    |> Enum.sort_by(& &1.score, :desc)
  end
end

And the public search function that ties it together:

def hybrid_search(query_text, opts \\ []) do
  limit = Keyword.get(opts, :limit, 10)
  candidates = Keyword.get(opts, :candidates, 40)
  keyword_weight = Keyword.get(opts, :keyword_weight, 1.0)
  semantic_weight = Keyword.get(opts, :semantic_weight, 1.0)

  keyword_task = Task.async(fn -> keyword_candidates(query_text, candidates) end)

  semantic_task =
    Task.async(fn ->
      case MyApp.Search.Embeddings.embed(query_text) do
        {:ok, embedding} -> semantic_candidates(embedding, candidates)
        {:error, _reason} -> []
      end
    end)

  keyword_rows = Task.await(keyword_task, 10_000)
  semantic_rows = Task.await(semantic_task, 25_000)

  [{keyword_rows, keyword_weight}, {semantic_rows, semantic_weight}]
  |> MyApp.Search.RRF.fuse()
  |> Enum.take(limit)
end

The two halves run concurrently, which is the whole point of doing this on the BEAM: the total latency is the slower of the two, not the sum. And notice that a failed embedding call degrades to keyword-only results instead of returning an error page. When OpenAI has a bad afternoon, your search box still works.

Pull candidates from a config value. Fetching 40 per side and returning 10 is a reasonable starting point; if your users complain that obviously relevant results are missing, raise the candidate pool before you touch the weights.

Doing the fusion in SQL instead

If you would rather not move 80 rows into the BEAM per query, Postgres can fuse for you with two CTEs:

WITH semantic AS (
  SELECT id, RANK() OVER (ORDER BY embedding <=> $1) AS rank
  FROM document_chunks
  ORDER BY embedding <=> $1
  LIMIT 40
),
keyword AS (
  SELECT id,
         RANK() OVER (
           ORDER BY ts_rank_cd(search_vector, websearch_to_tsquery('english', $2)) DESC
         ) AS rank
  FROM document_chunks
  WHERE search_vector @@ websearch_to_tsquery('english', $2)
  LIMIT 40
)
SELECT c.id,
       c.content,
       COALESCE(1.0 / (60 + semantic.rank), 0.0) +
       COALESCE(1.0 / (60 + keyword.rank), 0.0) AS score
FROM document_chunks c
LEFT JOIN semantic ON semantic.id = c.id
LEFT JOIN keyword ON keyword.id = c.id
WHERE semantic.id IS NOT NULL OR keyword.id IS NOT NULL
ORDER BY score DESC
LIMIT 10;

This is faster and it is one round trip, but it loses the concurrency (the embedding still has to be computed first) and it is harder to unit test. Start with the Elixir version, move to SQL if profiling says the transfer cost matters. In practice, for most apps, it does not.

Wiring it into LiveView

Search is exactly the case start_async was built for: the user keeps typing while the query runs, and stale results must not overwrite fresh ones.

defmodule MyAppWeb.SearchLive do
  use MyAppWeb, :live_view

  def mount(_params, _session, socket) do
    {:ok, assign(socket, query: "", results: [], loading?: false)}
  end

  def handle_event("search", %{"q" => q}, socket) when byte_size(q) > 2 do
    socket =
      socket
      |> assign(query: q, loading?: true)
      |> cancel_async(:search)
      |> start_async(:search, fn -> MyApp.Search.hybrid_search(q) end)

    {:noreply, socket}
  end

  def handle_event("search", %{"q" => q}, socket) do
    {:noreply, assign(socket, query: q, results: [], loading?: false)}
  end

  def handle_async(:search, {:ok, results}, socket) do
    {:noreply, assign(socket, results: results, loading?: false)}
  end

  def handle_async(:search, {:exit, reason}, socket) do
    require Logger
    Logger.error("hybrid search failed: #{inspect(reason)}")

    {:noreply,
     socket
     |> assign(loading?: false)
     |> put_flash(:error, "Search is having trouble. Please try again.")}
  end
end

cancel_async/2 before each new start_async/3 is what keeps the results consistent. Without it, a slow query for “err” can land after a fast query for “err_token_4021” and show the wrong list. Debounce the input in the template with phx-debounce="300" so you are not embedding every keystroke.

Building a document search product on top of this? Both pdfai and phx_ai_document ship the ingestion, chunking and pgvector wiring already done, so you can spend your time on ranking quality rather than on plumbing.

Testing that it actually works

The single highest-value test is the one that encodes why you built hybrid search at all: an exact identifier must win.

defmodule MyApp.SearchTest do
  use MyApp.DataCase, async: true

  import MyApp.SearchFixtures

  describe "hybrid_search/2" do
    test "an exact error code outranks semantically similar prose" do
      target = chunk_fixture(content: "ERR_TOKEN_4021 means the refresh token expired.")
      _decoy = chunk_fixture(content: "Authentication problems are usually caused by tokens.")

      assert [%{id: id} | _] = MyApp.Search.hybrid_search("ERR_TOKEN_4021")
      assert id == target.id
    end

    test "falls back to keyword results when embedding fails" do
      chunk_fixture(content: "ERR_TOKEN_4021 means the refresh token expired.")

      expect_embedding_error()

      assert [_ | _] = MyApp.Search.hybrid_search("ERR_TOKEN_4021")
    end
  end
end

Stub the embedding call with Req.Test rather than hitting the API in CI. Beyond unit tests, keep a small file of maybe 30 real queries with their expected top result and run it as a recall check whenever you change chunking, weights or the embedding model. It is not a sophisticated eval harness, and it will still catch the majority of ranking regressions.

Tuning, and where this approach runs out

A few things worth knowing before this hits production.

Weights are a blunt instrument. If your corpus is heavy on identifiers (logs, tickets, API docs) push keyword_weight to something like 1.5. If it is conversational prose, favour the semantic side. Change one weight at a time and check it against your query file.

k matters less than you think. Values between 20 and 100 barely move the ordering. Leave it at 60 unless you have a measurement telling you otherwise.

Language configuration is not cosmetic. to_tsvector('english', ...) applies English stemming and stop words. If your content is multilingual, either store a per-row language column and use it in the generated column, or use the simple configuration and accept worse stemming.

Chunking dominates everything. No fusion strategy rescues chunks that split a sentence in half or bundle six unrelated topics together. If results are bad, look at your chunks before you look at your ranking. Our post on extracting structured data from PDFs with AI in Elixir covers getting clean text out in the first place, which is where most of the quality is won or lost.

Reranking is the next step, not a replacement. Once hybrid retrieval gives you a solid top 20, a cross-encoder or a cheap LLM pass that reorders those 20 will add real precision. It also adds latency and cost per query, so add it only after retrieval is good. Fusing garbage more carefully still gives you garbage.

Know when to leave Postgres. At tens of millions of chunks with heavy concurrent write traffic, HNSW index maintenance starts to hurt and a dedicated vector database earns its keep. Below that, and that covers almost every SaaS, Postgres doing both jobs in one query is simpler, cheaper and easier to reason about than operating a second datastore.

Wrapping up

Hybrid search in Elixir is not a research project. It is one table, two indexes, two queries that run concurrently, and about thirty lines of fusion code. You get the recall of embeddings and the precision of keyword matching without adding a single service to your deployment, and the BEAM makes running the two halves in parallel a one-liner rather than an async refactor.

If you are feeding these results into a model, the retrieval quality you just gained is worth more than any prompt tweak. Pair it with the patterns in our guide to building an AI chatbot with Phoenix LiveView and you have a genuinely solid RAG stack on infrastructure you already run.

And if you would rather start from a working app than a blank mix phx.new, the Builder Pass gives you every PhxTemplates starter, including the AI and document templates, for one lifetime price. The AI-native, use-case-specific Phoenix starter, without the month of plumbing.