Instructor in Elixir: Validated, Structured LLM Output with Ecto

Every AI feature that does more than chat ends in the same place. You ask a model for something your code can use (a category, a priority, a list of line items, a yes or no) and you get back a string. Sometimes it is valid JSON. Sometimes it is JSON wrapped in a friendly sentence. Sometimes the priority field says "urgent-ish".

Instructor in Elixir fixes this by treating the model’s answer like any other untrusted input: you describe the shape you want as an Ecto schema, the library asks the model for exactly that shape, casts the response through a changeset, and if validation fails it sends the errors back to the model and asks again. You get either a struct you can pattern match on or an {:error, changeset} you can handle.

This guide walks through Instructor for Elixir from install to production: defining response models, writing validations the model has to satisfy, handling retries and failures, using it inside a Phoenix LiveView, and the honest cases where you should skip the dependency and do it by hand with Req.

Why structured output needs validation, not just a JSON mode

OpenAI, Anthropic and most local runtimes now offer some form of JSON mode or schema-constrained output. That helps a lot: it stops the “here is your JSON!” preamble and guarantees the response parses. It does not guarantee the response is correct for your domain.

A schema can say score is a number. It cannot easily say “score is between 0 and 1, and if category is :billing then invoice_id must be present”. Those are business rules, and in an Elixir app business rules already live in one place: Ecto.Changeset.

That is the core idea of Instructor. The JSON schema gets the model close, and the changeset decides whether it is close enough. When it is not, the changeset errors are the most precise feedback you could give the model, so Instructor feeds them back in a retry.

Installing Instructor

Add the dependency. The examples here use the 0.1 line of instructor; the API has moved before, so check the HexDocs for the exact version you install.

# mix.exs
defp deps do
  [
    {:instructor, "~> 0.1"},
    {:req, "~> 0.5"}
  ]
end

Then tell it which adapter to use and where the key lives. Keep the key in runtime config, never in a compiled config file:

# config/runtime.exs
config :instructor,
  adapter: Instructor.Adapters.OpenAI,
  openai: [api_key: System.fetch_env!("OPENAI_API_KEY")]

Instructor ships adapters for OpenAI and several other providers (including local runtimes such as llama.cpp and Ollama), so the same response model can be pointed at a different backend later by changing config. If you are already running models locally, our guide to Ollama in Elixir covers that side.

Your first response model

Say you run a SaaS with a support inbox and want every incoming ticket triaged: a category, a priority, a one-line summary, and a flag for whether a human must look at it today. Describe that as an embedded schema:

defmodule MyApp.Support.Triage do
  use Ecto.Schema
  use Instructor

  @llm_doc """
  Triage of a single customer support ticket.

  ## Fields
  - category: the area of the product the ticket is about.
  - priority: 1 is the most urgent, 4 the least.
  - summary: one sentence, at most 140 characters, in plain English.
  - needs_human_today: true only if the customer is blocked or losing money.
  """

  @primary_key false
  embedded_schema do
    field :category, Ecto.Enum, values: [:billing, :bug, :account, :feature_request, :other]
    field :priority, :integer
    field :summary, :string
    field :needs_human_today, :boolean
  end

  @impl true
  def validate_changeset(changeset) do
    changeset
    |> Ecto.Changeset.validate_required([:category, :priority, :summary])
    |> Ecto.Changeset.validate_inclusion(:priority, 1..4)
    |> Ecto.Changeset.validate_length(:summary, max: 140)
  end
end

Three things are doing the work here.

1. The schema becomes a JSON schema sent to the model. Ecto.Enum turns into an enum constraint, so the model is told the five allowed categories rather than guessing.
2. @llm_doc is the description attached to that schema. It is a prompt, so write it like one: say what each field means and what a good value looks like.
3. validate_changeset/1 runs after the response is cast. Anything that fails here triggers a retry with the error messages included.

Now call it:

defmodule MyApp.Support do
  alias MyApp.Support.Triage

  def triage(ticket_body) do
    Instructor.chat_completion(
      model: "gpt-4o-mini",
      response_model: Triage,
      max_retries: 2,
      messages: [
        %{role: "system", content: "You triage support tickets for a B2B SaaS."},
        %{role: "user", content: ticket_body}
      ]
    )
  end
end

The return value is the part that makes this pleasant in Elixir:

case MyApp.Support.triage(body) do
  {:ok, %Triage{priority: 1, needs_human_today: true} = triage} ->
    page_on_call(triage)

  {:ok, %Triage{} = triage} ->
    file_ticket(triage)

  {:error, %Ecto.Changeset{} = changeset} ->
    # still invalid after the retries: fall back to a human queue
    file_untriaged(body, changeset)

  {:error, reason} ->
    # network, rate limit, bad key
    {:error, reason}
end

No Jason.decode!, no Map.get(response, "priority"), no defensive String.to_existing_atom. The model’s output came through the same casting rules as a form submission.

How the retry loop works

When validate_changeset/1 returns an invalid changeset, Instructor does not just call the model again with the same prompt. It appends the model’s previous answer and the changeset errors to the conversation, roughly:

The response did not pass validation. Please try again and fix the following errors:
priority: is invalid
summary: should be at most 140 character(s)

Models are good at fixing errors stated this precisely. In practice most failures resolve on the first retry, which is why max_retries: 2 is a sensible default. Higher values mostly buy you latency and cost. If a field fails three times in a row, the problem is usually the prompt or the schema, not bad luck.

A few rules make retries cheaper and rarer:

• Constrain in the schema, not only in validation. An Ecto.Enum goes into the JSON schema and stops most bad values before they happen. A free :string validated with validate_inclusion only catches them after the fact.
• Describe limits in @llm_doc. If the summary must be under 140 characters, say so in the doc as well as in the validation. The model cannot read your changeset until it fails.
• Keep validations deterministic. A validation that calls the database or another API turns every retry into a side effect. Do that work after you have a valid struct.

Nested data: extracting line items from an invoice

Flat structs are the easy case. The place Instructor really earns its keep is nested data, because embedded schemas compose:

defmodule MyApp.Invoices.LineItem do
  use Ecto.Schema

  @primary_key false
  embedded_schema do
    field :description, :string
    field :quantity, :integer
    field :unit_price_cents, :integer
  end
end

defmodule MyApp.Invoices.Extraction do
  use Ecto.Schema
  use Instructor

  @llm_doc """
  Data extracted from one supplier invoice.
  Amounts are integers in the smallest currency unit (cents).
  """

  @primary_key false
  embedded_schema do
    field :supplier_name, :string
    field :invoice_number, :string
    field :currency, :string
    field :total_cents, :integer
    embeds_many :line_items, MyApp.Invoices.LineItem
  end

  @impl true
  def validate_changeset(changeset) do
    changeset
    |> Ecto.Changeset.validate_required([:supplier_name, :total_cents])
    |> Ecto.Changeset.validate_format(:currency, ~r/^[A-Z]{3}$/)
    |> validate_total_matches_lines()
  end

  defp validate_total_matches_lines(changeset) do
    items = Ecto.Changeset.get_embed(changeset, :line_items, :struct) || []
    total = Ecto.Changeset.get_field(changeset, :total_cents)

    sum = Enum.reduce(items, 0, fn i, acc -> acc + (i.quantity || 0) * (i.unit_price_cents || 0) end)

    if items != [] and total != sum do
      Ecto.Changeset.add_error(
        changeset,
        :total_cents,
        "is #{total} but the line items add up to #{sum}; re-read the totals"
      )
    else
      changeset
    end
  end
end

That last validation is the kind of check a JSON schema cannot express, and it catches the most common extraction error in practice: a model reading a subtotal as the total, or dropping a line. The error message is written for the model, which is a small but useful habit. “Re-read the totals” gives it something to act on.

Pair this with text pulled out of a PDF and you have a working document pipeline. Our earlier post on Elixir PDF AI extraction covers getting text and images out of the file in the first place; Instructor is the step that turns that text into rows you can insert.

Using Instructor in a Phoenix LiveView

LLM calls take seconds, so never make one inside handle_event/3 directly. Start it as an async task and let LiveView tell you when it lands. With LiveView’s start_async/3:

defmodule MyAppWeb.TicketLive do
  use MyAppWeb, :live_view

  alias MyApp.Support

  def mount(_params, _session, socket) do
    {:ok, assign(socket, triage: nil, status: :idle, body: "")}
  end

  def handle_event("triage", %{"body" => body}, socket) do
    socket =
      socket
      |> assign(status: :working, body: body)
      |> start_async(:triage, fn -> Support.triage(body) end)

    {:noreply, socket}
  end

  def handle_async(:triage, {:ok, {:ok, triage}}, socket) do
    {:noreply, assign(socket, triage: triage, status: :done)}
  end

  def handle_async(:triage, {:ok, {:error, _reason}}, socket) do
    {:noreply,
     socket
     |> assign(status: :failed)
     |> put_flash(:error, "Could not triage that ticket. It has been queued for a person.")}
  end

  def handle_async(:triage, {:exit, _reason}, socket) do
    {:noreply, assign(socket, status: :failed)}
  end
end

Because the result is a struct, the template stays boring in the best way:

<div :if={@triage}>
  <span class="badge"><%= @triage.category %></span>
  <span>P<%= @triage.priority %></span>
  <p><%= @triage.summary %></p>
</div>

For ticket volumes beyond a handful a minute, move the call into an Oban job instead, store the struct’s fields on the ticket row, and broadcast over PubSub when it is done. The job gets retries and backoff for provider errors for free, which is a different failure from a validation retry and deserves a different policy.

Streaming partial results

Instructor can also stream: you ask for stream: true and receive partial structs as fields fill in, or a stream of complete items when the response model is a list. It is a nice touch for long extractions in a UI (line items appearing one by one), but it adds complexity. Validation only means something on the finished object, so treat partial results as a preview and only persist the final, validated struct. If you are new to streaming in LiveView, start with streaming OpenAI responses in Phoenix LiveView, then add structure.

Doing it by hand: Req plus a changeset

Instructor is small and focused, but it is still a dependency on a fast-moving area. If you only have one or two structured calls, the same pattern is about fifty lines with Req and Ecto, using OpenAI’s strict JSON schema mode:

defmodule MyApp.LLM.Structured do
  @url "https://api.openai.com/v1/chat/completions"

  def call(schema_mod, json_schema, messages, retries \\ 2) do
    body = %{
      model: "gpt-4o-mini",
      messages: messages,
      response_format: %{
        type: "json_schema",
        json_schema: %{name: "result", strict: true, schema: json_schema}
      }
    }

    with {:ok, %{status: 200, body: resp}} <- Req.post(@url, json: body, auth: {:bearer, api_key()}),
         content = get_in(resp, ["choices", Access.at(0), "message", "content"]),
         {:ok, params} <- Jason.decode(content) do
      changeset = schema_mod.changeset(struct(schema_mod), params)

      cond do
        changeset.valid? ->
          {:ok, Ecto.Changeset.apply_changes(changeset)}

        retries > 0 ->
          feedback = %{role: "user", content: "Fix these errors: " <> errors_to_text(changeset)}
          assistant = %{role: "assistant", content: content}
          call(schema_mod, json_schema, messages ++ [assistant, feedback], retries - 1)

        true ->
          {:error, changeset}
      end
    end
  end

  defp errors_to_text(changeset) do
    changeset
    |> Ecto.Changeset.traverse_errors(fn {msg, opts} ->
      Enum.reduce(opts, msg, fn {k, v}, acc -> String.replace(acc, "%{#{k}}", to_string(v)) end)
    end)
    |> Enum.map_join("; ", fn {field, msgs} -> "#{field}: #{Enum.join(msgs, ", ")}" end)
  end

  defp api_key, do: Application.fetch_env!(:my_app, :openai_api_key)
end

The trade is honest: you write and maintain the JSON schema yourself (Instructor derives it from the Ecto schema), you handle each provider’s response format yourself, and you lose streaming of partial structs. You gain zero extra dependencies and complete control over the prompt that carries the errors back. For one feature, hand-rolled is fine. Once you have five response models, Instructor’s schema generation pays for itself.

Testing structured output

Two layers of tests are worth having, and they test different things.

Unit test the response model without a model. validate_changeset/1 is plain Ecto, so feed it params and assert on errors. This is where you prove the total-versus-line-items rule works.

test "rejects a total that does not match the line items" do
  params = %{
    "supplier_name" => "Acme",
    "currency" => "USD",
    "total_cents" => 1_000,
    "line_items" => [%{"description" => "Widget", "quantity" => 2, "unit_price_cents" => 300}]
  }

  changeset =
    %MyApp.Invoices.Extraction{}
    |> Ecto.Changeset.cast(params, [:supplier_name, :currency, :total_cents])
    |> Ecto.Changeset.cast_embed(:line_items, with: &Ecto.Changeset.cast(&1, &2, [:description, :quantity, :unit_price_cents]))
    |> MyApp.Invoices.Extraction.validate_changeset()

  assert %{total_cents: [msg]} = errors_on(changeset)
  assert msg =~ "add up to 600"
end

Evaluate the prompt against real examples. A small set of real tickets or invoices with the expected output, run against the live model on demand (tagged so it stays out of the default mix test), tells you whether a prompt change made things better or worse. We wrote up that setup in Elixir LLM evals with ExUnit.

Costs and gotchas

• Every retry is a full request. The conversation grows with each attempt, so a third try costs noticeably more than the first. Log how often retries happen; a rising rate is an early warning that a model update changed behaviour.
• Small models need tighter schemas. gpt-4o-mini and local 8B models do well with enums and short docs, and badly with long, open-ended string fields. Split a vague field into several specific ones.
• Do not trust it with authority. A validated struct is well formed, not true. If needs_human_today decides whether a customer gets a refund, keep a human in the loop. Validation protects your code from the model’s shape, not your business from its judgement.
• Prompt injection still applies. Ticket bodies and invoices are user content. A strict schema limits what an injected instruction can do, which is a real benefit, but it does not stop the model being persuaded to pick priority: 1 for everything.

Shipping it

The pattern is simple to state: describe the output as an Ecto schema, let the model fill it, let the changeset judge it, and retry with the errors. Instructor in Elixir packages that loop so each new AI feature is a schema and a function call rather than another bespoke parser.

If you would rather start from an app where this is already wired up, phx_ai_document gives you the document half already built (authenticated uploads, an in-browser PDF viewer, per-document chat and a typed AI provider boundary with a fake provider for tests), which is exactly where a response model like Extraction slots in. Meanwhile phx_ai covers the chat and streaming side. And if you plan to charge customers per extraction, Aurora Meter meters each call per customer and enforces plan limits in-process.

Building more than one of these? The Builder Pass gives you every PhxTemplates starter, current and future, for one price.