Six Lines, Zero API Calls: Running LLMs On-Device in React Native

I'm using LFM2.5 1.2B here because it's the library's own default and small enough to behave on mid-range hardware. You've got real choices though. The bundled lineup includes:

• Text models: Qwen 3 (0.6B / 1.7B / 4B), Llama 3.2 (1B / 3B), Phi 4 Mini, SmolLM 2, Hammer 2.1




• Vision-capable: Gemma 4 and LFM2.5-VL

Why I'd start small: a 4B model is noticeably smarter and noticeably more likely to crash with an out-of-memory error on a budget Android. Pick the smallest model that clears your quality bar, then size up only if you must.

The hook gives you state to drive your UI:

• llm.downloadProgress: 0 to 1 while the model downloads on first launch




• llm.isReady: flips true when it's loaded and usable




• llm.error: populated if anything blows up




• llm.isGenerating: true while tokens are streaming




• llm.response: the generated text, updated token by token

Wiring the chat

There are two ways to use this hook, and the docs name them well: functional vs managed. The distinction matters, so don't skim it.

Functional: you own the state

You pass the full message array every time, you keep the history, you get a token stream back. Nothing is remembered for you.

import { models, useLLM, type Message } from "react-native-executorch";



import { View, Text, Button } from "react-native";



function Chat() {

  const llm = useLLM({ model: models.llm.lfm2_5_1_2b_instruct() });



  const handleGenerate = async () => {

    const chat: Message[] = [

      { role: "system", content: "You are a concise, helpful assistant." },



      { role: "user", content: "Explain a closure in one sentence." },

    ];



    // resolves to the full string; llm.response updates live as it streams



    const final = await llm.generate(chat);



    console.log("done:", final);

  };



  if (!llm.isReady) {

    return (

      <Text>Loading model… {Math.round(llm.downloadProgress * 100)}%</Text>

    );

  }



  return (

    <View>

      <Button title="Generate" onPress={handleGenerate} />



      <Text>{llm.response}</Text>

    </View>

  );

}

Note the shape of generate: it both returns a promise and streams into llm.response. So you render llm.response for the live typewriter effect, and await the return value when you need the finished string for, say, saving to a DB. Same call, two consumption patterns.

Managed: the library owns the state

If you're building an actual back-and-forth chat, you don't want to hand-roll the history array. sendMessage plus messageHistory plus configure does it for you:

import { useEffect } from "react";



import { models, useLLM, DEFAULT_SYSTEM_PROMPT } from "react-native-executorch";



function ManagedChat() {

  const llm = useLLM({ model: models.llm.lfm2_5_1_2b_instruct() });



  const { configure } = llm;



  useEffect(() => {

    configure({

      chatConfig: {

        systemPrompt: `${DEFAULT_SYSTEM_PROMPT} Keep answers short.`,

      },



      generationConfig: {

        temperature: 0.7,



        topP: 0.9,

      },

    });

  }, [configure]);



  const send = () => llm.sendMessage("Who are you?");



  return (

    <View>

      {llm.messageHistory.map((m, i) => (

        <Text key={i}>

          {m.role}: {m.content}

        </Text>

      ))}



      <Button title="Send" onPress={send} disabled={!llm.isReady} />

    </View>

  );

}

configure only affects the managed path. chatConfig and toolsConfig do nothing to generate(). That's a subtle footgun: set a system prompt in configure, then call generate and wonder why it's ignored. Mode and config have to match.

My take: use managed for chat, functional for one-shot transforms (summarize this, rewrite that, extract JSON from this). The text-enhancement tool I mentioned was pure functional. There's no conversation, just input string to improved string, and managed state would've been overhead I'd have to fight.

Reality check

This is the section a docs-paraphrase can't write, so here's the honest list of what actually bit me.

1. Dismounting mid-generation crashes the app. Hard crash, not a warning. If the user navigates away while tokens are still streaming, you go down. The fix is to interrupt and wait:

// before unmount / on a stop button



llm.interrupt();



// then wait until llm.isGenerating === false before tearing down

Wire a stop button to interrupt() and gate isGenerating into your navigation guards. I learned this the way everyone does, with a back-button press during a long answer.

2. First launch downloads a model. A big one. These files run from roughly 700MB to over a gigabyte. The hosted models stream down on first use and cache in your app's documents directory, but if you don't render downloadProgress, the user stares at a dead screen and force-quits. Build the loading UX first, not last. And consider letting users pick a model, or bundle a small one for offline-from-install.

3. RAM, not CPU, is your ceiling. Crashes on cheaper devices are almost always out-of-memory, not slowness. Use quantized models. If you're testing on an Android emulator and it dies, bump the emulator's RAM before you blame your code. I wasted real time debugging "my" bug that was just a starved emulator.

4. Expo Go will never work. Said it above, saying it again, because you will forget once and spend ten minutes confused. Native modules mean a custom dev build.

5. One model runner at a time. The architecture is built around a single active model instance. Don't try to stand up two useLLM components side by side and expect both to run.

6. Token batching exists for a reason. A fast model can push 60+ tokens/sec, and if every token triggers a React re-render, your UI jank-fest begins. The library batches token emissions (default around 10 tokens or 80ms, whichever first). If generation feels choppy or your list stutters, tune outputTokenBatchSize and batchTimeInterval in generationConfig rather than reaching for a FlatList rewrite.

Taking it further

Once the basic loop works, the library has more than chat:

• Tool calling. Define functions the model can invoke (check weather, toggle a setting, hit a local API). You give it tools plus an executeToolCallback, and in managed mode it parses and runs the calls for you. Use a model whose chat template actually supports it. Hammer 2.1 is purpose-built for function calling.




• Structured output. Need clean JSON instead of prose? There's a helper that turns a schema (plain JSON Schema or Zod) into formatting instructions, plus a validator to fix and check the result. This is how you'd build an offline "extract fields from this text" feature.




• Vision and audio. Gemma 4 and LFM2.5-VL take a capabilities array and accept an imagePath or audio buffer on sendMessage. On-device OCR into an LLM is a genuinely good offline-translation pattern.




• RAG. There's a companion @react-native-rag/executorch package that plugs this LLM (and on-device embeddings) into a vector store for fully local retrieval-augmented generation. If your "model is one component" instinct is itching, that's the package that proves the point.

Next steps

Get the basic loop running, then start making it real. A sensible order:

• Validate on real hardware first. Clone the repo, run examples/llm on an actual phone (not the simulator), and watch the first-launch download happen. The number that matters is cold-start time on a mid-range device, and it should drive your model choice more than any benchmark table.




• Study a real, shipped app. The minimal example gets you running; a production app shows you the parts the docs skip. ·



• Repo: software-mansion/react-native-executorch