LiteRT is Google's on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with litert_torch.convert matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05).

LFM2.5-VL-1.6B — LiteRT-LM

LiquidAI/LFM2.5-VL-1.6B converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime — the middle of the family, between LFM2.5-VL-450M and LFM2.5-VL-3B.

Text + image work end-to-end on the released litert-lm 0.16.0 pip runtime: the bundle carries the vision encoder, the vision adapter and the LFM2 image-placeholder metadata, so litert-lm run … --attachment photo.png just works.

Known issue: positional answers are wrong on the released runtime

On litert-lm 0.16.0 — and on current main — only the top quarter of the image reaches the model. Captioning, colour questions and OCR of a large dominant subject still work. Anything positional — locate, count, enumerate, "which one is at the bottom" — comes back wrong, with no error and well-formed output.

The cause is a shrink-factor assumption in the runtime's vision path, reported upstream as LiteRT-LM#3246. It affects every LFM2.5-VL bundle, not just this one: the family performs its 2×2 pixel-unshuffle in the vision adapter rather than the encoder, and the runtime assumes the opposite.

Quick check — a 512×512 image with 16 numbered horizontal bands, asked to list every number from top to bottom:

expected: 1 … 16    actual: a runaway count that never stops at 16

The 1.6B degenerates rather than cutting off cleanly; the repaired build returns 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16.

A repaired int4 build is in this repo: LFM2.5-VL-1.6B_int4_fixB.litertlm — the stock bundle with its two vision graphs re-exported so the 2×2 pixel-unshuffle happens inside the encoder (text weights, tokenizer and metadata unchanged; scripts: reexport_vision_unshuffle.py + repack_vision.py). On the ruler above it returns 1 … 16 on litert-lm 0.16.1 and 0.17.0 (2026-09-16). It restores the visible area, not coordinate grounding — that stays a 3B capability in this family. The stock files stay published in case the runtime fix lands and makes them the right choice again.

LFM2.5-VL-1.6B pairs the hybrid LFM2 text backbone (16 layers: gated short-convolutions + grouped-query attention, hidden 2048, 64k vocab) with the same large SigLIP2 vision tower as the 3B (27 layers, hidden 1152). An image is processed at 512×512 into 256 soft tokens (single image per prompt; the runtime resizes for you) — but see the known issue above: on 0.16.0 those 256 tokens carry only the top quarter of the picture.

Load it by id on Android (hfmodels)

hfmodels is an independent, third-party Android library that takes this repository's id, downloads and verifies the .litertlm file, and opens a LiteRT-LM session; this repository ships an hfmodels.json that declares its files and profiles.

// app/build.gradle.kts (minSdk 31)
dependencies { implementation("io.github.john-rocky.hfmodels:hfmodels-litertlm:0.1.1") }

val models = HfModels(applicationContext)
val chat = models.fromPretrained(ModelRef("litert-community/LFM2.5-VL-1.6B"), Tasks.Chat)
val session = chat.createConversation(ConversationConfig(systemInstruction = Contents.of("You are a helpful assistant.")))
session.stream(Contents.of(Content.Text("What is 17 + 25? Answer briefly."))).collect { message -> append(message) }
withContext(NonCancellable) { chat.closeAndJoin() }

Verified on a Pixel 8a (Android 16, LiteRT-LM 0.16.1, 2026-09-07) on the int4 variant's gpu (language on the GPU, vision on the CPU) and cpu profiles through the library's own catalog gate (a real download from this repository with Range resume and a sha256 check, then one text turn). The image path answered an image question on both profiles; the positional defect described below applies to the runtime, not to the loader.

File	Recipe	Size
LFM2.5-VL-1.6B_int8.litertlm	int8 dynamic (text linears + convs + embedding, vision tower)	1.81 GB
LFM2.5-VL-1.6B_int4.litertlm	text int4 blockwise-32 OCTAV linears, int8 embedding + lm_head; vision tower int8	1.30 GB
LFM2.5-VL-1.6B_int4_fixB.litertlm	int4 as above, vision graphs re-exported with the pooling inside the encoder (full image visible on 0.16.x/0.17.0)	1.30 GB

Context (KV cache)	4096 max
Image input	1 per prompt, resized to 512×512 → 256 tokens; PNG/JPEG via --attachment . On 0.16.0 those tokens cover only the top quarter — see the known issue
Backend	CPU, and GPU with litert-lm ≥ 0.16.0 (macOS and Android OpenCL measured below; iOS Metal fails at engine creation for this family, tracked upstream in LiteRT-LM#3129 — use CPU on iOS)
Template	bundled — ChatML-style; image placeholders are inserted by the runtime's LFM2 data processor (non-thinking model)
Base model	LiquidAI/LFM2.5-VL-1.6B (LFM Open License v1.0)

Quality

Sanity gates on the 0.16.0 pip CLI (greedy, fresh engine per question, --cache no). Vision: five deterministic synthetic fixtures (dominant color, large-text OCR, shape, counting three squares, largest word). Text: the 8-question gate used across our LiteRT conversions.

Configuration	text 8Q	image 5Q
PyTorch bf16 (reference)	—	5/5
LiteRT int4-b32 (cpu & gpu)	8/8	3/5
LiteRT int8 (cpu & gpu)	8/8	3/5

Text is perfect across every configuration and backend. On the image side, color, large-text OCR and largest-word are answered correctly, and additional geometry probes (which corner an object is in, horizontal-vs-vertical stripes) also come back correct — but the fine-grained shape and counting fixtures miss on-device across all quantization levels including an unquantized probe build, while the PyTorch reference gets them right. We verified the conversion itself is exact (the exported vision tower and projector match PyTorch at cosine 1.0000 on identical inputs, and the prompt/token stream matches the HF processor token-for-token), so this is a runtime effect, not conversion loss. Correction (2026-08-14): we have since identified it — it is the defect described in the known issue above, and the shape and counting fixtures sit below the visible quarter. The 3B scores 5/5 on these fixtures despite being affected identically, because a larger model answers them from global context rather than from the pixels. Do not read the 3B's 5/5 as evidence that it sees more of the image.

Usage

pip install litert-lm
litert-lm run ./LFM2.5-VL-1.6B_int4.litertlm --prompt "What does the text in this image say?" --attachment photo.png

Text-only prompts work the same way without --attachment. --vision-backend cpu|gpu selects the vision encoder backend independently of the text backend.

Speed

litert-lm benchmark … --cache no, litert-lm 0.16.0 pip, Apple M4 Max (128 GB), text path (image encoding is a separate one-shot vision-encoder call at prompt time):

CPU backend:

Variant	Prefill (256)	Prefill (1024)	Decode	TTFT
int8	365 tok/s	796 tok/s	70.5 tok/s	0.72 s
int4	329 tok/s	405 tok/s	78.2 tok/s	0.79 s

GPU backend (--backend gpu; both variants verified to generate on GPU before quoting):

Variant	Prefill (256)	Decode	TTFT
int8	3601 tok/s	221.5 tok/s	0.08 s
int4	3792 tok/s	275.3 tok/s	0.07 s

On Android the same bundles run GPU-accelerated. Pixel 8a (Tensor G3), litert_lm_main built from the v0.16.0 release tag, 296-token prompt, decode run to EOS (1.5k–3.8k tokens sustained), --disable_cache:

Variant	Backend	Prefill (296 tok)	Decode	TTFT
int4	GPU (OpenCL)	201 tok/s	22.1 tok/s	1.5 s
int4	CPU	37 tok/s	13.6 tok/s	8.1 s
int8	GPU (OpenCL)	403 tok/s	15.9 tok/s	0.80 s
int8	CPU	60 tok/s	8.4 tok/s	5.0 s

The text graph delegates fully on Android OpenCL (543/543 nodes, zero rejected ops). As with the 450M, int4 prefills slower than int8 on CPU (blockwise-int4 repacking) but decodes markedly faster — pick by whether your prompts or your outputs dominate.

Galaxy S26 — GPU backend

Both published bundles run on the Android GPU backend and generate.

file	GPU backend	delegation	peak
LFM2.5-VL-1.6B_int4.litertlm	runs	6443 / 6443 ops across 12 subgraphs on LiteRT GPU	792 MB
LFM2.5-VL-1.6B_int8.litertlm	runs	6443 / 6443 ops across 12 subgraphs on LiteRT GPU	721 MB

Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) sampled during that same run. Gated 2026-08-25.

The op counts above are the LiteRT GPU partitions. XNNPACK additionally takes 1 of the 4 nodes in main; the runtime accepts that split.

The gate prompt carries no image, so this covers engine creation and the text path. The vision path on the GPU is not measured here.

No speed rows, on purpose. On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.

GPU wiring, including the Gallery import toggle: GPU guide.

Conversion

Converted with the open pipeline in hf-to-litertlm (lfm_work/convert_lfm25_vl.py, litert-torch 0.9.3 --task image_text_to_text): the exact recipe, the int4 post-processing (OCTAV int4-b32 + int8 embedder + zero-scale repair + executor metadata, all vision sections preserved) and the text/image gate harnesses are in the repo's REPRODUCE.md.