Building RAG on Android
Why local retrieval-augmented generation is hard on a tablet, and how the pipeline is actually put together.
Android Systems & Edge AI Engineer
I'm Shubham Singh Bisht, an Android engineer based in Bengaluru. Currently building high-performance system software and interactive displays at Teachmint (ex-Pratilipi), with a focus on runtime internals, low-latency graphics, and on-device AI.
A few things I've built, investigated, and learned from.
Why local retrieval-augmented generation is hard on a tablet, and how the pipeline is actually put together.
Dispatchers.IO is not a pool of IO threads. What the CoroutineScheduler really does, and the mental model that replaced mine.
If the thread is released while a request is in flight, what is actually waiting for the response? Coroutines, threads, sockets, and continuations, separated.