I've hit this wall more times than I can count. I need to post to a creator account, verify an SMS code on a virtual phone, or stitch a mobile workflow into a backend pipeline. Then I go looking for the API and find one of three things: there isn't one, there is one but it's locked behind a partnership form, or there is one but it's missing the exact thing I need :(
For a long time, the answer was "tough luck, scrape the UI." I think that answer is finally changing, because I think now the AI agents have gotten good enough to drive a phone the same way I would. This post is about how that actually works in practice.
Why so many apps don't have APIs
Let's talk about why this gap exists. Most consumer apps—banking, social, messaging, dating don't ship public APIs because their business model wants you to use the app not the web.
APIs introduce things they don't want like scraping, abuse, churn from third-party clients, and it creates regulatory headaches. Even when an API exists (eg. Instagram Graph, WhatsApp Business, the various fintech sandboxes), it's almost always a narrow slice of what the app itself can do.
So if I wanted to automate real-world consumer mobile flows, an API usually wouldn't work. I needed a different abstraction.
My old workaround, and why it kept breaking
In the past, I used UI automation frameworks like Appium, Espresso, XCUITest, and Selenium-for-mobile. The problem was they're brittle in exactly the way that matters. I'd write a script that taps a button at specific coordinates, and when the app ships a redesign, the selector moves, and my automation dies.
I spent a lot of time fixing selectors than actually shipping something. I added retries, then sleeps, then "wait for element," then a custom DOM diff tool, and eventually realized I'd built a small, sad browser inside my test harness.
The deeper issue is that these tools require me to describe the UI structurally, but apps don't actually have stable structure (iykyk). They have intent. When I look at a banking app, I don't think tap the third button in the second tab, I think open Payouts.
What changed now?
LLMs with vision finally made it cheap to do what I do naturally: look at a screen, decide what to tap, do it, look at the result, and repeat. Combine vision with the a11y tree the OS already exposes, and you have a model that can both see the pixels and read the semantic structure, which is much more robust imo. Add a planner that breaks post this clip to an app and respond to comments into the dozen taps and swipes it actually requires, and you have a real mobile agent. That's the category Mobilerun works in.
But what is Mobilerun?
Mobilerun is an open-source framework that lets you point an LLM at a real Android or iOS device and tell it what to do.
It reads the screen via a11y plus a vision model, plans multi-step workflows, and drives the device with taps, swipes, and text input. There's a CLI for one-off tasks, a Python SDK for embedding it into my own code, and a hosted cloud if I don't want to babysit physical phones.
A few things make Mobilerun interesting if you're a developer evaluating this space:
Mobilerun is LLM-agnostic which means you can choose whichever LLM best fits your needs. The interesting part imo is that it works not only on emulators but also on real devices. That matters more than it sounds. Many apps like banking, dating, healthcare, anything with high security can sniff for emulator signals, missing SIMs, or suspicious network properties and quietly degrade.
Running on a real device with a real SIM and a real residential network means the app behaves the way it would in a user's hand. Mobilerun provides exactly that: persistent, dedicated mobile devices with eSIMs and locality, so the automation runs reliably.
Where this actually paid off
A few workflows where this approach has beaten either an API integration or traditional UI tests for me:
- End-to-end testing of my own app where flows change weekly and I'm tired of fixing selectors. I describe the test in English, and let the agent figure out the taps.
- Account workflow automation at scale: scheduling posts, replying in DMs, managing creator inboxes across dozens of accounts. Long-lived, multi-profile sessions on real devices.
- Turning a native app into a structured data pipeline. If the data lives only inside an app I don't own, an agent that reads the screen and returns JSON is a perfectly reasonable adapter and often the only one available.
- Customer support: WhatsApp Business, Telegram, SMS. I drive them from a SIM that won't get blocked the way a third-party API gateway might.
What's the conclusion?
I don't think the no API problem is going away. If anything, it's getting worse as more value moves inside walled mobile apps. What's changed is that I no longer have to choose between a brittle XPath script and giving up.
Vision capable LLMs plus a properly engineered runtime (a stable a11y tree, a real device, structured output, and a planner that recovers from failure) make UI-driven automation actually production-grade for the first time.
If you want to try it, pip install mobilerun is the fastest path. The framework is open source, and the cloud is there when you outgrow your laptop.
SOCIAL SHARE CARD GENERATOR