Cutting My App's AI Cost by 90% — Build Log #5

My iOS task app has a feature that takes whatever you typed and fills it out until it's actually actionable. Write "doctor," and it splits that into what to bring, how long it takes, and the sub-steps.

That feature cost me about ¥66 per month for every paid user who maxed out their quota. Against the subscription price, that's a structure where profit evaporates the moment your users are enthusiastic. Solo project — so the shortfall is entirely mine.

The result up front: I got it to ¥6.8 (worst case), roughly a 90% cut. Here's the log, including what I threw away.

First: where was the money actually going?

The architecture is simple — app → my own server → an LLM API. I was using Anthropic's Haiku tier (small, fast, cheap).

Looking at the breakdown, the dominant line item was web search. To fill in "a place nearby" or "how long this takes" properly, you have to let the model search. The per-search charge dwarfed the raw token cost (tokens = the chunks of text an AI reads and writes; the billing unit).

So the cost problem was never "the model is expensive." It was "having the model look things up is expensive." Get that backwards and you'll swap in a cheaper model and watch the bill barely move.

Change 1: switch the model

I moved from Anthropic Haiku to Google's Gemini 3.1 Flash-Lite. The unit price is an order of magnitude apart, so this part just works.

One trap, though. I originally targeted the older gemini-2.5-flash-lite, and it wouldn't run from a freshly created Google account. The error said, in effect, that the model is no longer offered to new users. A model can be present in the docs and still be unavailable to you depending on when your account was created. Don't treat any model choice as settled until you've made a real API call.

The other quietly valuable change: making the API guarantee the response shape (constraining output with a schema). Previously I asked for JSON politely and parsed whatever string came back. Letting the API enforce the type cut the number of failed-and-retried calls — and a retry is just cost, so that's a price cut too.

Change 2: turn web search off by default

This was the big one. Search is now off by default, with an environment variable to flip it back.

With search off, the model has to answer from what it already knows. So I rewrote every prompt (the instruction text you send an AI) accordingly: instead of "look this up and give me the exact figure," they now say "give a reasonable ballpark."

Duration, typical cost, what to bring, travel estimates — none of that actually needs a live lookup. "Bring your insurance card to the doctor" requires no search. The real work was separating the features that genuinely need search from the ones that were searching out of habit.

I also set the thinking budget to zero. Modern models can deliberate internally before answering. Quality goes up, and you're billed for that deliberation as tokens. Splitting a task into sub-steps doesn't warrant deep thought, so I turned it off.

The price: restaurant suggestions went quiet

Being honest about this one. With search off, restaurant suggestions basically stopped appearing.

The reason is that the model is being responsible. If it can't verify a place exists, naming one would be inventing it — so it says nothing. That's correct behavior. A model that confidently names restaurants without searching is the dangerous one.

So "dinner tonight" style tasks now come back quiet. Meanwhile, the core — sub-task breakdown, what to bring, when to leave — didn't degrade.

That's the whole lesson: cost reduction isn't thinning every feature evenly, it's deciding which features to drop. Cutting one thing completely leaves a better product than making everything slightly worse.

I kept the option to restore search. But turning it back on would push the total above where I started, and it's mutually exclusive with the schema-enforced output, so it would need a two-call design. It's parked as "possible, but it's a real project."

Results, and how existing users got it

The best part: this needed no app update. The AI is called server-side, so the moment I deployed the server, every already-installed user was on the new setup. No review, no update prompt, no waiting.

Choosing early on to "never embed the heavy AI calls in the app, always go through my server" is what paid off here. Had the app called the API directly, every model switch would need to clear app review — and users on old versions would have stayed on the expensive path indefinitely.

If you're doing the same thing

• Look at the breakdown first. If search dominates, swapping models won't save you.
• Retries are cost. Constrain the response shape and kill parse-failure reruns.
• Turn deliberation off by default. Re-enable it only where it earns its keep.
• Decide what to drop. Thinning everything makes everything mediocre.
• Keep the AI call server-side. The freedom to switch is worth far more than it looks.

FAQ

Did quality drop? Not in the core (task breakdown, what to bring, timing estimates). What dropped is anything that assumed live search — real-world venue suggestions. That's the trade I knowingly made.

Why keep search as a toggle instead of deleting it? Because I might reverse the decision. Leaving it one setting away means I won't have to rebuild it if the revenue picture changes.

Are those cost figures measured? They're a worst-case estimate from published unit prices and the usage of someone maxing their quota. Real average users land lower. I'll publish measured averages once enough usage has accumulated.

The app in this post

Everything above is from building THE TASK: AI To-Do Prep (App Store, free), an iOS app I develop solo. The idea is that AI fills a written task out until it's actually actionable. The rest of the series lives in the build log.

Related reading

• My forced update reached nobody — Build Log #6
• The number I hid from the UI was wide open in the API — Build Log #7