One of our all-time clients owns a multi-brand marketplace. Earlier this year, he had an idea to revolutionize his platform by observing how his users behave and turning that information into actionable insights for his customers (the brands). The core idea was to track user events, turn each user’s events into a single, computed profile that identifies them (internally called a fingerprint), and use this information to build a brand brief, including user segments and actionable insights.
For this project, our client wanted to use AI throughout the development process: he built the plan with AI assistance, designed the UI with AI, and planned to use AI for most of the features he wanted to build. This presented many challenges: creating a plan with the AI, without having the project in its context, allowed incorrect assumptions to shape the plan, ultimately causing it to drift away from the original business requirements. Our job was to validate and shape the plan against the actual state of the project, and assess whether AI was fit for every feature where it was planned to be used.
The project consisted roughly of 5 phases:
- tracking events
- updating the fingerprints
- synthetic data generation (for demos)
- updating the brand briefings
- building the briefing dashboard
Let’s dissect each one of them.
Tracking events
The platform was already tracking some events, but they weren’t exactly in the shape we needed for the project. In the past, some events were tracked as part of a proof of concept, and they were left “just in case we need them”. The first task, then, was to assess which events we needed, based on the shape of the fingerprint and the information we needed to aggregate. Once this was done, it was just a matter of tracking them and testing.
Updating the fingerprints
As mentioned above, fingerprints are a computed profile for a user, derived from their behavior and affinities. It is used to build the briefings and estimate their value within the platform. Initially, our client wanted to use AI to compute these fingerprints, but we assessed that a good old deterministic algorithm was a better choice. There are many things to consider:
- computing thousand of users would be slow and potentially prohibitely expensive.
- the input size depends on how many events were tracked during the period we’re observing.
- LLMs are not always good at counting, and the fingerprint is heavily designed around counts.
Our client came up with the fingerprint specification (how to compute each field), created with the help of AI. With the event catalog at hand, plus the information he wanted to compute, the AI came up with a pretty solid specification that a priori made a lot of sense. In the next section, we’ll see it wasn’t exactly like that.
Synthetic data generation
The client also needed a way to show the project to potential clients and investors. In order to show a realistic picture of how the system would work under a realistic amount of data, our client designed several user archetypes (a high-level behavioral type of user) and variants (a sub-group inside an archetype). Then, these synthetic users would be distributed across different countries and would show different activity levels (stronger engagement signals with the platform).
They looked really good on paper, but having worked on the fingerprints, we could sense that some of the archetypes weren’t going to be feasible. After a thorough investigation, we understood what the problem with the fingerprint was: a fingerprint is created from several signals (user behavior), the AI treated them as independent, but in reality, a single event could affect multiple signals simultaneously.
Since having many archetypes and variants was key to showing meaningful user segments and actionable insights, we decided to change the way the fingerprint is computed so one event can’t affect many signals.
With this key change, synthetic data generation was possible. One of the hardest parts was over; now onto the most fun part.
Updating the brand briefings
AI was definitely right for this part of the project. The whole point of the briefings was to identify up to 10 custom user segments based on the fingerprints associated in one way or another with the brand, considering behavior, affinities, geographic localization, etc. For example, a user segment may be “Users who engaged with this brand but haven’t converted”, “Emerging fans of this brand in South America”, etc. This kind of creativity can’t be done deterministically. This is the kind of problem a human can solve by looking at an analytics tool and running different reports manually. Creatively, instinctively. If we went the deterministic way, we’d spend a lot of time building a segmentation engine that would never look right.
With 10k synthetic users, we had plenty of information to show the brands. We knew we were going to iterate on this step, as coming up with the perfect prompt is hard (at best, you can come up with a good-enough prompt).
Getting started
We needed to begin with something, so we asked GPT:
Our platform does this, a fingerprint is this, and these are all of the fingerprints associated with this brand, give me up to 10 user segments and up to 3 actionable insights for each one of the user segments.
Done. Easy, right?
The problem with this prompt is that most brands had no fewer than 5k synthetic users associated with them. This happens because associations are created on behavior (visited a brand, seen a product) and affinities (liked a brand, liked a product). And attaching 5k fingerprints would hit the ITM (Input Tokens per Minute) limit very soon. Testing the prompt in the chat agent is one thing, and testing it via the API is a different thing. The chat agent doesn’t even complain about the 5k-line CSV and responds as if it were a simple question, while the API has more usage restrictions and limits.
From prompt to agent
The prompt above is what we call a “one-shot”: we ask a question and wait until we get a response. That’s all.
The actual prompt was more complex. Instead of asking for segments directly, we asked the model to behave like a commercial analyst: research the brand’s market, compare those external signals with the brand’s internal audience summary, package the matches into cohorts, and write the commercial case for each one. From there, it had to produce tactics, call out the most urgent opportunity, write an executive summary, and return the whole thing in the structured shape the dashboard expected.
We could obviously one-shot this flow by doing the web search and computing the brand summary beforehand and sending that information as part of the prompt, but RubyLLM makes it so much easier, trivial: just create a WebSearch and a FetchFingerprintsSummary tool, and allow the agent to do a multi-turn response.
Building the briefing dashboard
The briefing dashboard, as mentioned before, was designed with AI. The design is acceptable, and the time it took to implement it was just a fraction of what it would’ve been without AI. With the design exported, it was just a matter of instructing the coding agent to implement it and just refine some rough edges. Much more customized and professional than a UI built out of components from a stock UI kit like we used to do before. This was a huge win.
Conclusion
This was one of the most fun AI projects we’ve worked on this year, mostly because AI wasn’t confined to one corner of the work. It shaped the client’s planning, the fingerprint spec, the synthetic data, the briefing design, and eventually the briefing generation itself.
That also meant we couldn’t treat “use AI” as the default answer. Part of the job was deciding where it made sense and where it didn’t. Fingerprints needed deterministic code. Synthetic data needed careful modeling. Briefings, on the other hand, needed exactly the kind of synthesis AI is good at: reading behavior, finding patterns, and turning them into commercial narratives.
On our side, the agentic development workflow made the project move much faster than it would have otherwise. Without AI, the briefing feature probably wouldn’t have been viable in this shape. Without coding agents, the rest of the project would have taken much longer to build, test, and iterate.
That was the real lesson for us: AI worked best when we were selective about it. Not as a blanket solution, but as another tool in the system: sometimes the runtime, sometimes the assistant, and sometimes the thing we deliberately chose not to use.