For this project, our client wanted to use AI throughout the development process: he built the plan with AI assistance, designed the UI with AI, and planned to use AI for most of the features he wanted to build. This presented many challenges: creating a plan with the AI, without having the project in its context, allowed incorrect assumptions to shape the plan, ultimately causing it to drift away from the original business requirements. Our job was to validate and shape the plan against the actual state of the project, and assess whether AI was fit for every feature where it was planned to be used.

The project consisted roughly of 5 phases:

• tracking events
• updating the fingerprints
• synthetic data generation (for demos)
• updating the brand briefings
• building the briefing dashboard

Let’s dissect each one of them.

Tracking events

The platform was already tracking some events, but they weren’t exactly in the shape we needed for the project. In the past, some events were tracked as part of a proof of concept, and they were left “just in case we need them”. The first task, then, was to assess which events we needed, based on the shape of the fingerprint and the information we needed to aggregate. Once this was done, it was just a matter of tracking them and testing.

Updating the fingerprints

As mentioned above, fingerprints are a computed profile for a user, derived from their behavior and affinities. It is used to build the briefings and estimate their value within the platform. Initially, our client wanted to use AI to compute these fingerprints, but we assessed that a good old deterministic algorithm was a better choice. There are many things to consider:

1. computing thousand of users would be slow and potentially prohibitely expensive.
2. the input size depends on how many events were tracked during the period we’re observing.
3. LLMs are not always good at counting, and the fingerprint is heavily designed around counts.

Our client came up with the fingerprint specification (how to compute each field), created with the help of AI. With the event catalog at hand, plus the information he wanted to compute, the AI came up with a pretty solid specification that a priori made a lot of sense. In the next section, we’ll see it wasn’t exactly like that.

Synthetic data generation

The client also needed a way to show the project to potential clients and investors. In order to show a realistic picture of how the system would work under a realistic amount of data, our client designed several user archetypes (a high-level behavioral type of user) and variants (a sub-group inside an archetype). Then, these synthetic users would be distributed across different countries and would show different activity levels (stronger engagement signals with the platform).

They looked really good on paper, but having worked on the fingerprints, we could sense that some of the archetypes weren’t going to be feasible. After a thorough investigation, we understood what the problem with the fingerprint was: a fingerprint is created from several signals (user behavior), the AI treated them as independent, but in reality, a single event could affect multiple signals simultaneously.

Since having many archetypes and variants was key to showing meaningful user segments and actionable insights, we decided to change the way the fingerprint is computed so one event can’t affect many signals.

With this key change, synthetic data generation was possible. One of the hardest parts was over; now onto the most fun part.

Updating the brand briefings

AI was definitely right for this part of the project. The whole point of the briefings was to identify up to 10 custom user segments based on the fingerprints associated in one way or another with the brand, considering behavior, affinities, geographic localization, etc. For example, a user segment may be “Users who engaged with this brand but haven’t converted”, “Emerging fans of this brand in South America”, etc. This kind of creativity can’t be done deterministically. This is the kind of problem a human can solve by looking at an analytics tool and running different reports manually. Creatively, instinctively. If we went the deterministic way, we’d spend a lot of time building a segmentation engine that would never look right.

With 10k synthetic users, we had plenty of information to show the brands. We knew we were going to iterate on this step, as coming up with the perfect prompt is hard (at best, you can come up with a good-enough prompt).

Getting started

We needed to begin with something, so we asked GPT:

Our platform does this, a fingerprint is this, and these are all of the fingerprints associated with this brand, give me up to 10 user segments and up to 3 actionable insights for each one of the user segments.

Done. Easy, right?

The problem with this prompt is that most brands had no fewer than 5k synthetic users associated with them. This happens because associations are created on behavior (visited a brand, seen a product) and affinities (liked a brand, liked a product). And attaching 5k fingerprints would hit the ITM (Input Tokens per Minute) limit very soon. Testing the prompt in the chat agent is one thing, and testing it via the API is a different thing. The chat agent doesn’t even complain about the 5k-line CSV and responds as if it were a simple question, while the API has more usage restrictions and limits.

From prompt to agent

The prompt above is what we call a “one-shot”: we ask a question and wait until we get a response. That’s all.

The actual prompt was more complex. Instead of asking for segments directly, we asked the model to behave like a commercial analyst: research the brand’s market, compare those external signals with the brand’s internal audience summary, package the matches into cohorts, and write the commercial case for each one. From there, it had to produce tactics, call out the most urgent opportunity, write an executive summary, and return the whole thing in the structured shape the dashboard expected.

We could obviously one-shot this flow by doing the web search and computing the brand summary beforehand and sending that information as part of the prompt, but RubyLLM makes it so much easier, trivial: just create a WebSearch and a FetchFingerprintsSummary tool, and allow the agent to do a multi-turn response.

Building the briefing dashboard

The briefing dashboard, as mentioned before, was designed with AI. The design is acceptable, and the time it took to implement it was just a fraction of what it would’ve been without AI. With the design exported, it was just a matter of instructing the coding agent to implement it and just refine some rough edges. Much more customized and professional than a UI built out of components from a stock UI kit like we used to do before. This was a huge win.

Conclusion

This was one of the most fun AI projects we’ve worked on this year, mostly because AI wasn’t confined to one corner of the work. It shaped the client’s planning, the fingerprint spec, the synthetic data, the briefing design, and eventually the briefing generation itself.

That also meant we couldn’t treat “use AI” as the default answer. Part of the job was deciding where it made sense and where it didn’t. Fingerprints needed deterministic code. Synthetic data needed careful modeling. Briefings, on the other hand, needed exactly the kind of synthesis AI is good at: reading behavior, finding patterns, and turning them into commercial narratives.

On our side, the agentic development workflow made the project move much faster than it would have otherwise. Without AI, the briefing feature probably wouldn’t have been viable in this shape. Without coding agents, the rest of the project would have taken much longer to build, test, and iterate.

That was the real lesson for us: AI worked best when we were selective about it. Not as a blanket solution, but as another tool in the system: sometimes the runtime, sometimes the assistant, and sometimes the thing we deliberately chose not to use.

Vacas, Rails y Blockchain

Ignacio Cesarani presented his thesis project: a cattle trading platform built with Ruby on Rails and blockchain. The interesting part is the architecture: it uses Ethereum RSK as a sidechain to record transactions immutably, while keeping operational data in a traditional database.

Break

After the first talk, we had time to eat, drink, and chat. The sponsors put together a good spread, and the atmosphere was the classic meetup vibe: people sharing what they’re building, making connections, and discussing both old and new problems.

De Heroku a Kamal

Fernando Silva Jacquier showed how he migrated a multi-tenant Rails app from Heroku to a VPS (Virtual Private Server) with Kamal. The most interesting part of the talk wasn’t the migration itself, but how he did it. Fernando shared that he worked with a Claude Code to perform the migration. And he didn’t show only the happy path: he shared all the problems he encountered, the failed attempts, the fixes, and the real experience of migrating infrastructure with AI assistance.

This meetup was possible thanks to the organizers, the sponsors (SINAPTIA, Rootstrap, Ombulabs, and Crunchloop), and everyone who showed up. We’re already looking forward to June’s online meetup.

A client would come to us with an existing application and a list of things they needed done. Sometimes that list was mostly bugs. Sometimes it was features. Sometimes it was a mix of vague product wishes, or urgent production issues.

So we would begin with the safest tickets.

Fix a small bug. Change a minor behavior. Add a low-risk feature. Read the code around it. Ask questions. Ship. Repeat.

Starting with low-risk work lets us deliver value to the client while we build confidence and learn the codebase we’re working on.

As the weeks went by, we would move toward more complex parts of the system. By then, we had seen enough of the data model, the workflows, the weird edge cases, and the business rules to make better calls. After one or two months, we usually had enough confidence to work almost anywhere in the application.

That approach still works. But it has a limitation: it assumes the application can be explored gradually.

Sometimes it can’t.

The old approach depended on the app working

A big part of our traditional onboarding was using the application itself.

We would log in, click around, follow user journeys, compare what the client said with what the product actually did, and then connect those screens back to the code. That gave us a practical map of the system.

But recently we had a case where that was impossible.

A few months ago, Joseph came to us with an application he had spent roughly eight months building. The product featured AI agents and workflows, had a meaningful business idea behind it, and already had a lot of code written. The problem was that it barely worked.

Login was broken or unreliable. Core flows failed. The parts we needed to inspect were hidden behind errors. The usual “let’s use the app and learn from it” path was blocked almost immediately.

Fixing it meant investing time into a codebase that might later be discarded. Rebuilding it meant accepting the cost of starting over. Both options could be right. Both could be wasteful.

Our new AI-assisted onboarding process

This is where our onboarding process changed. Instead of relying only on tickets and manual exploration, we started using AI-assisted analysis to generate a first layer of documentation from the codebase.

We used AI to help us build structured documents around questions like:

• What are the main user journeys?
• Which roles exist in the platform?
• What are the core entities?
• Which parts of the application are connected to each other?
• Where does the business logic actually live?
• Which flows appear incomplete, duplicated, or fragile?
• What external services does the system depend on?
• Which architectural choices are going to make future work expensive?

With the old methodology, we reduced risk by starting with small tasks and slowly building confidence. With the new methodology, we can build a much better map before we touch production behavior, and we can get there in much less time.

What we learned from Joseph’s app

In Joseph’s case, the AI-assisted documents gave us enough context to evaluate the product without first making the app usable.

If we had started by fixing bugs one by one, we would have spent paid hours inside a codebase that might not survive the assessment. Worse, every fix would have created momentum toward keeping it. Once you invest enough time patching a system, rewriting it starts to feel like admitting waste, even when rewriting is the better option.

The architecture had enough deep problems that fixing the existing codebase would likely take about as long as rebuilding the first version properly. Our estimate was that a rebuild would take somewhere between two weeks and a month. Trying to stabilize the existing application could land in the same range, with more uncertainty and less confidence at the end.

So our recommendation was to rebuild it.

The point is reaching confidence faster

The value we provide during onboarding has not changed: we want to deliver useful work early, while we build enough confidence to take care of the system responsibly.

What changed is how quickly we can get there.

Legacy systems are full of unknowns. Some parts are safe to touch. Some are risky. Some look harmless until you understand what depends on them. The sooner we can tell the difference, the sooner we can choose the right first tickets and start producing visible results.

AI gives us an initial map before the traditional onboarding loop begins. It helps us see which areas matter, which changes are likely to be low-risk, and which parts of the system deserve more care before anyone touches them.

The confidence we used to build in one or two months can now start to appear in one or two weeks.

That is the real change. We still like earning confidence through shipped work. But now we can reach that confidence much earlier, before we decide where the first tickets should be.

Empirical software engineering: the scientific compass in the age of LLMs

The first talk was “Ingeniería de software empírica: La brújula científica en la era de los LLMs” (“Empirical software engineering: the scientific compass in the age of LLMs”) by Florencia Riva, a sociologist working at LIFIA.

Florencia’s talk pushed against a common temptation: asking LLMs for answers when what we actually need is a reproducible method. Shaped by her background in sociology, she argued that LLMs can be useful assistants, but we should not rely on them to do the actual math for us. Instead, they can help us reach deterministic methods (like code) that perform calculations and analysis in a reproducible way.

For empirical software engineering, that distinction matters. The work still depends on careful observation, measurement, and verifiable results.

A custom-made coding agent

The second talk was “Agente de código hecho a medida” (“A custom-made coding agent”) by Fernando Martínez from SINAPTIA.

Fernando presented Detritus, a deliberately tiny coding agent built in Ruby. The point was not to build the most capable assistant, but almost the opposite: to make the agent as simple as possible in order to understand what really makes these tools useful. His talk separated the problem into three parts—the model, the harness, and the workflow—and argued that when the model and harness are intentionally small, the workflow becomes much easier to see and reason about.

That experiment helped him test which practices survive across tools and models. If a workflow works with Detritus, a weak and minimal agent, it will probably work with more powerful agents too. The takeaway was less about replacing existing tools and more about gaining freedom: understanding the harness, avoiding vendor lock-in, and building a small personal lab for experimentation.

Multi-agent architectures: methodology and practice

The final talk was “Arquitecturas multiagente: metodología y práctica” (“Multi-agent architectures: methodology and practice”) by Lara González from Lumen Lab.

Lara focused on the parts of multi-agent systems that get hard quickly: deciding what each agent owns, how they coordinate, and where the handoffs can fail. She covered both methodology and practice, including how responsibilities should be split and what trade-offs appear when multiple AI components have to work together toward a goal.

Community, sponsors, and thanks

After the talks, there was time to keep talking over beers and food. These community spaces make it easier to compare notes, ask questions, and see how other people are applying AI in real projects.

Huge thanks to the AI4Devs organizers, FaHCE for hosting the event, and the sponsors who made the meetup possible: SINAPTIA, Fudo, Purrfect AI, NaNLABS, Genom IT, SEDICI, and FaHCE.

The useful part of meetups like this is not the slide deck. It’s hearing how other people are testing the same tools, where they’re skeptical, and what they’ve managed to make work.

The event kicked off with Ariel Juodziukynas’ talk “Warnings en Ruby: NO LOS IGNORES!!!”. Ariel focused on an easy part of everyday Ruby work to ignore: warnings. His talk was a reminder that the yellow text in our terminal often points to real problems. He showed practical examples of common warnings and how to fix them.

After Ariel’s talk, we took a break to catch up, meet new people, and keep the conversations going over drinks and empanadas.

The lightning talks took the night in a looser direction:

• Simon shared how he fixed a tricky performance issue with a fulltext search, walking us through his debugging process and the solution he found.
• Viktor presented aj, a tool he built to improve his AI workflow, showing how Ruby developers can build utilities to streamline their daily work.
• Santiago taught us “how to do juggling”. Yes, actual juggling with balls, not code.
• Ariel came back for a second round with “how to tie your shoes”. It was even better watching everyone try Ariel’s method on the spot.

Thanks to the organizers, the sponsors (SINAPTIA, Rootstrap, Ombulabs, and Eryx), and everyone who showed up. We’re already looking forward to April’s online meetup.

LOL, no. We are not the next tier of Luddites! We started using AI-assisted code generation a little over a year ago, and from the looks of it, we are going to use it even more in the near future. AI is here to stay, and programming will never be the same, what a time to be alive and so on, and so on…

But also, in a couple of years, we’ll all lose our jobs. What a time to be alive, indeed.

We continually ask ourselves: what’s the AI capable of? Can it really replace us? Can it change the way we work? Will it make our problems simpler? Or there will be another kind of problems, more problems? Are we attending the death of software engineering?

There’s only one way to find out.

Initially, we were somewhat skeptical about coding agents. We tried them, but the autocomplete feature was usually annoying, and the chat mode wasn’t smart enough to understand the context of the code. But then, at some point, Claude Code and Cursor became smart enough to be of massive help while building features: they could explain things easily in the context they were in, and they could edit text nicely.

Since then LLMs and coding agents became a key part of our daily routine and we started building more and more features using AI, and growing tools for the RubyLLM ecosystem (such as RubyLLM::Instrumentation, RubyLLM::Monitoring, and RubyLLM::Evals) that we use in our projects and we open-sourced for the larger community in hope to make Ruby one of the top languages for building with AI.

The problem with AI-generated code

I bet at some point you stumbled upon a piece of code that achieved something, but it was hard to understand, poorly thought out, or just ugly (eg, for us, Rubyists, seeing Ruby code that doesn’t feel Ruby). That happened a lot in the StackOverflow era.

With AI-generated code, it’s the same as with any code you didn’t write yourself. You might find AI-generated code hard to understand, poorly thought out, or even solving problems that no one asked to solve.

We’ve seen coding agents working without oversight and proper feedback going down paths that ended up in code that no one would be able to manage. Neither the coding agents nor the humans. Discard and regenerate is a possibility, yes, but tokens are not actually free, and budgeting and financials are something coding agents cannot fix either.

Models and agents will become better over time, and the barrier might be farther away each time, but the problem will always be there.

The hard parts

After more than a year of using coding agents for our daily work, developing AI-powered features, and running several experiments, we feel that AI is truly a multiplier force, but it isn’t changing the most fundamental bits of programming and software production: the hard parts of software production are still hard.

The power of solving a complex problem with a simple and elegant solution/architecture is what makes a good engineer a great one. And AI can help one do so, but is not able to come up with such architectural decisions on its own (at least not yet).

But in the areas where the AI excels, we should try to leverage it. I, honestly, cannot care less if you wrote the code we just deployed to production by punching holes in a card, taping keys in a keyboard letter by letter, copying-pasting snippets from StackOverflow, or you used a high-rate probabilistic word predictor that can produce hundreds of words per second. Tools are not at trial here. But, regardless of how that code came to life, I do care you don’t fall prey to the laziness of not curating the code you produce, of validating it and understanding it (There are many ways to do this, and they are changing, but isn’t this what engineering is about?)

Striving for quality and good architectural decisions is still central. Simplicity is still the only way software remains workable in the long run. For humans, yes, but for AI agents too.

Simple was hard in the pre-AI era and is still hard today (perhaps even harder), but what’s simpler for humans is also simpler for agents. And it is still worth all the effort.

So we say: There is no such thing as AI code. We just have code: good, bad, simple, or complex. And we have tools and processes to deal with it. Like we always did.

So, long live power tools. Long live software engineering.

At SINAPTIA, we specialize in helping businesses implement AI solutions that deliver real value. If you’re facing challenges with prompt engineering or AI integration, we’d love to help.

# db/migrate/XXXXXXXXXXXXXX_create_reports.rb
class CreateReports < ActiveRecord::Migration[7.1]
  create_table :reports do |t|
    t.string :title
    t.integer :reason, default: 0, null: false
  end
end

# app/models/report.rb
class Report < ApplicationRecord
  enum :reason, {
    spam: 0,
    harassment: 1,
    inappropriate_content: 2,
    copyright: 3,
    misinformation: 4
  }, prefix: true
end

Everything worked fine until the client requested the ability to select multiple reasons for a report. A user could report content for being spam and containing misinformation. The set of possible reasons is fixed and defined by developers.

We tried four approaches.

Four ways to solve it

• Bitwise Operations: store multiple values in a single integer using bit-level flags.
• PostgreSQL Array: use native PostgreSQL array columns with enum-like syntax.
• JSONB: store reasons as a JSON array inside a JSONB column.
• HABTM: the classic many-to-many approach with a join table.

1. Bitwise Operations

They use bit-level operations to store multiple values in a single integer. Each reason occupies a specific bit. The active_flag gem provides a clean DSL for this:

# db/migrate/XXXXXXXXXXXXXX_add_reasons_to_reports.rb
class AddReasonsToReports < ActiveRecord::Migration[7.1]
  def change
    add_column :reports, :reasons, :bigint, default: 0, null: false

    # migrate data to new format

    remove_column :reports, :reason
  end
end

# app/models/report.rb
class Report < ApplicationRecord
  flag :reasons, [:spam, :harassment, :inappropriate_content, :copyright, :misinformation]
end

# Usage
report = Report.new(reasons: [:spam, :misinformation])
report.reasons # => [:spam, :misinformation]
report.reasons.spam?   # => true
report.reasons = [:spam]
report.save!

# Read: check if spam is included
report.reasons.spam? # => true

# Validation: invalid values raise ArgumentError
report.reasons = [:invalid] # => raises ArgumentError

# Query: find all reports with spam AND misinformation
Report.where_reasons(:spam, :misinformation)

The benefits here are clear: extremely compact storage, very fast boolean checks, and no GIN index required, just a standard integer index. But the trade-offs become apparent quickly. Database values are inexpressive (what does value 4 mean?). You’re limited to 64 values with bigint. And query operations are less intuitive than standard ActiveRecord.

Note: Most Ruby gems (including active_flag) are based on integers with a 64-bit limit. While PostgreSQL supports the bit string data type, which can store many more flags without this limitation, the Ruby ecosystem doesn’t have widely adopted gems for this approach.

2. PostgreSQL Array Field (Multivalued Column)

PostgreSQL has native array support. We can store reason strings directly in a string array column without any gem:

# db/migrate/XXXXXXXXXXXXXX_add_reasons_to_reports.rb
class AddReasonsToReports < ActiveRecord::Migration[7.1]
  def change
    add_column :reports, :reasons, :string, array: true, default: []
    add_index :reports, :reasons, using: 'gin'

    # migrate data to new format

    remove_column :reports, :reason
  end
end

# app/models/report.rb
class Report < ApplicationRecord
  VALID_REASONS = %w[spam harassment inappropriate_content copyright misinformation].freeze

  validate :reasons_must_be_valid

  private

  def reasons_must_be_valid
    return if reasons.blank? || reasons.all? { |r| VALID_REASONS.include?(r) }

    errors.add(:reasons, "contain invalid values")
  end
end

# Usage
report = Report.create(title: "Report 1", reasons: ["spam", "misinformation"])

# Read: check if spam is included
report.reasons.include?("spam") # => true

# Query: find all reports with spam AND misinformation
Report.where("reasons @> ARRAY[?]::varchar[]", ["spam", "misinformation"])

The upside: no gem dependency, human-readable values in the database, and easy to expand. The downside: it’s PostgreSQL-specific, validations require manual implementation, and it’s less familiar to developers who don’t use PostgreSQL regularly.

3. JSONB (Multivalued Column)

We store reasons as a JSON array inside a JSONB column.

# db/migrate/XXXXXXXXXXXXXX_add_reasons_to_reports.rb
class AddReasonsToReports < ActiveRecord::Migration[7.1]
  def change
    add_column :reports, :reasons, :jsonb, default: [], null: false
    add_index :reports, :reasons, using: 'gin'

    # migrate data to new format

    remove_column :reports, :reason
  end
end

# app/models/report.rb
class Report < ApplicationRecord
  VALID_REASONS = %w[spam harassment inappropriate_content copyright misinformation].freeze

  validate :reasons_must_be_valid

  private

  def reasons_must_be_valid
    return if reasons.blank? || reasons.all? { |r| VALID_REASONS.include?(r) }

    errors.add(:reasons, "contain invalid values")
  end
end

# Usage
report = Report.create(title: "Report 1", reasons: ["spam", "misinformation"])

# Read: check if spam is included
report.reasons.include?("spam") # => true

# Query: find all reports with spam AND misinformation
Report.where("reasons @> ?", '["spam", "misinformation"]')

The benefit is GIN indexing for fast searches. The cost: the flexible structure requires stronger validations to enforce the expected format, it’s slightly more verbose than alternatives, and there’s JSON parsing overhead in some cases.

4. HABTM Table

The classic many-to-many approach with a join table.

# db/migrate/XXXXXXXXXXXXXX_create_reasons_and_join_table.rb
class CreateReasonsAndJoinTable < ActiveRecord::Migration[7.1]
  def change
    create_table :reasons do |t|
      t.string :name, null: false
      t.timestamps
    end
    add_index :reasons, :name, unique: true

    create_join_table :reports, :reasons do |t|
      t.index [:report_id, :reason_id]
      t.index [:reason_id, :report_id]
    end

    # migrate data to new format

    remove_column :reports, :reason
  end
end

# app/models/report.rb
class Report < ApplicationRecord
  has_and_belongs_to_many :reasons
end

# app/models/reason.rb
class Reason < ApplicationRecord
  has_and_belongs_to_many :reports

  validates :name, inclusion: { in: %w[spam harassment inappropriate_content copyright  misinformation] }
end

# Usage
report = Report.create(title: "Report 1", reasons: Reason.where(name: ["spam", "misinformation"]))

# Read: check if spam is included
report.reasons.exists?(name: "spam") # => true

# Query: find all reports with spam AND misinformation
Report.joins(:reasons)
      .where(reasons: { name: ["spam", "misinformation"] })
      .group(:id)
      .having("COUNT(DISTINCT reasons.id) = 2")

Benefits include total flexibility to add additional metadata, extreme familiarity for Rails developers, and unlimited scalability. The drawback: slower writes (create/update) and the need for more queries or joins to read data.

Comparison

Here’s a summary of how each approach compares across the attributes that matter most:

Property vs Entity

In data modeling terms, reasons are a property of a report, not an entity with its own identity and lifecycle. Nobody queries “show me all attributes of the spam reason”. That makes the multivalued column approaches (array, JSONB, bitwise) more true to the conceptual model, even though they break First Normal Form. The HABTM is the more purely relational approach, but it treats a property more as if it were an entity.

Performance

We ran benchmarks with 1000 operations for create, find, and update scenarios. You can find the full benchmark code and results in the multivalued_attributes repository.

The HABTM is ~4x slower on creates and ~3-5x slower on updates compared to the other methods. However, it’s worth noting that most performance issues related to HABTM and JOINs aren’t actually caused by the JOIN itself. Databases like PostgreSQL are highly optimized for these operations. The real culprits are usually N+1 queries, missing indexes, or poorly designed queries. With proper indexing and eager loading, the read performance gap narrows significantly. The first three approaches (bitwise, array, and jsonb) have very similar performance. On reads, the difference is marginal (only 2-3 extra seconds over 1000 queries).

Query Simplicity

One often overlooked aspect is query complexity at the call site. If your codebase filters by these values extensively, in scopes, serializers, admin interfaces, and column-based approaches, keep queries simple and on a single table. With array_enum or bitwise, you can do:

Report.with_reason(:spam)
Report.where.not(reasons: [])

With a HABTM, every call site would need a .joins(:reasons), adding complexity and increasing the risk of N+1 queries if eager loading is forgotten. While you can mitigate this with default scopes or associations, it adds friction that column-based approaches simply don’t have.

Default Values

With array or JSONB columns, you can set default values directly in the migration:

t.integer :reasons, array: true, default: ["spam", "harassment"]

New records automatically get these values. No callbacks required. With a HABTM, you’d need a callback to seed join table rows, adding an extra step that can be forgotten or misconfigured.

Database-Level Validation

With a HABTM, foreign keys enforce valid references automatically:

INSERT INTO reasons_reports (report_id, reason_id)
VALUES ('some-uuid', 999);
-- ERROR: Key (reason_id)=(999) is not present in table "reasons"

You also get cascading behavior. ON DELETE CASCADE automatically cleans up join table rows when a reason is removed.

With array or JSONB, you can use PostgreSQL CHECK constraints to enforce valid values at the database level:

ALTER TABLE reports
ADD CONSTRAINT valid_reasons
CHECK (reasons <@ ARRAY['spam', 'harassment', 'inappropriate_content', 'copyright', 'misinformation']);

UPDATE reports
SET reasons = ARRAY['spam', 'harassment', 'oops_typo'];
-- ERROR: new row for relation "reports" violates check constraint "valid_reasons"

Both approaches can enforce valid values at the database level. The difference is ergonomics: foreign keys handle this naturally, while CHECK constraints require explicit maintenance when adding new values.

Extensibility

• Bitwise: Limited to 64 values. To add more, you need to change to bigint or use bit strings.
• Array: Easy to expand, but storing many values (dozens+) becomes unwieldy. Row size grows, and GIN indexes start having performance issues.
• JSONB: Flexible, but the same problem as arrays when storing many values.
• HABTM: The most extensible. You can add additional columns (e.g., reason_details, reported_at in the join table). Also ideal for dynamic or user-defined values.

Familiarity

• Bitwise: Strange for most. Requires explaining bit-level operations.
• Array: Intuitive if you know Ruby/PostgreSQL.
• JSONB: Familiar to those who use modern REST APIs.
• HABTM: The most idiomatic. Any Rails developer immediately understands has_and_belongs_to_many.

Ecosystem

• Bitwise/Array/JSONB: Require custom form inputs, serializers, and manual validations. Some support from gems.
• HABTM: Works out-of-the-box with ActiveAdmin, RailsAdmin, nested forms, and bulk operations.

Database Compatibility

PostgreSQL arrays are PostgreSQL-specific. Bitwise and HABTM work across any database. JSON works across MySQL, SQLite, and PostgreSQL (though with different syntax and index types. MySQL uses GIN indexes similarly to PostgreSQL, while SQLite has the JSON1 extension).

Conclusion

If your value set is static and small, PostgreSQL array is the better choice over bitwise. They offer nearly identical performance without introducing unfamiliar bit-level operations that confuse most Rails developers.

But don’t dismiss HABTM. For most applications, the write overhead is irrelevant compared to the cost of maintaining “clever” code that the next developer doesn’t understand.

In our specific case, a reporting system where users can select multiple reasons, we chose HABTM. Not because it was the fastest, it wasn’t. But because it didn’t need justification. Any new developer on the team immediately understands it, and that clarity is worth more than the performance gains we’d rarely notice in practice.

The best technical choice is the one that doesn’t need a justification.

Every change on a prompt opens up a tree of choices. Which provider should we use? Which model? How detailed should the instructions be? Would more samples in the prompt work better? How much context per message? Should we use a reasoning model? Or augment the data available to the model with multi-modal input? There’s also the cost vs. accuracy tradeoff: is 10x the price worth a 5% improvement for this specific feature?

The combinatorial explosion gets overwhelming fast, and the result of the process has this feeling of uncertainty… is there a branch I missed that works better? Or that costs less?

The pragmatic choice: spreadsheets

We needed a methodology to track changes across iterations so the team can follow along. Naturally, we took a pragmatic stance: we started using spreadsheets for each feature tracking results across prompt/provider/model configurations, all run against the same data. It worked quite well, and over several features, we started seeing a workflow emerge, but…

Spreadsheets don’t scale

We knew the limits going in, but they became harder to ignore over time:

• They fragment. People make copies. When you’re sharing with non-technical collaborators, you end up with multiple sources of truth.
• No enforced structure. Each feature ended up with its own format. You have to re-learn how to read each one, and not all of them track the same metrics the same way.
• Hard to compare. Eyeballing results across configurations isn’t intuitive, and people get confused.
• No regression baseline. Once you settle on a configuration, how do you catch regressions later?
• Prompts drift. Someone edits the spreadsheet and forgets to update the code. Nobody notices until something breaks.
• Disconnected from code. Prompts and evaluations should live where the application lives.

In one project with many AI features, this all came apart. Links got lost, copies multiplied across different people’s drives with small divergences. Building eval datasets meant downloading images and re-uploading them to sheets. Running prompts required manual dev work because the data lived in Google Drive, but prompts had to go through the LLM provider. We built some internal tooling to help, but since every sheet and feature had a different format, nothing was reusable.

But they were useful to uncover what we needed: a place where you can couple a prompt configuration with a curated dataset extracted from real data, that helps you find the right balance between accuracy and costs for the feature at hand. Ideally, without leaving the Rails app.

So we built RubyLLM::Evals, a Rails engine for testing, comparing, and improving LLM prompts directly inside your application.

RubyLLM::Evals

Since we’re using RubyLLM, it made sense to build on top of it.

The core abstractions are prompts and samples. A prompt captures a full configuration: provider, model, system instructions, message template (with Liquid variables), tools, and output schemas. If you already have tools or schemas in your app, you can reuse them. Samples are your test cases: each one defines an evaluation type (exact match, contains, regex, LLM judge, or human judge) and an expected output.

The interesting design choice was making the LLM-as-judge a first-class eval type. For features like summarization or classification with fuzzy boundaries, exact matching doesn’t cut it. You need another model to assess whether the response is good enough. It’s not perfect, the judge has its own biases and failure modes, but for iterative prompt development, it’s a pragmatic tradeoff: fast feedback now, human review on the edge cases.

Each run saves a snapshot of the prompt settings and records accuracy, cost, and duration. A comparison tool lays all runs of a prompt side by side, so you can spot what changed and why.

Real application data

One thing we really wanted was the ability to populate samples from the application’s data. For example, in our image categorization feature, we can:

prompt = RubyLLM::Evals::Prompt.find_by(slug: "image-categorization")

Image.uncategorized.limit(50).each do |image|
  sample = prompt.samples.create(eval_type: :human)
  sample.files.attach(image.attachment.blob)
end

Now you’re iterating on your prompt with actual production data, not synthetic examples.

The temptation is to throw hundreds of samples at a prompt and see what sticks. In practice, a smaller curated set that covers your edge cases tells you more than a large random one. We typically start with 20-30 samples: a mix of straightforward cases, known hard cases from production, and a few adversarial examples. If accuracy looks promising, we expand. If not, the small set is faster to iterate on.

In production

Once you’re happy with a prompt, you can use it directly in your application:

response = RubyLLM::Evals::Prompt.execute(
  "image-categorization",
  files: [image.attachment.blob]
)
response.content  # => "deck"

The configuration lives in the database, versioned through your evaluation runs, always in sync with what you tested. Rolling back to a previous version or A/B testing a new iteration becomes straightforward.

Where this leaves us

Production data has a way of surprising you: new usage patterns, edge cases you never curated a sample for, a provider silently updating a model or its pricing… your prompt’s accuracy can degrade, or your cost can skyrocket without a single line of code changing. This is a challenge that has no single solution, but monitoring a prompt’s performance in production is key. Each feature will require something different and use different metrics, but you need feedback, so when your metrics surface a drift, lower quality results, or higher error cases, higher costs, you can pull new samples into RubyLLM::Evals and adjust the prompt to the new reality.

The pattern we keep seeing across projects is that prompts are never done. Models get updated, data distributions shift, and what worked last month might silently degrade and fail over time. Continuous testing and monitoring are critical.

RubyLLM::Evals and RubyLLM::Monitoring are how we go from concept to production. Both are open source and built for Rails.

At SINAPTIA, we specialize in helping businesses implement AI solutions that deliver real value. If you’re facing challenges with prompt engineering or AI integration, we’d love to help.

“The first version of the sublayer gem was actually a coding agent, but it was coming together so quickly… I was like, wait… if this is so easy for me, it’s going to be easy for everybody, and everybody is going to be making these…”

Last week, we open-sourced a minimal but feature-packed coding agent. We were after the simplest, straightforward, stupidly effective agent possible, so we named it Detritus, after Lance Constable Detritus of the Ankh-Morpork City Watch from Discworld (thanks for so much and so many, Sir Terry).

Detritus is built in just 250 lines of code, yet it packs a CLI with history, custom slash commands and skills (sort of), save/resume chats, subagents, and a two-level configuration (project and global). A full-featured coding agent.

While building this basic agent, we confirmed, firsthand, what Scott had said. And I kept wondering:

Why? What makes it so easy?

Is it the LLMs? Is it Ruby? Is it that it’s fun, so you don’t really feel the pain? Or is it something else?

After giving it some thought and talking about this with teammates, we converged on two key factors:

The first key: general availability of LLMs

General availability of LLMs changed the nature of the problem of building something like Detritus. Before, building a coding AI was unthinkable, but current LLMs made impossible things almost trivial:

Code some utility functions for the LLM to call (one for editing files, one for bash commands), hook up an LLM via API, put it all in a loop, and that’s it. You have a coding agent.

What used to be a research problem is now an integration problem. The problem migrated from the lab to the workshop.

The second key: Ruby’s power

Ruby is well known for its historical focus on developer happiness: “A programmer’s best friend”. I think this is a fundamental characteristic of the language, but sometimes I feel it’s a little superficial, and it doesn’t tell you why.

I think Ruby brings something else that is a much more fundamental property that emerges out of its design and philosophy: Power.

Originally, the idea of “powerful programming languages” came to me via Amir Rajan, creator of DragonRuby, when he shared this article from Paul Graham, “Beating the averages”. We talked about how and why Lisp was the most powerful language, with Ruby being a close second. Graham’s key insight — what he calls the “Blub paradox” — is that power in programming languages sits on a continuum, and you can only recognize a more powerful language from above, never from below.

Any general-purpose programming language is nowadays more or less equivalent, equally capable. You can build Detritus with the exact same features in Python, Go, JavaScript, or even in C. And yet, the experience of building this in Ruby feels fluid and frictionless. Like cutting a wooden block with a hand saw or a circular saw. Both will cut the wood just fine (they are equally capable), and you can probably enjoy both (personal taste is not the matter here), but one will make you feel more powerfully invested than the other.

I think power in programming languages is not just capability, but the relation between using the capability and the effort the developer has to invest in wielding it.

In this sense, Ruby has the ability to maximize the capability/effort ratio. The amount of power condensed in a few lines of code feels extraordinary.

If you take a look at Detritus’ source code, this is how you set up the agent:

def create_chat(instructions: state.instructions, tools: [EditFile, Bash, WebSearch, SubAgent], persist: true)
  chat = RubyLLM::Chat.new(model: state.model, provider: $state.provider)
  chat.with_instructions(instructions) if instructions
  chat.on_end_message { |msg| save_chat } if persist
  chat.with_tools(*tools)
end

Five lines of RubyLLM set the model, system prompt, and tools. That’s all you need to set the agentic loop ready to go.

And the rest of the code is the same: chat persistence is Marshal.dump. The CLI router is a case statement. The subagent is a tool that calls create_chat. None of this code is clever or magical; it’s just plain Ruby. That’s exactly the point. When the language is powerful enough, building an AI agent doesn’t require anything special, just the mundane. And Ruby makes the mundane exquisite, short.

Detritus’ history started when Thorsten Ball published The Emperor Has No Clothes, a guide to building a super basic coding agent in Go. My immediate thought after the head explosion was: if we did this in Ruby, it would take a fraction of the code and give us twice the features. So, as Thorsten suggested, “I went and tried how far I could get”. I got this far.

Raised to the power

LLMs’ general availability turned AI from a “research problem” into an “integration problem”. The nature of the work changed to match Ruby’s strengths: orchestration, expressiveness, and fast iteration.

When you combine Ruby with LLMs, you get compounding power. Power * Power. Power squared.

The key to building an agent is defining what to delegate to the LLM and what to handle in code. For example, Detritus’ skills feature: the code just provides a list of instructions and scripts. The actual skill, knowing when to use each one, how to combine them, that’s all the LLM.

This is where both keys meet. LLMs do the hard part; our job is orchestration. And Ruby makes the orchestration so clean you can see just how little code is actually needed. Compounding power.

The Opportunity

The Ruby AI ecosystem is young, but it’s growing fast. RubyLLM, the gem that powers Detritus, is already spawning its own ecosystem: MCP support, monitoring, agent frameworks, etc. Andrew Kane has quietly built an entire ML infrastructure layer for Ruby: transformers, torch, embeddings, vector search, and ONNX runtime. Officially supported SDKs from OpenAI, Anthropic, and MCP. The foundations are being laid right now, the Ruby way: simple, expressive, and delightful to use.

In the coming years, most of us, Ruby developers, won’t be training models. We will be orchestrating API calls, building agents, capabilities, features, and designing systems. Building products on top of a dynamic, ever-changing landscape. We’ll be doing what Ruby does best: making powerful capabilities accessible through elegant, expressive interfaces. And because of Ruby’s power, we can do those things naturally, frictionlessly, easily.

The same things that made Ruby great for web development 15 years ago are perfectly aligned again, but now with a more mature, faster, and modern Ruby. The potential is huge.

The Ruby community has decades of experience building products and delightful tools. The AI landscape is wide open, the tools are here, and the problem fits like a glove. So… what are we, Rubyists, going to do?

But how did we do it? What mechanism does RubyLLM provide that we can use to capture all LLM requests? Or did we use something else?

RubyLLM event handlers

RubyLLM provides event handlers out of the box. You can use them to capture an event when a message is sent to the LLM and, for example, calculate its cost. This is how you’d use them:

# Provided that you have gemini configured in config/initializers/ruby_llm.rb
chat = RubyLLM.chat provider: "gemini", model: "gemini-2.5-flash"

chat.on_end_message do |message|
  Event.create(
    provider: chat.model.provider,
    model: chat.model.id,
    input_tokens: message&.input_tokens || 0,
    output_tokens: message&.output_tokens || 0
  )
end

response = chat.ask("Write a short poem about Ruby")

In the code above, an event record is created when a message is completed, and the cost is calculated in an ActiveRecord callback. The solution is pretty simple and works perfectly, but it doesn’t scale very well:

• You need to add this manual tracking everywhere. Every chat instance requires this callback to be set up; otherwise you will lose that data. You can simplify it even more, but you’ll always have to set up the callback.
• Your instrumentation code and your business logic are tightly coupled, which makes both harder to maintain.
• This only works for RubyLLM::Chat instances. What about embeddings, image generation, and other operations? You’d need different mechanisms for each.
• Tracking full request metrics like latency needs more complex and intrusive code.

We needed something more comprehensive and automatic that doesn’t rely on us remembering to hook the instrumentation code everywhere. Luckily, Rails has something neat for us to use baked in.

ActiveSupport::Notifications

ActiveSupport::Notifications is Rails’ instrumentation API. It’s what Rails uses internally to track things like database queries, view rendering, controller executions, and more.

Using it is simple: you make your code emit events by calling ActiveSupport::Notifications#instrument(...), and subscribers can consume those events to do logging, monitoring, or whatever else you need. An interesting example is Prosopite, which hooks into sql.active_record events to detect N+1 queries.

This mechanism is especially important for libraries, as it decouples the business logic of the library and the business logic of the application that uses it. In the case of RubyLLM::Monitoring, the monitoring logic lives separately and subscribes to what it cares about. No coupling between RubyLLM and RubyLLM::Monitoring.

So, this is what we did in RubyLLM::Instrumentation to make RubyLLM emit events after each LLM call. RubyLLM::Monitoring, on the other hand, provides an event subscriber that captures the events and feeds them into its dashboard.

RubyLLM::Instrumentation

Instrumentation should be automatic and invisible. RubyLLM::Instrumentation achieves that: just add it to your Gemfile, run bundle install, and you’re done. RubyLLM will start emitting events for you to subscribe to.

Now, following the example above, the code becomes:

# in config/initializers/ruby_llm.rb
ActiveSupport::Notifications.subscribe(/ruby_llm/) do |event|
  # Do whatever you want with the event, in RubyLLM::Monitoring we store the event data in the database for later use
  Event.create(
    provider: event.payload[:provider],
    model: event.payload[:model],
    input_tokens: event.payload[:input_tokens] || 0,
    output_tokens: event.payload[:output_tokens] || 0
  )
end

# Provided that you have gemini configured in config/initializers/ruby_llm.rb
chat = RubyLLM.chat provider: "gemini", model: "gemini-2.5-flash"

# RubyLLM will emit the event, and it'll be captured by the subscriber above
response = chat.ask("Write a short poem about Ruby")

The code remains practically the same as in the original example, but the instrumentation becomes much simpler and decoupled, and there’s no need to repeat the same hook in multiple places.

In the example above, all ruby_llm events are captured, but you can subscribe to specific events. You can read more about the instrumented events and their payload in the project’s repository.

Wrapping up

RubyLLM::Instrumentation takes off the burden of manually instrumenting the code from the users’ shoulders. Originally written as part of RubyLLM::Monitoring, we extracted it into its own gem because we thought it was a fundamental tool, and as we needed it, other people might need it too to build a different monitoring tool, or an analytics tool, or set up logging differently.

Give it a try, send us feedback, and contribute if you want to!

If you’re building AI-powered applications with Rails and need help with architecture, optimization, or observability, get in touch.