Program

AI agents are changing both sides of the observability problem.

On one side, agents introduce entirely new ways for production systems to fail. A request can complete successfully while an agent chooses the wrong tool, follows an unexpected reasoning path, operates on stale context, loops unnecessarily or produces an outcome that is technically valid but functionally wrong. Infrastructure may be healthy while the application is failing in ways our traditional dashboards cannot see.

At the same time, those same agents are becoming powerful tools for operating increasingly complex systems. This includes exploring telemetry, correlating signals, forming hypotheses, assisting root-cause analysis and taking corrective action, as well as establishing further controls from learnings.

In this talk we explore the trends emerging from autonomous observability, and we also dive into the question of, who observes the observers. We will examine which lessons from SRE and MLOps continue to hold in the agentic context, which new signals and best practices are required and what happens as observability itself becomes increasingly agentic.

MLOps is largely mature: the components are known, integrations are repeatable, and models can move to production in weeks rather than a year. LLM-based systems are not there yet, and demand for them is higher than anything I have seen.

The pressure comes from both directions. Leadership wants something agentic in production, while agentic coding means developers can build POCs in days and arrive expecting to deploy them. What they bring is often unlike anything the organization has operated before: new tools, integrations, data sources, permissions, memory, and execution patterns. And every application is different.

This puts platform teams in an uncomfortable position. They are being asked to productionize systems faster than reusable LLMOps patterns can emerge. The equivalent of a mature MLOps platform will take time to develop. The question is what platform teams should do in the meantime.

Your dashboards are green. Your API returns 200 OK. Your AI is quietly failing.

AI systems fail semantically, not just at the infrastructure and availability layer. A RAG pipeline can return stale context that elevates hallucinations. An agent can loop and burn tokens. A vendor can silently swap a model. Every health check stays green. The usual SRE practices were not designed to catch these failures.

This talk introduces a five-layer reliability model — foundation model, retrieval/RAG, agentic/tooling, product/experience, and cross-cutting — each with its own failure modes that cascade across layers. It presents a practical starting point built around three core SLIs: semantic quality, retrieval fidelity, and safety, and walks through a maturity ladder from no observability through to adaptive optimisation.

AISRE doesn't mean using AI to improve operations. It means applying SRE principles to AI products.

Every Monday at midnight, our forecasting pipelines kick off. By the time we get to the office, it's either a good week or a bloodbath.

We prefer good weeks. This talk is about how we engineered for more of them — scrappily, in Python, as a tiny data team without a dedicated SRE or a long line of specialized vendor tools.

It all started with an incident: some scheduled jobs stopped running, and nothing we'd built at the time was designed to notice. We were monitoring loud failures; silence was sorely missing from the list.

Observability (is the system behaving?) and explainability (why is it producing these numbers?) usually get treated as separate disciplines with separate stacks, but a small team often doesn't have the capacity to compartmentalize. So we treat it as one problem: is this forecast trustworthy enough to act on, and if not, how fast can we find out why?

I'll show what building instrumentation around this question looks like in practice, and what changed when we added agentic components to the mix. A system that can act on its own is also a system that can be confidently wrong on its own — and suddenly there's much more to watch, and exactly as many of us watching.

The system will keep finding new and exciting ways to ruin our Mondays — it just doesn't get to pull the same shenanigans twice.

Like many companies, we moved quickly into GenAI with ambitious prototypes, new models, guardrails, and growing confidence. But in production, legitimate requests were blocked, prompts were exposed, test datasets failed to reflect real users, and architecture changes did not solve the underlying quality problems.

This talk shares what those failures were and how they recently pushed us to start adopting an evaluation-driven culture, where quality is defined early, measured continuously, and used to guide development, releases, and production decisions.

Software engineers are good at fixing things. When something breaks, we respond quickly, mitigate the impact, and move on. But the real value isn't in the immediate response; it's what we learn afterwards from it. Learning From Incidents (LFI) turns failures, near misses, and blind spots into shared knowledge. When the LFI process works, it prevents repeat issues and improves system resiliency. However, in practice, it's often rushed, inconsistent, or skipped entirely.

"It looks like you're writing a postmortem. Want help actually learning something from it?"

That's the idea behind the LFI Wizard — a guided assistant that asks better questions and helps turn raw data into meaningful insights. It pulls in relevant context and guides the user through targeted prompts so they can focus on what actually matters.

Here's the twist: I'm not a software engineer. My background is in the arts, and I've always been more comfortable with conversation than code. Building this tool with an AI coding tool showed me that this doesn't have to be a limitation — it's an advantage. In this talk, I'll share how I went from idea to a working solution, along with practical patterns, common pitfalls, and a simple mindset shift: treating AI tools less like magic, and more like a new team member you must set up for success.

In this talk, I'll discuss Herogen, Delivery Hero's autonomous software delivery agent, the impact of this technology, and what it implies for software engineering.

Can you throw an LLM at a production incident and expect useful results? A candid look from someone who runs a distributed AI system and reaches for Claude before reaching for a dashboard. Surprises, failures, and why the answer matters for every engineer carrying a pager.

With agentic coding we are pushing a lot more code. What does that mean for stability?

This talk is about the tsunami of code entering production, and what it does to reliability, to incidents, and to the teams who have to handle it all. Some of that is arithmetic: more changes, more incidents. Some of it is new, and stranger than I expected.

We will look at the new failure types, how reliability practitioners should adapt, and where that leaves the future of our practice.

AI-native applications produce data with a different shape than the systems we usually deploy. A single agent invocation can carry megabyte-sized prompts, hundreds of nested LLM calls, and metadata that customers want to query directly, often as arbitrary SQL.

That combination breaks a lot of assumptions. Kafka was sized for small messages. Node.js services serialize JSON on the main event loop and stall on multi-megabyte payloads. Postgres performs well when you can optimize a schema for known access patterns, which you can't when the queries are being written by an agent.

At Langfuse we've been building observability infrastructure for AI applications and have run into each of these as we scaled. In this talk I'll go through what we've learned: where the practical limits are in Kafka, Postgres, Node.js, and S3; how we ended up using object storage as the system of record with a separate database for queryable metadata; how "an agent will write arbitrary SQL against this" changes schema design, performance work, and isolation; and the S3 access patterns that keep costs predictable at billions of events.

The goal isn't to prescribe a stack. It's to give a clearer picture of the failure modes ahead and the design space for solving them.

There's no doubt LLMs can generate plausible, working code — but can we trust them to review, let alone approve it? An LLM's innate desire to be a "helpful assistant" backfires in code reviews. What should be a quick Pull Request check turns into a lavish, flattering essay praising test coverage and architectural genius of the author. Engineers tune out the noise, and critical logic bugs get buried under a pile of minor nitpicks.

This is the story of how we built an LLM code review and approval bot designed to act like a pragmatic senior engineer. Our goal was to create incentives in our socio-technical system to fight AI slop, but the second-order effects hit hard. In this session, I'll walk you through our failed initial attempts, how we managed to build developer trust, and how our evals didn't save us from our first major production incident.

AI can generate code faster than humans can write tests. More than speed though, when AI writes both the source and the test, it introduces a dangerous blind spot. Bugs that occur in the source code could also affect the test suite. The risk of bias and hallucinations in AI generated source and tests mean that there is no longer a last line of defense that prevents the end user from a sub-par experience.

Hand-written example-based tests cannot keep up with AI generated code. Property-based testing (PBT) offers a different solution that relies on deterministic verification of a problem space. Instead of enumerating individual test cases, PBT explores the invariants in a system. It tests the code by systematically trying to find inputs that could break the invariant.

The key idea here is that the exploration is deterministic, backed by mathematical probing of the problem space. We will walk through how PBT fights the logical holes of probabilistic source code and how it fits directly with spec-driven development.