Study status (2026-08-16): Methodology published. The first-phase study is not yet scheduled; the gating dependency is recruiting an independent participant cohort. We are targeting a first-phase result in the first half of 2027. This line is updated as the study moves; if the date slips or the scope narrows, it will say so here.

A claim is not evidence, and we know it

We have made a specific claim across this site and our manifesto: that rendering a system as the navigable topology it actually is, rather than flattening it onto dashboard panels, helps an engineer find a problem faster. We believe it. Belief is not evidence, and a beautifully argued thesis is still just a thesis.

The honest thing to do is measure it against the tools you already use, and publish the result whether or not it flatters us. We are not there yet; the study takes real incidents and real engineers, and rigging the conditions would make any number we produced worthless. So before the numbers, here is the thing we can publish today and be held to: exactly how we intend to be measured.

If the method below is rigged, tell us, and we will fix the method. That is the point of publishing it first.

What "faster" has to mean

"Faster" is the most abused word in observability marketing, because it is almost never defined. We are defining it narrowly and testably.

The metric is time to locate the responsible element: from the moment an engineer is told "something is wrong" to the moment they correctly identify the specific service, dependency, or absent call responsible. Not time to acknowledge an alert. Not time to full resolution, which depends on the fix and is outside any observability tool's control. The one thing a diagnostic tool can legitimately claim to change is how fast you find the thing that is wrong. That is what we measure.

We separate this into the case we think is ordinary (a service is slow or erroring, which flat tools handle well) and the case we think is hard (an "absence" incident, the call that never returned, the dependency that should be in the picture and is not). We expect the flat tools to be competitive on the first and we expect to be tested hardest on the second. We will report both, separately, so the result cannot hide behind an average.

The test design

• Tasks: a fixed set of seeded scenarios in three classes: ordinary-degradation cases, absence cases, and benign-change cases where nothing is actually wrong, a deploy or an autoscaling event that changes the shape of the system without breaking it. The third class exists so a tool can be caught inventing a fault, not only missing one: on those runs the correct answer is "nothing is wrong", and reporting a phantom is a wrong answer. All are generated reproducibly so every tool sees the identical system state. We use Snowglobe, our open-source telemetry generator, and we will publish the exact scenario configuration files alongside the results so the specific experiment is reproducible, not just the generator.
• Tools compared: DeepCube against the flat tools engineers actually use day to day, starting with Datadog and at least one other widely used platform confirmed before the study runs. We name the full set in the published study and we do not compare against strawmen.
• Participants: practicing engineers, not our own staff, sourced from the OpenTelemetry community and a screened volunteer pool rather than our own followers, so the sample is not skewed toward people already sold on the idea. We will state the final count, and we will not publish a first phase with fewer than a dozen engineers per tool, because below that a timing difference is a story rather than a finding. Within the study, each participant is randomly assigned which tool they use for which scenario, so familiarity does not silently decide the outcome. We also record each participant's first exposure to each tool and report that cohort separately, because randomizing learning effects out of the headline number is not the same as pretending they do not exist. If first contact with DeepCube is slower than an engineer's hundredth hour on their own dashboard, that number gets published too.
• Measure: time to correctly locate the responsible element, plus whether they located it at all within the window. A fast wrong answer is not a win, so correctness gates the timing. Ground truth for each scenario is fixed in the published configuration files before any participant is assigned, so the vendor does not grade its own exam. On benign-change runs we report the false-positive rate: how often each tool led an engineer to name a culprit when there was none.
• Controls: the same telemetry feeds every tool (it is all OpenTelemetry, which is the whole point of being OTel-native), the same incidents, randomized order to wash out learning effects.
• Scope and honesty about cost: this is a real study, not a blog graphic, and a credible version of it takes independent coordination and time. We would rather run a smaller first phase well, with a defined bar for proceeding, than rush a big number we cannot defend. The bar is simple and stated in advance: a clear, repeatable directional difference on the absence-case condition. No directional signal there, and we do not scale the study up, we go back and ask why.

How this could come back against us

A benchmark you cannot lose is not a benchmark. Here are the outcomes we are exposing ourselves to:

• The flat tools may match us on ordinary degradation. We expect that. If we cannot show a meaningful difference on the absence cases, the central claim weakens, and we will say so.
• Spatial navigation may carry a learning cost that shows up as slower first-use times. If a first-time DeepCube user is slower than an expert on their familiar dashboard, that is a real finding about onboarding, and the first-exposure cohort above is how it gets reported rather than quietly screened out.
• The expectation model that renders absence is designed to tell a genuinely missing call apart from a normal change like a deploy or an autoscaling event, and that is one of the hardest parts of the problem. If it raises phantoms during ordinary change, the benign-change class is there to catch it, and that false-positive rate belongs in the result too.

We would rather publish a mixed result honestly than a clean one we cannot defend.

Where we draw the line on the category

An honest benchmark also means an honest claim about novelty. Service maps, topology views, and dependency graphs are not new; Datadog, Honeycomb, and others have had visual topology for years, and we are not going to pretend otherwise. The line we are testing is narrower and specific: whether rendering a system spatially, and rendering the absence of expected behavior as a visible divergence you can navigate to, changes how fast a human locates the responsible element. That is the claim on trial. If a flat topology view gets you there just as fast, the benchmark will say so.

What you can hold us to

This is the commitment: a published study, named incumbents, independent participants, a public reproducible scenario set, both incident classes reported separately, and the failure modes above reported if they occur. When the numbers exist, they will appear here with the full method attached, not as a headline percentage with the method hidden.

Timeline and status

We are targeting a first-phase result in the first half of 2027. We are naming the thing that gates it rather than just naming a date: the study needs an independent participant cohort, and recruiting practicing engineers who do not already work for us or follow us is the slow part. A methodology published in public is only honest if its status is too, so we will keep a dated status line at the top of this post and update it as the study moves, including if we have to push the date or narrow the scope. If you come back and the date has slipped with no note, that is on us, and you should hold us to it. The whole point of publishing the method before the numbers is that we cannot quietly let it lapse.

Until then, you have the right thing to be skeptical of: not "trust us, data coming," but a specific test you can critique today and check us against tomorrow. If you run production systems and would consider taking part, email us at info@immersivefusion.com with "benchmark cohort" in the subject and we will follow up about screening and scheduling. If you think the method is wrong, the same address works for that, and that mail is more useful to us than the other kind.

Try the scenarios yourself

The incident scenarios are generated by Snowglobe, our open-source telemetry generator, so you do not have to wait for our study to start forming your own opinion. It is a single binary and it is not gated behind a signup. Generate an absence incident with snowglobe -no-consumers (producers fire, consumers never pick up, so the trace ends where the missing call should have been), or snowglobe -errors 5 for probabilistic lost messages. Send the identical trace to the flat tool you use now, then send it to DeepCube, and judge the "time to locate" difference for yourself on your own terms.