The Practical Case for Independent Benchmarking

Our team has spent years working with AI infrastructure in production environments before forming Substrate Co. This means we understand failure modes in benchmark design — warm-up underestimation, memory locality effects, batch size sensitivity — and we build procedures that account for them rather than ignore them.

We keep current with hardware releases and software stack changes, including driver updates and framework-level optimisations that affect measured throughput. Expertise here is not a background credential; it is the reason the method notes are useful rather than decorative.

AI accelerator hardware and the software stacks that run on it move quickly. A benchmark procedure designed for last year's models and drivers may not produce meaningful results this year. Our procedures are reviewed and updated as the landscape shifts.

This is particularly relevant for the Advisory Retainer service, where the living method library is designed to stay current over a three-month engagement — rather than becoming stale documentation from a one-time study.

We keep engagement sizes manageable. When you contact Substrate Co, you are working with the people who will run the benchmarks, not a support layer that escalates internally. Scoping discussions are direct, and deliverables are reviewed with you rather than sent and forgotten.

Questions about methodology — before, during, or after an engagement — receive considered answers. We do not treat method discussion as out of scope.

All three services have published prices in Ringgit Malaysia. The scope of each engagement is defined before work begins, and we do not add deliverables without agreement. The Benchmark Setup starts at RM 620, the Comparative Study at RM 1,510, and the Advisory Retainer at RM 3,020 for three months.

There are no hidden fees. If a piece of work falls outside the agreed scope, we flag it and discuss whether to extend or defer it, rather than billing after the fact.

A benchmark result is only as useful as it is defensible. If a procurement decision is later questioned, the method note should be able to answer the question. We write our documentation with that standard in mind.

The datasets we deliver are structured for further analysis. If your team wants to slice the results differently or run their own visualisations, the underlying figures support that without additional processing on our end.