Audit Methodology
This document defines exactly what a Validash audit examines, how it is performed, and what falls outside its scope. It is incorporated by reference into our Terms of Service. The version in effect at the time of your purchase governs your audit, and your report identifies that version. Superseded versions are not published on this site: we retain each one and provide it on request to anyone whose audit it governs, at hello@validash.com. See /methodology/archive.
§ 1What an audit is
A Validash audit is an automated, point-in-time analysis of a single snapshot of your repository, performed by multiple independent large-language-model engines and synthesized into a scored report by a separate judge model. Each engine analyzes your code in isolation, with no visibility into any other engine's work. The judge never analyzes code itself; it evaluates and reconciles the engines' reports into a single findings list and written verdict. The score is not chosen by the judge: it is computed from those findings by the published rule in Section 6.
Your report identifies the exact commit (or uploaded snapshot) analyzed. Findings speak only to that snapshot as of the report date.
§ 2What each tier runs
| BASIC | PLUS | DEEP | |
|---|---|---|---|
| Independent engines | 2 | 3 | 3 |
| Reasoning depth | Standard | Standard | Extended (maximum configured reasoning budget per engine) |
| Repository size | Up to 50,000 LOC | Up to 100,000 LOC | Up to 100,000 LOC; larger by arrangement |
| Scoring rubric | Identical 0-100 scale at every tier | Identical 0-100 scale at every tier | Identical 0-100 scale at every tier |
Engine lineups may be updated over time. Every party that can receive your code is listed on the subprocessors page, and your report records the number of independent engines that produced it. Each engine analyzes your repository in isolation, with no visibility into any other engine's work, and the judge receives their reports with no marking of which engine produced which. Each engine slot is backed by a vetted understudy pool and the judge runs on a fallback chain, so most single-engine or single-provider failures are absorbed rather than fatal.
§ 3What the analysis examines
The engines receive your repository's source files and are directed to assess, at minimum: security vulnerabilities in application code, code quality and maintainability, error handling, dependency manifests (declared dependencies, not resolved trees), testing posture as visible from the submitted code, and overall architecture as expressed in the code itself.
Every finding must cite the file it refers to and quote the code it refers to, copied verbatim from that file, so you can find it in your own repository and check it against what the finding claims. Engines are directed not to state a line number: the payload they read carries no line markers, and a counted-by-eye line number would look exact while being unverifiable. Findings that engines flag with low confidence are labeled as informational or advisory in the report and are distinguished from definite findings.
Engines are also given a manifest of the paths that exist in your repository but were removed from the payload by our packager, as described in § 4. They are directed not to report the absence of those paths as a finding and not to conclude from the payload that your project lacks tests, a lockfile, CI configuration, or documentation. An audit tells you what it found; where it could not look, it says so rather than inferring.
§ 4What is excluded from analysis
The following are outside the scope of every audit tier. A Validash report makes no statement about them. Where a repository actually contains one of the excluded paths below, the engines are told which ones were removed, so an exclusion is never mistaken for a gap in your project:
node_modules, .git, dist, build, vendor, .next, out, coverage, docs, tests, __tests__, .agents, .github. Note that because test directories are excluded, reports do not evaluate the contents of your test suite, only testing posture visible from the analyzed code.package-lock.json, pnpm-lock.yaml, yarn.lock, bun.lockb) are not analyzed. Binary files and files containing null bytes are skipped. Individual files larger than 5 MB are skipped.§ 5Size and processing limits
Repository downloads are capped at 100 MB (compressed) and 50 MB (extracted). The compiled analysis payload is capped at 6 MB of text and 100,000 lines of code across analyzed files; the per-tier LOC limits in Section 2 are enforced within these bounds. Code files are counted by file extension. Submissions exceeding these limits fail before payment is consumed.
§ 6How the score is produced
Each engine produces an independent report. The judge model receives all engine reports (and only the reports, never the code), evaluates their agreement and divergence, and reconciles them into one findings list with a written verdict. Where engines disagree, the judge's synthesis notes the disagreement rather than silently averaging it away.
The judge does not choose your score. The score is computed from the findings by a fixed rule. Every audit starts at 100, and each finding whose citation resolved against the audited snapshot deducts a set number of points determined by which section it falls in and how confident it is. Security findings weigh heaviest, then architecture, then resilience. Each section can remove at most a fixed share of the 100, so no one section can sink a score by itself, and a finding we could not locate in your code deducts nothing. Your report shows this arithmetic line by line, so you can recompute your own number from your own findings.
What a tier changes, and what it does not. A tier determines how much of your codebase is examined and how deeply. It does not change the deduction table: a finding of a given section and confidence costs the same points at every tier, and no tier carries a floor or a ceiling the score cannot pass. But a deeper tier runs more engines, more engines surface more findings, and more findings deduct more. So a deeper audit of the same code will generally produce a lower number than a shallower one. That is a consequence of measuring more thoroughly, not of grading more harshly, and we would rather say so than let you discover it.
Read scores within a tier, not across them. A deep-tier 70 and a basic-tier 85 are not measurements of the same thing, and the deep number is the better-evidenced of the two.
Scores are comparative signals, not certifications. A high score is not a guarantee of security or quality, and a low score is not proof of defect; see Section 3 of the Terms of Service.
§ 7Failure and billing
Every engine slot draws from a vetted understudy pool, and the judge runs on a fallback chain, so most single-engine or single-provider failures are absorbed: a substitute steps in and the run continues. This includes safety-guardrail declines: an engine that refuses to analyze legitimately security-related code (security tooling, exploit research) is replaced by an understudy and the audit proceeds. A run halts only when an engine slot's primary and every eligible understudy fail, or when engine output cannot be validated against our report schema. Whenever a run cannot complete, the audit fails and your payment token is not consumed. There are no partial audits and no partial reports.
§ 8Versioning
This methodology is versioned. Material changes produce a new version with a new effective date; your audit is governed by the version in effect at purchase, which is identified in your report.
Superseded versions are not published on this site. We retain every superseded version in full, and where one governs an audit of yours we will send you that text on request: write to hello@validash.com.