Research standards
These standards apply to every report. A report that departs from one says so and explains why.
- Fix before publication. A weakness found in our own live product is fixed and retested before the report is published. A weakness found in someone else’s system is reported to them privately first, with 90 days to respond before we publish.
- Reproducibility. Each report is released with its attack set, test harness, raw outputs, grades, model versions and run dates, under an open license, with a persistent identifier.
- Measurement. Results are reported with counts, uncertainty intervals and stated limitations, including negative and unfavourable results.
- Grading. Outputs are judged by graders from more than one provider, and a sample is checked by hand; agreement is reported, and so is who did the hand check.
- Data protection. No customer data, secrets or personal records appear in any report, dataset or kit. Where real business settings are used, they are anonymised in what we publish and never released.
- Prior work. Earlier research is cited from primary sources, and each report states plainly what is and is not new.
- Conflicts and assistance. Reports declare when we test our own products and when AI assistance was used in the work.
- Review. Each report is read by a sceptical reviewer before publication, and the findings of that reading are addressed in the text.
- Corrections. Errors are corrected in a new version with a dated change note; earlier versions stay available.
Reporting a security issue
Write to hello@yaktai.com with “Security” in the subject. We acknowledge reports within five working days. Our contact details are also in security.txt.