# Meritcs

> Meritcs investigates when image-quality metrics like SSIM (the Structural Similarity Index) can be trusted. The work derives closed-form mathematical boundaries where SSIM's reliability breaks down, and verifies them against tens of millions of real image blocks (e.g., 81,564,041 blocks per model, identical across all four models, used to detect the Structural Masking boundary) from independent, no-join verification databases built for four AI upscaling models. All results are published as a numbered technical report series on Zenodo (DOIs). Every report includes the exact SQL queries and hash values used; newer reports also include a screen-recorded video of the queries actually running, as reproducibility evidence.

## Core pages

- [Home](https://merit-metrics.org/): Overview of Meritcs and its mission.
- [About](https://merit-metrics.org/about.html): How Meritcs started, the team, and how results are designed to be independently verified.
- [Approach](https://merit-metrics.org/approach.html): Open-science practices - publishing mistakes alongside corrections, and CC BY-NC licensing.
- [Technology](https://merit-metrics.org/technology.html): Technical foundations - DuckDB for large-scale local analysis, the no-join verification pipeline, and the comp_* feature framework (eight reconstruction-side diagnostic features per image block, used to cross-check content agreement independently of the SSIM score).
- [What Is SSIM?](https://merit-metrics.org/ssim.html): Plain-language introduction to SSIM for readers unfamiliar with it.
- [Publications](https://merit-metrics.org/publications.html): The full, chronological technical report series with Zenodo links and SQL walkthroughs.
- [Demonstrations](https://merit-metrics.org/demonstrations.html): Screen-recorded videos of live SQL verification and pipeline tooling.
- [FAQ](https://merit-metrics.org/faq.html): Common questions, including how to reproduce results on your own data or model.
- [Documentation](https://merit-metrics.org/docs/index.html): Index of pipeline code, SQL reference, and a glossary of terms/variable names.

## Key concepts

- Closed-form boundaries: mu_c (Luminance Critical Point, approx. 0.0071) and sigma_c (Texture Critical Point, approx. 0.0212) are derived directly from SSIM's own formula as characteristic scales at which the stabilization constants become comparable to the corresponding signal-dependent terms: mu_c = sqrt(C1/2) and sigma_c = sqrt(C3). Using this project's corrected constants C1=0.0001, C2=0.0009, and C3=C2/2=0.00045 (documented in DOI 1.10), the reported values can be independently recomputed. Original derivation: [SSRN preprint](http://ssrn.com/abstract=7291266).
- Luminance Bias: below mu_c (approx. 0.0071), a fixed absolute reconstruction error produces a disproportionately larger change in SSIM's luminance term -- a direct consequence of the formula, since the stabilization constant C1's contribution comes to dominate the denominator relative to the squared mean-brightness terms. In practice, this can cause scores to change sharply; published reports find that such changes are often unrelated to perceptible quality.
- Structural Masking: whenever a ground-truth block is perfectly flat (sigma_gt=0), SSIM's structure term is exactly 1 regardless of reconstruction quality, no longer reflecting structural differences -- an exact identity derived directly from the formula. Separately, whenever ground truth and reconstruction have equal standard deviation (sigma_gt=sigma_rec), the contrast term is exactly 1, and whenever they have the same mean brightness (mu_gt=mu_rec), the luminance term is exactly 1. More loosely, below sigma_c (approx. 0.0212), the relevant terms become less discriminative and tend toward 1 as a block's local variation shrinks.
- See the [About](https://merit-metrics.org/about.html) page for the discovery narrative behind both concepts, and [What Is SSIM?](https://merit-metrics.org/ssim.html) for background on the metric itself.

## Key publications (technical report series, chronological)

Each entry links directly to its Zenodo DOI record.

- [DOI 1.0 - First report in the series](https://doi.org/10.5281/zenodo.18830286)
- [DOI 1.1 - Statistical infrastructure for the 82-million-block SSIM blind-spot database](https://doi.org/10.5281/zenodo.18883267)
- [DOI 1.2 - Structural failure theory: visible and hidden collapse across four architectures](https://doi.org/10.5281/zenodo.18917078)
- [DOI 1.3 - Axis1 bias direction is architecture-dependent, not universal](https://doi.org/10.5281/zenodo.18949243)
- [DOI 1.5 - Viewer systems and a corrected collapse-type classification](https://doi.org/10.5281/zenodo.19040901)
- [DOI 1.6 - Universal Tier3 classification coverage, with a DOI 1.0-1.6 research overview](https://doi.org/10.5281/zenodo.19058174)
- [DOI 1.7 - Axis5 collapse intensity and Tier3 state distribution by Axis1 polarity](https://doi.org/10.5281/zenodo.21425511)
- [DOI 1.8 - The comp_* feature framework: definitions and implementation](https://doi.org/10.5281/zenodo.21433447)
- [DOI 1.10 - Correction of the C1/C2/C3 constant bug (consolidates DOI 1.0/1.4/1.9/1.9.1)](https://doi.org/10.5281/zenodo.21658621)
- [DOI 1.11 - The masking blind spot theorem](https://doi.org/10.5281/zenodo.21837260)
- [DOI 1.12 - sigma_stat correction](https://doi.org/10.5281/zenodo.21894585)
- [DOI 1.12.1 - Verification SQL supplement for the SSIM=1.0 transitivity corollary](https://doi.org/10.5281/zenodo.21903024)
- [DOI 1.13 - Two empirical tests of score-vs-content agreement](https://doi.org/10.5281/zenodo.21960019)
- [DOI 1.14 - Four-model verification of the Failure Fingerprint statistics](https://doi.org/10.5281/zenodo.22012409)
- [DOI 1.15 - Testing the converse: does a feature match guarantee SSIM=1.0?](https://doi.org/10.5281/zenodo.22013446)
- [DOI 1.16 - Generalizing the masking identity beyond sigma_gt=0](https://doi.org/10.5281/zenodo.22029346)
- [DOI 1.18 - Completing the SSIM sub-term picture: the luminance term](https://doi.org/10.5281/zenodo.22230202)
- [DOI 1.19 - Four-model video evidence supplement for two discreteness claims](https://doi.org/10.5281/zenodo.22252098)
- [DOI 1.19.1 - Release of the DB-building pipeline, for reuse on any dataset](https://doi.org/10.5281/zenodo.22838600)
- [DOI 1.20 - A ground-truth-free binary classifier for Amplification Phase risk, validated across four models](https://doi.org/10.5281/zenodo.22968572)
- [DOI 1.20.1 - A ground-truth-free binary classifier for Structural Masking risk, validated across four models](https://doi.org/10.5281/zenodo.23023733)
- [DOI 1.21 - A higher-precision pipeline for measuring sigma_gt/mu_gt's distinct-value count, correcting for floating-point ghost duplication](https://doi.org/10.5281/zenodo.23271571)

See [Publications](https://merit-metrics.org/publications.html) for the complete list with descriptions, SQL walkthroughs, and superseded-file notes.

## Documentation and code

- [build_notta_tta_comp_db.py](https://merit-metrics.org/docs/build_notta_tta_comp_db.html): The script that builds the no-join verification databases used from DOI 1.12 onward.
- [build_notta_tta_comp_db.py, explained for beginners](https://merit-metrics.org/docs/build_notta_tta_comp_db_explained.html): Section-by-section walkthrough for readers new to the codebase.
- [How to read SQL](https://merit-metrics.org/docs/how_to_read_sql.html): Explains the SQL patterns used repeatedly across the published verification queries.
- [Glossary of terms](https://merit-metrics.org/docs/glossary.html): Plain-language definitions of symbols and variable names used throughout the reports and SQL.
- [SQL Reference](https://merit-metrics.org/docs/sql_reference.html): Index of every verification SQL file published alongside the technical reports.
- [Catching SSIM's Blind Spots Without Ground Truth](https://merit-metrics.org/docs/gt_free_diagnostic_classifiers.html): Plain-language walkthrough of DOI 1.20 and DOI 1.20.1 for readers without a technical background.
- [8-Bit Quantization and a Floating-Point Trap](https://merit-metrics.org/docs/eight_bit_quantization_explained.html): Plain-language explanation of DOI 1.16 and DOI 1.21 for readers without a technical background -- a structural property of any model trained on 8-bit images, not specific to this project.

## Notes for automated readers

- All technical claims are backed by SQL queries executed against source databases; the databases themselves are not publicly released due to size, but the schema, pipeline scripts, and queries are.
- The technical report series is published under the pen name "mohumohu neco"; this is disclosed on the About page.
- This site does not reference any in-progress or under-review journal submissions - only the published DOI technical report series above.
- A Japanese-language version of this entire site is available at https://merit-metrics.org/ja/, with its own AI-oriented index at https://merit-metrics.org/ja/llms.txt.