DOI 1.20.1 — A ground-truth-free binary classifier for Structural Masking risk

View DOI 1.20.1 on Zenodo: https://doi.org/10.5281/zenodo.23023733

New DBBuilt on the current, join-free pipeline (build_notta_tta_comp_db.py), one independent database per model.

comp_contrast, one of this series' eight independently-measured quality features (Technical Report v1.8), turns out to be RMS contrast computed on the raw 8-bit pixel scale rather than the [0,1]-normalized scale used for σ_gt/σ_notta throughout this series — dividing by 255 recovers σ_gt from comp_contrast_gt exactly, with zero error across all four models' full σ_gt≤0.16 populations (81,564,041 blocks each). By the same construction, comp_contrast_notta/255 is an exact, algebra-free measurement of σ_notta — not an approximation, unlike an earlier candidate approach (a quadratic-equation recovery with an unresolvable root-selection ambiguity, the same kind of method used for μ_notta in DOI 1.20).

The classifier — `verification_sql_final_4model_comp.sql`

The identical three-query sequence below (sanity check, correlation, confusion matrix) repeats for all four models, only the σ_stat constant and database path change (shown once here for Upscayl Standard):

Screen recording: verification_sql_final_4model_comp.mp4 (all four models, one continuous session).

Sanity check — is `comp_contrast/255` really exact?

Before using it as a classifier input, the identity itself is checked directly. The domain CTE rescales comp_contrast_gt onto the same [0,1] scale σ_gt already uses (dividing by 255), restricted to the σ_gt≤0.16 population this whole report works in. The outer SELECT then compares the two directly: avg_abs_diff_rescaled and max_abs_diff_rescaled measure how far apart they are on average and in the worst single block, and n_meaningfully_different counts blocks where that gap exceeds a generous 0.0001 tolerance. All three come back at exactly zero, for every model, with no exceptions.

.open --readonly output_std/std_notta_tta_comp.duckdb

SELECT COUNT(*) AS n_total_blocks_in_db FROM blocks;

-- Sanity check: comp_contrast_gt/255 == sigma_gt exactly?
WITH domain AS (
    SELECT sigma_gt, comp_contrast_gt / 255.0 AS comp_contrast_gt_rescaled
    FROM blocks
    WHERE sigma_gt <= 0.16
)
SELECT
    COUNT(*) AS n_total,
    ROUND(AVG(ABS(sigma_gt - comp_contrast_gt_rescaled)), 8) AS avg_abs_diff_rescaled,
    ROUND(MAX(ABS(sigma_gt - comp_contrast_gt_rescaled)), 8) AS max_abs_diff_rescaled,
    COUNT(*) FILTER (WHERE ABS(sigma_gt - comp_contrast_gt_rescaled) > 0.0001) AS n_meaningfully_different
FROM domain;

Table 1 — correlation

sigma_notta_est is comp_contrast_notta, rescaled the same way — the model's own reconstructed contrast, measured exactly rather than estimated. CORR() is computed three times against σ_gt: once over the whole domain, and once each restricted, via FILTER, to blocks on either side of σ_stat — to see whether the model's output tracks the ground truth differently inside the masking-risk region than outside it.

-- Table 1: correlation check
WITH domain AS (
    SELECT sigma_gt, comp_contrast_notta / 255.0 AS sigma_notta_est
    FROM blocks
    WHERE sigma_gt <= 0.16
)
SELECT
    COUNT(*) AS n_total,
    ROUND(CORR(sigma_gt, sigma_notta_est), 4) AS corr_overall,
    ROUND(CORR(sigma_gt, sigma_notta_est) FILTER (WHERE sigma_gt <= 0.1010), 4) AS corr_masking_region,
    ROUND(CORR(sigma_gt, sigma_notta_est) FILTER (WHERE sigma_gt > 0.1010), 4) AS corr_non_masking
FROM domain;

Table 2 — binary classifier confusion matrix

The labeled CTE reduces both sides to a single boolean each: true_masking_region asks whether the ground truth actually falls at or below σ_stat, and predicted_masking_region asks the same question of the model's own reconstruction (via sigma_notta_est). Comparing the two, block by block, gives the four confusion-matrix counts (true_positive, false_negative, false_positive, true_negative), from which accuracy_pct, recall_masking_pct, and recall_non_masking_pct are each computed as the usual percentage of the relevant denominator.

-- Table 2: binary classifier confusion matrix
WITH domain AS (
    SELECT sigma_gt, comp_contrast_notta / 255.0 AS sigma_notta_est
    FROM blocks
    WHERE sigma_gt <= 0.16
),
labeled AS (
    SELECT
        (sigma_gt <= 0.1010) AS true_masking_region,
        (sigma_notta_est <= 0.1010) AS predicted_masking_region
    FROM domain
)
SELECT
    COUNT(*) AS n_total,
    COUNT(*) FILTER (WHERE true_masking_region) AS n_true_masking,
    COUNT(*) FILTER (WHERE NOT true_masking_region) AS n_true_non_masking,
    COUNT(*) FILTER (WHERE true_masking_region AND predicted_masking_region) AS true_positive,
    COUNT(*) FILTER (WHERE true_masking_region AND NOT predicted_masking_region) AS false_negative,
    COUNT(*) FILTER (WHERE NOT true_masking_region AND predicted_masking_region) AS false_positive,
    COUNT(*) FILTER (WHERE NOT true_masking_region AND NOT predicted_masking_region) AS true_negative,
    ROUND(COUNT(*) FILTER (WHERE true_masking_region = predicted_masking_region) * 100.0 / COUNT(*), 4) AS accuracy_pct,
    ROUND(COUNT(*) FILTER (WHERE true_masking_region AND predicted_masking_region) * 100.0
        / NULLIF(COUNT(*) FILTER (WHERE true_masking_region), 0), 4) AS recall_masking_pct,
    ROUND(COUNT(*) FILTER (WHERE NOT true_masking_region AND NOT predicted_masking_region) * 100.0
        / NULLIF(COUNT(*) FILTER (WHERE NOT true_masking_region), 0), 4) AS recall_non_masking_pct
FROM labeled;

-- Repeats identically for SwinIR (sigma_stat=0.0800), High-Fidelity
-- (sigma_stat=0.0860), and Digital-Art (sigma_stat=0.0750), each
-- preceded by its own .open line.

Results (Tables 1–2)

All four models: n_total_blocks_in_db = 81,985,536 (unfiltered, identical across models); sanity-check population — and every other query below — is restricted to σ_gt≤0.16, n=81,564,041 per model (about 326 million blocks combined). The sanity check returns avg_abs_diff_rescaled=0.0, max_abs_diff_rescaled=0.0, n_meaningfully_different=0 for all four models, with no exceptions.

Upscayl Standard (σ_stat=0.1010): correlation 0.9251 overall (0.9411 masking region, 0.2231 non-masking); classifier 99.4936% accuracy, 99.8302% masking recall, 44.9572% non-masking recall. SwinIR (σ_stat=0.0800): correlation 0.9597 (0.9737, 0.5154); classifier 99.5337% accuracy, 99.8508% masking recall, 70.7943% non-masking recall. Upscayl High-Fidelity (σ_stat=0.0860): correlation 0.9214 (0.9409, 0.2628); classifier 99.4263% accuracy, 99.9524% masking recall, 42.7923% non-masking recall. Digital-Art (σ_stat=0.0750): correlation 0.8834 (0.8895, 0.3142); classifier 98.8300% accuracy, 99.0871% masking recall, 78.7616% non-masking recall.

All four models achieve 98.8%+ accuracy and 99.1%+ recall on the masking region, using a single untrained threshold, σ_notta≤σ_stat, with σ_notta itself measured exactly (not estimated) via comp_contrast_notta/255. Recall on the minority non-masking class is lower and model-dependent (42.79%–78.76%): because σ_notta is measured exactly here, this gap reflects real reconstruction behavior, not measurement error. σ_stat is calibrated from ground truth only once per model-and-dataset combination (this calibration was done against one specific dataset, NIH ChestX-ray8; whether the same values hold on a different dataset is untested); once that one-time calibration exists, the same threshold applies to any subsequent, ground-truth-free reconstruction on that same dataset. This is offered as a validated diagnostic method for a specific, narrow use case, not as a claim that ground-truth-free SSIM diagnosis is possible in general — see the full report for scope and caveats. A plain-language walkthrough of these results, with a photo-smoothing analogy, is available on the GT-free diagnostic tools explainer page.