← Documentation / SQL Reference

DOI 1.13 — Two empirical tests of score-vs-content agreement

View DOI 1.13 on Zenodo: https://doi.org/10.5281/zenodo.21960019

New DBBuilt on the current, join-free pipeline (build_notta_tta_comp_db.py), one independent database per model.

SSIM multiplies three components together (L·C·S) into a single number — a lossy compression by construction, so two blocks with different underlying content can land on exactly the same score. This report asks whether that theoretical possibility actually happens in practice, via two independent tests of the same underlying question at increasing perturbation size: does an SSIM score tie also mean the eight independently- measured comp_* quality features agree?

Finding 1: same model, TTA on/off — notta_tta_same_score_same_content_check_4model.sql

Holds the model fixed and toggles test-time augmentation. Among blocks where a model's noTTA and TTA reconstructions score identically against the same ground truth, do the eight comp_* features also agree between the two runs? This version ATTACHes all four models' databases in one session and UNIONs the results, so all four can be verified in one continuous recording.

WITH
std_result AS (
    SELECT 'Standard' AS model,
        COUNT(*) AS total_blocks,
        SUM(CASE WHEN (ssim_l_notta*ssim_c_notta*ssim_s_notta)
                     = (ssim_l_tta*ssim_c_tta*ssim_s_tta) THEN 1 ELSE 0 END) AS n_same_score,
        SUM(CASE WHEN (ssim_l_notta*ssim_c_notta*ssim_s_notta)
                     = (ssim_l_tta*ssim_c_tta*ssim_s_tta)
                  AND comp_sharpness_notta = comp_sharpness_tta
                  AND comp_edges_notta = comp_edges_tta
                  AND comp_artifact_ringing_notta = comp_artifact_ringing_tta
                  AND comp_artifact_block_notta = comp_artifact_block_tta
                  AND comp_contrast_notta = comp_contrast_tta
                  AND comp_texture_notta = comp_texture_tta
                  AND comp_entropy_notta = comp_entropy_tta
                  AND comp_noise_notta = comp_noise_tta
                 THEN 1 ELSE 0 END) AS n_same_score_same_features
    FROM std.blocks
)
-- ...the same CTE, unchanged, repeated for swinir.blocks, high.blocks,
-- and digital.blocks, then UNIONed together as all_models below.
SELECT
    model, total_blocks, n_same_score,
    ROUND(n_same_score * 100.0 / total_blocks, 4) AS pct_same_score_of_total,
    n_same_score_same_features,
    (n_same_score - n_same_score_same_features) AS n_same_score_different_features,
    ROUND(n_same_score_same_features * 100.0 / NULLIF(n_same_score, 0), 4) AS pct_same_features_of_same_score,
    ROUND((n_same_score - n_same_score_same_features) * 100.0 / NULLIF(n_same_score, 0), 4) AS pct_different_features_of_same_score
FROM all_models
ORDER BY model;

Each model's CTE counts three things over its whole database: total blocks, how many have an identical noTTA/TTA score (n_same_score), and, among those, how many also match on all eight comp_* fields (n_same_score_same_features). The final SELECT turns the difference between those two counts into a percentage of same-score blocks whose features actually diverge. Result: feature agreement holds for 98.75%–99.28% of same-score blocks in three of the four models, but only 94.21% in Upscayl High-Fidelity — the one consistent outlier.

Finding 2: different models, same coordinate — cross_model_notta_exact_match_recheck.sql

A larger perturbation than Finding 1: instead of the same model with augmentation toggled, this compares two different models at the same (image_id, row, col), restricted to exact SSIM ties (all four models share the same ground-truth images and block grid, so this coordinate is a 1:1 correspondence across every pair). Run once for each of the 6 possible model pairs.

WITH
std_swinir AS (
    SELECT
        'Standard' AS model_a, 'SwinIR' AS model_b,
        COUNT(*) AS n_total,
        SUM(CASE WHEN (a.ssim_l_notta*a.ssim_c_notta*a.ssim_s_notta)
                     = (b.ssim_l_notta*b.ssim_c_notta*b.ssim_s_notta)
                 THEN 1 ELSE 0 END) AS n_exact_match,
        SUM(CASE WHEN (a.ssim_l_notta*a.ssim_c_notta*a.ssim_s_notta)
                     = (b.ssim_l_notta*b.ssim_c_notta*b.ssim_s_notta)
                  AND a.comp_sharpness_notta = b.comp_sharpness_notta
                  AND a.comp_edges_notta = b.comp_edges_notta
                  AND a.comp_artifact_ringing_notta = b.comp_artifact_ringing_notta
                  AND a.comp_artifact_block_notta = b.comp_artifact_block_notta
                  AND a.comp_contrast_notta = b.comp_contrast_notta
                  AND a.comp_texture_notta = b.comp_texture_notta
                  AND a.comp_entropy_notta = b.comp_entropy_notta
                  AND a.comp_noise_notta = b.comp_noise_notta
                 THEN 1 ELSE 0 END) AS n_exact_match_same_features
    FROM std.blocks a
    JOIN swinir.blocks b ON a.image_id=b.image_id AND a.row=b.row AND a.col=b.col
)
-- ...the same pattern repeated for the other 5 pairs, then UNIONed.

Same idea as Finding 1, but the join is now across two models' databases on the shared coordinate key instead of noTTA/TTA columns within one. Restricting to exact score ties, feature agreement is 99.85%–99.98% for the three pairs that don't involve High-Fidelity, but only 67.14%–81.93% for the three that do — the same model standing out as in Finding 1, now at a much larger perturbation.

Tracing the mechanism — cross_model_notta_exact_match_sigma_breakdown.sql

A follow-up query, not a third independent finding: why does an exact SSIM tie between two different models happen at all, and where does the residual feature disagreement concentrate? Reuses DOI 1.11's masking blind spot theorem (σ_gt=0 ⇒ S=1 for every model simultaneously) as the working hypothesis: forcing S=1 removes one of the two factors that would otherwise have to coincide by chance for L·C·S to tie exactly, so exact matches should concentrate at σ_gt=0 and nearby.

per_pair AS (
    SELECT
        model_a, model_b,
        COUNT(*) AS n_exact_match,
        SUM(CASE WHEN sigma_gt_a = 0 THEN 1 ELSE 0 END) AS n_at_sigma0,
        ROUND(SUM(CASE WHEN sigma_gt_a = 0 THEN 1 ELSE 0 END) * 100.0 / COUNT(*), 2) AS pct_at_sigma0,
        SUM(CASE WHEN sigma_gt_a <= 0.16 THEN 1 ELSE 0 END) AS n_in_masking_regime,
        ROUND(SUM(CASE WHEN sigma_gt_a <= 0.16 THEN 1 ELSE 0 END) * 100.0 / COUNT(*), 2) AS pct_in_masking_regime,
        SUM(1 - same_features) AS n_different_features,
        SUM(CASE WHEN same_features = 0 AND sigma_gt_a <= 0.16 THEN 1 ELSE 0 END) AS diff_feat_in_masking_regime,
        ROUND(SUM(CASE WHEN same_features = 0 AND sigma_gt_a <= 0.16 THEN 1 ELSE 0 END) * 100.0 /
              NULLIF(SUM(1 - same_features), 0), 2) AS pct_diff_feat_in_masking_regime
    FROM all_pairs
    GROUP BY model_a, model_b
)
SELECT model_a, model_b, metric, value
FROM per_pair
UNPIVOT (value FOR metric IN (
    n_exact_match, n_at_sigma0, pct_at_sigma0,
    n_in_masking_regime, pct_in_masking_regime,
    n_different_features, diff_feat_in_masking_regime,
    pct_diff_feat_in_masking_regime
));

Confirmed for every one of the 6 pairs: 100% of exact matches fall within the masking regime (σ_gt≤0.16). The final UNPIVOT reshapes the per-pair results into one row per metric so every column stays readable regardless of terminal width, rather than a wide table that gets truncated. Within that regime, matches away from the exact identity point (σ_gt=0 itself) show 1.8–78× more feature divergence than matches sitting exactly at it — the residual disagreement isn't a separate phenomenon, it's concentrated at the edge of the same masking mechanism.