# Topic 07 — MSIP Metadata — Run 2 (Fonts + Terms, FIXED)

This Run 2 pack extends Topic 07 by analyzing **tracking fonts** and **language terms** *only* within the MSIP-labeled cohort.

## Critical Fix (why the prior Run 2 exported empties)

Some documents have MSIP metadata present in:

- `children.metadata.msip.has_msip_flag = true`

…but **do not** consistently include `"META_MSIP"` in `children.doc_set[]`.

Therefore the MSIP cohort in this fixed Run 2 is defined by:

> **MSIP cohort = `COALESCE(children.metadata.msip.has_msip_flag,false) = true`**

A boolean column `doc_set_has_meta_msip` is retained for auditing inconsistencies.

---

## What this produces

### Fonts
- `v07r2_msip_font_objects__docs_long`  
  Doc × object long table for MSIP docs.

- `v07r2_msip_shared_fonts__hash_stats`  
  Reuse stats per `object_sha256` (docs/cases/clusters, first/last dates).

- `v07r2_msip_shared_fonts__cross_focal`  
  Multi-cluster font hashes that include focal cluster **1570**.

### Terms (presence-based; ignores string_quantity)
- `v07r2_msip_terms__docs_long`  
  Doc × term rows for MSIP docs (uses `children_doc` view).

- `v07r2_msip_shared_terms__stats`  
  Reuse stats per `string_norm` (+ group fields).

- `v07r2_msip_terms__focal_vs_others`  
  Classifies terms as `focal_only`, `focal_plus_others`, `others_only`.

### Compound anomaly queue (strict)
- `v07r2_msip_compound_anomalies`  
  MSIP docs that simultaneously have:
  - shared font hashes (`n_clusters >= 2`)
  - shared terms (`n_clusters >= 2`)
  - signature anomaly gate:
    - any pdfsig non-valid OR watermark missing/invalid

This is intentionally strict and may be empty depending on your sample size.

---

## How to run

From DuckDB CLI (project root), after bootstrap + starter views:

```sql
.read sql/topics/07_msip_metadata/07_msip_metadata__run02_fonts_terms__views.sql
.read sql/topics/07_msip_metadata/07_msip_metadata__run02_fonts_terms__exports.sql
```

Outputs land in:

`reports/07_msip_metadata/`

---

## Quick validation query

To sanity-check the cohort is non-empty:

```sql
SELECT COUNT(*) AS n_msip_docs
FROM v07r2_msip_docs_fixed;
```

And to see how often `doc_set` disagrees:

```sql
SELECT doc_set_has_meta_msip, COUNT(*) 
FROM v07r2_msip_docs_fixed
GROUP BY 1
ORDER BY 1;
```