# The Succession — build pipeline

Four steps. Only the first needs the source data.

    python3 measure.py --cra <cra-csv-dir> --db /tmp/succession.sqlite \
                       --out ../measure.json --extracts ../data
    python3 render.py
    node gen_tiles.mjs
    node ../../scripts/generate-narration.mjs --root succession --qa

`measure.json` is the only source of figures. `render.py` writes `../index.html` from it and
`gen_tiles.mjs` writes the cards from it, so the page, the cards and the data cannot disagree.
The narration script in `../narration.json` is hand-written in spoken form; re-check its figures
against `measure.json` after any rebuild.

## Inputs

CRA's annual *List of charities* releases on open.canada.ca, one dataset per year, 2000–2024
(search the catalogue for "List of charities"; 2024 is dataset
`80c00cdb-1358-415c-bb8b-0de7f12675b8`). Name each file `<year>_<kind>.csv`:

| kind | CRA resource | used for |
|---|---|---|
| `donees` | Qualified donees (T1236) | every gift edge, read from the giver's side |
| `ident` | Identification | names, designation, registry presence |
| `fin` | Financial data (2009+) | lines 4200, 4350, 4500, 4510, 4700, 5000, 5010, 5100 |
| `s2det` | Activities outside Canada – Details on financial (2009+) | line 200 |
| `s2res` | Activities outside Canada – Financial resources used | intermediaries by country |
| `dir` | Directors/Officers | board overlap and the arm's-length flag |

About 3 GB in total; deliberately not committed.

**The 2011 release has no qualified-donee file.** It is not on the dataset and not anywhere else in
the catalogue's resource search. `measure.py` drops every event whose window touches 2011 rather
than filling it, because a missing year reads as a collapse and a 2011 gift would read as new in 2012.

## The rule

Fixed before the named charities were scored, and stated in full at the top of `measure.py`:
a gift counts at $5,000 or more; a collapse needs ≥ $1M a year from charities in one of the two
prior years and loses at least half over the next two; at least ten funders; a follower starts
giving to a recipient it had not funded in the previous four years.

## Sources outside the data

| What | Where |
|---|---|
| Beth Oloth revocation, effective 12 Jan 2019 | Canada Gazette Part I, Vol. 153 No. 2, Commissions |
| Gates of Mercy revocation, effective 13 Apr 2019 | Canada Gazette Part I, Vol. 153 No. 15, Commissions |
| CRA's grounds against Beth Oloth | The CJN, R. Csillag, 29 Jan 2019 (CRA's 94-page letter) |
| The study and its Tables 1–3 | Howe & Sylvestre (2022), doi:10.1177/2631309X221138974, CC BY-NC 4.0 → `paper_tables.json` |
| Revocation tax, eligible donee | ITA s.188(1.1)–(1.3) |
| Ineligible individual; revocation grounds | ITA s.149.1(1), (4.1) |

## What was tried and dropped

`mirror_test.py` runs the logic backwards — for every charity whose funding surged, which single
charity's funder list supplied its new funders? The median surge scored 59 %, because nearly any new
charity's funders already give to some large popular charity, and it could not score the named cases
at all. Its summary is `rise_test.json` and the page publishes the failure as a section.

## What the page refuses to do

Name the funders. Counts and dollars are published; the 115 charities are not, because a gift to a
registered charity is lawful and a list on a page about a revocation would imply what the data does
not show. `../data/collapse-events.csv` labels only the named case.