Methods

The same people, the same years, five ways of deciding whether someone holds a role. If the historical story depended on the method, it would show here.

Method	How it decides
A · lexical window	Any role noun within ten tokens of the name, in the same sentence. One sense per word.
B · dependency rules	Titles on the name, appositives, “was elected president of…”, organisational heads typed by the organisation they head, passive harm predicates. Never inherits a relative’s role.
C · bag-of-words classifier	Logistic regression on the words around the person (name masked), trained on reference labels and cross-fitted.
D · contextual embeddings	MiniLM token states pooled over the name and its context, with a logistic head; cross-fitted on reference labels.
E · LLM (Qwen3-8B)	An open-weights language model reading the same context under a fixed JSON prompt at temperature 0. One measurement among five, never ground truth.

Women's share of authority roles, measured five ways on the same peopleEach panel is one method applied to the same application sample of 109,518 gender-signalled people (up to 5,000 per year). The LLM (E) labelled a year-balanced random subsample of 266 of them, so its panel pools four periods (1900–15, 1918–30, 1933–45, 1948–63) and is still noisy.

Table view

Method	Year	Women's share in role	Role prevalence	People in role
A · lexical window	1900.0	20.7%	15.2%	757
A · lexical window	1903.0	21.4%	14.2%	704
A · lexical window	1906.0	29.6%	16.5%	822
A · lexical window	1909.0	24.1%	14.6%	725
A · lexical window	1912.0	32.1%	15.2%	754
A · lexical window	1915.0	32.9%	12.8%	639
A · lexical window	1918.0	33.4%	11.7%	584
A · lexical window	1921.0	35.4%	12.5%	621
A · lexical window	1924.0	37.7%	13.7%	682
A · lexical window	1927.0	46.7%	13.0%	644
A · lexical window	1930.0	48.4%	14.7%	732
A · lexical window	1933.0	45.7%	16.3%	810
A · lexical window	1936.0	47.6%	12.8%	639
A · lexical window	1939.0	49.2%	14.7%	731
A · lexical window	1942.0	46.1%	12.5%	625
A · lexical window	1945.0	42.9%	13.9%	693
A · lexical window	1948.0	44.4%	15.2%	757
A · lexical window	1951.0	47.0%	14.9%	743
A · lexical window	1954.0	46.8%	15.3%	761
A · lexical window	1957.0	49.9%	12.9%	641
A · lexical window	1960.0	36.0%	14.6%	727
A · lexical window	1963.0	35.8%	15.6%	777
B · dependency rules	1900.0	10.6%	7.9%	396
B · dependency rules	1903.0	11.1%	7.5%	371
B · dependency rules	1906.0	10.2%	8.0%	400
B · dependency rules	1909.0	11.0%	8.1%	401
B · dependency rules	1912.0	12.6%	7.5%	372
B · dependency rules	1915.0	13.3%	6.3%	315
B · dependency rules	1918.0	17.3%	5.9%	294
B · dependency rules	1921.0	23.5%	6.1%	306
B · dependency rules	1924.0	17.6%	6.3%	313
B · dependency rules	1927.0	29.2%	6.2%	308
B · dependency rules	1930.0	26.1%	6.9%	341
B · dependency rules	1933.0	28.7%	8.0%	400
B · dependency rules	1936.0	31.9%	6.7%	332
B · dependency rules	1939.0	31.8%	6.8%	340
B · dependency rules	1942.0	28.0%	5.9%	293
B · dependency rules	1945.0	28.0%	7.5%	371
B · dependency rules	1948.0	28.6%	7.9%	392
B · dependency rules	1951.0	31.5%	7.5%	371
B · dependency rules	1954.0	30.7%	7.5%	371
B · dependency rules	1957.0	34.2%	5.9%	295
B · dependency rules	1960.0	22.2%	7.6%	379
B · dependency rules	1963.0	20.9%	8.4%	417
C · bag-of-words classifier	1900.0	15.7%	15.0%	747
C · bag-of-words classifier	1903.0	15.4%	13.6%	675
C · bag-of-words classifier	1906.0	14.2%	12.1%	605
C · bag-of-words classifier	1909.0	18.8%	12.9%	644
C · bag-of-words classifier	1912.0	20.2%	10.8%	535
C · bag-of-words classifier	1915.0	21.3%	11.0%	550
C · bag-of-words classifier	1918.0	23.0%	10.0%	501
C · bag-of-words classifier	1921.0	24.7%	8.9%	445
C · bag-of-words classifier	1924.0	22.7%	10.7%	533
C · bag-of-words classifier	1927.0	26.3%	8.3%	414
C · bag-of-words classifier	1930.0	26.2%	8.8%	439
C · bag-of-words classifier	1933.0	29.7%	9.3%	461
C · bag-of-words classifier	1936.0	29.0%	8.6%	431
C · bag-of-words classifier	1939.0	30.1%	9.7%	481
C · bag-of-words classifier	1942.0	24.0%	9.5%	475
C · bag-of-words classifier	1945.0	23.3%	10.9%	545
C · bag-of-words classifier	1948.0	22.1%	10.6%	529
C · bag-of-words classifier	1951.0	25.9%	10.2%	509
C · bag-of-words classifier	1954.0	25.4%	10.2%	507
C · bag-of-words classifier	1957.0	24.2%	8.7%	434
C · bag-of-words classifier	1960.0	23.5%	9.8%	490
C · bag-of-words classifier	1963.0	19.9%	12.3%	612
D · contextual embeddings	1900.0	17.2%	37.6%	1,876
D · contextual embeddings	1903.0	20.5%	36.6%	1,819
D · contextual embeddings	1906.0	21.5%	34.0%	1,695
D · contextual embeddings	1909.0	22.2%	32.4%	1,614
D · contextual embeddings	1912.0	25.2%	32.9%	1,635
D · contextual embeddings	1915.0	28.1%	27.8%	1,386
D · contextual embeddings	1918.0	28.9%	26.7%	1,332
D · contextual embeddings	1921.0	33.6%	26.8%	1,336
D · contextual embeddings	1924.0	30.1%	27.2%	1,354
D · contextual embeddings	1927.0	32.0%	24.2%	1,202
D · contextual embeddings	1930.0	36.1%	24.3%	1,209
D · contextual embeddings	1933.0	34.9%	24.9%	1,241
D · contextual embeddings	1936.0	31.0%	22.3%	1,111
D · contextual embeddings	1939.0	31.3%	23.6%	1,177
D · contextual embeddings	1942.0	29.7%	24.6%	1,227
D · contextual embeddings	1945.0	30.0%	28.4%	1,413
D · contextual embeddings	1948.0	30.5%	26.7%	1,329
D · contextual embeddings	1951.0	33.0%	26.6%	1,324
D · contextual embeddings	1954.0	34.0%	25.8%	1,284
D · contextual embeddings	1957.0	35.9%	23.0%	1,147
D · contextual embeddings	1960.0	31.3%	25.9%	1,289
D · contextual embeddings	1963.0	29.0%	29.0%	1,440
E · LLM (Qwen3-8B)	1900.0	33.3%	23.1%	3
E · LLM (Qwen3-8B)	1903.0	14.3%	58.3%	7
E · LLM (Qwen3-8B)	1906.0	0.0%	33.3%	4
E · LLM (Qwen3-8B)	1909.0	20.0%	38.5%	5
E · LLM (Qwen3-8B)	1912.0	16.7%	46.2%	6
E · LLM (Qwen3-8B)	1915.0	66.7%	23.1%	3
E · LLM (Qwen3-8B)	1918.0	0.0%	25.0%	3
E · LLM (Qwen3-8B)	1921.0	0.0%	16.7%	2
E · LLM (Qwen3-8B)	1924.0	50.0%	16.7%	2
E · LLM (Qwen3-8B)	1927.0	66.7%	25.0%	3
E · LLM (Qwen3-8B)	1930.0	16.7%	50.0%	6
E · LLM (Qwen3-8B)	1933.0	0.0%	8.3%	1
E · LLM (Qwen3-8B)	1936.0	50.0%	16.7%	2
E · LLM (Qwen3-8B)	1939.0	0.0%	8.3%	1
E · LLM (Qwen3-8B)	1942.0	0.0%	25.0%	3
E · LLM (Qwen3-8B)	1945.0	25.0%	33.3%	4
E · LLM (Qwen3-8B)	1948.0	33.3%	27.3%	3
E · LLM (Qwen3-8B)	1951.0	33.3%	27.3%	3
E · LLM (Qwen3-8B)	1954.0	0.0%	25.0%	3
E · LLM (Qwen3-8B)	1957.0	50.0%	16.7%	2
E · LLM (Qwen3-8B)	1960.0	50.0%	16.7%	2
E · LLM (Qwen3-8B)	1963.0	20.0%	41.7%	5

AmericanStories (Dell et al. 2023, CC-BY-4.0), built on Chronicling America (Library of Congress). 10,000 sampled articles per year, 22 years.

Trend in women's share of authority roles, by methodHAC 95% CIs on the yearly series.

Table view

Method	Slope (pp/decade)	95% CI
A · lexical window	+3.37	95% CI +1.02 to +5.72
B · dependency rules	+3.22	95% CI +1.34 to +5.10
C · bag-of-words classifier	+1.30	95% CI +0.08 to +2.51
D · contextual embeddings	+2.00	95% CI +0.80 to +3.19
E · LLM (Qwen3-8B)	+1.35	95% CI −1.02 to +3.71

AmericanStories (Dell et al. 2023, CC-BY-4.0), built on Chronicling America (Library of Congress). 10,000 sampled articles per year, 22 years.

How far apart are the methods? (H4)

For each role, the spread between the smallest and largest method slope on the same population. The pre-registered test counts a role as materially different when the ratio of the largest to the smallest absolute slope exceeds 2, or when the signs disagree.

Role	Smallest slope	Largest slope	Ratio	Signs agree	Methods
Public office	+0.33	+3.49	10.58	yes	A,B,C,D
Military	+1.38	+5.18	3.74	yes	A,B,C,D
Business	+1.56	+2.08	1.33	yes	A,B,C,D
Labor	−0.19	+8.70	45.86	no	A,B,C,D
Professional & expert	+3.09	+4.18	1.35	yes	A,B,C,D
Arts & sports	−3.03	+0.74	31.59	no	A,B,C,D
Civic organisations	+2.17	+5.29	2.43	yes	A,B,C,D
Society pages	+0.50	+2.93	5.82	yes	A,B,C,D
Family (kin-identified)	+0.47	+3.64	7.68	yes	A,B,C,D
Crime & accident	−0.01	+1.00	110.85	no	A,B,C,D
Authority (public office, business, professional)	+1.30	+3.37	2.60	yes	A,B,C,D

Agreement between methods on who holds an authority role (Cohen's κ)Per period, on the shared application sample.

Table view

Pair	1900-15	1918-30	1933-45	1948-63
A–B	0.61	0.58	0.56	0.57
A–C	0.19	0.20	0.20	0.23
A–D	0.20	0.19	0.20	0.23
A–E	0.33	0.29	0.48	0.53
B–C	0.17	0.21	0.22	0.24
B–D	0.16	0.17	0.19	0.20
B–E	0.22	0.22	0.29	0.42
C–D	0.19	0.22	0.23	0.26
C–E	0.08	0.47	0.26	0.36
D–E	0.36	0.29	0.48	0.43

AmericanStories (Dell et al. 2023, CC-BY-4.0), built on Chronicling America (Library of Congress). 10,000 sampled articles per year, 22 years.

Does agreement change over time?

A trend in κ means the methods disagree more in some decades than others, so disagreement is itself time-dependent.

Pair	Role	κ change per decade	95% CI
A–B	Public office	−0.008	−0.016 to −0.000
A–B	Civic organisations	0.010	−0.004 to 0.025
A–B	Authority (public office, business, professional)	−0.009	−0.014 to −0.003
A–C	Public office	0.008	0.001 to 0.016
A–C	Civic organisations	0.022	0.014 to 0.029
A–C	Authority (public office, business, professional)	0.009	0.003 to 0.015
A–D	Public office	0.007	−0.002 to 0.016
A–D	Civic organisations	0.013	0.009 to 0.017
A–D	Authority (public office, business, professional)	0.008	0.002 to 0.014
B–C	Public office	0.012	0.004 to 0.020
B–C	Civic organisations	0.013	0.004 to 0.021
B–C	Authority (public office, business, professional)	0.013	0.005 to 0.022
B–D	Public office	0.007	−0.002 to 0.016
B–D	Civic organisations	0.007	0.002 to 0.011
B–D	Authority (public office, business, professional)	0.007	0.002 to 0.012
C–D	Public office	0.018	0.012 to 0.024
C–D	Civic organisations	0.027	0.014 to 0.040
C–D	Authority (public office, business, professional)	0.014	0.011 to 0.017

The LLM condition, for the record

Output file	Calls	Parsed	Model	Prompt hash	Dates
data/annotation/llm_reference.jsonl	875	600	Yuu no Sekai	d077f00586a629fc	2026-09-10 to 2026-09-10
data/interim/llm_apply.jsonl	575	558	Yuu no Sekai	d077f00586a629fc	2026-09-10 to 2026-09-11