HitungKerusi Forecast Methodology

A layered Bayesian model for Malaysian parliamentary election forecasting

Last updated: 2026-10-17 (alliance-watch detection note in §11) · Version 0.1.3 · October 2026 · This document is the authoritative specification of the model implemented in backend/hitungkerusi/analysis_engine/. The public site renders this same document on its Methodology page; if the code and this paper ever disagree, that is a bug.

---

1. Scope and principles

HitungKerusi ("count the chairs") forecasts Malaysian legislative elections under first-past-the-post:

• Dewan Rakyat — 222 federal constituencies (GE16 outlook).
• State assemblies (DUN) — chamber-specific forecasts with their own arithmetic: Melaka (28 seats, majority 15 — PRN polling day 14 November 2026), Sarawak (82, majority 42 — PRN due by February 2027), Johor (56 — next PRN ~2031), Negeri Sembilan (36 — ~2031), and Sabah (73 — ~2030; the 2025 PRN left a hung assembly). Each DUN chamber runs the same layered model on its own baseline (its latest PRN), own state-scoped polls, and own news window; the remaining states follow the same pattern as their PRNs approach.

Four principles govern the design:

1. Reproducibility. Every published number comes from an immutable forecast_run row that stores the full parameter set and a SHA-256 hash over all input frames. Re-running with the same inputs reproduces the same output.
2. Layered evidence. Information enters in layers of decreasing reliability — electoral history first, then by-elections, then polls, then news — and each layer can only adjust what the previous layers established. No single layer, however loud, can dominate.
3. Honesty about scarcity. Malaysian public polling is sparse, methodologically uneven, and concentrated in peninsular Malay-majority seats. The model must produce a defensible forecast from fundamentals alone, with polls and news as refinements.
4. Bloc-level realism. Malaysian elections are contested by coalitions whose membership changes between and even during parliamentary terms (the Sheraton Move of 2020 redrew the entire chessboard without an election). The unit of forecasting is therefore the bloc — currently PH, BN, PN, GPS, GRS, WARISAN, and a residual OTHERS bucket — not the party. The bloc set is maintained as data (the blocs table plus curated memberships), so a new bloc is a curation event, not a code change; see the development guide.

2. Data foundation

2.1 Sources

2.2 Notional results: re-bucketing history onto today's blocs

A vote cast at GE15 for Bersatu (then in PN) sits correctly in today's PN column. A vote for Pejuang under the Gerakan Tanah Air (GTA) banner belongs in today's OTHERS bucket, because GTA no longer exists as a competitive force. Notional results answer the question: given the votes actually cast at election E, how would they be distributed across today's blocs?

Two paths produce them:

• Bloc-aligned ingestion (used for GE15): the TindakMalaysia dataset already reports votes per today's blocs, so each candidacy carries its bloc directly and notional results are the straight aggregation of candidate votes into (seat, bloc) pairs.
• Party-history mapping (used for GE14): each 2018 candidacy's party is mapped to the bloc that party belongs to as of the load date, via the dated coalition_membership table and coalition_bloc overrides. A 2018 PPBM vote counts for PN today; a 2018 BN vote in Sarawak (PBB/PRS/PDP/SUPP) counts for GPS; a 2018 PAS vote counts for PN; PBS's 2018 BN seat counts for GRS. Parties with no current home (UPKO, PCS, BERJASA, ...) fall into OTHERS.

The result is a genuine realignment-aware history: GE14's notional seat allocation reads PH 100, PN 56, BN 34, GPS 19, WARISAN 8 — the PPBM+PAS votes that became the 2022 "green wave" were already visible inside the 2018 vote distribution once re-bucketed onto today's blocs.

The notional table is the only electoral input the model consumes. This isolates the realignment problem: when the coalition map changes, only notional results are recomputed, never the model.

2.3 Data-quality gates

Before any forecast runs, every election in the database passes QA: per-seat bloc vote totals must reconcile with official valid-vote counts (blocker if off by more than 0.1%), ballots cannot exceed electorate, recorded winners must hold the seat's maximum votes, and turnout must sit in a plausible band. Blockers abort the pipeline rather than poison the forecast.

3. Model overview

For each seat $s$ and bloc $b$ the model produces an expected vote share $\mu_{s,b}$, then a Monte Carlo simulation converts shares into seat outcomes. The expected share is built in four layers:

$$\mu_{s,b} = \underbrace{\text{baseline}{s,b}}{L1} \oplus \underbrace{\delta^{\text{nat}}b + \delta^{\text{state}}{s,b}}{L2\ \text{swing}} \oplus \underbrace{\text{poll-implied }\delta^{\text{nat}}b}{L3} \oplus \underbrace{\nu{s,b}}_{L4\ \text{semantic}}$$

where $\oplus$ denotes addition in share space followed by re-normalisation, and each layer's contribution is bounded (§7 caps L4; §6 caps L3’s trust weight).

4. Layer 1 — Baseline (electoral history)

Time-decay weighting. For each past election $e$ at distance $\Delta_e$ years from the forecast date:

$$w_e = \exp(-\lambda \Delta_e), \qquad \lambda = 0.5 \ \text{by default}$$

One election half-life of two years: GE14 (2018) would enter a GE16-era forecast at roughly 15% weight alongside GE15's 100%... and GE13 at 2%. Malaysian politics re-aligns fast; old elections inform the seat distribution's texture, not the contest.

Effective votes. With $v_{e,s,b}$ the notional votes and $n_{e,s}$ the valid votes:

$$v^{\text{eff}}{s,b} = \sum_e w_e, v{e,s,b}, \qquad n^{\text{eff}}s = \sum_e w_e, n{e,s}$$

Hierarchical shrinkage. The raw time-weighted share $\hat{p}{s,b} = v^{\text{eff}}{s,b} / n^{\text{eff}}_s$ is shrunk toward the bloc's national share $p_b^{\text{nat}}$ with pseudo-vote mass $m$:

$$\text{baseline}{s,b} = \frac{v^{\text{eff}}{s,b} + m, p_b^{\text{nat}}}{n^{\text{eff}}_s + m}, \qquad m = 800 \ \text{by default}$$

This is a two-level empirical-Bayes hierarchy (seat → nation; a state level is planned). Its job is regularisation: seats whose history is thin or stale move toward the national norm, while data-rich seats keep their own numbers. With $m=800$ against typical effective turnouts of 20,000–60,000, the pull is about 1.5–4% — enough to soften idiosyncratic seats without re-drawing the map. With GE12–GE15 now loaded, cross-election smoothing is active: at λ=0.5 the pre-2018 elections carry only ~2% weight, so their main contribution is texture in the seat distribution rather than the contest.

Calibration check: run against GE15 alone (no swings, no polls, no news), the model's 80% seat-count intervals contain every bloc's actual result — PH actual 81 vs forecast median 84 [80–88]; PN 74 vs 71 [68–75]; BN 30 vs 32 [29–36]; GPS 23 vs 20 [19–22]; GRS 6 vs 6; WARISAN 3 vs 3; OTHERS 5 vs 5.

5. Layer 2 — Swing (by-elections)

By-elections between general elections are Malaysia's only observed vote movements. Three Dewan Rakyat by-elections have occurred since GE15 — Kuala Terengganu (Aug 2023), Pulai (Sep 2023), Kemaman (Dec 2023) — and each is loaded with candidate-level votes. (Padang Serai, held 18 days after GE15 following a candidate's death, is counted as part of GE15.)

Like-for-like measurement. The unity government means blocs routinely sit out by-elections they cede to their partner, which mechanically inflates the remaining blocs' shares: PAS "won" 84.96% at Kuala Terengganu partly because BN did not contest. Measuring the naive share difference would report a phantom +20-point PN swing. Instead, for each seat we restrict to the set of blocs that contested both the by-election and the base election, renormalise both elections' shares to that contested set, and measure the swing within it:

$$\text{swing}{s,b} = \frac{v^{\text{BE}}{s,b}}{\sum_{b' \in C_s} v^{\text{BE}}{s,b'}} - \frac{\text{baseline}{s,b}}{\sum_{b' \in C_s} \text{baseline}_{s,b'}}, \qquad C_s = {\text{blocs contesting both}}$$

On this like-for-like basis the 2023 by-elections read: PN +5.3 points nationally, PH −4.2, BN −8.0 (Kemaman only) — real PN consolidation in the Malay belt, an order of magnitude smaller than the naive reading.

L2b — state-election swings. When a state votes more recently than the base general election, its PRN is itself electoral evidence for that state's parliamentary seats. Eight states now feed this layer: Johor and Negeri Sembilan (PRNs mid-2026, candidate-level), Sabah (PRN 2025, candidate-level), and the six August-2023 peninsular PRNs (Selangor, Negeri Sembilan, Penang, Kedah, Kelantan, Terengganu) loaded as aggregate state shares — the NS 2023 evidence is superseded by its 2026 PRN under a latest-per-state rule. For each such state the swing per bloc is

$$\delta^{\text{PRN}}{j,b} = \left(p^{\text{PRN}}{j,b} - \bar{p}^{\text{baseline}}{j,b}\right) \cdot \underbrace{0.5^{a_j / 365\text{d}}}{\text{age decay}} \cdot \underbrace{0.5}_{\text{structure shrink}}$$

shrunk twice: evidence fades with a 365-day half-life, and a 50% structure discount because DUN contests have different multi-corner structures and seat deals than federal ones (straight BN+PN fights in Negeri Sembilan, absent blocs in Johor). The half-life was recalibrated from an initial 120 days after the Johor 2022→2026 replay (§10) showed that four-year-old state votes still allocate 94.6% of seats correctly — a 120-day half-life would erase fresh PRN evidence years before the next federal contest. The swing is added to the by-election state residual and applied to the state's parliamentary seats through the same uniform-swing mechanism. As of October 2026 this layer reads Johor BN +11 / PN −9, Sabah GRS +4.6 / PH −5.6, and small age-decayed PN gains (+0.5–0.9) in the northern states from the 2023 evidence.

From seats to a national swing. The national swing per bloc is the mean of its by-election seat swings, then shrunk toward zero by sample size with $k_{be} = 4$ pseudo-seats:

$$\delta^{\text{nat}}_b = \frac{n}{n + 4},\overline{\text{swing}}_b$$

so three seats carry 43% of their face value. It is then blended with the poll-implied swing (§6) by weight. State swings shrink toward the national value with $k=2$ pseudo-seats, giving the uniform-plus-regional decomposition (Butler's UNS generalised). This matters in Malaysia — the 2022–23 "green wave" was regionally concentrated, and the backtest (§10) quantifies how much that regional concentration costs a purely uniform model.

When no by-elections are loaded, L2 contributes zero and the forecast is baseline + L3 + L4 only.

6. Layer 3 — Polls

Polls never touch seat-level numbers directly. They adjust the national vote shares, which propagate through the same uniform-swing mechanism as L2 — one mechanism, two evidence sources.

Decay weighting. For poll $i$ with age $a_i$ days and sample size $n_i$:

$$w_i = 0.5^{a_i / 45} \cdot \sqrt{n_i} \quad (\text{45-day half-life; } n_i \text{ defaults to 500 if unreported})$$

Polls older than six half-lives (270 days) are dropped entirely. Per-poll bloc shares are renormalised to sum to 1 before aggregation.

Trust cap. With $N$ distinct polls in the window:

$$\kappa = \min(0.7,\ 0.25,N)$$

So one poll moves the national estimate by 25%, three polls by 70% (the cap), and the baseline always retains at least 30%. The blended national share is renormalised across blocs, and the implied national swing $\delta^{\text{poll}}b = p^{\text{blended}}b - p^{\text{baseline}}b$ is merged with the by-election swing from L2 in proportion to their weights ($w{\text{polls}} = \kappa$; $w{\text{BE}} = \min(1, n{\text{BE seats}}/8)$).

This is a deliberately conservative version of dynamic poll aggregation in the spirit of Linzer (2013); house effects are not yet modelled because the number of usable Malaysian pollsters per cycle barely exceeds two, which cannot identify house-effect parameters. When a poll series accumulates, house offsets and pollster grades (already stored) will enter the weighting.

7. Layer 4 — Semantic (news, weekly)

The semantic layer converts tagged news into small, capped seat-level adjustments. It runs every week and is the layer users feel between elections.

Pipeline per weekly cycle:

1. Collect — RSS feeds (Google News queries per topic, direct publisher feeds) → news_articles (immutable, deduplicated by URL).
2. Tag — each article is linked to entities (seats, parties, coalitions) and scored for per-bloc sentiment and relevance. Two interchangeable taggers share one schema:
   • fallback (always on): deterministic keyword/regex matching against seat and party registries, with a Malay/English political lexicon for sentiment. Zero cost, fully reproducible.
   • LLM (when LLM_API_KEY is set): an OpenAI-compatible model performs entity extraction and per-bloc sentiment scoring under a strict JSON contract. The deterministic tagger remains the floor — the LLM only refines.
3. Embed (when a key is set): article vectors stored in pgvector for semantic dedup and retrieval of seat-relevant coverage.
4. Score — for each seat $s$ and bloc $b$ within the 28-day window:

$$\text{signal}{s,b} = \sum{a \in \text{articles}(s)} \text{relevance}a \cdot \text{sentiment}{a,b}$$

5. Standardise and cap — per bloc across seats:

$$z_{s,b} = \frac{\text{signal}{s,b} - \overline{\text{signal}}b}{\sigma_b}, \qquad \nu{s,b} = \operatorname{clip}\left(0.01 \cdot \kappa{\text{sem}} \cdot z_{s,b},\ -0.02,\ +0.02\right)$$

with $\kappa_{\text{sem}} = 1.0$. One standard deviation of news signal therefore moves a seat by one share point, and no amount of news can move any seat-bloc share by more than two points — the formal guarantee that headlines alone cannot swing the forecast. In share space the nudges sum with L1–L3 and the row is renormalised.

Why standardise rather than use raw sentiment? Absolute sentiment scores are tagger-dependent and unbounded; z-scores measure relative news pressure across seats, which is the decision-relevant quantity ("which seats is the news cycle moving?"), and the clip bounds the units regardless of tagger calibration drift.

8. The weekly analysis cycle

The worker runs on an adaptive cadence (HK_CYCLE_DAYS=auto): every 7 days in quiet periods, every 3 days once a dated election sits within 60 days, and every 24 hours through the campaign window (from nomination ≈ polling − 14 days until polling day). A daily decision tick re-resolves the interval from the electoral calendar, so a snap dissolution shortens the cycle within a day. Each cycle runs:

collect → tag → embed → notional-refresh → QA gates → forecast → persist

Notional results are recomputed each cycle (cheap) so that coalition-map edits take effect immediately. QA blockers abort before forecasting. The forecast run then executes L1 → L2 → L3 → L4 → simulation in that fixed order and persists:

• forecast_seat_probs: per seat and bloc — win probability, expected share, σ.
• forecast_summaries: per bloc — median/mean seats and the p05/p10/p90/p95 interval.
• forecast_metrics: P(hung parliament), P(government blocs ≥ 112 / ≥ 149), P(largest bloc).
• run metadata: model version, all hyperparameters, and the input hash.

Runs are append-only. The site always displays the latest run; the full history is retained, so the forecast's own movement over time is auditable after the fact.

9. From shares to seats: simulation

Malaysian contests are multi-cornered by default (GE15 averaged ~4 candidates per seat, with 3+ serious blocs in Malay-majority seats). Pairwise win-probability shortcuts do not survive this; HitungKerusi simulates each seat's entire share vector jointly, with regionally correlated uncertainty:

1. Draw a state shock per state and bloc: $z_{j,b} \sim \mathcal{N}(0, \sigma_{\text{state}})$, applied to every seat in state $j$ — seats in the same state move together within a simulation.
2. Draw an independent seat perturbation: $\varepsilon_{s,b} \sim \mathcal{N}(0, \sigma_{\text{seat}})$.
3. $\tilde{p}{s,b} = \mu{s,b} + z_{\text{state}(s),b} + \varepsilon_{s,b}$, clipped at a 0.2% floor and renormalised.
4. The winner of seat $s$ in draw $d$ is $\arg\max_b \tilde{p}_{s,b}$.

Horizon-scaled volatility. Uncertainty grows with time to the election as a damped random walk:

$$\sigma_{\text{total}}(t) = \min\left(0.16,; 0.045\sqrt{1 + t/168\text{ days}}\right)$$

anchored at both ends: 4.5 share points near-term (a weekly cycle moves little), ~14–15 points at a four-year horizon (the empirical GE14→GE15 per-bloc swing σ is 16.2). The split between the two components is $\sigma_{\text{seat}} = \sqrt{1-\rho},\sigma_{\text{total}}$ and $\sigma_{\text{state}} = \sqrt{\rho},\sigma_{\text{total}}$ with $\rho = 0.4$: 40% of variance is regional. Expected polling days per chamber drive $t$ — Melaka 14 Nov 2026, Sarawak by early 2027, Sabah ~2030, Johor and Negeri Sembilan ~2031, parliament at its auto-dissolution deadline (all env-overridable via HK_POLLING_DAY_*).

Across 10,000 draws this yields per-seat win probabilities, full seat-count distributions per bloc, and the headline metrics: P(no single bloc ≥ majority), P(government ≥ majority / two-thirds), and P(largest bloc) per bloc.

10. Backtesting and calibration

With GE14 and GE15 both loaded, the harness replays the model as if only GE14's notional results were known, applies the actual national swing that occurred by 2022 (the "oracle swing" — isolating seat-allocation error from national-movement error), and scores the resulting seat win probabilities against the true GE15 winners.

Empirical volatility first. The GE14→GE15 seat-level swing distribution has standard deviation 16.2 points per contested bloc (mean absolute swing 12.7; winner-share swing σ 10.3). Malaysian electoral volatility over a four-year, realigning, mid-pandemic cycle is enormous by international standards — this is the anchor for how wide long-horizon intervals must be.

Chained replay across eras (calibrated model — horizon σ + state shocks, oracle national swing; the GE14→GE15 ablation that motivated the calibrated model is retained below it):

GE14→GE15 ablation (why the calibrated model is the live one):

Reading: the model's skill tracks the stability of the party system. In the stable 2008→2013 era it is excellent (Brier 0.24, 85% accuracy, fully-calibrated intervals); it degrades precisely at the two systemic breaks — the 2018 regime change and the 2022 realignment — where share-level noise cannot represent bloc chemistry changing (notional re-bucketing mitigates but cannot eliminate this). GE16's replay joins this table when results are in, whatever it says.

State-chamber validation (2023–2026 ground truth). The same harness replays each loaded DUN election from the previous DUN election of that state — the first genuinely out-of-sample test against elections the model never saw:

Johor is the model at its best: BN's consolidation was already visible in the 2022 state votes and the calibrated replay called 53 of 56 seats. Negeri Sembilan is the systemic-break finding in miniature — the 2026 BN+PN alliance fought straight contests that recombined two blocs' votes, exactly the chemistry change share-noise cannot represent. The practical reading matches the federal chain: within a stable bloc structure the state forecasts are sharp; when alliances recombine, intervals widen honestly but medians lean on stale structure.

These scores are computed with the oracle national swing, so they measure the seat-allocation machinery, not the (much harder) national-movement forecast. The in-sample calibration check in §4 (every bloc's actual GE15 count inside the model's 80% band at election-day parameters) remains the complementary evidence.

11. Assumptions and limitations

1. Boundaries are frozen. Constituency boundaries (unchanged since 2003... with the 2018 delimitation the last exercise) are assumed constant through the forecast horizon.
2. Candidate effects are local noise. A particularly strong individual candidacy is absorbed into σ, not modelled; candidate announcements enter only via the semantic layer.
3. Turnout is implicit. Shares are of valid votes; differential turnout (e.g. the 2022 drop to 73.9%, or PAS-base turnout surges) is a known systematic risk not yet modelled. A turnout layer is planned once by-election turnout data is loaded.
4. Polls are scarce and possibly biased. The trust cap and heavy shrinkage reflect this; the model never requires polls.
5. The semantic layer is bounded by design — it detects relative news pressure shifts, not regime changes. A genuine earthquake (a coalition collapse) enters the model through the curated coalition map and notional recomputation, not through sentiment scores. Detection is assisted, not automated: a daily alliance watch (hk unknown-terms) flags trending proper-noun phrases that no registry knows, prompting the reviewed curation — the tagger's whitelist guarantees no unreviewed entity ever reaches the data.
6. Sarawak/Sabah opacity. GPS/GRS interior dynamics (seat deals, local parties) are minimally observed by national news; their baseline comes almost entirely from history, which is arguably correct. A related DUN artifact: the OTHERS bucket aggregates all minor candidates, so in fragmented seats the bucket's combined vote can exceed the GPS winner's — notional "winner" counts then understate the dominant bloc (Sarawak 2021 reads GPS 70 notional vs 76 actual). Hierarchical shrinkage toward the chamber-wide share corrects most of this (the current Sarawak forecast median is GPS 80), but it is a known aggregation bias, documented here rather than hidden.
7. Party-hopping between elections changes the "current composition" view but not the forecast, which is about the next election's bloc contest. Notional recomputation handles historical realignments.
8. DUN baselines are single-election. Each state chamber rests on its latest PRN alone (Melaka 2021, Sarawak 2021, Johor 2026, Negeri Sembilan 2026, Sabah 2025; older PRNs would enter with heavy time-decay anyway), so between elections their movement comes almost entirely from the semantic layer until by-elections or polls arrive. During a live campaign this makes them responsive to news and honest about it.

12. Academic anchors

• Butler, D. (1951). The British General Election of 1951 — origin of uniform swing.
• Linzer, D. (2013). "Dynamic Bayesian Forecasting of Presidential Elections in the States." Journal of the American Statistical Association 108(501).
• Jackman, S. (2005). "Incumbency Advantage and Candidate Choice." American Journal of Political Science — seat-level swing/variance modelling.
• Gelman, A., Carlin, J., Stern, H., Dunson, D., Vehtari, A., Rubin, D. (2013). Bayesian Data Analysis, 3rd ed. — hierarchical shrinkage.
• Tavits, G. & others on multiparty seat-vote models — multi-bloc share normalisation.
• Ufen, A. (2020). "Opposition Coalitions in Democratizing Southeast Asia." Government and Opposition — coalition realignment context for Malaysia.

Appendix A — Parameter reference

Appendix B — Reproducing a run

make weekly                      # collect → tag → notional → QA → forecast
psql> SELECT id, created_at, inputs_hash FROM forecast_runs ORDER BY id DESC LIMIT 1;

The inputs_hash covers every input frame plus the parameter set; two runs with equal hashes produce equal outputs up to Monte Carlo seed (fixed per run by default for exact reproducibility).