Open methodology

How VoxSim works

The methodology guide explaining how VoxSim works, how it validates its results, and where its limits lie.

01

Primary axis: 12 political clusters + echo chamber

VoxSim's primary axis is political tendency, not demographics. Turkish society is split into 12 political clusters (social democrat, religious conservative, nationalist conservative, Kurdish political, Gen-Z undecided, etc). Each cluster is fed by its own echo chamber.

For each cluster, news sources are scanned across four time windows; the AI reads this time-weighted news environment and distils the cluster's current mood, grievances, and dominant narrative frames. Discourse evaluation runs against this live context.

The population share (% figure) shown for each cluster is an estimate derived from cross-tabulated national public-opinion surveys and election results. A lifestyle typology from Türkiye's leading independent public-opinion research institution (referenced as LKA under a commercial data licence) provides the core framework; the institution's name is kept confidential per licence terms, raw tables are never republished — only statistical inferences are carried over.

12

political clusters

10

news sources / cluster

4

time windows

Note: Source names (newspapers, TV channels, podcasts) are NEVER published in the product UI or PDF reports — only codenames (ULS_01, MRS_02, ...). Publishers' own RSS feeds are used (publisher-self-published = implicit license); no custom scraping (EU AI Act provenance + ToS compliance).

Bourdieu 3-Capital × LKA Subtypology

Faz 1.8 C4 (2026-05-27): each cluster now carries Bourdieu's three-capital framework (Distinction, 1984 — economic · cultural · social) plus a subtype from LKA's publicly available October 2024 subtypology study (Modern Secular, Conservative Modern, Religious Conservative, etc.). The table reflects aggregate cluster behaviour; never an individual persona. SyntheticPollPrompt optionally injects these profiles to improve the LLM's Turkey-specific habitus modelling. As of 2026-09-17 the profile is no longer injected into production queries; its ablation contribution could not be distinguished from noise.

Cluster Economic Cultural Social LKA subtype
ulusalci_sol 3 4 4 Modern Seküler
sosyal_demokrat 4 5 4 Modern
liberal_kentli 5 5 3 Liberal Modern
merkez_sag 3 3 4 Muhafazakar Modern
dindar_muhafazakar 3 3 5 Dindar Muhafazakar
islamci_taban 2 3 5 Dini Muhafazakar
milliyetci_muhafazakar 2 3 4 Geleneksel Muhafazakar
iyi_parti_merkez 4 4 3 Liberal Muhafazakar
kurt_siyasi 2 3 5 Geleneksel-Etnik
genc_kararsiz 2 4 2 Modern Genç
kentli_kararsiz 3 4 2 Modern
anadolu_kararsiz 2 2 4 Geleneksel Muhafazakar

1 = very low, 5 = very high. Navy shades = high capital, amber shades = low capital.

Academic references: Bourdieu (1984) — Distinction: A Social Critique of the Judgement of Taste (Harvard UP); Çarkoğlu & Kalaycıoğlu (2021) — Fragile but Resilient? Turkish Electoral Dynamics, 2002–2015 (U. Michigan Press); Çarkoğlu & Kalaycıoğlu (2020) — Elections and Public Opinion in Turkey (Routledge); Çarkoğlu & Toprak (2007) — Religion, Society and Politics in a Changing Turkey (TESEV).

Explore the 12 political clusters

Each cluster's profile, information ecology, circulating narratives and weekly snapshot history.


01.5

Information ecology mapping (3-tier hybrid sourcing)

The natural quality filters of major news services exclude some of the most defining sources for specific clusters. Niche publishers — especially ideologically focused outlets — are absent from global news catalogs, yet they reflect the actual mood and concerns of certain political clusters far more accurately than mainstream sources.

VoxSim closes this blind spot with a multi-layer sourcing approach: publishers' own RSS feeds form the foundation of a layered news base (mainstream + niche + echo chamber). Each piece of content is classified by source quality and type.

Information Ecology & Echo Chambers

Mainstream verified

Large news organisations with broad reach — high editorial standards and established content verification processes.

Niche verified

Niche publishers targeting specific audiences while maintaining editorial standards.

Niche unverified

Smaller publishers — unverified but may carry cluster signal.

Echo chamber

Ideological publishers that reflect the inner world of a specific political cluster — critical sources for accurately reading cluster mood.

Disinformation (cluster consumes)

Sources carrying disinformation patterns — VoxSim does not validate their content; they are measured solely as a sociological observation of what the cluster consumes.

Important methodology note

VoxSim performs information ecology mapping — it measures the truth a cluster CONSUMES, not which truth is correct. This is the standard methodological framing of academic echo chamber research (Sunstein, Bail, Levy).

  • ✓Disinformation tier content is not published by VoxSim as truth; it is processed as a "cluster consumes this" signal.
  • ✓A clear directive is given to the AI: the task is information ecology mapping, not truth verification.
  • ✓Customer reports show ecological composition transparently with visuals (mainstream %X, echo chamber %Y, disinformation %Z).
Workspace opt-out: Karargah tier customers who want pure mainstream-only data can opt out of niche + disinformation sources via Workspace settings.

Academic references: Sunstein (2017) — #Republic: Divided Democracy in the Age of Social Media (Princeton UP); Bail (2021) — Breaking the Social Media Prism (Princeton UP); Levy (2021) — Social Media, News Consumption, and Polarization (American Economic Review); Cinelli et al. (2021) — The Echo Chamber Effect on Social Media (PNAS); Bakshy, Messing & Adamic (2015) — Exposure to Ideologically Diverse News on Facebook (Science); Garrett (2009) — Echo Chambers Online? Selective Exposure Among Internet News Users (JCMC).


01.6

Daily echo-chamber pipeline

Each cluster's snapshot is rebuilt nightly through a four-stage pipeline. Total runtime ≈ 30 minutes; happens between 02:30 and 03:30 Istanbul time. Daytime simulations always read the previous night's context. Important constraint: breaking crises that emerge during the day are not reflected in the snapshot until the same-night pipeline completes — Module B should be paired with Event Injection (Module E, Karargah+) for same-day crisis scenarios.

Nightly workflow (Istanbul time)

1

Ingest articles

Publishers' own RSS feeds (mainstream + niche). Last ~2 days of articles per source.

02:30 TR · runs automatically overnight

2

Per-article summary

Each article is condensed into a structured summary of ≤ 200 words. Original text is not retained.

02:45 TR · LLM inference stage

3

Vector embedding

Title + summary → high-dimensional semantic vector. Powers semantic search and cluster centroid computation.

02:50 TR · semantic indexing

4

Cluster snapshot rebuild

Per cluster: time-weighted sample of summaries → LLM composes mood, grievances, and lexicon.

03:00 TR · 12 clusters · nightly snapshot

Stages run sequentially with overlap-prevention locks. If a stage fails, the next one still runs against yesterday's data — no silent corruption.

Exponential forgetting curve (w = exp(−t/τ))

Each article receives a weight <strong>w(t) = exp(−t / τ)</strong> based on its age; bucket weights are derived deterministically by integrating this curve over window boundaries. τ (tau) is the single source of truth — half-life is τ·ln(2). Grounded in Murre &amp; Dros (2015 PLOS ONE) Ebbinghaus replication and Wu et al. (2007 PNAS) news attention decay.

Last 2 weeks 26.7%
Last 3 months 59.8%
Last 6 months 11.7%
Last 12 months 1.8%
w(t) = exp(−t / τ)  ·  τ = 45.0 days  ·  half-life = 31.2 days

Bucket weights are computed live — they update automatically when config changes. The AI is given the average article freshness per window (window integral ÷ window length); the total mass is used only for the article quota and token budget.

Source anonymity

All public-facing surfaces — config files, API responses, PDF reports, audit logs, this page — refer to news sources by codename only (e.g. ULS_01, MRS_03). The real source URI mapping lives in an encrypted file accessible only to Talivio compliance staff.

Public (in code, logs, reports)

ULS_01, MRS_02, DNM_05, …

Codenames stable across taxonomy versions — clients can build dashboards on them.

Private mapping (encrypted)

[encrypted · ops-only]

Real publisher URIs. Rotation policy + access audit on every read.

Backtest temporal pin

When VoxSim simulates a historical event (e.g. February 2023 earthquake), it would be misleading to use today's cluster snapshot — clusters' beliefs in 2023 were different. Backtest selects the snapshot version active on or before the event_date.

Snapshots are append-only

Old snapshots are never overwritten. A simulation against a 2024-09 event reads the 2024-09-15 snapshot, not the latest. This makes Accuracy Score calibration honest — we don't get to retroactively 'fix' a cluster's past beliefs. ⚠ Current state: the echo-chamber archive started on 2026-04-30; every historical event in the calibration set predates it, so backtests currently run WITHOUT echo-chamber context (cluster identity + demographic anchor only). Feeding a present-day snapshot to a past event would leak the future, so this is deliberate. The echo chamber's contribution is therefore measured separately, via an ablation arm on current polls (04.7).

Daily update cost

The daily echo-chamber update runs at minimal cost: RSS content collection is free of charge; AI processing for all 12 clusters runs within a fixed daily budget. This architecture ensures long-term platform sustainability without variable per-customer overhead.

Academic references: Murre & Dros (2015) — Replication and Analysis of Ebbinghaus' Forgetting Curve (PLOS ONE); Tversky & Kahneman (1973) — Availability: A Heuristic for Judging Frequency and Probability (Cognitive Psychology).


02

Secondary axis: demographic nuance

Alongside the political-cluster axis, a secondary demographic axis runs as well. Türkiye's adult population is split across three independent dimensions to build a statistical model: geography (NUTS-2 regions), socio-economic tier (SES), and cultural-lifestyle cluster. Each combination forms a distinct "cell" in a 26 × 4 × 3 matrix.

Within each cell, synthetic agents inherit statistical distributions derived from publicly available demographic surveys and census data. No real individuals are represented — every agent is a procedurally generated statistical composite.

Synthetic Population Matrix

26

NUTS-2 geographic regions

4

Socio-economic tiers (AB · C1 · C2 · DE)

3

Cultural-lifestyle clusters

The three lifestyle clusters reflect research on Turkish society's self-identification patterns with respect to secularism, tradition, and religiosity. The approximate national-level proportions are:

Modern / Secular ~34%

Urban-centred, digitally native, secular value system, higher education concentration

Traditional Conservative ~42%

Anatolian roots, family and community-oriented, moderate religiosity, cautious of rapid change

Religious Conservative ~24%

High religiosity, community-bound, values-driven, concentrated in eastern and central regions

Note: All segment proportions are synthetic approximations derived from publicly available aggregate survey research. VoxSim does not hold or process individual-level survey respondent data.
Current implementation (Faz 1.8+): The 312 cells are generated in two stages: (1) conditional prior + Dirichlet sampling (concentration=50) for per-cell stochastic variation, then (2) Beckman 1996 / Müller-Axhausen 2011 IPF (Iterative Proportional Fitting) fits the NUTS-2 + SES + Lifestyle marginals to TÜİK ADNKS 2023 + LKA October 2024 targets (engine/app/services/ipf.py + population.py). IPF tolerance is 10⁻⁵; marginals match targets within 0.001. The conditional structure (Eastern = more dindar, urban AB = more modern) is preserved by IPF (Beckman §3.2).

Academic references: Beckman, Baggerly & McKay (1996) — Creating Synthetic Baseline Populations (Transportation Research A); Müller & Axhausen (2011) — Hierarchical IPF: A Synthetic Population for Switzerland (ERSA); Park et al. (2023) — Generative Agents: Interactive Simulacra of Human Behavior (UIST); Argyle et al. (2023) — Out of One, Many (Political Analysis); Aher, Arriaga & Kalai (2023) — Using LLMs to Simulate Multiple Humans (ICML).


03

Historical event calibration

A simulation engine is only as good as its calibration. VoxSim validates its population model against historical events where an objective outcome — an election result or a credible polling snapshot — provides ground truth. This process is called backtest calibration.

  1. 1

    Collect historical events

    Each event is documented with date, category, a plain-language description, and a URL pointing to public sources (Wikipedia, official gazette, press archives).

  2. 2

    Assign a measurement anchor

    Only events with a qualifying anchor — a national election result, a credible nationwide poll, or official statistics — enter the calibration set. This is the distinguishing criterion.

  3. 3

    Run population simulation

    The event description is passed to the simulation engine, which predicts emotional response (anger, fear, hope, apathy, support) across all population cells.

  4. 4

    Compare against ground truth

    Predicted distributions are compared against the anchor measurement. For elections, vote-share shifts are mapped to segment-level support estimates. For polls, direct sentiment questions provide validation.

  5. 5

    Iterate and re-calibrate

    Systematic deviations reveal model biases (e.g. underweighting rural religiosity). Parameters are adjusted and the calibration set is re-run until mean absolute error stabilises.

The anchor condition is strict: an event enters the calibration set only if a qualifying measurement (election or poll) falls within approximately six months of the event. This ensures the predicted societal response can be compared against an independently observed signal.

Election results

YSK box-level and national vote-share data for all elections since 1983

Survey polling

Published aggregate results from credible nationwide polling firms

Official statistics

TÜİK, CBRT and other state-published aggregate data

Academic references: Brier (1950) — Verification of Forecasts (Monthly Weather Review); Tetlock (2005) — Expert Political Judgment (Princeton UP); Hyndman & Koehler (2006) — Forecast accuracy measures; Gneiting & Raftery (2007) — Strictly Proper Scoring Rules (JASA).


04

Accuracy score calculation

Each backtest run produces a per-event accuracy estimate. The overall accuracy score reported in VoxSim is the mean across all calibration events, weighted by event reliability.

Accuracy metrics (four scores reported side-by-side)

Brier = (1/N) · Σ (pnorm − anorm)² proper
Log-score = − Σ anorm · log(pnorm) proper
RMSE = √((1/N) · Σ (p − a)²) proper (mean-prediction)
CRPS = (1/m)·Σ|xi − y*| − (1/(2m²))·Σ|xi − xj| proper
MAE = (1/N) · Σ |p − a| legacy
Accuracy = 1 − MAE / 100 legacy

Faz 1.7+: Brier (1950) and Log-score (Gneiting-Raftery 2007) reported as primary strictly proper scoring rules over the normalized emotion distribution. RMSE is proper for mean-prediction. Faz 1.8 C5: CRPS (Continuous Ranked Probability Score, Gneiting-Raftery 2007 §4.2) — strictly proper for continuous outcomes using a replication-sweep ensemble against ground truth; applies directly to intensity scales. MAE and the legacy Accuracy (1−MAE/100) are preserved for backward compatibility. In an LLM-based synthetic system, the Brier score measures model consistency; it is not a validation of the frequentist voter distribution in the field. Important limit: emotion scores are independent 0-100 intensities, not a probability distribution; because Brier/Log-score force them to sum to 1, they only measure the RELATIVE shape of the emotion profile — (80,80,10,10,10) and (40,40,5,5,5) get the same Brier. The primary metric that does not lose raw intensity error is MAE; Brier/Log-score are secondary shape-fidelity indicators.

The score is decomposed by topic category (economic, political, security, etc.) so users can see exactly which policy domains the model performs best and worst on. Every score links back to the specific events that produced it — full transparency, not a black box.

≥ 85%
High confidence Predictions closely match anchor measurements; safe for decision support
70–84%
Medium confidence Notable deviation in some cells; treat output as directional rather than precise
< 70%
Low confidence Significant model bias detected; calibration ongoing
An accuracy score reflects how well the model predicted past events with known outcomes. It does not guarantee future predictive performance. Societal dynamics shift — the model is updated as new calibration data becomes available.
Latest validation measurement (2026-05-27) — reference tables are inferred EMPİRİK

A 46-event full A/B sweep using the updated methodology achieved 81.84% weighted accuracy, weighted MAE 18.16 points. The previous methodology averaged 62.31% across 50 events. Fair comparison (same 6 events): 80.94% → 82.60% = +1.66 points. Two honesty notes: (1) the cluster × emotion reference tables for these events are not directly measured cross-tabs but tables inferred from published survey and election data; the accuracy score measures internal consistency against this reference. (2) Accuracy alone is not sufficient: a trivial baseline (climatology) that predicts, for each cluster × emotion, the average of the other events can achieve similar accuracy. That is why the model's skill relative to that baseline is reported separately (below; shows "not yet measured" if unmeasured).

%81.84

Faz 1.8 (46 olay)

%62.31

Faz 1.5/1.6 (50 olay)

+1.66 pp

same-events (6)

Skill score (relative to climatology baseline)

+0.03

skill

15.26 / 15.72

MAE model / baseline

36 / 67

events beating climatology

Matches the baseline

1 − MAE_model / MAE_baseline. 0 = as good as the baseline; positive = better. Baseline: for each cluster × emotion, the average of the other events (leave-one-out).

Academic references: Willmott & Matsuura (2005) — Advantages of MAE over RMSE (Climate Research); Hyndman & Koehler (2006) — Another look at measures of forecast accuracy (Int'l J. Forecasting); Gneiting & Raftery (2007) — Strictly Proper Scoring Rules (JASA).


04.5

Consistency of cluster seed distributions with YSK (2002–2024, 8 anchors)

Each cluster's hand-curated historical vote distribution is weighted by its population share, aggregated nationally, and compared against the actual YSK result. This is NOT an independent validation: because the distributions were curated by looking at these very elections, the result is an in-sample consistency measure — it shows that the clusters sum arithmetically to the official result, not that the model has forecasting power. It involves no AI call. The 8 anchors below (2002, 2007, 2011, June 2015, November 2015, 2018, 2023, 2024 local) produce the composite score; pre-2014 anchors are back-projected onto today's cluster structure.

2002 Milletvekili

88/100

2007 Milletvekili

94/100

2011 Milletvekili

94/100

2015 Haziran Milletvekili

98/100

2015 Kasım Milletvekili

100/100

2018 Milletvekili

99/100

2023 Milletvekili

98/100

2024 Yerel Seçim

98/100

COMPOSITE SCORE 96/100 (High confidence)

Average of the 8 anchors. A high score shows internal consistency of the curated distributions; for forecasting quality see the measurement program in §04.7.

Forecast quality breakdown

proper

0.0002

Brier

1.698

Log-score

0.009

RMSE

Brier (1950) + Log-score (Gneiting-Raftery 2007) strictly proper. RMSE point-prediction proper. calibration_score TVD-tabanlı (gevşek-proper).

Scores are recalculated automatically every day and update instantly when model parameters change. These computations require no AI inference — they are purely deterministic mathematical operations.

Türkiye political fault lines (2015 — today)

The chain of events behind the calibration anchors: elections, judicial rulings, economic shocks — a sourced chronology.


04.6

Replication variance & algorithmic fidelity

Two foundational academic warnings are addressed with explicit system safeguards. Both tests below run on live data.

Prompt instability — Bisbee (2024) replication sweep

Bisbee et al. (2024, Political Analysis) warn that LLM-based synthetic surveys exhibit low variance + high prompt instability. Relying on a single shot is dangerous; the same prompt must be repeated N times. VoxSim runs an N=5 replication sweep per template, measures the coefficient of variation (CV), and classifies into a three-tier band. The weekly automated sweep measures the production template on a sample basis; separately, the 2026-08 measurement program found the run-to-run noise floor of a single-shot poll to be 4.7–5.8 points (see §04.7).

Template Context N CV Stability
synthetic_poll anadolu_kararsiz 5 0.1451 moderate
synthetic_poll anadolu_kararsiz 5 0.1495 moderate
synthetic_poll merkez_sag 5 0.0694 moderate

Monte Carlo uncertainty band & LLM stochasticity

The ± values in report PDFs are not an internal confidence measure of a single run; they are derived from the standard deviation of N independent Monte Carlo runs of the same simulation. Large language models are not deterministic: due to sampling temperature and context effects, the same input produces somewhat different output on every run. Rather than suppressing this inherent stochasticity, VoxSim measures it and reports it as an uncertainty band.

A narrow band is not a guarantee of accuracy — it only shows the model is internally consistent; systematic bias does not show up in band width. A wide band signals the discourse sits in a "sensitive zone" for that cluster. The band should therefore be read as a model-consistency indicator, not as a real-world margin of error.

Algorithmic fidelity — Argyle (2023)

Argyle et al. (2023) central claim: synthetic population models must reproduce real-world demographic subgroup distributions. VoxSim aggregates its 12 cluster priors into the three LKA lifestyle segments (modern / traditional / religious) and compares them against published cross-tabs from LKA's monthly public opinion survey. Fidelity score is based on Total Variation Distance (TVD): 1.0 = perfect, 0.0 = completely off.

2023

0.767

HIGH · 3 seg

2018

0.761

HIGH · 3 seg

2024_local

0.809

HIGH · 3 seg

Per-segment detail table

2023

modern

0.842

gelenekci

0.690

dindar

0.770

2018

modern

0.807

gelenekci

0.654

dindar

0.823

2024_local

modern

0.927

gelenekci

0.677

dindar

0.821

Cultural bias mitigation — Atari (2023)

Atari et al. (2023) demonstrate that LLMs (GPT/Gemini included) lose cross-cultural validity in non-WEIRD societies (r=-0.70 for collectivist cultures). Türkiye sits in the non-WEIRD bracket, making this drift critical. Mitigation: 36 Turkish aggregate behaviour patterns (drawn from publicly available YSK + LKA aggregates; never individual personas) are injected into the prompt as in-context examples (Brown et al. 2020). Mood and vote-rationale generation are calibrated to the Turkish context.

12

Clusters

36

Examples

Note

As of 2026-09-17 these example patterns are no longer appended to production queries: across two independent polls the ablation contribution could not be distinguished from noise. The library is kept as a measurement arm.

Academic references: Bisbee et al. (2024) — Synthetic Replacements for Human Survey Data? The Perils of LLMs (Political Analysis); Argyle et al. (2023) — Out of One, Many: Using LLMs to Simulate Human Samples (Political Analysis).


04.7

Measurement integrity program — what does the score actually measure?

Module K's calibration score can be misleading on its own, because the seed distribution given to the model is derived from the official result: even a model that simply echoes the seed back would score well. In August 2026, four tests were built to measure this question directly, and were run under a single pinned model across two independent production polls. Results and the decisions taken are below; the measurement files are kept under version control.

Repeat stability and noise floor

The same poll was run three times with identical inputs, bypassing cache. Noise floor = twice the largest party standard deviation. Result: 4.65, 5.79, and 9.31 points (three polls). Decision: a single-shot poll cannot carry party-level differences below this band; time-series charts render the band in grey, and polls are now generated by default as the average of independent runs.

Placebo baseline (the parrot test)

Three arms: the real seed distribution · no seed distribution · a placebo distribution with party labels shuffled. The hypothesis was that the model parrots the seed back; the opposite was found: the model moves markedly away from the seed and produces a recall further from the official result (skill −11.1 and −11.9; negative in 28 of 28 retrospective polls, median −6.7). In the second poll it also could not be confirmed that the score distinguishes the placebo from the real seed. Decision: when skill is negative the confidence band stays low regardless of the score; the recall comparison is now presented as a diagnostic, not a "validation".

Answer-order sensitivity

The party list and schema order are shuffled deterministically on every call (to guard against position bias). The largest difference between polls run with two different order seeds was 3.7 points — below the noise floor. Decision: shuffling stays on; no further correction is applied.

Backstory-injection ablation

The contribution of the Turkish few-shot behaviour-pattern injections and the Bourdieu habitus profile injection was measured across four arms (both · neither · only one · only the other), across two independent polls. Net contribution: +0.013 and −0.011 fidelity points — within noise in both. Decision (2026-09-17): the injections were removed from production; kept as a measurement arm.

Time-series band: 9.3 points

Evidence hierarchy — how is each claim supported?

Claim Evidence type Status
The national total is not anchored to any poll average By construction (from 2026-09-21) — raw population-weighted total. Open issue: some cluster starting distributions were hand-tuned against previously published polls and have not yet been moved to a poll-independent source Active
Cluster seed distributions are consistent with YSK In-sample consistency (same data as the curation target) Active, not an independent validation
Lifestyle segments align with published field findings Measured — Argyle fidelity score, 3 elections Active
The model beats the seed on the recall question Measured — placebo and skill score (for anchors whose seed distribution is given in the prompt) Refuted (skill is negative)
Historical event reactions are correctly predicted Accuracy against inferred reference tables; climatology-baseline skill score — backtests run without context because the echo-chamber archive postdates the events Partial — skill score newly added
Election outcome is predicted in advance Out-of-sample test (pre-election poll vs. official result) Not yet measured — at the next election
Forgetting-curve time constant fits Turkey Literature assumption; empirical fit diagnostic available Assumption
The model can recall an election whose seed distribution and result were withheld Measured — every poll holds out one anchor (parliamentary: 2024 local, presidential: 2018 round 1, municipal: 2019) Active — since 2026-09-17
Echo-chamber context meaningfully changes the distribution Measured — arms with and without echo-chamber context on the same poll, judged against the noise floor In measurement (2026-09-17)

All measurement commands and decision tables are documented in the product repository. Measurements are valid under a single pinned model; they are re-run whenever the model fingerprint changes.

Academic references: Bisbee et al. (2024) — Synthetic Replacements for Human Survey Data? (Political Analysis); Dominguez-Olmedo, Hardt & Mendler-Dünner (2024) — Questioning the Survey Responses of Large Language Models (NeurIPS); Distribution-First Population Simulation: Collapse, Calibration, and Recall in Non-WEIRD LLM Persona Modeling; Murphy (1988) — Skill Scores Based on the Mean Square Error (Monthly Weather Review 116:2417).


05

Historical event coverage

The calibration set consists of hand-curated events spanning 1999 to 2025; the current count is computed live below.

38 Election outcomes
24 Economic events
21 Social & education
18 Foreign policy
28 Crisis & disaster
21 Political events

150 events total

26-year historical coverage (1999-2025)

Module D (Backtest) and Module K (Synthetic Poll) calibration rests on a 26-year political cycle, from 1999 to the present. Selected events — including the Feb 2001 crisis, AKP's rise in 2002, the 2007 e-memorandum, the 2010 constitutional referendum, the 2013 Gezi protests, the 2016 coup attempt, the 2018 lira crisis, COVID-19 in 2020, the 2023 Kahramanmaraş earthquake, and the 2025 İmamoğlu process — carry ground-truth reaction distributions across 12 clusters × 5 emotions (anger, fear, apathy, hope, support).

1999

Start

26 yr

Span

7

Election anchors

Pre-2014 events are back-projected using the current cluster model — flagged as "legacy". Post-2014 cluster behaviour is measured directly. The current calibration composite score is published in the live panel on this page.

Earlier events (pre-2010) use electoral data as their primary anchor, since polling archives are sparser. Post-2013 events benefit from richer survey research, producing more granular per-segment ground truth.

The anchor rule

Every event in the calibration set has a qualifying measurement anchor — an election result, nationwide poll, or official statistical release — within approximately six months of the event date. Events without a validating anchor are excluded, regardless of their historical significance. This keeps the accuracy score honest.

The calibration set is continuously expanded. New events added in each release are noted in the changelog. The target is 500+ events by end of year two.

Academic references: King & Lowe (2003) — Automated Information Extraction for Conflict Data (International Organization); Schrodt (2012) — Precedents, Progress & Prospects in Political Event Data (Int'l Interactions); Boschee et al. (2015) — ICEWS Coded Event Data (Harvard Dataverse); Leetaru & Schrodt (2013) — GDELT: Global Data on Events, Location and Tone.


05.5

Market Signal Profile

Karargah · Karargah+

In its default mode, Module K generates an intra-cluster party distribution for each of the 12 political clusters in a separate AI session; the national estimate is the population-weighted sum of these distributions. Sampling across 26 NUTS-2 × 4 SES × 3 lifestyle = 312 demographic cells is used only in the cell-level mode available on the Karargah+ plan. It provides a decision-support signal based on synthetic simulation for the question "what would happen if elections were held today?" — not a public opinion poll, but a strategic internal assessment tool.

  1. 1

    Cluster-based parallel analysis

    Each of the 12 political clusters — given its cluster identity, lifestyle anchor, ideology axes, and that cluster's echo-chamber snapshot — is evaluated in a dedicated session. The model revises the cluster's seed distribution based on the live agenda in its echo chamber.

  2. 2

    Recall calibration

    Each cluster is also asked for its recalled distribution in the 2024 local, 2023, 2018, and November 2015 elections; the weighted national total is compared against actual YSK results. This comparison is NOT a corrective, and since the 2026-08 measurement program it is no longer treated as a standalone confidence signal either: alongside the score, a skill value is reported that measures what the model gains relative to the seed distribution it was given — if skill is negative, the confidence band stays low regardless of the score. Every poll holds out one anchor: the model is given neither its seed distribution nor its result; the confidence band is primarily derived from this anchor.

  3. 3

    Population-weighted national aggregate

    All segment outputs are weighted by population share and combined into a national estimate. The national total is not anchored to any polling firm's results or to a poll average. Historical election results are used only as a calibration-quality indicator (recall); they do not pull the estimate toward past outcomes. Invalid or missing segments are automatically excluded.

Until 2026-09-21 the national total was anchored to a consensus of published poll averages via Iterative Proportional Fitting (raking). That method has been removed: the synthetic result no longer targets any polling firm's results or uses a poll average; the published figure is the raw population-weighted total. The blog, Pulse page and home page all show this single series. Client reports produced before that date remain archived under the method of their time and are labelled "consensus-anchored (legacy method)".

7

election anchors

25+

years of historical coverage

Not a public opinion poll

Module K output is not a public opinion poll; it cannot be published without the "synthetic simulation" framing and watermark, and cannot be presented as an official/authoritative poll. Per Turkish electoral law, the module automatically disables itself when fewer than 10 days remain before an upcoming election. Users must explicitly acknowledge the synthetic nature before each run.

05.5b — Presidential variant

Karargah · Karargah+

Presidential election 2-round synthetic signal

The presidential variant runs candidate-based 2-round analysis. All 12 political clusters are evaluated against 5 fixed candidate roles (incumbent candidate, main opposition, Kurdish political bloc, nationalist alternative, religious right). Teams can customise candidate display names for their specific scenario.

  • • Auto-computed R2 if no candidate clears 50% in R1 (R2 always filled hypothetically; aggregator decides whether it's required).
  • • Calibration anchors: 2018 R1, 2023 R1, and 2023 R2 official YSK results. The current calibration score is published in the live panel on this page.
  • • Model flexibility is preserved for 2028 candidate uncertainty; teams can enter recognisable candidate names to make outputs more actionable.

⚠ All ethical guardrails of the parliamentary variant apply to CB equally: NOT a public presidential poll, watermark mandatory, T-10 ban window, PDF 30-day expiry, recall is calibration-only.

05.5c — Municipal (mayoral) variant

Karargah · Karargah+

Province-level mayoral synthetic signal

The municipal variant offers province-level (NUTS-3, all 81 provinces) mayoral election analysis. All 12 national political clusters are evaluated against 6 fixed candidate roles (incumbent mayor, main opposition, Kurdish political bloc, nationalist alternative, religious right, independent-local). National cluster weights are preserved; the AI estimates each cluster's province-specific behaviour.

  • • Province selection (plate 01..81) is mandatory. Each prompt is injected with province name, region, tier (metropolitan/province), and anchor data (when available).
  • • SINGLE ROUND — Turkish mayoral elections have no R2. Highest valid-vote share wins. Margin (1st − 2nd) is reported separately.
  • • Calibration: 2019 + 2024 YSK municipal results (30 provinces direct from YSK, 51 from TR Wikipedia citing YSK). All 81 provinces get a calibration score; low-confidence provinces are flagged UNCERTAIN in the config.
  • • Province-level cluster weights remain fixed; output should be read as 'province X viewed through the 12-cluster lens'. This is a deliberate design choice to avoid introducing noise from insufficient local data.

⚠ All ethical guardrails of the parliamentary variant apply to the municipal variant equally: NOT a public mayoral poll, watermark mandatory, T-10 ban window, PDF 30-day expiry, recall is calibration-only.

Academic references: Argyle et al. (2023) — Out of One, Many: Using Language Models to Simulate Human Samples (Political Analysis); Horton (2023) — Large Language Models as Simulated Economic Agents: Homo Silicus (NBER WP 31122); Bisbee et al. (2024) — Synthetic Replacements for Human Survey Data? (Political Analysis); Aher, Arriaga & Kalai (2023) — Using LLMs to Simulate Multiple Humans (ICML); Deville & Särndal (1992) — Calibration Estimators in Survey Sampling (JASA 87:376-382); Deville, Särndal & Sautory (1993) — Generalized Raking Procedures in Survey Sampling (JASA 88:1013-1020).


06

Known limitations

VoxSim is a decision-support tool, not a prediction oracle. Honest communication of limitations is central to responsible use.

  • Synthetic agents ≠ real people

    Responses come from statistical agents, not actual citizens. The model cannot capture individual stories, micro-community dynamics, or real-time mood shifts.

  • Calibration lag

    Model parameters reflect the most recent calibration cycle. Rapidly evolving situations — breaking crises, sudden leadership changes — may not yet be reflected.

  • Correlation, not causation

    A high anger signal does not mean your policy caused that anger. Other concurrent factors may explain the historical pattern; the model attributes to the event, not to external context.

  • Coverage gaps

    Pre-1999 events have no polling anchor and are excluded. Regional events affecting only specific NUTS-2 zones may be under-represented in aggregate accuracy.

  • LLM variability (mitigation: Bisbee + Atari)

    Simulation outputs are generated by large language models and carry inherent LLM limitations: hallucination risk, sensitivity to prompt wording, lack of genuine world-understanding. Faz 1.8 introduced active mitigations — Bisbee replication sweep (N=5 σ measurement, see 04.6) and Atari WEIRD-bias mitigation via a Turkish few-shot library (12 clusters × 3 aggregate patterns). Single-shot output remains limited; if the production replication detects an unstable band, the PDF embeds a warning.

  • A synthetic poll is not an opinion poll

    Module K produces an LLM-based distribution estimate across 12 political clusters; it is NOT a poll of real voters and can only be shared under the synthetic-simulation framing, with a watermark. Past-election recall distributions are used solely as a diagnostic indicator — never as a corrector that drags predictions back to the last election. Module K is auto-disabled during the 10 days preceding any announced election.

  • Municipal polls do not localise cluster shares per province

    In the municipal poll variant, the national shares of the 12 political clusters are not re-estimated per province (a deliberate decision driven by sample-sparsity risk in small provinces). Province context is injected into the LLM prompt and the model revises within-cluster behaviour accordingly; in atypical provinces (where a cluster's true local share deviates strongly from its national share) the error margin of this approach grows. Calibration is measured per province against 2019 + 2024 YSK anchors; anchors for 51 provinces are compiled from secondary sources and low-confidence provinces are flagged UNCERTAIN in config.

Academic references: Bisbee et al. (2024) — Synthetic Replacements for Human Survey Data? The Perils of LLMs (Political Analysis); Santurkar et al. (2023) — Whose Opinions Do Language Models Reflect? (ICML); Park et al. (2023) — Generative Agents: Interactive Simulacra of Human Behavior (UIST); Atari et al. (2023) — Which Humans? (WEIRD bias in LLMs).


07

Compliance & ethical framework

VoxSim is designed with EU and Turkish data protection, AI transparency, and ethical requirements in mind.

  • EU AI Act (High-Risk Classification)

    AI systems that may influence democratic processes fall under high-risk classification. VoxSim addresses this with transparent accuracy scoring, human review gates, and ethical filters against misuse.

  • GDPR (General Data Protection Regulation)

    The system uses synthetic (artificial) data — no real individual personal data is used. Customer documents are processed under a DPA. Data retention is 12 months; right to erasure is provided via self-service.

  • KVKK (Turkish Personal Data Protection Law)

    A KVKK-compliant data processing agreement applies for Turkey-based customers. Talivio Technology OÜ acts as data processor; the customer is the data controller.

  • Synthetic data & privacy

    Generated simulation outputs are not based on any real person's or survey respondent's data. Statistical distributions (TÜİK and public research) are used only at aggregate level.

For compliance and legal enquiries: [email protected]

Ready to run your first simulation?

Start with the Discourse Test module — results in under 5 minutes.

Fresh insights and sample analyses

Stay in the loop with VoxSim.

New features, sample analyses and notes on political communication.

Please check your inbox and confirm your subscription.

Cookie preferences

VoxSim uses essential cookies (session, language, theme) that are always active. We would also like to use Google Analytics to understand aggregate site usage — it runs only if you accept, and you can change your choice at any time. See our Privacy Policy