Combating Polarization with ML
DeliBERT: A technical framework for analyzing deliberative polling at scale.
Linux Foundation AI Developers Conference, 2024
Alice Siu (Stanford University) + Daniel Hu (Fileread AI)

by Josh Joseph

Polarization
  • Polarization is undoubtedly increasing in our country.
  • Poli Sci literature argues that political isolation is a significant factor.
Deliberative Polling
Deliberative Polling
Pre
Survey
Random Sample of population completes a poll
Deliberation
Event
  • Plenary Session
  • Group Deliberation
Post
Poll
Informed opinions assessed post-deliberation
America in One Room (2019)
Challenges

1

Manual Analysis Limitations
20 groups * 6 hrs = 120 hours!

2

Black Box
What actually changes someone's mind?
DeliBERT Model Architecture
BERT Component
Handles tokenization and argument classification.
GPT-4 Component
Responsible for summarization and interpretation.
Pipeline Flow
Data preprocessing → Argument Extraction → Argument Classification → Interpretation.
Training Classification
1
Debate Dataset
US Presidential Debates
2
Speakers
Modify normal training - split by speaker's dialogue
3
Claims / Premises
Split into "claims" and supporting "premises"
Claim Entailment
TOT Mapping
Tokenized Text → Original Text
Global Attribution
ALBERT SQUAD 2.0
ALBERT Tokenization
SQUAD Span Labeling
LCS Mapping
TOT Pipeline
The Span Labeler
The Span Labeler
Problems with SQUAD
  • Most claims are not in the form of questions
  • SQUAD Dataset doesn't contain complex questions
  • SQUAD Dataset is too clean → false positives
GPT Solves!
New Training proc.
Add Synthetic Noise
Unentailed claims
Unattributable!
Results (Ranked Choice)
Strongest For
  • Vote for more than one candidate.
  • Prioritize their choices and ensure that their vote still counts even if their top choice loses.
  • Fresh options outside the 2-party system
Strongest Against
  • #1 pick of most people can actually lose
  • Elects the least objectionable candidate vs. the most popular
  • Takes over a week to determine the outcome
Challenges and Future Work

1

Current Challenges
Dealing with noisy data, long deliberations, and improving argument detection.

2

Future Improvements
Enhancing TOT Mapping algorithm (more efficiency)

3

Expansion Plans
Extending DeliBERT's application to other domains beyond deliberative polling.
Q&A
Thanks for listening!
Made with Gamma