Skip to content

Commit 1f8b635

Browse files
agent-smith-ddpramonperezDDPclaude
authored
MI: fix House votes never scraped due to regex mismatch and tab-separated names (#5696)
Two bugs introduced by #5379 (merged 2025-05-01) that together cause all Michigan House vote records to be silently dropped. Bug 1 — regex mismatch (scrape_votes, line 197): The roll call regex `r"Roll Call # (\d+)"` requires a space between "#" and the number. House journal action strings use "Roll Call #44" (no space); Senate uses "ROLL CALL # 1" (with space). Because `rcmatch` is always None for House actions, the `if rcmatch:` block never executes and no House votes are ever scraped. Fixed with `r"Roll Call #\s*(\d+)"`. Bug 2 — tab-separated voter names (parse_roll_call, line 351): House journal <p> elements separate voter names with single tab characters ("Alexander\tFarhat\tMartus\tSchmaltz"), while Senate journals use multi-space padding. The split `re.split(r"(?<!,)\s{2,}", p)` requires 2+ consecutive whitespace characters, so single tabs go unsplit — all four names on a line are stored as one fused string. Note: a session-specific workaround for the identical tab issue already existed for 2017-2018 but was never generalised. Fixed by changing the split to `r"\t|(?<!,)\s{2,}"`, which handles both formats. Tested against 45 Michigan 2025-2026 bills (38 House, 7 Senate) from a known-missing dataset. 31 House votes recovered with correct counts and fully split individual voter name lists (e.g. HB4707: 57 yeas, 44 nays). Senate bills with no floor votes correctly return 0 votes. Co-authored-by: Ramon Perez <ramon.perez@digitaldemocracyproject.org> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
1 parent 5e405af commit 1f8b635

1 file changed

Lines changed: 4 additions & 3 deletions

File tree

scrapers/mi/bills.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -194,7 +194,8 @@ def scrape_votes(self, bill: Bill, page: lxml.html.HtmlElement):
194194
actor = "upper"
195195

196196
# Process roll call votes
197-
rcmatch = re.search(r"Roll Call # (\d+)", action, re.IGNORECASE)
197+
# House format: "Roll Call #44" (no space); Senate: "ROLL CALL # 1" (space)
198+
rcmatch = re.search(r"Roll Call #\s*(\d+)", action, re.IGNORECASE)
198199
if rcmatch:
199200
rc_num = rcmatch.groups()[0]
200201
vote_url = None
@@ -347,8 +348,8 @@ def parse_roll_call(self, url, rc_num, session):
347348
elif p.startswith("In The Chair:"):
348349
break
349350
elif vtype:
350-
# Split on multiple spaces not preceded by commas
351-
for line in re.split(r"(?<!,)\s{2,}", p):
351+
# Split on tabs (House journals) or multiple spaces (Senate journals)
352+
for line in re.split(r"\t|(?<!,)\s{2,}", p):
352353
if line.strip():
353354
if session == "2017-2018":
354355
for leg in line.split():

0 commit comments

Comments
 (0)