[Bug] SEC 13F-HR: & silently dropped from issuer names ("S&P500" → "SP500")
Describe the bug
obb.equity.ownership.form_13f (provider sec) silently corrupts every nameOfIssuer / titleOfClass containing an ampersand:
| EDGAR source XML |
OpenBB output |
S&P500 EQL WGT |
SP500 EQL WGT |
Ishares S&P Gsci Commodity- |
Ishares SP Gsci Commodity- |
BABCOCK & WILCOX ENTERPRISES |
BABCOCK WILCOX ENTERPRISES (double space) |
No error is raised — the corrupted names flow straight into downstream consumers. Any filer holding S&P ETFs (very common) is affected every quarter.
Root cause
parse_13f_hr in openbb_sec/utils/parse_13f.py feeds the entire Complete Submission TXT file to BeautifulSoup(filing, "xml") (~L137). That file is SGML-wrapped (<SEC-DOCUMENT>, <DOCUMENT>, <TYPE> tags...) and is not well-formed XML, so lxml falls back to recover mode — and recover mode drops character entities such as & entirely.
The proof: extracting only the embedded well-formed <XML>...</XML> block and feeding that to the same parser preserves all entities perfectly.
How to reproduce
from openbb import obb
res = obb.equity.ownership.form_13f(symbol="1536411", date="2026-03-31", provider="sec")
# Duquesne holds several S&P ETFs; every one of them comes back as "SP..." / "S P..."
[r.issuer for r in res.results if "S&P" in (r.issuer or "")] # == [] (should be non-empty)
Suggested fix
In parse_13f_hr, after downloading the complete submission, reassemble the embedded well-formed XML blocks (form header + information table) under a synthetic root before parsing, instead of souping the raw SGML:
xml_blocks = re.findall(r"<XML>(.*?)</XML>", filing, re.DOTALL | re.IGNORECASE)
if xml_blocks:
decl = re.compile(r"<\?xml[^>]*\?>")
filing = "<root>" + "".join(decl.sub("", b) for b in xml_blocks) + "</root>"
soup = BeautifulSoup(filing, "xml")
Inputs that are already bare XML (no <XML> wrapper) pass through unchanged. parse_header / get_period_ending keep working because the header block is included under the synthetic root.
Verified locally against openbb-sec 1.6.6/1.6.7 (the affected lines are identical in main): a 70-position filing round-trips with all & intact and sum(weight) == 1.0; empty filings and options positions (putCall) unaffected.
Happy to open a PR (together with the one-line fix for the empty-filing weight ValidationError — see companion issue).
[Bug] SEC 13F-HR:
&silently dropped from issuer names ("S&P500" → "SP500")Describe the bug
obb.equity.ownership.form_13f(providersec) silently corrupts everynameOfIssuer/titleOfClasscontaining an ampersand:S&P500 EQL WGTSP500 EQL WGTIshares S&P Gsci Commodity-Ishares SP Gsci Commodity-BABCOCK & WILCOX ENTERPRISESBABCOCK WILCOX ENTERPRISES(double space)No error is raised — the corrupted names flow straight into downstream consumers. Any filer holding S&P ETFs (very common) is affected every quarter.
Root cause
parse_13f_hrinopenbb_sec/utils/parse_13f.pyfeeds the entire Complete Submission TXT file toBeautifulSoup(filing, "xml")(~L137). That file is SGML-wrapped (<SEC-DOCUMENT>,<DOCUMENT>,<TYPE>tags...) and is not well-formed XML, so lxml falls back to recover mode — and recover mode drops character entities such as&entirely.The proof: extracting only the embedded well-formed
<XML>...</XML>block and feeding that to the same parser preserves all entities perfectly.How to reproduce
Suggested fix
In
parse_13f_hr, after downloading the complete submission, reassemble the embedded well-formed XML blocks (form header + information table) under a synthetic root before parsing, instead of souping the raw SGML:Inputs that are already bare XML (no
<XML>wrapper) pass through unchanged.parse_header/get_period_endingkeep working because the header block is included under the synthetic root.Verified locally against openbb-sec 1.6.6/1.6.7 (the affected lines are identical in
main): a 70-position filing round-trips with all&intact andsum(weight) == 1.0; empty filings and options positions (putCall) unaffected.Happy to open a PR (together with the one-line fix for the empty-filing
weightValidationError — see companion issue).