Skip to content

Commit eedb8b3

Browse files
committed
Generic fallback: don't append a country to a street address that skipped the city
A schema.org PostalAddress that carries a streetAddress but leaves addressLocality empty has a hole in its hierarchy, and the generic location composer then jumped straight from the street to the country: "City Museum Tel Aviv-Yafo, Bialik Street 27, Israel". That names no findable place — the country adds nothing once you are at street level without a city — and it is exactly what the dedicated source omits. So append the country only when nothing finer already stands in for it: no addressRegion (the existing rule), and either the address named its city or it gave no street at all, leaving the country the most specific thing we have. A complete street/city/country hierarchy (livenation.de's shape) and a bare city/country address both keep the country, unchanged. Generic: keys off the schema.org PostalAddress convention, not a host. The city-skipping shape is what ASP.NET/SharePoint CMSes emit when the venue record has no city field, so any such site benefits. Lifts both visit.tel-aviv.gov.il fallback locations from a diff to an exact match; no other corpus case changes and no field regresses. Fallback-coverage: criticalFieldsPct 75.2->77.1, allFieldsPct 55.9->56.9 (location matches 22->24). The #728 control-chars test asserted the same city-skipping page's composed location; its expected value moves to the new composition. That assertion is a proxy for "the tolerant parse succeeded" (a value assembled from separate JSON-LD fields, so it cannot leak from the raw <script> text), and it stays an exact multi-field equality assertion — just as strong. Refs #366 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018975YrYb81uQmAvG315fTE
1 parent 1f9860f commit eedb8b3

4 files changed

Lines changed: 67 additions & 14 deletions

File tree

dev/requirements/extractor/fallback/fallback-coverage.GENERATED.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,8 @@ Headline coverage over all 35 cases in the corpus:
1212

1313
| Metric | Coverage | Hits / gradeable |
1414
| --- | --: | --: |
15-
| **Critical fields** (title + start + location) | **75.2%** | 79 / 105 |
16-
| **All fields** | **55.9%** | 114 / 204 |
15+
| **Critical fields** (title + start + location) | **77.1%** | 81 / 105 |
16+
| **All fields** | **56.9%** | 116 / 204 |
1717
| Event coverage *(informational)* | 25.3% | 41 / 162 |
1818

1919
### Gate
@@ -22,8 +22,8 @@ The gate (`dev/requirements/extractor/fallback/fallback-coverage.baseline.GENERA
2222

2323
| Metric | Watermark | Current (shared) | |
2424
| --- | --: | --: | :-: |
25-
| Critical fields | 75.2% | 75.2% ||
26-
| All fields | 55.9% | 55.9% ||
25+
| Critical fields | 77.1% | 77.1% ||
26+
| All fields | 56.9% | 56.9% ||
2727

2828
Gated over **35** shared case(s).
2929

@@ -36,7 +36,7 @@ Event coverage is reported but **not gated** (a few listing pages the fallback c
3636
| `title` **(critical)** | 35 | 30 | 3 | 2 | 85.7% |
3737
| `start` **(critical)** | 35 | 27 | 6 | 2 | 77.1% |
3838
| `end` | 23 | 15 | 1 | 7 | 65.2% |
39-
| `location` **(critical)** | 35 | 22 | 6 | 7 | 62.9% |
39+
| `location` **(critical)** | 35 | 24 | 4 | 7 | 68.6% |
4040
| `ctz` | 32 | 12 | 0 | 20 | 37.5% |
4141
| `eventLengthInMinutes` | 11 | 0 | 0 | 11 | 0% |
4242
| `description` | 33 | 8 | 18 | 7 | 24.2% |
@@ -67,7 +67,7 @@ Event coverage is reported but **not gated** (a few listing pages the fallback c
6767
| `tel-aviv.gov.il` | 1 | 1/1 | 66.7% | 60% |
6868
| `thinkdrink.co.il` | 1 | 1/1 | 100% | 60% |
6969
| `ticketmaster.co.il` | 1 | 1/1 | 100% | 80% |
70-
| `visit.tel-aviv.gov.il` | 2 | 2/2 | 50% | 41.7% |
70+
| `visit.tel-aviv.gov.il` | 2 | 2/2 | 83.3% | 58.3% |
7171

7272
## By exemplar
7373

@@ -108,5 +108,5 @@ Legend: ✓ match · ~ different value · ✗ missing (source had it, fallback d
108108
| `telavivcinematheque-taiwan-week` | 1/6 | ~ |||||| ~ |
109109
| `thinkdrink-quantum-lecture` | 1/1 ||||||| ~ |
110110
| `ticketmaster-ravid-plotnik` | 1/1 ||||||||
111-
| `visit-tel-aviv-timed` | 1/1 |||| ~ ||||
112-
| `visit-tel-aviv` | 1/1 || ~ | ~ | ~ ||| ~ |
111+
| `visit-tel-aviv-timed` | 1/1 |||| ||||
112+
| `visit-tel-aviv` | 1/1 || ~ | ~ | ||| ~ |

dev/requirements/extractor/fallback/fallback-coverage.baseline.GENERATED.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"_comment": "High-watermark gate for the generic fallback extractor's coverage, asserted by fallback-coverage.test.js and explained in dev/requirements/extractor/fallback/fallback-coverage.GENERATED.md. criticalFieldsPct/allFieldsPct are the best field-match percentages reached against the dedicated sources, and cases is the list they were computed over. The gate compares the current run to these over the SHARED cases, so adding a new case never fails it; the watermark ratchets up on an unchanged case set and re-anchors to the current aggregate when the set changes. Re-baseline by hand (lower a number) only for a deliberate, reviewed drop.",
3-
"criticalFieldsPct": 75.2,
4-
"allFieldsPct": 55.9,
3+
"criticalFieldsPct": 77.1,
4+
"allFieldsPct": 56.9,
55
"cases": [
66
"bandsintown-berry-sakharof",
77
"barby",

extension-test/event-extractors/extraction.test.js

Lines changed: 48 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,48 @@ test("JSON-LD location: a Place name holding only the city (not the street) stil
140140
assert.equal(e.times[0].location, "Zappa Tel Aviv - Midtown, Menachem Begin Road 144, 6492102");
141141
});
142142

143+
test("JSON-LD location: a country is not appended to a street address that skipped the city", () => {
144+
// A PostalAddress that carries a street but leaves addressLocality EMPTY has a
145+
// hole in its hierarchy, and the country then jumps a level: "Bialik Street 27,
146+
// Israel" names no findable place, and a dedicated source writes the street
147+
// without it. So the country is appended only when the address didn't skip the
148+
// city (or when the country is all we have) — the same "don't tack on a part
149+
// that adds nothing locatable" reasoning that drops a country code after a
150+
// region. Mirrors visit.tel-aviv.gov.il's JSON-LD.
151+
const html = `
152+
<script type="application/ld+json">
153+
{ "@type": "Event", "name": "Savta Stories", "startDate": "2026-07-14T10:00:00",
154+
"location": { "@type": "Place", "name": "City Museum Tel Aviv-Yafo",
155+
"address": { "@type": "PostalAddress", "streetAddress": "Bialik Street 27",
156+
"addressLocality": "", "addressCountry": "Israel" } } }
157+
</script>`;
158+
159+
const e = firstEvent(html, "https://www.somecity.example/events/savta-stories");
160+
assert.equal(e.times[0].location, "City Museum Tel Aviv-Yafo, Bialik Street 27");
161+
});
162+
163+
test("JSON-LD location: a country IS kept when the address names its city too", () => {
164+
// The guard for the rule above: a complete street/city/country hierarchy is
165+
// exactly the case the country belongs in (livenation.de's shape), and a
166+
// city-and-country address with no street keeps it as well.
167+
const withCity = `
168+
<script type="application/ld+json">
169+
{ "@type": "Event", "name": "Harbour Show", "startDate": "2026-08-01T20:00:00",
170+
"location": { "@type": "Place", "name": "Harbour Hall",
171+
"address": { "streetAddress": "12 Dock Rd", "addressLocality": "Cork",
172+
"addressCountry": "Ireland" } } }
173+
</script>`;
174+
assert.equal(firstEvent(withCity, "https://www.x.example/a").times[0].location, "Harbour Hall, 12 Dock Rd, Cork, Ireland");
175+
176+
const cityOnly = `
177+
<script type="application/ld+json">
178+
{ "@type": "Event", "name": "City Fest", "startDate": "2026-08-02T20:00:00",
179+
"location": { "@type": "Place", "name": "Somewhere Central",
180+
"address": { "addressLocality": "Cork", "addressCountry": "Ireland" } } }
181+
</script>`;
182+
assert.equal(firstEvent(cityOnly, "https://www.x.example/b").times[0].location, "Somewhere Central, Cork, Ireland");
183+
});
184+
143185
test("JSON-LD location: an addressCountry Country object (not a plain string) contributes its name, not '[object Object]'", () => {
144186
// schema.org allows addressCountry to be a Country object ({ "@type":
145187
// "Country", "name": "..." }) rather than a plain string — seen on
@@ -183,6 +225,11 @@ test("JSON-LD with raw (unescaped) control chars inside a string value is still
183225
// can never leak out of the raw <script> text as one contiguous string — is the
184226
// assertion that pins the parse actually succeeding. The literal newline below
185227
// is intentional (it is the defect being tolerated).
228+
//
229+
// This page's address is also the city-skipping shape (a streetAddress with no
230+
// addressLocality), so the composed location stops at the street instead of
231+
// tacking on the country — see the country/city tests above. Still two separate
232+
// JSON-LD fields joined together, so it pins the parse just as tightly.
186233
const html = `
187234
<script type="application/ld+json">
188235
{ "@context": "http://www.schema.org", "@type": "Event",
@@ -200,7 +247,7 @@ Second line, after a raw newline.",
200247
assert.equal(e.title, "Heritage Talk");
201248
assert.equal(e.times[0].start, "2026-07-05T18:00:00");
202249
assert.equal(e.times[0].end, "2026-07-05T20:00:00");
203-
assert.equal(e.times[0].location, "City Museum, Bialik St 27, Israel");
250+
assert.equal(e.times[0].location, "City Museum, Bialik St 27");
204251
assert.ok(
205252
e.description.includes("First line") && e.description.includes("Second line"),
206253
"both lines of the raw-newline description survive"

extension/event-extractors/helpers/embedded-events.js

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -160,9 +160,15 @@ globalThis.GCal = Object.assign(globalThis.GCal || {}, (() => {
160160
p.add(addr);
161161
} else if (addr && typeof addr === "object") {
162162
p.add(addr.streetAddress).add(addr.addressLocality).add(addr.addressRegion).add(addr.postalCode);
163-
// A country code after a region is noise ("..., NY, us"); keep the
164-
// country only when it's the most specific thing we have.
165-
if (!addr.addressRegion) p.add(countryName(addr.addressCountry));
163+
// A country code after a region is noise ("..., NY, us"), and so is a
164+
// country bolted straight onto a street whose city the address left out:
165+
// "Bialik Street 27, Israel" skips a level of the hierarchy and names no
166+
// findable place, which is why a dedicated source writes the street alone.
167+
// So append the country only when nothing finer already stands in for it —
168+
// no region, and either the address named its city or it gave no street at
169+
// all (leaving the country the most specific thing we have).
170+
const skipsCity = GCal.clean(addr.streetAddress) && !GCal.clean(addr.addressLocality);
171+
if (!addr.addressRegion && !skipsCity) p.add(countryName(addr.addressCountry));
166172
}
167173
return p.join();
168174
}

0 commit comments

Comments
 (0)