Skip to content

A full-width percent sign is a percentage, so a "12.5%" object becomes a value instead of an entity named 12.5% - #1440

Open
rudycelekli wants to merge 1 commit into
deeplethe:devfrom
rudycelekli:extract-full-width-percent
Open

rudycelekli wants to merge 1 commit into
deeplethe:devfrom
rudycelekli:extract-full-width-percent

Conversation

@rudycelekli

Copy link
Copy Markdown
Contributor

Why

values_are_not_things moves an object that is really a value ("5000万元", "52%", "2015年3月12日") back into the value slot, so the open extraction does not create an entity named after a figure. It decides with literal_object, which asks scan_quantity whether the whole name is one quantity. scan_quantity only recognizes the ASCII percent sign. Chinese typesetting very often uses the full-width one: "增长12.5%", "毛利率为38.6%". For "12.5%" the scan leaves % as an unknown trailing word, the strict check rejects it, and literal_object says it is not a value.

So when the model writes "12.5%" as the object (the case this function exists for), it stays an object. The statement becomes a relation to an entity called "12.5%" instead of a percentage, and it can never bind to the numeric attribute it states.

What changes

scan_quantity strips either % or % as the percent suffix. A full-width percentage now reads exactly like the ASCII one, with the unit %. Nothing else in the scan changes.

How it was checked

  • A new test, a_full_width_percent_sign_is_a_percentage, checks that literal_object("12.5%", true) is true, that parse_quantity("12.5%") is (12.5, "%"), that "-3.2 %" reads like "-3.2%", and that the value under a % unit is still 12.5.
  • With the fix reverted the test fails on literal_object("12.5%", true).
  • cargo test -p utopia-extract --lib: 113 passed.
  • cargo fmt -p utopia-extract --check and cargo clippy -p utopia-extract --all-targets -- -D warnings pass.

Before review

  • Every commit is signed off (git commit -s)
  • cargo fmt --all --check, cargo clippy and cargo test pass for the touched crate

…s a value instead of an entity named 12.5%

Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant