This document reports the results for the BESDUI benchmark for the PepeSearch tool.
| PEPESEARCH | Capability | K (0.2s) | P (1.1s) | H (0.4s) | Operator Count | Time | Task Efficiency |
|---|---|---|---|---|---|---|---|
| Task 1 | 0% | ||||||
| Task 2 | 0% | ||||||
| Task 3 | 100% | 7 | 3 | 2 | 12 | 5.5 | 10.9 |
| Task 4 | 0% | ||||||
| Task 5 | 0% | ||||||
| Task 6 | 0% | ||||||
| Task 7 | 100% | 7 | 2 | 1 | 10 | 4 | 15.0 |
| Task 8 | 0% | ||||||
| Task 9 | 0% | ||||||
| Task 10 | 0% | ||||||
| Task 11 | 0% | ||||||
| Task 12 | 0% | ||||||
| SUM | 200% | 14 | 5 | 3 | 22 | 9.5 | 25.9 |
| AVERAGE | 17% | 7 | 2.5 | 1.5 | 11 | 4.8 | 2.1 |
Task 1. Find products for a given set of features combined
"Look for products of type sheeny with product features stroboscopes and gadgeteers, and a productPropertyNumeric1 greater than 450".
This task cannot be performed because PepeSearch does not allow to set several values for a property. More specifically, it is possible to set the product feature stroboscopes or gadgeteers, but not the two of them at the same time.
Task 2. Find products for a given set of alternative features
"List products of type sheeny with product features stroboscopes OR gadgeteers, and a productPropertyNumeric1 greater than 450".
This task cannot be performed because PepeSearch does not allow to set several values for a property. More specifically, it is possible to set the product feature stroboscopes or gadgeteers, but not the two of them at the same time.
Task 3. Retrieve basic information about a specific product for display purposes
"Get details about product boozed".
| Interaction Steps | K | P | H |
|---|---|---|---|
| 1. Select "Product" in the list of concepts | 1 | 1 | |
| 2. Click on the product label field and type "bo" | 3 | 1 | 1 |
| 3. Select "boozed" from autocomplete | 2 | ||
| 4. Click on the product link "boozed" | 1 | 1 | 1 |
| Total Operators | 7 | 3 | 2 |
| Time | 5.5 |
Task 4. Find products having some specific features and not having one feature
"Look for products of type sheeny with product features stroboscopes but NOT gadgeteers, and productPropertyNumeric1 value greater than 300 and productPropertyNumeric3 smaller than 400".
This task is not currently possible because PepeSearch does not support negation.
Task 5. Find products matching two different sets of features
"Look for products of type sheeny with product features stroboscopes and gadgeteers and a productPropertyNumeric1 value greater than 300 plus those of the same product type with product features stroboscopes and rotifers and a productPropertyNumeric2 greater than 400".
This task cannot be performed because PepeSearch does not allow to set several values for a property. More specifically, it is possible to set the product feature stroboscopes or gadgeteers, but not the two of them at the same time.
Task 6. Find products that are similar to a given product
"Look for products similar to boozed, with at least one shared feature, and a productPropertyNumeric1 value between 427 and 627 (100 more or less than its value for boozed, 527) and a productPropertyNumeric2 value between 545 and 945 (200 more or less than its value for boozed, 745)".
This task cannot be performed because PepeSearch does not allow to set several values for a property.
Task 7. Find products having a name that contains some text
"Search products whose name contains ales".
| Interaction Steps | K | P | H |
|---|---|---|---|
| 1. Select "Product" in the list of concepts | 1 | 1 | |
| 2. Click on the product label field and type "ales" | 5 | 1 | 1 |
| 3. Press enter to get results | 1 | ||
| Total Operators | 7 | 2 | 1 |
| Time | 4 |
Task 8. Retrieve in-depth information about a specific product including offers and reviews
"For the product waterskiing sharpness horseshoes list details for all its offers by Chinese vendors and still valid by 2008-05-28 plus details for all reviews for this product having either rating1 or rating2".
This task cannot be solved with PepeSearch due to:
- Countries are filtered out because they do not have a type in the dataset
- It is not possible to specify an OR clause for a datatype property value
Task 9. Give me recent reviews in English for a specific product
"For the product waterskiing sharpness horseshoes list the 20 more recent reviews in English".>
PepeSearch does not provide a mechanism to filter literals by language.
Task 10. Get information about a reviewer
"Get all available information about the author of Review5481".
The label "Review5481" is included in the URI of a review, but there is no other data property value with this information. Since PepeSearch does not allow searches in resource URIs, the task cannot be fulfilled.
Task 11. Get offers for a given product which fulfill specific requirements
"Look for the cheapest and still valid by 2008-06-15 offer for the product waterskiing sharpness horseshoes by a US vendor that is able to deliver within 3 days".
This task cannot be solved with PepeSearch because the countries of the dataset are filtered out (as they do not have a specific type).
Task 12. Export the chosen offer into another information system which uses a different schema
"Save in the local computer the information about the vendor for Offer3499, this is half the task. To complete it, restrict the output to just label, homepage and country and map them to schema.org terms name, url and nationality".
PepeSearch does not provide a way to map data and export it using a different schema.