[Bug]: A ; seems to be being added to the custom metadata search

[alexander-paterson]

What happened?

I'm using abs-tract to add in extra metadata providers. The search being sent to abs-tract appears to have a ; before the word query.

I created a but report in abs-tract's GitHub and the dev said it appears to be an ABS issue and to create a bug here.

What did you expect to happen?

After adding a metadata provider through abs-tract I should be able to get search results.

Steps to reproduce the issue

1. Create a metadata provider pointing to abs-tract
2. Perform a search, no results are returned
3. Check the logs and you can see a ; before the word query

Audiobookshelf version

2.19.3

How are you running audiobookshelf?

Docker

What OS is your Audiobookshelf server hosted from?

Linux

If the issue is being seen in the UI, what browsers are you seeing the problem on?

None

Logs

2025-02-05T21:34:37.079432255Z INFO Response: 200 OK service: "abs-tract" httpRequest: {url: "http://192.168.8.219:5555/kindle/us/search?mediaType=book&;query=Die+Trying&author=Lee+Child" method: "GET" path: "/kindle/us/search" remoteIP: "172.23.0.1:59900" proto: "HTTP/1.1" requestID: "5c2b925ac873/rZ2rQH0VgA-000001" scheme: "http" header: {accept: "application/json, text/plain, */*" user-agent: "axios/0.27.2" connection: "keep-alive"}} httpResponse: {status: 200 bytes: 3511 elapsed: 3524.220387}

Additional Notes

No response

[advplyr]

I'm not able to reproduce this. Do you get this result when you use other search text?

[alexander-paterson]

Okay, this is weird.

I have Goodreads, Kindle US and Kindle UK set up in abs-tract

Looking at the logs, they all have the same ; in the URL sent to abs-tract

2025-02-20T09:10:29.704551625Z INFO Response: 200 OK service: "abs-tract" httpRequest: {url: "http://192.168.8.219:5555/kindle/uk/search?mediaType=book&;query=Cujo&author=Stephen+King" method: "GET" path: "/kindle/uk/search" remoteIP: "172.23.0.1:48080" proto: "HTTP/1.1" requestID: "c70c485fc52f/6Wt7NidunR-000001" scheme: "http" header: {connection: "keep-alive" accept: "application/json, text/plain, */*" user-agent: "axios/0.27.2"}} httpResponse: {status: 200 bytes: 2111 elapsed: 2909.317652} 
2025-02-20T09:11:54.665493131Z INFO Response: 200 OK service: "abs-tract" httpRequest: {url: "http://192.168.8.219:5555/kindle/us/search?mediaType=book&;query=Cujo&author=Stephen+King" method: "GET" path: "/kindle/us/search" remoteIP: "172.23.0.1:33950" proto: "HTTP/1.1" requestID: "c70c485fc52f/6Wt7NidunR-000002" scheme: "http" header: {accept: "application/json, text/plain, */*" user-agent: "axios/0.27.2" connection: "keep-alive"}} httpResponse: {status: 200 bytes: 3544 elapsed: 1027.499949} 
2025-02-20T09:12:00.877708181Z INFO Response: 200 OK service: "abs-tract" httpRequest: {url: "http://192.168.8.219:5555/goodreads/search?mediaType=book&;query=Cujo&author=Stephen+King" method: "GET" path: "/goodreads/search" remoteIP: "172.23.0.1:33950" proto: "HTTP/1.1" requestID: "c70c485fc52f/6Wt7NidunR-000003" scheme: "http" header: {accept: "application/json, text/plain, */*" user-agent: "axios/0.27.2" connection: "keep-alive"}} httpResponse: {status: 200 bytes: 11087 elapsed: 2887.215947}

The two Kindle ones don't return any results in ABS, but Goodreads does.

If I take all three URL's, copy them and remove the ; then they all return results.

[ahobsonsayers]

Hey! Yeah I am also having problems replicating this with ABS sadly 😞

The reason I think this is a (possible) ABS bug rather than to do with abs-tract, is that the lines printed are simply logs of exactly the requests made to the abs-stract server - which in theory means if that is what is being logged, the request is incorrectly being made.

Worth checking (although I believe it should have no impact), you mention you are running ABS using docker, and I assume you are running abs-tract using docker, but in the logs I see an IP address being used rather than a container name to connect them. Are you using a host network?

[alexander-paterson]

The logs I posted were from abs-tract, this is what I see in ABS's logs

[2025-02-21 09:41:55.484] ERROR: Error searching Custom provider 'custom-4ef706d2-7f61-4f50-ab79-60dc94f66ea0': Error: sanitizeHtml: input must be a string
    at Object.sanitize (/server/utils/htmlSanitizer.js:12:11)
    at /server/providers/CustomProviderAdapter.js:78:36
    at Array.map (<anonymous>)
    at CustomProviderAdapter.search (/server/providers/CustomProviderAdapter.js:70:20)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async BookFinder.getCustomProviderResults (/server/finders/BookFinder.js:207:21)
    at async ApiRouter.findBooks (/server/controllers/SearchController.js:37:21)
[2025-02-21 09:42:22.762] ERROR: Error searching Custom provider 'custom-460512ec-f1bf-410a-a1d2-c77597f3040f': Error: sanitizeHtml: input must be a string
    at Object.sanitize (/server/utils/htmlSanitizer.js:12:11)
    at /server/providers/CustomProviderAdapter.js:78:36
    at Array.map (<anonymous>)
    at CustomProviderAdapter.search (/server/providers/CustomProviderAdapter.js:70:20)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async BookFinder.getCustomProviderResults (/server/finders/BookFinder.js:207:21)
    at async ApiRouter.findBooks (/server/controllers/SearchController.js:37:21)

[nichwall]

Is it possible that something between ABS and your abs-tract instance is changing the request to include a semicolon? In doing some research, it looks like older versions of reverse proxies and some server implementations may use a semicolon as the delimiter instead of an ampersand, so could be something rewriting the path?

https://www.w3.org/TR/html4/appendix/notes.html#h-B.2.2

If I take all three URL's, copy them and remove the ; then they all return results.

How did you test this? Was this through a browser or using something like curl?
curl -v "http://192.168.8.219:5555/kindle/uk/search?mediaType=book&query=Cujo&author=Stephen+King"

Either way, we should add logging of the actual query that ABS is generating to match the other provider debug printouts. I just opened #4034 to add this logging.

[alexander-paterson]

I just copied the URL, pasted it into the browser, removed the ; and pressed enter. I then got a json response in my browser.

I'm not aware of anything that would mess with the request. I'm just confused as to why the Goodreads one works but the two Kindle ones don't.

They were working at one point back when I initially set up abs-tract a few months ago.

[Vito0912]

I just tested it with both Goodreads and Kindle using
https://provider.vito0912.de/kindle/uk and https://localhost:5555/kindle/uk, neither of which sent a ";".

I also got the error:

Error searching Custom provider 'custom-f2e3bf15-3ebd-4258-b724-f65d058c8e4f': Error: sanitizeHtml: input must be a string at Object.sanitize (/server/utils/htmlSanitizer.js:12:11) at /server/providers/CustomProviderAdapter.js:78:36 at Array.map (<anonymous>) at CustomProviderAdapter.search (/server/providers/CustomProviderAdapter.js:70:20) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async BookFinder.getCustomProviderResults (/server/finders/BookFinder.js:207:21) at async ApiRouter.findBooks (/server/controllers/SearchController.js:37:21)

I assume this is because my provider returns a 500 error most of the time for Kindle (because Amazon is Amazon). Goodreads works. However, when I enabled logging in my proxy, I saw that it sent the correct data, and since other custom providers work with parsing, I assume this will work too.

How did you add the provider in ABS? Perhaps there is a character that ABS does not expect which causes it to panic.

[alexander-paterson]

This is what I have in ABS

[Vito0912]

If you want, you could try using https://provider.vito0912.de/kindle/uk or https://provider.vito0912.de/goodreads with abs as the Auth value? I will enable logging/access log and check if ; is sent. Then we at least could sort out if this is something with your setup of abs-tract or/and abs.
Kindle will most likley not return anything

[alexander-paterson]

Just added and tried. Kindle didn't work (as you said) and Goodreads did.

[Vito0912]

Here are the results:
/uk/search?mediaType=book&query=Cujo&author=Stephen+King
/search?mediaType=book&query=Cujo&author=Stephen+King
the provider is stripped from the logs, because of technical reasons
LGTM, so probably some issue with local networking or the abs-tract image you are running?