Do you find papers only for mentions in the text, or also for mentions in links?

[bockthom]

Do you find papers only for mentions in the text (i.e., a string separated by spaces), or also for mentions in links?

I do not have an example as I don't have a paper in mind that mentions software only in the URL but not in the text as a separate word.

But for exmaple, would softcite detect mentions like the following ones:

https://github.com/tidyverse/ggplot2
https://github.com/tidyverse/ggplot2/issues

In both cases, ggplot2 is not a separate string, but just part of an URL.

Not sure what the proper way of handling would be, though–handling it as a mention or not.

[jameshowison]

I do not think that we would find software within links. I'll be on the lookup for an example paper to try (could generate one using LaTeX).

[shaunak-pusalkar]

I investigated whether Softcite detects software mentions solely from URLs or primarily from running text.

At the dataset level, ~278k (~0.5% of total 54.8M)mentions co-occur with URLs, and ~4.8k (~0.009%) mentions appear “URL-only” within their local context windows. However, a detailed inspection of a representative case (ggplot/ggplot2) shows that the sole URL-only ggplot instance (found in PAPER ID: 21979332) occurs in a paper that contains multiple clear textual ggplot mentions elsewhere.

The URL-only classification in this case is probably due to context window boundaries (the textual mention falls outside the extracted window), not because the software was detected from a URL alone. This single instance does not provide strong evidence on its own that URLs are systematically used as the primary trigger for software extraction.

I think this suggests that Softcite is generally not extracting software solely from URLs, though software names may co-occur within URLs and context windows. Based on the current evidence, software mentions in Softcite generally co-occur with explicit textual mentions, while cases classified as “URL-only” appear to be rare and often ambiguous at the level of extracted context windows.

Importantly, the parquet representation does not preserve token-level provenance (e.g., whether a mention was triggered by running text versus a URL). Determining whether URLs can independently trigger extraction would therefore require inspection of upstream extraction artifacts, which are not exposed in the parquet files.

A detailed analysis of URL-only cases is required to confirm whether any true URL-only detections exist.