Issue Report - ncore - Fix requested

[kriziw]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Type {Torrent|Usenet}: Torrent
Tracker {Name}: Ncore
URL {URL}: ncore.pro

Additional Information:

The current implementation of the ncore indexer integration is using a pull of the /browse.php path and using CSS selectors to get the content. This is triggering a rate limiting on the ncore platform which results in the indexer becoming unavailble (429:TooManyRequests).

Expected Behavior

The recommended fix is to move to a JSON-pull via https://ncore.pro/torrents.php?jsons=true to overcome this.

Note: ncore support staff has been contacted - issue resolution is suggested by ncore technical team

Steps To Reproduce

No response

Environment

- OS: Debian 13
- Prowlarr: Latest
- Docker Install: Yes
- Using Reverse Proxy: only for local access
- Browser: Chrome

What branch are you running?

Master

Trace Logs? Not Optional

prowlarr.txt

prowlarr.debug.txt

Trace Logs have been provided as applicable. Reports will be closed if the required logs are not provided.

• I have read and followed the steps in the wiki link above and provided the required trace logs - the logs contain trace - that are relevant and show this issue.

[kriziw]

prowlarr.trace.txt

ADDED TRACE**

[ilike2burnthing]

ncore.yml

---
id: ncore
name: nCore
description: "nCore is a HUNGARIAN Private Torrent Tracker for MOVIES / TV / GENERAL"
language: hu-HU
type: private
encoding: UTF-8
links:
  - https://ncore.pro/
legacylinks:
  - https://ncore.cc/

caps:
  categorymappings:
    # E-book
    - {id: ebook, cat: Books, desc: "E-book/Eng"}
    - {id: ebook_hun, cat: Books, desc: "E-book/Hun"}
    # Játék
    - {id: console, cat: Console, desc: "Játék/Console"}
    - {id: game_iso, cat: PC/Games, desc: "Játék/ISO"}
    - {id: game_rip, cat: PC/Games, desc: "Játék/Rip"}
    # Filmek
    - {id: xvid_hun, cat: Movies/SD, desc: "Film/SD/Hun"}
    - {id: xvid, cat: Movies/SD, desc: "Film/SD/Eng"}
    - {id: dvd_hun, cat: Movies/DVD, desc: "Film/DVD/Hun"}
    - {id: dvd, cat: Movies/DVD, desc: "Film/DVD/Eng"}
    - {id: dvd9_hun, cat: Movies/DVD, desc: "Film/Dvd9/Hun"}
    - {id: dvd9, cat: Movies/DVD, desc: "Film/Dvd9/Eng"}
    - {id: hd_hun, cat: Movies/HD, desc: "Film/HD/Hun"}
    - {id: hd, cat: Movies/HD, desc: "Film/HD/Eng"}
    # Sorozatok
    - {id: xvidser, cat: TV/SD, desc: "Sorozat/SD/Eng"}
    - {id: xvidser_hun, cat: TV/SD, desc: "Sorozat/SD/Hun"}
    - {id: dvdser, cat: TV/Other, desc: "Sorozat/DVDR/Eng"}
    - {id: dvdser_hun, cat: TV/Other, desc: "Sorozat/DVDR/Hun"}
    - {id: hdser, cat: TV/HD, desc: "Sorozat/HD/Eng"}
    - {id: hdser_hun, cat: TV/HD, desc: "Sorozat/HD/Hun"}
    # Zene
    - {id: mp3_hun, cat: Audio/MP3, desc: "Zene/MP3/Hun"}
    - {id: mp3, cat: Audio/MP3, desc: "Zene/MP3/Eng"}
    - {id: lossless_hun, cat: Audio/MP3, desc: "Zene/Lossless/Hun"}
    - {id: lossless, cat: Audio/MP3, desc: "Zene/Lossless/Eng"}
    - {id: clip, cat: Audio/Video, desc: "Zene/Klip"}
    # Program
    - {id: iso, cat: PC/ISO, desc: "Win/Program/ISO"}
    - {id: misc, cat: PC/0day, desc: "Win/Program/Rip"}
    - {id: mobil, cat: PC/Mobile-Other, desc: "Win/Program/Rip"}
    # Felnőtt tartalom
    - {id: xxx_xvid, cat: XXX/SD, desc: "XXX/Film/SD"}
    - {id: xxx_dvd, cat: XXX/DVD, desc: "XXX/DVDR"}
    - {id: xxx_hd, cat: XXX/UHD, desc: "XXX/HD"}
    - {id: xxx_imageset, cat: XXX/ImageSet, desc: "XXX/Kép"}

  modes:
    search: [q]
    tv-search: [q, season, ep, imdbid, genre]
    movie-search: [q, imdbid, genre]
    music-search: [q, genre]
    book-search: [q, genre]

settings:
  - name: username
    type: text
    label: Username
  - name: password
    type: password
    label: Password
  - name: 2facode
    type: text
    label: 2FA code
  - name: info_2fa
    type: info
    label: "About 2FA code"
    default: "Only fill in the <b>2FA code</b> box if you have enabled <b>2FA</b> on the nCore Web Site. Otherwise just leave it empty."
  - name: info_tpp
    type: info
    label: Results Per Page
    default: For best results, change the <b>Torrent oldalanként:</b> setting to <b>100</b> on your account profile.
  - name: info_activity
    type: info
    label: Account Inactivity
    default: "Anyone who does not access the site for 8 weeks from registration or has been more than 8 weeks since their last stay will be automatically suspended. Anyone who does not log in (stays inactive) for 12 weeks after registration or after their last visit will be removed automatically."

login:
  method: post
  path: login.php
  inputs:
    nev: "{{ .Config.username }}"
    pass: "{{ .Config.password }}"
    2factor: "{{ .Config.2facode }}"
    ne_leptessen_ki: "1"
    set_lang: "en"
    submitted: "1"
  error:
    - selector: div#hibauzenet table tbody tr
  test:
    path: index.php
    selector: a[href*="profile.php"]

search:
  paths:
    - path: torrents.php
      response:
        type: json

  inputs:
    mire: "{{ if .Query.IMDBID }}{{ .Query.IMDBID }}{{ else }}{{ .Keywords }}{{ end }}"
    miben: "{{ if .Query.IMDBID }}imdb{{ else }}name{{ end }}"
    tipus: "{{ if .Categories }}kivalasztottak_kozott{{ else }}all_own{{ end }}"
    submit.x: 1
    submit.y: 1
    tags: "{{ .Query.Genre }}"
    $raw: "{{ range .Categories }}&kivalasztott_tipus[]={{.}}{{end}}"
    jsons: true

  rows:
    selector: results

  fields:
    category:
      selector: category
    title:
      selector: release_name
    details:
      selector: details_url
    download:
      selector: download_url
    imdbid:
      selector: imdb_id
    size:
      selector: size
    seeders:
      selector: seeders
    leechers:
      selector: leechers
    date:
      text: now
    downloadvolumefactor:
      text: 0
    uploadvolumefactor:
      text: 1
    minimumratio:
      text: 1.0
# MST is 48 hours + 24 minutes (.4 hours) per GB of downloaded size
#    minimumseedtime:
#      # 2 days (as seconds = 2 x 24 x 60 x 60)
#      text: 172800
    description_imdb:
      selector: imdb_rating
      filters:
        - name: replace
          args: ["0", ""]
    description:
      text: "{{ .Result.description_imdb }}"
      filters:
        - name: prepend
          args: "{{ if .Result.description_imdb }}IMDB: {{ else }}{{ end }}"
# json engine n/a

Only things missing are the date, whether the torrent has been checked/verified, and the alternate title. If you're in contact with the staff already, can you ask them to at least add the date? Thanks.

[kriziw]

Hey @ilike2burnthing ,
Just to make sure I understand you correctly;
In the JSON response you are not getting the above mentioned 3 values as a response, and those would be needed (at minimum the date) to make it work via JSON query?
Thanks!

[ilike2burnthing]

JSON result example:

    {
      "torrent_id": "123456",
      "category": "hdser",
      "release_name": "Title.Here.S01.1080p.WEB.h264-RlsGrp",
      "details_url": "https://ncore.pro/torrents.php?action=details&id=123456",
      "download_url": "https://ncore.pro/torrents.php?action=download&id=123456&key=1234567890abcdef",
      "freeleech": true,
      "imdb_id": "tt12345678",
      "imdb_rating": "0",
      "size": "5599679454",
      "type": "show",
      "leechers": "16",
      "seeders": "1"
    },

Compared to what we are currently using from the HTML, the JSON is missing 3 pieces of information (with date being the vital one).

Date:

    date:
      selector: div.box_feltoltve2

Alternate title:

    description_alt:
      selector: div.siterank > span
      attribute: title

Checked/verified:

    description:
      case:
        div.torrent_ok: "Verified"
        div.torrent_unchecked: "Unverified"

I can technically make the change without the date, but all results will show as being 0 minutes old, which isn't particularly useful.

[ilike2burnthing]

@kriziw any update?

[kriziw]

Hi @ilike2burnthing ,
I have asked the dev team for a response, and asked them to directly contributre/update here. Please leave it open/pending - as it's depending on them implementing updates in their JSON response.
Thanks!

[kriziw]

Hi @ilike2burnthing ,
Alternative api was provided, which should have the right response (IMDB missing).
https://ncore.pro/api/v2/torrents

[ilike2burnthing]

ncore.yml

---
id: ncore
name: nCore (API)
description: "nCore is a HUNGARIAN Private Torrent Tracker for MOVIES / TV / GENERAL"
language: hu-HU
type: private
encoding: UTF-8
links:
  - https://ncore.pro/
legacylinks:
  - https://ncore.cc/

caps:
  categorymappings:
    # E-book
    - {id: ebook, cat: Books, desc: "E-book/Eng"}
    - {id: ebook_hun, cat: Books, desc: "E-book/Hun"}
    # Játék
    - {id: console, cat: Console, desc: "Játék/Console"}
    - {id: game_iso, cat: PC/Games, desc: "Játék/ISO"}
    - {id: game_rip, cat: PC/Games, desc: "Játék/Rip"}
    # Filmek
    - {id: xvid_hun, cat: Movies/SD, desc: "Film/SD/Hun"}
    - {id: xvid, cat: Movies/SD, desc: "Film/SD/Eng"}
    - {id: dvd_hun, cat: Movies/DVD, desc: "Film/DVD/Hun"}
    - {id: dvd, cat: Movies/DVD, desc: "Film/DVD/Eng"}
    - {id: dvd9_hun, cat: Movies/DVD, desc: "Film/Dvd9/Hun"}
    - {id: dvd9, cat: Movies/DVD, desc: "Film/Dvd9/Eng"}
    - {id: hd_hun, cat: Movies/HD, desc: "Film/HD/Hun"}
    - {id: hd, cat: Movies/HD, desc: "Film/HD/Eng"}
    # Sorozatok
    - {id: xvidser, cat: TV/SD, desc: "Sorozat/SD/Eng"}
    - {id: xvidser_hun, cat: TV/SD, desc: "Sorozat/SD/Hun"}
    - {id: dvdser, cat: TV/Other, desc: "Sorozat/DVDR/Eng"}
    - {id: dvdser_hun, cat: TV/Other, desc: "Sorozat/DVDR/Hun"}
    - {id: hdser, cat: TV/HD, desc: "Sorozat/HD/Eng"}
    - {id: hdser_hun, cat: TV/HD, desc: "Sorozat/HD/Hun"}
    # Zene
    - {id: mp3_hun, cat: Audio/MP3, desc: "Zene/MP3/Hun"}
    - {id: mp3, cat: Audio/MP3, desc: "Zene/MP3/Eng"}
    - {id: lossless_hun, cat: Audio/MP3, desc: "Zene/Lossless/Hun"}
    - {id: lossless, cat: Audio/MP3, desc: "Zene/Lossless/Eng"}
    - {id: clip, cat: Audio/Video, desc: "Zene/Klip"}
    # Program
    - {id: iso, cat: PC/ISO, desc: "Win/Program/ISO"}
    - {id: misc, cat: PC/0day, desc: "Win/Program/Rip"}
    - {id: mobil, cat: PC/Mobile-Other, desc: "Win/Program/Rip"}
    # Felnőtt tartalom
    - {id: xxx_xvid, cat: XXX/SD, desc: "XXX/Film/SD"}
    - {id: xxx_dvd, cat: XXX/DVD, desc: "XXX/DVDR"}
    - {id: xxx_hd, cat: XXX/UHD, desc: "XXX/HD"}
    - {id: xxx_imageset, cat: XXX/ImageSet, desc: "XXX/Kép"}

  modes:
    search: [q]
    tv-search: [q, season, ep, imdbid, genre]
    movie-search: [q, imdbid, genre]
    music-search: [q, genre]
    book-search: [q, genre]
  allowrawsearch: true

settings:
  - name: username
    type: text
    label: Username
  - name: password
    type: password
    label: Password
  - name: passkey
    type: text
    label: Passkey
  - name: info_passkey
    type: info
    label: "About Passkey"
    default: "Find your Passkey by going to your profile and clicking on the <b>Egyéb adatok</b> section."
  - name: 2facode
    type: text
    label: 2FA code
  - name: info_2fa
    type: info
    label: "About 2FA code"
    default: "Only fill in the <b>2FA code</b> box if you have enabled <b>2FA</b> on the nCore Web Site. Otherwise just leave it empty."
  - name: info_tpp
    type: info
    label: Results Per Page
    default: For best results, change the <b>Torrent oldalanként:</b> setting to <b>100</b> on your account profile.
  - name: info_activity
    type: info
    label: Account Inactivity
    default: "Anyone who does not access the site for 8 weeks from registration or has been more than 8 weeks since their last stay will be automatically suspended. Anyone who does not log in (stays inactive) for 12 weeks after registration or after their last visit will be removed automatically."

login:
  method: post
  path: login.php
  inputs:
    nev: "{{ .Config.username }}"
    pass: "{{ .Config.password }}"
    2factor: "{{ .Config.2facode }}"
    ne_leptessen_ki: "1"
    set_lang: "en"
    submitted: "1"
  error:
    - selector: div#hibauzenet table tbody tr
  test:
    path: index.php
    selector: a[href*="profile.php"]

search:
  paths:
    - path: api/v2/torrents
      response:
        type: json

  inputs:
    q: "{{ .Keywords }}"
#    mire: "{{ if .Query.IMDBID }}{{ .Query.IMDBID }}{{ else }}{{ .Keywords }}{{ end }}"
#    miben: "{{ if .Query.IMDBID }}imdb{{ else }}name{{ end }}"
#    tipus: "{{ if .Categories }}kivalasztottak_kozott{{ else }}all_own{{ end }}"
#    tags: "{{ .Query.Genre }}"
#    $raw: "{{ range .Categories }}&kivalasztott_tipus[]={{.}}{{end}}"

  rows:
    selector: torrents

  fields:
    category:
      text: hd
#      selector: category
    title:
      selector: releaseName
    _id:
      selector: id
    details:
      text: "/torrents.php?action=details&id={{ .Result._id }}"
    download:
      text: "/torrents.php?action=download&id={{ .Result._id }}&key={{ .Config.passkey }}"
    size:
      selector: size
    seeders:
      selector: seeders
    leechers:
      selector: leechers
    grabs:
      selector: downloaded
    date:
      selector: createdAt
    downloadvolumefactor:
      text: 0
    uploadvolumefactor:
      text: 1
    minimumratio:
      text: 1.0
# MST is 48 hours + 24 minutes (.4 hours) per GB of downloaded size
#    minimumseedtime:
#      # 2 days (as seconds = 2 x 24 x 60 x 60)
#      text: 172800
    description:
      selector: verified
      case:
        1: Verified
        0: Unverified
# json api

Is there any documentation available for their API? The only parameter I've been able to work out is q for the keyword search, but currently we can also search by IMDB ID, categories, and tags.

As you mentioned, IMDB ID is missing from the results information, but so is the category ID, we just get a category number, which I can't find anywhere else on the site, and is going to be a pain to use instead. So can the category ID (e.g. hdser_hun) be added?

Users will also now need to add their passkey, as we're having to piece together the download link. This isn't the end of the world, but it will mean most users will have a bunch of failed downloads before they realise they need to reconfigure the indexer and add it. The previous download_url was easier to work with, in case that can be added as well.

[ilike2burnthing]

?