allow to disable validation

[milahu]

i want to disable validation like

torrent_models.Torrent.read(some_path, validate=False)

so i can get the v1 and v2 hashes of broken torrent files like

$ python
>>> import torrent_models
>>> torrent_models.Torrent.read("no-piece-layers.torrent")
...
pydantic_core._pydantic_core.ValidationError: 1 validation error for Torrent
  Assertion failed, Hybrid and v2 torrents must have piece layers [type=assertion_error, input_value={'info': {'file tree': {'...2\xa8\xeb\x86:yV\x95G'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.11/v/assertion_error

[sneakers-the-rat]

see #10

there are some validations that we can probably disable and other we can't.
so e.g. we could just disable all validation by using model_construct, but that wouldn't do the validation that handles converting the bytes to strings and so on.

As a format, torrent files are an "open world" format where things that aren't anticipated by the spec are allowed - extra fields, etc. but it is still a specification that makes requirements on the things that are specified. one of the ways this package is different from other torrent file packages is that we don't maintain some private copy of the bencoded dictionary and instead have a full-fledged python representation of the torrent. the models are designed to reflect the format - extra values are permitted, but specified values must be valid. that could be written around, but it would require a bit of work and i don't have time for it just now.

luckily if what you're after is just getting the infohashes, that's pretty simple:

from pathlib import Path
import hashlib
from bencode_rs import bdecode, bencode

data = bdecode(Path('my_torrent.torrent').read_bytes())
info = bencode(data[b'info'])

v1_infohash = hashlib.sha1(info).hexdigest()
v2_infohash = hashlib.sha256(info).hexdigest()

[milahu]

info = bencode(data[b'info'])

that decode-encode roundtrip can produce wrong hashes

i solved this with the CST parser tree-sitter-bencode
in qbittorrent-move-to-cas.py

i later found that qbittorrentapi
does expose torrent.infohash_v1 and torrent.infohash_v2
so in my case, i dont need this any more
but it may be useful in other cases

[sneakers-the-rat]

that decode-encode roundtrip can produce wrong hashes

how? As far as I know that bencode implementation is correct, but I'd like to know if its not

[milahu]

for example, what happens when a dict key is repeated?

d1:ai1e1:ai2ee

a lossless decoder would produce a list of items (keys and values)

[ ("a", 1), ("a", 2) ]

bencode-rs throws BencodeDecodeError on duplicated keys

todo: does libtorrent allow such broken torrents?
at first sight, i could not find a duplicate_dict_keys.torrent

[sneakers-the-rat]

ah sure, yes for invalid bencoded data the roundtrip will fail with an error, which imo is correct behavior. libtorrent looks like it just takes the value of the first appearance of the key and ignores subsequent appearances of the key.

one can consider this package 'opinionated' if one considers 'invalid data should noisily raise an error rather than pass silently' an opinion. sorry if that's unhelpful. if there is need for repairing invalid torrents, i would be into that, but i don't think the design of this package allows for handling invalid data that stays invalid.

out of curiosity - what torrent creator is emitting invalid torrents? i'm curious about the torrents you're dealing with, because i want to know how common that is/whether we need to add something to the docs here. (also general curiosity about the state of the bittorrent tooling ecosystem)

[milahu]

what torrent creator is emitting invalid torrents?

good question...
i dont remember how i created no-piece-layers.torrent
but libtorrent did not complain when loading that torrent

how common that is

i would guess that 0.1% of all torrents are invalid

for a better estimate, you could parse 25 GB of torrent files from
DHT database dumps