200/connection pool Error's

[DF4708]

Good afternoon,

Thanks for taking a moment to look at this, I'm not sure why I keep receiving the following. Below are two example sites with the same issue. I know the cookies are valid because I'll be logged in during the auto-cookie pull and a manual local.txt cookie check. Manual username/password doesn't change the outcome.

CMD's attempted:

1. gallery-dl --cookies-from-browser firefox -o browser=firefox --filter "extension == 'jpg' or extension == 'png' or extension == 'gif' or extension == 'mp4'" -v --sleep-429 300 --write-tags --write-info-json -s 15 -i /Users/******/aryion_urls_ALL.txt
2. gallery-dl --cookies-from-browser firefox --filter "extension == 'jpg' or extension == 'png' or extension == 'gif' or extension == 'mp4'" -v --sleep-429 300 --write-tags --write-info-json -s 15 -i /Users/*****/aryion_urls_ALL.txt
3. gallery-dl --u A******* -p ************* -o browser=firefox --filter "extension == 'jpg' or extension == 'png' or extension == 'gif' or extension == 'mp4'" -v --sleep-429 300 --write-tags --write-info-json -s 15 -i /Users/*****/aryion_urls_ALL.txt

I visited the URL's and they are legit and there is an image there. It's clearly identifying images below but not pulling them:

Aryion:
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=1010430 HTTP/1.1" 200 0
[urllib3.connectionpool][debug] https://aryion.com:443 "GET /g4/view/1010430 HTTP/1.1" 200 None
[aryion][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
1010430 Monkeyin' Around - by Oogzie.png
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/987239'
[aryion][debug] cookies: Using cached cookies from ('firefox', '', '', '', '')
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=987239 HTTP/1.1" 200 0
[urllib3.connectionpool][debug] https://aryion.com:443 "GET /g4/view/987239 HTTP/1.1" 200 None
[aryion][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
987239 Holt Bustin' - by GrayscaleRain.png
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/987238'
[aryion][debug] cookies: Using cached cookies from ('firefox', '', '', '', '')
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=987238 HTTP/1.1" 200 0
[urllib3.connectionpool][debug] https://aryion.com:443 "GET /g4/view/987238 HTTP/1.1" 200 None
[aryion][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
987238 Holt and his Spunk - by GrayscaleRain.png
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/987237'
[aryion][debug] cookies: Using cached cookies from ('firefox', '', '', '', '')
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=987237 HTTP/1.1" 200 0
[urllib3.connectionpool][debug] https://aryion.com:443 "GET /g4/view/987237 HTTP/1.1" 200 None
[aryion][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
987237 Holt and Andy - by GrayscaleRain.png
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/953580'
[aryion][debug] cookies: Using cached cookies from ('firefox', '', '', '', '')
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=953580 HTTP/1.1" 200 0
[urllib3.connectionpool][debug] https://aryion.com:443 "GET /g4/view/953580 HTTP/1.1" 200 None
[aryion][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
953580 Unovan food is so fattening - by EcchiPandaa.png
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/770052'
[aryion][debug] cookies: Using cached cookies from ('firefox', '', '', '', '')

Furaffinity:
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/42248097/ HTTP/1.1" 200 None
42248097 The best bed for my little guest.jpg
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/45839237/ HTTP/1.1" 200 None
45839237 Star(shallows)bucks.jpg
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/24311945/ HTTP/1.1" 200 None
24311945 My Boyfriend's page 1.jpg
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/29782384/ HTTP/1.1" 200 None
29782384 Timed Comm- Gaming Distractions.png
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/30688623/ HTTP/1.1" 200 None
30688623 MouseCakes #48 - Meanwhile, before...Pt. 3.png
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/47694167/ HTTP/1.1" 200 None

[mikf]

-s

  -s, --simulate              Simulate data extraction; do not download
                              anything

-v

  -v, --verbose               Print various debugging information

`--filter "extension == 'jpg' or extension == 'png' or extension == 'gif' or extension == 'mp4'"

--filter "extension in ('jpg', 'png', 'gif', 'mp4')"

[DF4708]

I changed the filter with the one provided, -v was on there before, so no change, and I added -s. Below is the output for the two sites, respectively:

Example CMD output:

1. gallery-dl --cookies-from-browser firefox --filter "extension in ('jpg', 'png', 'gif', 'mp4')" -s -v --sleep-429 300 --write-tags --write-info-json -s 15 -i /Users/**/aryion_urls_ALL.txt
   [gallery-dl][debug] Version 1.30.10 - Git HEAD: fbfdbaba00
   [gallery-dl][debug] Python 3.14.0 - macOS-26.1-arm64-arm-64bit-Mach-O
   [gallery-dl][debug] requests 2.32.5 - urllib3 2.5.0
   [gallery-dl][debug] Configuration Files ['${HOME}/.config/gallery-dl/config.json']
   [2/10000001] https://aryion.com/g4/view/277278
   [gallery-dl][debug] Starting SimulationJob for 'https://aryion.com/g4/view/277278'
   [aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/277278'
   [cookies][debug] Extracting cookies from /Users//Library/Application Support/Firefox/Profiles/8wgj0ze9.default-release-1764016610628/cookies.sqlite
   [cookies][debug] Only loading cookies not belonging to any container
   [cookies][info] Extracted 238 cookies from Firefox
   [urllib3.connectionpool][debug] Starting new HTTPS connection (1): aryion.com:443

Furaffinity:
[cookies][debug] Extracting cookies from /Users/*****/Library/Application Support/Firefox/Profiles/8wgj0ze9.default-release-1764016610628/cookies.sqlite
[cookies][debug] Only loading cookies not belonging to any container
[cookies][info] Extracted 238 cookies from Firefox
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.furaffinity.net:443
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "POST /search/ HTTP/1.1" 200 None
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/17371095/ HTTP/1.1" 200 None
[furaffinity][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
[furaffinity][debug] Sleeping 0.99 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/13353103/ HTTP/1.1" 200 None
[furaffinity][debug] Sleeping 0.99 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/57639640/ HTTP/1.1" 200 None
57639640 Trucker Problems.png
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/57638756/ HTTP/1.1" 200 None
57638756 Out with the Outlaws.png
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/56613456/ HTTP/1.1" 200 None
56613456 Tahm's snack.gif
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/55354394/ HTTP/1.1" 200 None
55354394 Big Kitty.jpg
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/59865630/ HTTP/1.1" 200 None
59865630 A Meager Snack.png
[furaffinity][debug] Sleeping 0.99 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/59865640/ HTTP/1.1" 200 None
59865640 A Meager Snack (Clothes).png
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/28250123/ HTTP/1.1" 200 None
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/9373803/ HTTP/1.1" 200 None
[furaffinity][debug] Sleeping 0.99 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/8728328/ HTTP/1.1" 200 None
8728328 Weight Gain RPG Text Adventure Game Download-Dev 0.33.1.png
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/16901881/ HTTP/1.1" 200 None
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/61143477/ HTTP/1.1" 200 None
61143477 -i got something in my teeth -.png
[furaffinity][debug] Sleeping 1.00 seconds (request)
[urllib3.connectionpool][debug] https://www.furaffinity.net:443 "GET /view/28249917/ HTTP/1.1" 200 None
28249917 Vore Game: I8UBETA.png

Aryion
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/277278'
[aryion][debug] cookies: Using cached cookies from ('firefox', '', '', '', '')
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=277278 HTTP/1.1" 200 0
[urllib3.connectionpool][debug] https://aryion.com:443 "GET /g4/view/277278 HTTP/1.1" 200 None
[aryion][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
277278 Tasty Saoirse.jpg
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/277276'
[aryion][debug] cookies: Using cached cookies from ('firefox', '', '', '', '')
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=277276 HTTP/1.1" 200 0
[urllib3.connectionpool][debug] https://aryion.com:443 "GET /g4/view/277276 HTTP/1.1" 200 None
[aryion][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
277276 Linda's Picnic.jpg
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/257165'
[aryion][debug] cookies: Using cached cookies from ('firefox', '', '', '', '')
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=257165 HTTP/1.1" 200 0
[urllib3.connectionpool][debug] https://aryion.com:443 "GET /g4/view/257165 HTTP/1.1" 200 None
[aryion][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
257165 Effigia's Slumbering Snacks.jpg
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/257163'
[aryion][debug] cookies: Using cached cookies from ('firefox', '', '', '', '')
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=257163 HTTP/1.1" 200 0
[urllib3.connectionpool][debug] https://aryion.com:443 "GET /g4/view/257163 HTTP/1.1" 200 None
[aryion][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
257163 Gaja's 3-Course Meal.jpg

[mikf]

-v is the reason you get 200/connection pool "errors".
-s is the reason it doesn't download anything.

[DF4708]

Thanks so much for your quick replies! I was able to fix it with your help :)

I accidentally thought I could use -s as shorthand for sleep request, and the fact that other sites downloaded with -s contributed to the confusion.

I have noticed that the twitter/x tags text file isn't tagging images even if they have a hashtag. So I was thinking that I'll need to run a local tagger for them, unless I'm doing something wrong?

gallery-dl --cookies "/Users/*****/cookies.txt" -o skip=true -o tags=translated --filter "extension in ('jpg', 'png', 'gif', 'mp4', 'webp')" --filter "'artwork' in filename" -v --sleep 5-15 --sleep-429 300 --write-tags --write-info-json "https://x.com/search?..."

^ My assumption is that this will pull cookies from the text file, skip files already downloaded, translate non-English tags, only download files with a jpg, gif, png, gif, and mp4 extension, not include images with "artwork" in the filename, etc.

I also have a similar search for aryion, but it still includes files like rtf, doc, etc.

[mikf]

I have noticed that the twitter/x tags text file isn't tagging images even if they have a hashtag

--write-tags writes tags metadata. Twitter hashtags are provided as hashtags and is therefore not included in the --write-tags output.

You can rename the metadata field from hashtags to tags with

-P metadata/modify@post -O fields='{"tags": "{hashtags}"}'

--filter "extension in ('jpg', 'png', 'gif', 'mp4', 'webp')" --filter "'artwork' in filename"

The second --filter overwrites the first. Combine them with and:

--filter "extension in ('jpg', 'png', 'gif', 'mp4', 'webp') and 'artwork' not in filename"

translate non-English tags

Nope, that's not a thing.

not include images with "artwork" in the filename

It does the opposite. Use not in not exclude filenames with "artwork"

and the fact that other sites downloaded with -s contributed to the confusion.

They don't, as long as -s isn't overwritten by some other option.

[DF4708]

That was extremely helpful! Thank you so much!

Ah, okay, I was using -o tags=translated based on what I was seeing others do with pixiv, for example. I've removed its use after your insight.

-s must have been overwritten then, based on the order of the CMD in those queries.

I took what you taught me and integrated it into the config file rather than the command, like so:

"twitter":{
"users": "https://x.com/",
"username": "",
"password": "**",
"cookies": ["/Users//cookies.txt"],
"include": "media",
"tags": "{hashtags}",
"image-filter": "extension in ('jpg', 'png', 'gif', 'mp4', 'webp', 'jpeg') and ('artwork', 'cat', 'feline') not in filename and contains (hashtags, ('dog', 'puppy', 'cute'))",
"skip": true,
"text-tweets":true,
"quoted":true,
"retweets":false,
"logout":true,
"replies":true,
"pinned":true,
"retry-codes": [429, 430],
"metadata": true,
"syndication":"extended",
"filename":"{author[name]}-{tweet_id|retweet_id}-{date:%Y%m%d}-{extension}{num}.{extension}"
},

After making those changes, the code executed fine and had a much shorter command! Something else I learned, if using a long URL with an advanced search you can ' | less' to still be able to execute it after the CMD buffer is exhausted.

So, I wanted to test if the updated config and CMD would generate new text files with the hashtags inside. So I went to a folder with 5 images, deleted all its contents (json, txt, and jpgs), and ran a CMD to pull the photos from the same account, since I knew it had images and had previously performed all the functions correctly except the tag.

Unfortunately, it did not pull any of the images or generate text files, but it did create a new JSON file. So, I tried turning skip off in the config ("skip":false,) and CMD (-o skip=false) without any luck (also tried true just cuz). The output would repeat whether I had updated the config or the CMD to not skip and regardless of whether the files were in or not in the folder.
- [twitter][debug] Skipping 1688990412265099265 (previously seen)

I also tried to find a JSON or CSV file in the root folder and then the entire directory that was keeping a record that I could clear, but couldn't find one. I didn't use a database, but I searched for a local '.sqlite3' file anyway and couldn't find one. Do you know where I can see this log (MacOS), or is there a CMD I can use to overwrite the record?

I saw some folks using a >time stamp for newer images but that wouldn't apply here.

Last question: Is there a means to update the other folders' blank text files without redownloading the images and add their username as a tag in the text file (cite)?