Having trouble grabbing very old replies? I wrote a script to help grab old tweets #4643
anthonyj-codestuff
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
If you are trying to grab very old text tweets, you run into the problem that you need to run gallery-dl for potentially hours or even days to get through the API rate limit. And when you grab from "/with_replies", it sometimes doesn't even try to grab parent tweets to replies beyond a certain point. BUT if you can get the regular replies, you can reconstruct the link for the parent tweet and ask gallery-dl for it by name.
That being said, I was able to generate 11k missing links resulting in 9k rescued JSON files +3000 media files
Might not be the most professional code (and it's WAY slower than running gallery-dl on the user), but it'll output a list of commands that you can run at your leisure.
As always, set up a debugger before running strange code.
https://github.com/fireheadlazzo/gallery-dl-orphan-parent-grabber/blob/main/script.py
Beta Was this translation helpful? Give feedback.
All reactions