Output unfollowed links to a list? GDL not grabbing youtube and other embedded links from Patreon.

[kendrakirai]

There's a patreon I follow that has their videos hosted elsewhere. Those links can be downloaded via yt-dlp, but I can't figure out how to make GDL actually just write those urls into a single file that I can pick them out of. --write-unsupported didn't do anything, --resolve-urls only touched the ones from patreon.

[kendrakirai]

Still wondering about this. I need a list of links to throw at yt-dlp because GDL isn't grabbing youtube or cloudflare video link embeds.

[mikf]

#8174 (comment)

[kendrakirai]

Any way to make it output to a single file rather than one per post?

[mikf]

Add "open": "a" to the metadata options
https://gdl-org.github.io/docs/configuration.html#metadata-open

and use a constant / static filename.

[kendrakirai]

Cool, thanks. ...Is there a way to comment out entries in GDL's .conf? It's performing the post processor even without the switch being on and putting I know I'm not going to remember to put the --no-postprocessor switch on manually.

[mikf]

You can rename the option to something like #postprocessors or anything that's not recognized as an actual option.

Or you add the postprocessor to the global postprocessor block and enable it with -P
(-P links in this case)

{
    "postprocessor": {
        "links": {
            "name": "metadata@post",
            "filename": "links.txt",
            "open"    : "a",
            "filter"  : "content",
            "format"  : "{content!R:J\n}"
        }
    }
}

[kendrakirai]

Ooooohhhhh, thank you!

Ah, new small problem. The output doesn't put line breaks between the links it finds. IE

http://www.youtube.com/URL1http://www.youtube.com/URL2http://www.twitter.com/URL3

I can probably process this with notepad++ if GDL can't put in the breaks. but if I can finagle it automatically....

[mikf]

Adding a newline \n at the end of format should fix it:

            "format"  : "{content!R:J\n}\n"

[kendrakirai]

Looks like it's adding blank lines to the file for every post checked, which is infinitely easier to deal with than trying to split out the results, thank you!

[mikf]

It should even be possible to prevent (most) blank lines by adding another condition to filter, I think:

            "filter"  : "content and 'http' in content",