Skip to content

Commit 6224e71

Browse files
ifoschifosch
committed
Use RSS feed libraries to mangle the RSS Feed
Using XML libraries to manipulate RSS feed is too complicated. This implementation uses [gofeed](https://github.com/mmcdole/gofeed) to read current feed, and [podcast](https://github.com/eduncan911/podcast) to add the new episode and write it to stdout. This fixes the bug on newlines in episode description (#44), while making the code easier to understand.
1 parent e90a5de commit 6224e71

File tree

5 files changed

+121
-29
lines changed

5 files changed

+121
-29
lines changed

‎README.md‎

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# appu
22

3-
Appu is a toolkit for podcast edition and publishing.
3+
**A**utomatic **P**odcast **PU**blisher, aka appu, is a toolkit for podcast edition and publishing.
44

55
## Rationale
66

7-
While running the [Entre Dev Y Ops podcast](https://www.entredevyops.es), the authors found interesting to start building a set of tools to make this easier. We hope this might help anyone else.
7+
While running the [Entre Dev y Ops podcast](https://www.entredevyops.es), the authors found interesting to start building a set of tools to make this easier. We hope this might help anyone else.
88

99
Currently we start preparing every episode by writing a simple script we store in a shared Drive folder.
1010

@@ -20,9 +20,8 @@ Finally, using the tools here, and our own podcast configuration file, we have m
2020

2121
You'll need the following:
2222
- Docker, for Windows and Mac, you should use Docker Desktop.
23-
- Publishing infrastructure based on an online storage, and a CDN with invalidation features.
24-
Currently only AWS S3 and CloudFront are supported.
25-
You'll need credentials for uploading the episodes' audio files and the RSS feed and then invalidating them on the CDN.
23+
- Publishing infrastructure based on an online storage, and a CDN with invalidation features (currently only AWS S3 and CloudFront are supported.)
24+
- Valid credentials for uploading the episodes' audio files and the RSS feed and then invalidating them on the CDN.
2625

2726
We recommend to use some secure online storage for your podcast configuration and specific details.
2827
It's also interesting for sharing the publishing process within teams.
@@ -58,16 +57,13 @@ docker build -t ghcr.io/edyo/appu:local appu/.
5857
#### Download and update the feed
5958

6059
```
61-
./feedupdater http://my.podcast.com/podcast/feed.xml mypodcast-XX.yaml
60+
./feedupdater http://my.podcast.com/podcast/feed.xml mypodcast-XX.yaml > new_feed.xml
6261
```
6362

64-
Open the original feed downloaded, `feed.xml` in this case, to avoid editing all entries, and add the last `item` element from the `new_feed.xml`.
65-
Then replace the new lines in the `description` tag's text for the last `item` with the corresponding HTML entity, i.e. `
`.
66-
6763
#### Upload the new feed
6864

6965
```
70-
./uploader feed.xml feed.xml mypodcast-XX.yaml
66+
./uploader new_feed.xml feed.xml mypodcast-XX.yaml
7167
```
7268

7369
First argument is the name of the XML file with the new episode entry. The second argument is the bucket key to upload the file to. The Third argument is the episode configuration file.

‎cmd/feedupdater/main.go‎

Lines changed: 5 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -24,21 +24,15 @@ func main() {
2424
log.Fatalf("Error downloading feed %s: %v", feedURL, err)
2525
}
2626

27-
doc, err := appu.ReadXML(feedFileName)
27+
feed, err := appu.ReadXML(feedFileName)
2828
if err != nil {
2929
log.Fatalf("Error parsing feed: %v", err)
3030
}
3131

32-
channel := doc.FindElement("./rss/channel")
33-
if channel != nil {
34-
newEpisodeTag, err := appu.CreateFeedItem(cfg)
35-
if err != nil {
36-
log.Fatalf("Error creating new episode tag: %v", err)
37-
}
38-
channel.AddChild(newEpisodeTag)
39-
} else {
40-
log.Fatalf("Error: could not find channel tagin feed XML")
32+
err = appu.AddNewEpisode(cfg, feed)
33+
if err != nil {
34+
log.Fatalf("Error adding new episode: %v", err)
4135
}
4236

43-
appu.WriteXML(doc)
37+
appu.WriteXML(feed)
4438
}

‎go.mod‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,10 @@ require (
1515
github.com/antchfx/xpath v1.2.0
1616
github.com/containerd/containerd v1.5.5 // indirect
1717
github.com/docker/go-connections v0.4.0 // indirect
18+
github.com/eduncan911/podcast v1.4.2
1819
github.com/gorilla/mux v1.8.0 // indirect
1920
github.com/ifosch/stationery v0.1.2
21+
github.com/mmcdole/gofeed v1.1.3
2022
github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 // indirect
2123
github.com/morikuni/aec v1.0.0 // indirect
2224
github.com/smartystreets/goconvey v1.6.4 // indirect
@@ -27,6 +29,8 @@ require (
2729
require (
2830
cloud.google.com/go v0.93.3 // indirect
2931
github.com/Microsoft/go-winio v0.4.17 // indirect
32+
github.com/PuerkitoBio/goquery v1.5.1 // indirect
33+
github.com/andybalholm/cascadia v1.1.0 // indirect
3034
github.com/docker/distribution v2.7.1+incompatible // indirect
3135
github.com/docker/go-units v0.4.0 // indirect
3236
github.com/go-ini/ini v1.25.4 // indirect
@@ -35,6 +39,10 @@ require (
3539
github.com/golang/protobuf v1.5.2 // indirect
3640
github.com/googleapis/gax-go/v2 v2.0.5 // indirect
3741
github.com/jmespath/go-jmespath v0.0.0-20160803190731-bd40a432e4c7 // indirect
42+
github.com/json-iterator/go v1.1.10 // indirect
43+
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf // indirect
44+
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
45+
github.com/modern-go/reflect2 v1.0.1 // indirect
3846
github.com/opencontainers/go-digest v1.0.0 // indirect
3947
github.com/opencontainers/image-spec v1.0.1 // indirect
4048
github.com/pkg/errors v0.9.1 // indirect

‎go.sum‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,8 @@ github.com/Microsoft/hcsshim/test v0.0.0-20201218223536-d3e5debf77da/go.mod h1:5
8080
github.com/Microsoft/hcsshim/test v0.0.0-20210227013316-43a75bb4edd3/go.mod h1:mw7qgWloBUl75W/gVH3cQszUg1+gUITj7D6NY7ywVnY=
8181
github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ=
8282
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
83+
github.com/PuerkitoBio/goquery v1.5.1 h1:PSPBGne8NIUWw+/7vFBV+kG2J/5MOjbzc7154OaKCSE=
84+
github.com/PuerkitoBio/goquery v1.5.1/go.mod h1:GsLWisAFVj4WgDibEWF4pvYnkVQBpKBKeU+7zCJoLcc=
8385
github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
8486
github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=
8587
github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d/go.mod h1:HI8ITrYtUY+O+ZhtlqUnD8+KwNPOyugEhfP9fdUIaEQ=
@@ -88,6 +90,8 @@ github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuy
8890
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
8991
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
9092
github.com/alexflint/go-filemutex v0.0.0-20171022225611-72bdc8eae2ae/go.mod h1:CgnQgUtFrFz9mxFNtED3jI5tLDjKlOM+oUF/sTk6ps0=
93+
github.com/andybalholm/cascadia v1.1.0 h1:BuuO6sSfQNFRu1LppgbD25Hr2vLYW25JvxHs5zzsLTo=
94+
github.com/andybalholm/cascadia v1.1.0/go.mod h1:GsXiBklL0woXo1j/WYWtSYYC4ouU9PqHO0sqidkEA4Y=
9195
github.com/antchfx/htmlquery v1.2.4 h1:qLteofCMe/KGovBI6SQgmou2QNyedFUW+pE+BpeZ494=
9296
github.com/antchfx/htmlquery v1.2.4/go.mod h1:2xO6iu3EVWs7R2JYqBbp8YzG50gj/ofqs5/0VZoDZLc=
9397
github.com/antchfx/xpath v1.2.0 h1:mbwv7co+x0RwgeGAOHdrKy89GvHaGvxxBtPK0uF9Zr8=
@@ -267,6 +271,8 @@ github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96/go.mod h1:Qh8CwZ
267271
github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE=
268272
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
269273
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
274+
github.com/eduncan911/podcast v1.4.2 h1:S+fsUlbR2ULFou2Mc52G/MZI8JVJHedbxLQnoA+MY/w=
275+
github.com/eduncan911/podcast v1.4.2/go.mod h1:mSxiK1z5KeNO0YFaQ3ElJlUZbbDV9dA7R9c1coeeXkc=
270276
github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc=
271277
github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=
272278
github.com/emicklei/go-restful v2.9.5+incompatible/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=
@@ -441,6 +447,7 @@ github.com/jmespath/go-jmespath v0.0.0-20160803190731-bd40a432e4c7/go.mod h1:Nht
441447
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
442448
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
443449
github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
450+
github.com/json-iterator/go v1.1.10 h1:Kz6Cvnvv2wGdaG/V8yMvfkmNiXq9Ya2KUv4rouJJr68=
444451
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
445452
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
446453
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
@@ -481,6 +488,10 @@ github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible/go
481488
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
482489
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
483490
github.com/mitchellh/osext v0.0.0-20151018003038-5e2d6d41470f/go.mod h1:OkQIRizQZAeMln+1tSwduZz7+Af5oFlKirV/MSYes2A=
491+
github.com/mmcdole/gofeed v1.1.3 h1:pdrvMb18jMSLidGp8j0pLvc9IGziX4vbmvVqmLH6z8o=
492+
github.com/mmcdole/gofeed v1.1.3/go.mod h1:QQO3maftbOu+hiVOGOZDRLymqGQCos4zxbA4j89gMrE=
493+
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf h1:sWGE2v+hO0Nd4yFU/S/mDBM5plIU8v/Qhfz41hkDIAI=
494+
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf/go.mod h1:pasqhqstspkosTneA62Nc+2p9SOBBYAPbnmRRWPQ0V8=
484495
github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc=
485496
github.com/moby/sys/mountinfo v0.4.0/go.mod h1:rEr8tzG/lsIZHBtN/JjGG+LMYx9eXgW2JI+6q0qou+A=
486497
github.com/moby/sys/mountinfo v0.4.1/go.mod h1:rEr8tzG/lsIZHBtN/JjGG+LMYx9eXgW2JI+6q0qou+A=
@@ -489,8 +500,10 @@ github.com/moby/term v0.0.0-20200312100748-672ec06f55cd/go.mod h1:DdlQx2hp0Ss5/f
489500
github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 h1:dcztxKSvZ4Id8iPpHERQBbIJfabdt4wUm5qy3wOL2Zc=
490501
github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6/go.mod h1:E2VnQOmVuvZB6UYnnDB0qG5Nq/1tD9acaOpo6xmt0Kw=
491502
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
503+
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
492504
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
493505
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
506+
github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI=
494507
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
495508
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
496509
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
@@ -640,6 +653,7 @@ github.com/urfave/cli v0.0.0-20171014202726-7bc6a0acffa5/go.mod h1:70zkFmudgCuE/
640653
github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
641654
github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
642655
github.com/urfave/cli v1.22.2/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
656+
github.com/urfave/cli v1.22.3/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
643657
github.com/vishvananda/netlink v0.0.0-20181108222139-023a6dafdcdf/go.mod h1:+SR5DhBJrl6ZM7CoCKvpw5BKroDKQ+PJqOg65H/2ktk=
644658
github.com/vishvananda/netlink v1.1.0/go.mod h1:cTgwzPIzzgDAYoQrMm0EdrjRUBkTqKYppBueQtXaqoE=
645659
github.com/vishvananda/netlink v1.1.1-0.20201029203352-d40f9887b852/go.mod h1:twkDnbuQxJYemMlGd4JFIcuhgX83tXhKS2B/PRMpOho=
@@ -727,6 +741,7 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
727741
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
728742
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
729743
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
744+
golang.org/x/net v0.0.0-20180218175443-cbe0f9307d01/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
730745
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
731746
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
732747
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=

‎pkg/appu/feed.go‎

Lines changed: 87 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,96 @@ package appu
22

33
import (
44
"fmt"
5+
"os"
6+
"strconv"
57
"time"
68

79
"github.com/beevik/etree"
10+
"github.com/eduncan911/podcast"
11+
"github.com/mmcdole/gofeed"
812
)
913

10-
// ReadXML loads XML from a file into an `etree.Document` for processing.
11-
func ReadXML(feedFileName string) (*etree.Document, error) {
12-
doc := etree.NewDocument()
13-
err := doc.ReadFromFile(feedFileName)
14+
// AddNewEpisode creates a new episode from `cfg` and adds it to `feed`.
15+
func AddNewEpisode(cfg *Config, feed *podcast.Podcast) error {
16+
description := cfg.Summary + "\n"
17+
for _, link := range cfg.Links {
18+
description += link + "\n"
19+
}
20+
newItem := podcast.Item{
21+
Title: cfg.Title,
22+
Description: description,
23+
PubDate: &cfg.PubDate,
24+
}
25+
length, err := GetEpisodeLength(cfg.EpisodeURL)
26+
if err != nil {
27+
return err
28+
}
29+
newItem.AddEnclosure(cfg.EpisodeURL, podcast.MP3, int64(length))
30+
31+
if _, err := feed.AddItem(newItem); err != nil {
32+
return err
33+
}
34+
35+
return nil
36+
}
37+
38+
// ReadXML loads XML from a file into a `podcast.Podcast` for processing.
39+
func ReadXML(feedFileName string) (*podcast.Podcast, error) {
40+
file, err := os.Open(feedFileName)
1441
if err != nil {
1542
return nil, err
1643
}
44+
defer file.Close()
1745

18-
return doc, err
46+
fp := gofeed.NewParser()
47+
feed, err := fp.Parse(file)
48+
if err != nil {
49+
return nil, err
50+
}
51+
52+
p := podcast.New(
53+
feed.Title,
54+
feed.Link,
55+
feed.Description,
56+
feed.PublishedParsed,
57+
feed.UpdatedParsed,
58+
)
59+
60+
p.AddAtomLink(feed.Link)
61+
p.ISubtitle = feed.ITunesExt.Subtitle
62+
p.AddAuthor(feed.ITunesExt.Author, feed.ITunesExt.Owner.Email)
63+
p.AddImage(feed.ITunesExt.Image)
64+
p.AddSummary(feed.ITunesExt.Summary)
65+
for _, category := range feed.ITunesExt.Categories {
66+
p.AddCategory(category.Text, nil)
67+
}
68+
p.IOwner = &podcast.Author{
69+
Name: feed.ITunesExt.Owner.Name,
70+
Email: feed.ITunesExt.Owner.Email,
71+
}
72+
p.IExplicit = feed.ITunesExt.Explicit
73+
p.Language = feed.Language
74+
p.Copyright = feed.Copyright
75+
76+
for _, item := range feed.Items {
77+
podcastItem := podcast.Item{
78+
Title: item.Title,
79+
Description: item.Description,
80+
PubDate: item.PublishedParsed,
81+
}
82+
for _, itemEnclosure := range item.Enclosures {
83+
length, err := strconv.ParseInt(itemEnclosure.Length, 10, 64)
84+
if err != nil {
85+
return nil, err
86+
}
87+
podcastItem.AddEnclosure(itemEnclosure.URL, podcast.MP3, length)
88+
}
89+
if _, err := p.AddItem(podcastItem); err != nil {
90+
return nil, err
91+
}
92+
}
93+
94+
return &p, nil
1995
}
2096

2197
// CreateFeedItem creates a new feed's Item from the Config information.
@@ -62,7 +138,10 @@ func CreateFeedItem(cfg *Config) (*etree.Element, error) {
62138
}
63139

64140
// WriteXML creates a new file on disk with the appropriate XML contents from `doc`.
65-
func WriteXML(doc *etree.Document) {
66-
doc.Indent(2)
67-
doc.WriteToFile("new_feed.xml")
141+
func WriteXML(p *podcast.Podcast) {
142+
143+
// Podcast.Encode writes to an io.Writer
144+
if err := p.Encode(os.Stdout); err != nil {
145+
fmt.Println("error writing to stdout:", err.Error())
146+
}
68147
}

0 commit comments

Comments
 (0)