-
Notifications
You must be signed in to change notification settings - Fork 13
Open
Description
I used the following go program to page through the v2 advisories API:
- main.go
package main
import (
"encoding/json"
"fmt"
"net/http"
"time"
)
type advisories struct {
Advisories []advisory `json:"advisories"`
LastUpdated time.Time `json:"lastUpdated"`
Page int `json:"page"`
Size int `json:"size"`
Total int `json:"total"`
}
type advisory struct {
Name string `json:"name,omitempty"`
}
func main() {
if err := run(); err != nil {
panic(err)
}
}
func run() error {
u := "https://errata.build.resf.org/api/v2/advisories?page=%d&limit=25"
for i := 0; ; i++ {
as, err := func() ([]advisory, error) {
resp, err := http.Get(fmt.Sprintf(u, i))
if err != nil {
return nil, err
}
defer resp.Body.Close()
var advs advisories
if err := json.NewDecoder(resp.Body).Decode(&advs); err != nil {
return nil, err
}
return advs.Advisories, nil
}()
if err != nil {
return err
}
if len(as) == 0 {
break
}
for _, a := range as {
fmt.Println(a.Name)
}
time.Sleep(1 * time.Second)
}
return nil
}$ go run main.go | sort | uniq -c > advs.txt
$ cat advs.txt
...
1 RLSA-2025:4051
2 RLSA-2025:4063
4 RLSA-2025:4170
1 RLSA-2025:4244
1 RLSA-2025:4263
1 RLSA-2025:4341
3 RLSA-2025:4362
1 RLSA-2025:4443
3 RLSA-2025:4458
3 RLSA-2025:4459
1 RLSA-2025:4460
3 RLSA-2025:4461
1 RLSA-2025:4487
...
$ cat advs.txt | wc -l
7400At the point where I ran the command, the total number of advisories was 8014. (No additional advisories were added during this time.)
The actual number of unique advisories that I was able to retrieve by paging was 7400.
Looking at advs.txt, I noticed that many of the advisories appeared duplicated while paging.
How can I get all advisories without any duplicates?
Metadata
Metadata
Assignees
Labels
No labels