Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Package: osmtaggingr
Title: OpenStreetMap's tagging datasets
Title: OSMtaggingR: Reproducible Datasets about OpenStreetMap's tagging
Version: 0.0.0.9000
Authors@R: c(
person(
Expand Down
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Generated by roxygen2: do not edit by hand

export(get_proposals_info)
export(get_proposals_votes)
export(get_tagging_proposals)
export(get_voting_summary)
22 changes: 21 additions & 1 deletion R/data.R
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#' OSM Tagging Proposals
#' OSM Tagging Proposals Data
#'
#' A data frame with `r nrow(proposals)` rows and `r ncol(proposals)` columns, containing all
#' Approved, Rejected and Proposed tagging proposals from the [OSM Wiki](https://wiki.openstreetmap.org), as well as their associated metadata.
Expand Down Expand Up @@ -42,3 +42,23 @@
#'
#' @source <https://wiki.openstreetmap.org/wiki/Proposal_process>
"proposals"

#' OSM Tagging Proposals' votes Data
#'
#' A data frame with `r nrow(proposals_votes)` rows and `r ncol(proposals_votes)` columns, containing all
#' the votes cast for every tagging proposal in `proposals`, generated by `get_proposals_votes()`.
#'
#'
#' @format
#'
#' columns:
#' \describe{
#' \item{url}{(string) the URL to the tagging proposal wiki page.}
#' \item{votes_raw}{a string with the wiki page containing the proposal.}
#' \item{vote}{(factor) vote type (approve, abstain, oppose).}
#' \item{user}{(factor) voter's username}
#'
#' }
#'
#' @source <https://wiki.openstreetmap.org/wiki/Proposal_process#Voting>
"proposals_votes"
90 changes: 90 additions & 0 deletions R/get_proposals_votes.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
#' Scrape votes for tagging proposals
#'
#' Given a vector of urls, scrapes the tagging proposals to retrieve all the votes received.
#'
#' @param urls a string containing the tagging proposal to retrieve voting history from.
#'
#' @returns a dataframe with the following columns:
#' - `fullurl`: (string) the URL to the tagging proposal wiki page.
#' - `votes_raw`: (string) the raw text describing the vote.
#' - `vote`: (factor) vote (approve, abstain, oppose), inferred from the votes_raw text.
#' - `user`: (factor) voter's username.
#' - `date_vote`: (date) date in which the vote was cast.

#' @export
#'
#' @examples
#' proposal_votes <- get_proposals_votes('https://wiki.openstreetmap.org/wiki/Proposal:Electricity')
#'
#' proposal_votes
#'
get_proposals_votes <- function(urls) {
votes_df <- data.frame()
pb <- cli::cli_progress_bar("Scraping proposals", total = length(urls))

for (url in urls) {
tmp_df <- NULL
result <- tryCatch(
{
page <- rvest::read_html(url)
li_votes <- page |>
rvest::html_elements(xpath = "//*[@id='Voting']/following::ul/li") |>
rvest::html_text2()

tmp_df <- data.frame(
url = url,
votes_raw = li_votes
) |>
dplyr::mutate(
vote = dplyr::case_when(
stringr::str_detect(
votes_raw,
"I approve this proposal"
) ~ "Approve",
stringr::str_detect(
votes_raw,
"I oppose this proposal"
) ~ "Oppose",
stringr::str_detect(
votes_raw,
"I have comments but abstain from voting on this proposal"
) ~ "Abstain"
),
user = stringr::str_match(
votes_raw,
"[.\\-]\\s*([A-Za-z0-9 ]+)\\s*\\(talk\\)"
)[, 2],
user = stringr::str_remove_all(user, "-"),
user = stringr::str_remove_all(user, "\\."),
user = stringr::str_remove(user, "\\(talk\\)"),
user = stringr::str_trim(user)
) |>
dplyr::filter(!is.na(vote))
tmp_df
},
error = function(e) {
# If error, create a row with url and empty columns
message(sprintf("Could not scrape %s: %s", url, e$message))
data.frame(
url = url,
votes_raw = NA_character_,
vote = NA_character_,
user = NA_character_
)
}
)

votes_df <- dplyr::bind_rows(votes_df, result)
cli::cli_progress_update()
Sys.sleep(runif(1, min = 5, max = 30))
}
votes_df <- votes_df |>
dplyr::mutate(
vote = as.factor(vote),
user = as.factor(user)
) |>
tibble::as_tibble()

cli::cli_progress_done()
return(votes_df)
}
16 changes: 10 additions & 6 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,25 @@ knitr::opts_chunk$set(
fig.path = "man/figures/README-",
out.width = "100%"
)
library(osmtaggingr)
```

# OpenStreetMap's tagging datasets
# OSMtaggingR: Reproducible Datasets about OpenStreetMap's tagging

<!-- badges: start -->

[![DOI](https://zenodo.org/badge/1165670803.svg)](https://doi.org/10.5281/zenodo.18833755)

[![Project Status: WIP – Initial development is in progress, but there has not yet been a stable, usable release suitable for the public.](https://www.repostatus.org/badges/latest/wip.svg)](https://www.repostatus.org/#wip)

<!-- badges: end -->

The goal of `{osmtaggingr}` is to provide reproducible datasets containing information about [OpenStreetMap](http://openstreetmap.org)'s tagging proposals. It does so by querying and webscrapping OSM's wiki via helper functions.

The package contains the following:
The package contains the following datasets, as well as associated metadata and functions to recreate them:

1. Datasets (`proposals`) and associated metadata (see `?proposals`)
2. Helper functions and description
1. `proposals`: A data frame with `r nrow(proposals)` rows and `r ncol(proposals)` columns, containing all Approved, Rejected and Proposed tagging proposals from the [OSM Wiki](https://wiki.openstreetmap.org), as well as their associated metadata (run `?proposals` to see its metadata).
2. `proposals_votes`: A data frame with `r nrow(proposals_votes)` rows and `r ncol(proposals_votes)` columns, containing all the votes cast for every tagging proposal in `proposals`, generated by `get_proposals_votes()` (run `?proposals_votes` to see its metadata).


## Acknowledgements
Expand All @@ -39,11 +42,11 @@ This is an output of the research project _["Can digital goods be neutral? Evalu
You are free to use and reuse this tool under the licence conditions.
If you use this package in your work, please cite it as below:

> Cámara-Menoyo C, Monteath T (2026). OSMtaggingR: Creates OpenStreetMap's tagging Datasets. R package version 0.0.0.9000, https://github.com/WarwickCIM/OSMtaggingR.
> Cámara-Menoyo C, Monteath T (2026). OSMtaggingR: Reproducible Datasets about OpenStreetMap's tagging. R package version 0.0.0.9000, https://github.com/WarwickCIM/OSMtaggingR.

```bibtex
@Manual{,
title = {OSMtaggingR: Creates OpenStreetMap's tagging Datasets},
title = {OSMtaggingR: Reproducible Datasets about OpenStreetMap's tagging},
author = {Carlos Cámara-Menoyo and Timothy Monteath},
year = {2026},
note = {R package version 0.0.0.9000},
Expand Down Expand Up @@ -71,6 +74,7 @@ library(osmtaggingr)

# Load the built in datasets.
data(proposals)
data(proposals_votes)

# Converting to tibble for better representation.
proposals <- tibble::as_tibble(proposals)
Expand Down
26 changes: 19 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,34 @@

<!-- README.md is generated from README.Rmd. Please edit that file and then run `devtools::build_readme()` to render with the latest version of the code.-->

# OpenStreetMap’s tagging datasets
# OSMtaggingR: Reproducible Datasets about OpenStreetMap’s tagging

<!-- badges: start -->

[![DOI](https://zenodo.org/badge/1165670803.svg)](https://doi.org/10.5281/zenodo.18833755)

[![Project Status: WIP – Initial development is in progress, but there
has not yet been a stable, usable release suitable for the
public.](https://www.repostatus.org/badges/latest/wip.svg)](https://www.repostatus.org/#wip)

<!-- badges: end -->

The goal of `{osmtaggingr}` is to provide reproducible datasets
containing information about [OpenStreetMap](http://openstreetmap.org)’s
tagging proposals. It does so by querying and webscrapping OSM’s wiki
via helper functions.

The package contains the following:
The package contains the following datasets, as well as associated
metadata and functions to recreate them:

1. Datasets (`proposals`) and associated metadata (see `?proposals`)
2. Helper functions and description
1. `proposals`: A data frame with 724 rows and 27 columns, containing
all Approved, Rejected and Proposed tagging proposals from the [OSM
Wiki](https://wiki.openstreetmap.org), as well as their associated
metadata (run `?proposals` to see its metadata).
2. `proposals_votes`: A data frame with 8487 rows and 4 columns,
containing all the votes cast for every tagging proposal in
`proposals`, generated by `get_proposals_votes()` (run
`?proposals_votes` to see its metadata).

## Acknowledgements

Expand All @@ -33,13 +44,13 @@ Good Research Fund 2024-25.
You are free to use and reuse this tool under the licence conditions.
If you use this package in your work, please cite it as below:

> Cámara-Menoyo C, Monteath T (2026). OSMtaggingR: Creates
> OpenStreetMap’s tagging Datasets. R package version 0.0.0.9000,
> Cámara-Menoyo C, Monteath T (2026). OSMtaggingR: Reproducible Datasets
> about OpenStreetMap’s tagging. R package version 0.0.0.9000,
> <https://github.com/WarwickCIM/OSMtaggingR>.

``` bibtex
@Manual{,
title = {OSMtaggingR: Creates OpenStreetMap's tagging Datasets},
title = {OSMtaggingR: Reproducible Datasets about OpenStreetMap's tagging},
author = {Carlos Cámara-Menoyo and Timothy Monteath},
year = {2026},
note = {R package version 0.0.0.9000},
Expand All @@ -66,6 +77,7 @@ library(osmtaggingr)

# Load the built in datasets.
data(proposals)
data(proposals_votes)

# Converting to tibble for better representation.
proposals <- tibble::as_tibble(proposals)
Expand Down
1 change: 1 addition & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ reference:
Datasets provided by this package
contents:
- proposals
- proposals_votes
- title: "Data retrieval"
desc: "Functions used to retrieve data"
contents:
Expand Down
10 changes: 10 additions & 0 deletions data-raw/tagging_proposals.R
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
## This code prepares `tagging_proposals` dataset
library(osmtaggingr)


# Tagging proposals -------------------------------------------------------

proposals <- get_tagging_proposals(max_items = 1000)

usethis::use_data(proposals, overwrite = TRUE)


# Proposals votes ---------------------------------------------------------

proposals_votes <- get_proposals_votes(proposals$fullurl)

usethis::use_data(proposals_votes, overwrite = TRUE)
Binary file added data/proposals_votes.rda
Binary file not shown.
30 changes: 30 additions & 0 deletions man/get_proposals_votes.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/proposals.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 27 additions & 0 deletions man/proposals_votes.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions osmtaggingr.Rproj
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,13 @@ SaveWorkspace: No
AlwaysSaveHistory: Default

EnableCodeIndexing: Yes
UseSpacesForTab: Yes
NumSpacesForTab: 2
Encoding: UTF-8

RnwWeave: Sweave
LaTeX: pdfLaTeX

AutoAppendNewline: Yes
StripTrailingWhitespace: Yes
LineEndingConversion: Posix
Expand Down