Skip to content

Problem with chromote package #159

Open
@sale4cast

Description

@sale4cast

Dear community,

First of all, I would like to thanks everyone for your previous support. I want to extract "hotel prices" from google. From your previous suggestions, I can extract the first page prices perfectly. However, I want to extract also second page prices. Here I attach the code below:

library(shiny)
library(curl)
library(chromote)
library(pagedown)

ui <- fluidPage(
  textOutput("targetId"),
  tableOutput("result")
)

r <- Chromote$new()

server <- function(session, input, output) {
  driver <- r$new_session()
  output$targetId <- renderText(paste0("Target ID: ", driver$get_target_id()))
  p <- driver$Page$loadEventFired(wait_ = FALSE)
  driver$Page$navigate("https://google.com", wait_ = FALSE)
  
  p$then(function(value){
    googleSearchText <- "4 star hotel in barcelona"
    driver$Runtime$evaluate(paste0('document.querySelector("textarea").value = "', googleSearchText,'"'))
    
    {
      p2 <- driver$Page$loadEventFired(wait_ = FALSE)
      driver$Runtime$evaluate('document.querySelector("input[aria-label=\'Google Search\']").click()', wait_ = FALSE)
    }
    
    p2
  })$then(function(value){
    driver$Runtime$evaluate('document.querySelector("div.R2w7Jd").click()')
    {
      p3 <- driver$Page$loadEventFired(wait_ = FALSE)
      driver$Runtime$evaluate('document.querySelector("div.JWXKNd").click()', wait_ = FALSE)
    }
    p3
  })$then(function(value){
    priceElement <- driver$Runtime$evaluate(
      'var elements = document.querySelectorAll(".K1smNd > c-wiz[jsrenderer=\'hAbFdb\'] .PwV1Ac");
                 var elementPrices = [];
                 elements.forEach(function(element) {
                   elementPrices.push(element.innerText);
                 });
                 elementPrices.join("@");'
    )
    driver$close()
    
    splittedPriceElements <- unlist(strsplit(priceElement$result$value, "@"))
    
    neighborHotelRoomPrices <- lapply(splittedPriceElements, function(aElement){
      ## Great Deal\n$80    Deal 5%\n$90
      roomPrice <- unlist(strsplit(aElement,"\n"))
      if(length(roomPrice) > 1){
        return(roomPrice[[2]])
      }
      return (aElement)
    })
    output$result <- renderTable(as.data.frame(unlist(neighborHotelRoomPrices)))
  })
}


shinyApp(ui, server)

Now, I want to extract second page prices as like the screenshot below:

Screenshot_368

Question: How can I extract second page prices in asynchronous way?

Note: Here I extract prices from a single page url and data updated in the particular results section only.

Best Regards,
SaleForecast

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions