R shiny with a package installed from github

[betatim]

This comes from jupyterhub/mybinder.org-deploy#907 reported by @Athanasiamo:

Hi. Is there any way to install a package from github in a binderhub shiny app deploy?

I've tried adding devtools::install_github() in the install.R file, (after devtools install ofc), but that > does not seem to do it? At least the build is failing.

[betatim]

The lines in install.R get run as an Rscript (I think that is the right terminology) so in principle you should be able to use devtools::install_github() there. We also use it to install other things for you. So this should work :-/ Maybe a link to the repository is indeed the next step.

[drmowinckels]

Hey! I found this issue on repo2docker, and I am trying to get it working with a commit tag

#397

[drmowinckels]

It completes building with the new commit tag to install_github(), but fails to launch the application.

# An error has occurred
The application failed to start.

The application exited during initialization.

It would be great if we could get more information during such instances, because it it really hard to debug what is going on based on this error message.

Github repo in question: https://github.com/LCBC-UiO/virtual_histology_2019

We're trying to launch the shiny application in the folder vh_new

[betatim]

It seems like this is a question about how to change the RShiny server configuration so that you get more details. Not sure how to do that, might be a question for the RShiny server documentation. Hopefully they let you configure the emaount of debug information via an environment variable which you can set in a start script. Or start your own RShiny server from a terminal once inside the binder and then visit .../proxy/<portnumber> to see what is happening.

For local debugging I find it useful to install http://repo2docker.readthedocs.io/ and run it locally, that way you get the most amounts of logs and faster turn around times compared to mybinder.org.

[drmowinckels]

I'm testing with repo2docker now, but am afraid the error message for the shiny instance is not helping alot.

In this case I'm trying to run the very simplified shiny in the folder vh_new_simp, after docker has been built and launched.

500 GET /shiny/vh_new_simp/?token=39a10562139bdebc608ed34ee561146ae688c66e82dbb7b4 (172.17.0.1) 424.10ms referer=None
[I 13:13:24.736 NotebookApp] No existing shiny found
[I 13:13:24.737 NotebookApp] Starting process...
[2019-03-08T13:13:25.272] [INFO] shiny-server - Shiny Server v1.5.7.907 (Node.js v8.10.0)
[2019-03-08T13:13:25.274] [INFO] shiny-server - Using config file "/tmp/tmpp9cpfi2k"
[2019-03-08T13:13:25.330] [INFO] shiny-server - Starting listener on 0.0.0.0:43243
[2019-03-08T13:13:25.776] [INFO] shiny-server - Created bookmark state directory: /var/lib/shiny-server/bookmarks
[2019-03-08T13:13:25.776] [INFO] shiny-server - Created user bookmark state directory: /var/lib/shiny-server/bookmarks/amowinckel
[2019-03-08T13:13:26.167] [INFO] shiny-server - Error getting worker: Error: The application exited during initialization.
[I 13:13:26.193 NotebookApp] shiny startup complete
[2019-03-08T13:13:26.580] [INFO] shiny-server - Error getting worker: Error: The application exited during initialization.
[E 13:13:26.589 NotebookApp] {
      "Host": "127.0.0.1:58945",
      "Connection": "keep-alive",
      "Upgrade-Insecure-Requests": "1",
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36",
      "Dnt": "1",
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
      "Accept-Encoding": "gzip, deflate, br",
      "Accept-Language": "en-GB,en;q=0.9,nb-NO;q=0.8,nb;q=0.7,en-US;q=0.6",
      "Cookie": "username-127-0-0-1-58945=\"2|1:0|10:1552050730|24:username-127-0-0-1-58945|44:YWM1NGNkYjM0ZWJiNDI2ODg1Mzc2ZDk2ZTA2MGIyMWM=|f91d0c4c6dbd0a8943a360cd2d224a9677fbc8eebba2daf6559f81d6ccccbd64\"; _xsrf=2|23c36901|45e56a1bd7128101fc69e2017f7218e5|1552050730"
    }
[E 13:13:26.589 NotebookApp] 500 GET /shiny/vh_new_simp/?token=39a10562139bdebc608ed34ee561146ae688c66e82dbb7b4 (172.17.0.1) 1854.13ms referer=None

[betatim]

My hunch is that this is a problem with the shiny app or the shiny server (or most likely the combination of repo2docker, shiny app and rshiny server). On mybinder.org I would have thought it could have been the maximum memory size we give people. You can modify the link to launch the app like so https://mybinder.org/v2/gh/LCBC-UiO/virtual_histology_2019/master?urlpath=shiny/ which just shows the directory contents (via the RShiny app). However that also fails so I think it isn't a problem in Binder that is causing this :-/

Do you know someone who knows more about RShiny (I've never used it or really debugged it) who could have ideas?

[drmowinckels]

the shiny app it self is running just fine locally, also in completely new R sessions. I have no experience with working with rshiny server though. This app also launches just fine on shinyapps.io, so I'm quite confident it is not the app it self, but likely something wrong in the specification between the app and repo2docker, but it is really hard to debug with the sparse information repo2docker is giving.

[betatim]

I tried running the shiny app from RStudio by visiting the repo on mybinder.org, going to RStudio, opening vh_new_simp/server.R and then pressing "Run app" at the top of the editor window.

That ends up printing the following in the console:

> shiny::runApp('vh_new_simp')
Loading required package: shiny
Error in library(shinydashboard) : 
  there is no package called ‘shinydashboard’

Do you know what shinydashboard is? Is this a package your app uses or something that shiny provides?

it would be good to get this more detailed output when you visit the shiny app directly in the browser. I suspect that there is a config option we could set when starting the shiny server to do that. I haven't yet found the documentation on how to find out what the options are though.

[drmowinckels]

omg! I'm missing shinydashboard from the install.R. That is pretty facepalmy. Just pushed a new version adding it, can't believe I missed it. Hoping this is the fix to all the issues, but as you note, being able to get more info on where things are failing would be amazing. I had not even thought that the RStudio instance might be able to run the shiny app, that is a very nice trick to know for future debugging.

[betatim]

I had not even thought that the RStudio instance might be able to run the shiny app, that is a very nice trick to know for future debugging.

How do people normally run shiny apps locally? I found it by clicking around the RStudio GUI and noticing that there was a "Run app" button. We should add a FAQ entry about this way of running a shiny app and getting more output so that people from the future can find it.

In addition to a FAQ entry a short how-to on Shiny apps (how to modify the mybinder.org URL, how to debug it, pitfalls when you come from shinyapps.io, etc) here would be great but a lot more work than adding a FAQ entry.

Do you want to make a Pull Request to add to the FAQ?

[drmowinckels]

absolutely, the rstudio things for debugging shiny is a good idea to have in the faq. I can start on something if you like. While my issue is not 100% resolved yet, I'm learning all sorts of tricks ;)

Right now, the shiny app in vh_new (the main shiny app we want to serve) is being launched through rstudio in mybinder, but is failing to launch with the shiny/vh_new url