Potential improvements: Should smile support SSG and could it be refactored into an npm package?

[jmuchovej]

👋 Firstly, thanks for building this! Certainly does quite a bit of heavy lifting and is far beyond psiTurk! 😃

I suppose there are two proposals here:

1. smile seems ripe for using static site generators (SSG) (like Netlify, Cloudflare Pages, etc.). I'm curious if there was a motivation to not do this?
2. What are your thoughts on converting smile into a proper npm package? While forking smile for use with a lab certainly makes sense, integrating changes will certainly prove more challenging – e.g. say some time has passed since @csc-yale has synced with the NYUCCL/smile repo, now we want to integrate the updates, this becomes a pretty significant task to not only correctly rebase but also to propagate the changes coming in from the NYUCCL/smile copy.

For (1), I'm actively testing this, though if this is desirable, it would make sense to use an existing SSG like Astro as they handle a lot of the quirks of developing SSGs. Astro is incredly fast uses Vite, and ships minimal JS by default. (I can elaborate further on Astro, but it's one of the few SSG frameworks that's actually component-library agnostic.)

For (2), tools like Astro provide for templates – so someone could spin-up a new experiment like so:

npm create astro@latest -- --template nyuccl/smile

Templates, in Astro, provide the user with a way to import components directly from the template, but they also provide a ways for components to be overridden and otherwise customized.

Additionally, there are ways to make the "start from a template experience" more seamless. There are also ways in which things like secrets management, shared configurations, etc. could be pulled in as well, but that's only relevant if (2) is something you find worth pursuing.

(Also, I'm happy to spearhead some/all of these things, just wanted to test the waters before I start chunking time for it. 🙂)

[gureckis]

Thanks for the feedback John!

On the first topic of SSG -- most smile experiments are heavy on animations and other dynamic content (e.g., full featured video games or other multiplayer settings). I understand the value of SSG for rendering static blogs or making web pages SEO but I'm not sure why SSG would be interesting for dynamic experiments (and even could be detrimental because you can't exert as much control over which pages are accessible if they each have a different url).

On the second topic - Smile could be a npm package or at least parts of it probably could. But there's also kind of a lot of stuff specific to github (e.g., using github actions to send deploy hooks to Slack, etc...). That might not work as well within a npm package. But the javascript libraries and components definitely could. More particularly I think one day it'd be nice if Smile had a component library like VueUse which people could import to get lots of the functionality of common components (consent forms, etc...) in a ways similar to how JsPsych is modular. I think you are suggesting something similar and it'd be interesting to look at how Vue is recommending people make component libraries (e.g., https://www.youtube.com/watch?v=qLAzMT3paA4).

Incorporating updates to the base code isn't usually a huge problem -- We have instructions on how to merge changes from the root repo into your current project. Of course there could be compatibility issues if months/years go by and you return to a project, but during the development cycle this is still quite easy. https://smile.gureckislab.org/developing.html#merging-changes-to-the-template-into-your-existing-project. In the future after a few more development goals stablize we might shift to a semantic versioning model for Smile template so you could trust that you can merge changes from 1.1.X but not 1.2..X of the template.

We went with this approach of installing the secrets in git itself, but encrypted (using git-secret). It's a little clunky but has some advantages because some of the current methods of doing this involve extra (paid) services, etc... (like dotenv stuff). But really the "start from template" is pretty seamless currently assuming you first configure your organization/lab set up. The only reason it is even a little bit involved is because it uses github actions to coordinate a few things like a hosting service and Slack messages/notifications. But in our daily workflow these have been pretty nice actually. A few other labs have gone through the steps of getting going with this and have found it pretty easy.

On a mild todo list is to document a bit more of this with videos this semester. So if you had some very strong suggestions on making things easier I'd be happy to discuss them because we could change things before we double down too much on the "easy install" instructions.

[jmuchovej]

👋 (Sorry for the incredibly slow reply! 😅)

Just to clarify – most SSGs (notably Nuxt.js and Astro, which both support Vue) have first-party support for SSR (Server-Side Rendering). These aren't SSGs like Jekyll, Hugo, Documenter.jl, etc.

I suppose my "proposal" (if you can even call it that) to use existing SSGs is to:

1. reduce self-rolled code, which needs to vigilantly be documented otherwise documentation debt begins to accrue.
2. using mainstream SSGs also opens the door for build-hooks beyond a lab's server (e.g., Netlify, Cloudflare Pages, Vercel, etc.) – these platforms have first-party support for nearly every mainstream SSG and requires about 5min to link to your repository and configure. (
3. using tools that are highly popular (typically) increases the likelihood of finding similar bugs (and fixes, hopefully) – while bugs that might crop-up in development with a Smile that uses most self-rolled code may require a deep understanding of Smile to correct (which could prove very time consuming to build such an understanding).

I see (2) and (3) as possible DX bumps that might push someone back to Qualtrics (but perhaps I'm jumping the gun on this 🙂).

---

I'm not sure if you've heard of sli.dev (GitHub), but when I proposed converting Smile into an installable package, I was thinking of something along similar lines. Essentially slidev has a few different packages. For something like smile, I was thinking it would be sharded along these lines:

• create-experiment – which would allow users to run something like npm init smile (or similar).
  • This would walk users through customizing the initial configuration and setting up a working experiment from a template (or setting up an empty directory structure).
• client – which would be the actual infrastructure for serving/building experiments (this would handle most vite-related and router-related things).
  • This would, to some degree, mirror psiturk server and the like.
• types – which would just have the various types configurations expect and the like.
  • Extracted out partially to avoid circular dependencies.
• components – which would be a component (and layout?) library like you mentioned earlier.
  • Each component would live here in the same way jsPsych "components" live in packages/. Each should definitely be published individually to allow for tree shaking to work its magic, too.

---

As for things like GitHub Actions, Slack notification hooks, etc. – those could all, very easily (and sensibly), be part of create-experiment! This heavily borrows on ideas from PkgTemplates.jl which is a Julia package that is strictly focused on configuring Julia projects with things like testing tools, documentation builders, GitHub (or GitLab or Travis CI, etc.) automation, etc.

---

In some of my free time, I'm currently exploring how this kind of sharding might be doable, so I'm happy to loop you in (or hop on a call if that would be helpful) as things progress.

[jmuchovej]

@gureckis friendly ping about this. 🙂

[gureckis]

Hi John,
Thanks for following up. We have a weekly meeting about this on Wednesday afternoon, if you email me I can send you a zoom link and you could hop on and maybe we could discuss and I could learn more about how you are thinking about this.

Thanks,
Todd