Plugin architecture brainstorm #215

dillonkearns · 2020-03-11T16:16:10Z

dillonkearns
Mar 11, 2020
Maintainer

I'm thinking about ways to extend elm-pages for plugin authors.

Goals

Prevent users from having to wire in several pieces to "install" a plugin - for example, if a plugin adds a global <head> and generates a file - should be able to pipeline that easily with the user's config and with other plugins
Configuration should all be in pure Elm
Plugin builder API should be secure, which means that it's probably not the exact same API as users are using to build up their Pages.Platform.application
The functions for users to init a Platform.Pages.application should be simple, intuitive, and make it natural to do the right thing. So for example, we want API authors to be able to add global <head> tags, and different plugins (maybe a sitemap and RSS plugin) might both add global head tags. But for the user, it probably makes sense to define their head tags from a central location, rather than being able to arbitrarily chain it and then have to trace where in their code it came from.

It might look something like this:

Pages.init { ... }
|> Pages.withPlugin (Rss.plugin rssOptions)
|> Pages.withPlugin (SiteMap.plugin sitemapOptions)

Security

StaticHttp
Accessing secrets

Potential Strategies

User accepts through config file or CLI

The user could generate a list of files

{
    "dillonkearns/elm-pages-rss": {
    },
    "dillonkearns/elm-pages-sanity": {
        "domains": ["https://*.sanity.io/*"],
        "secrets": ["SANITY_TOKEN"]
    }
}

This seems reasonable for domains. But for secrets, what if the user wants to call it something else? Not very configurable (though the user might want to configure it for domains as well).

User passes a value in that restricts possible values

For example, if you want to allow a plugin to use a secret, the only way a plugin author can access it is if you pass it in to the plugin:

Pages.GoogleCalendar.install (Secrets.get "GOOGLE_CALENDAR_TOKEN")

Use cases

Generate files (access to StaticHttp)
Add global head tag (access to StaticHttp)
Generate a config or JavaScript file that is run by something like an admin CMS page to configure it
Generate a config file for services like Netlify

File generators

Typically just generate files where the user may depend on StaticHttp, but they don't directly call it, only pass through the user's StaticHttp.

RSS reader (generate files - user may depend on StaticHttp)
Sitemap (generate files - user may depend on StaticHttp)
Netlify CMS routing rules (would need access to all page paths)

Example:

plugin
    |> RssPlugin.generate
        { siteTagline = siteTagline
        , siteUrl = canonicalSiteUrl
        , title = "elm-pages Blog"
        , builtAt = Pages.builtAt
        , indexPage = Pages.pages.blog.index
        }
        metadataToRssItem

metadataToRssItem :
    { path : PagePath Pages.PathKey
    , frontmatter : Metadata
    , body : String
    }
    -> Maybe Rss.Item
metadataToRssItem page =
    case page.frontmatter of
        Metadata.Article article ->
            if article.draft then
                Nothing

            else
                Just
                    { title = article.title
                    , description = article.description
                    , url = PagePath.toString page.path
                    , categories = []
                    , author = article.author.name
                    , pubDate = Rss.Date article.published
                    , content = Nothing
                    }

        _ ->
            Nothing


RssPlugin.generate :
    { siteTagline : String
    , siteUrl : String
    , title : String
    , builtAt : Time.Posix
    , indexPage : PagePath pathKey
    }
    ->
        ({ path : PagePath pathKey
         , frontmatter : metadata
         , body : String
         }
         -> Maybe Rss.Item
        )
    -> Builder pathKey userModel userMsg metadata view 
    -> Builder pathKey userModel userMsg metadata view 
RssPlugin.generate options metadataToRssItem builder =
    let
        feedFilePath =
            (options.indexPage
                |> PagePath.toPath
            )
                ++ [ "feed.xml" ]
    in
    builder
        |> Pages.Platform.withFileGenerator
            (\siteMetadata ->
                { path = feedFilePath
                , content =
                    Rss.generate
                        { title = options.title
                        , description = options.siteTagline

                        -- TODO make sure you don't add an extra "/"
                        , url = options.siteUrl ++ "/" ++ PagePath.toString options.indexPage
                        , lastBuildTime = options.builtAt
                        , generator = Just "elm-pages"
                        , items = siteMetadata |> List.filterMap metadataToRssItem
                        , siteUrl = options.siteUrl
                        }
                }
                    |> Ok
                    |> List.singleton
                    |> StaticHttp.succeed
            )
        |> Pages.Platform.addGlobalHeadTags [ Head.rssLink (feedFilePath |> String.join "/") ]

Source Plugin

StaticHttp is core
Likely will provide a Json.Decode.Exploration.Decoder to encapsulate the data that the service gives you (what if you want to strip out the data you don't use? Do we need a builder-style syntax that allows you to just add in the pieces you want, and store it in your preferred data type? Would be similar to how elm-graphql provides you decoders but you piece them together)
Could encapsulate reading from the filesystem
Might specify how to parse file body or metadata?

Examples:

Contentful source
Google Calendar source
File system source - Parse markdown content from content folder (or let user specify)
Sanity.io source - not much to this, though, it just points to a GraphQL endpoint, so maybe there's no need for this

Things that could just wrap the core APIs

With the designs I have and am imaginging for features, I think plugin authors can just wrap them if they want a different API from these primitives:

Encapsulate image optimizations
Manifest configuration

Tracing errors

Let's say you have some build-time error, like:

Generating files with duplicate names
Failed StaticHttp request or decoder

The error message should clearly state that the conflicting generated file is conflicting with the plugin named X, or that the HTTP failure came from the plugin making HTTP call Y.

ringods · 2020-10-20T11:36:01Z

ringods
Oct 20, 2020

Hello @dillonkearns,

when designing a plugin mechanism, it first of all must be clear what the sequence of actions are in the lifecycle of publishing a site. As an example, here are the different steps how Antora does it:

https://gitlab.com/antora/antora/-/blob/master/packages/site-generator-default/lib/generate-site.js#L15-30

Once you have this, it must be clear which are the pluggable points and what API each of these pluggable points has. What I like about Antora is that it is the only static site generator I currently know off which can collect content from different repositories. On top of that, it can retrieve multiple tags from each repository to publish versioned documentation.

Linking this back to the plugin architecture, retrieving from multiple repositories could be a "content loader" plugin implementation, which off course needs access to the overall config to retrieve which sources to read from.

Another point I am thinking about is that some plugin points might only accept a single implementation (rss fead generation?) while other plugin points can have a chain of implementations (e.g. content loaders)

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Plugin architecture brainstorm #215

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 1 comment

{{title}}

Select a reply

Plugin architecture brainstorm #215

dillonkearns Mar 11, 2020 Maintainer

Goals

Security

Potential Strategies

User accepts through config file or CLI

User passes a value in that restricts possible values

Use cases

File generators

Source Plugin

Things that could just wrap the core APIs

Tracing errors

Replies: 1 comment

ringods Oct 20, 2020

dillonkearns
Mar 11, 2020
Maintainer

ringods
Oct 20, 2020