How can I use this module as a part of a neovim plugin

[jghauser]

Hi!

I'm the author of the papis.nvim neovim plugin. It's a companion plugin for the papis cli reference manager. The plugin allows users to search their library, insert references into their documents, add notes, and so on. One functionality some users have asked for is to insert formatted citations. I've implemented a pretty basic version of this, but it would obviously be much nicer to be able to insert proper citations using CSL. Neovim plugins can now use luarocks and so I was thinking of leveraging your really neat looking module for this functionality. However, I'm a bit unsure how to do so because your focus in the readme seems to be on LaTeX documents.

Is there some way I could import your module, call some function with some bibliographical data, a citation style, and get back a formatted citation? I would really appreciate some pointers!

Let me know if I can provide some more information.

[zepinglee]

The current API is very likely to citeproc-js's (see https://citeproc-js.readthedocs.io/en/latest/running.html) since the beginning of this project but I'm not quite satisfied with it. I'm planning to improve the Lua API and that's why the documentation for Lua users is not written yet. It's appreciated if you give some suggestions on it. Also mention @Omikhleia if you are interested in this.

API

The following are important functions/methods.

citeproc.new()

local citeproc = require("citeproc")
local csl_engine = citeproc.new(citeproc_sys, style, lang, force_lang)

This function returns a CSL Engine instance from a given style. sys is a table with two functions: retrieveLocale(lang) and retrieveItem(id). The former returns the locale content of a given language code like en-US or zh-CN (see CSL's official locales repository https://github.com/citation-style-language/locales). The latter returns the item data of a given id in a lua table which is similar to CSL-JSON format (see CSL schema). The sys is mainly used in the Engine for loading data.

makeCitationCluster()

citation_str = csl_engine:makeCitationCluster(cite_items)

This is a simple method for generating a citation. It doesn't handle disambiguation (e.g., [Doe 2005; Doe 2005] => [Doe 2005a; Doe 2005b]) or citation position (e.g. ibid. and Supra note 4). The cite_items is a list of table each of which has an id field and optionally prefix, suffix, locator (along with its label) fields. See the CSL schema for details.

processCitationCluster()

local citation1 = {
  citationID = "CITATION-1",
  citationItems = {
    {
      id = "ITEM-2",
    }
  },
  properties = {
    noteIndex = 0
  }
}
local result = csl_engine:processCitationCluster(citation, citations_pre, citations_post)

The processCitationCluster() method is more complex. citation is a table with citationID, citationItems, properties fields. The citationItems is a list of citation items described above. The properties table must have a noteIndex field which is the note number if the citation in a footnote or endnote or 0 if the citation is in-text. The properties table can also have suppress-author or author-only field for extended formats.

The citations_pre and citations_post are lists of tuples of citation IDs and note numbers. For example, there are three citations in the following order: Citation-1, Citation-2, Citation-3. If Citation-4 is inserted between 2 and 3, the citations_pre is like { {"Citation-1", 1}, {"Citation-2", 2} } and citations_post is { {"Citation-3", 4} }. The note number can be changed.

The returned result is a tuple where result[1] is a table of parameters and result[2] is a list of changed citations. (I didn't know Lua function can return multiple objects when I wrote this.) At the moment, the result[1] is not implemented and its content doesn't make sense now. result[2] is a list of tables each of which is a tuple of citation index (not likely to be useful), citation string, and citation ID (e.g., {1, "[Doe 2005]", "Citation-1"}).

makeBibliography()

local result = csl_engine:makeBibliography()

This method returns the formatted bibliography. The returned result is a tuple where result[1] is a table of parameters and result[2] is a list of bibliography strings.

Improvements

I've thought of the following improvements. Any idea is welcomed.

1. The camel case function (method) names look odd in Lua projects. I prefer lowercase with underscores like process_citation_cluster().
2. Make processCitationCluster and makeBibliography returning multiple objects rather than a tuple (actually a table): bib_strs, params = engine:makeBibliography().

Full example

The following is a full example which can be run in the root of the repository (provided citeproc is installed via luarocks build).

local citeproc = require("citeproc")
local inspect = require("inspect")

-- Item data
local item_dict = {
  ["ITEM-1"] = {
    id = "ITEM-1",
    type = "book",
    author = {
      {
        family = "D’Arcus",
        given = "Bruce"
      }
    },
    issued = {
      ["date-parts"] = {
        {
          2005,
          11,
          22
        }
      }
    },
    publisher = "Routledge",
    title = "Boundaries of dissent: Protest and state power in the media age"
  },
  ["ITEM-2"] = {
    id = "ITEM-2",
    type = "article-journal",
    author = {
      {
        family = "Bennett",
        given = "Frank G.",
        suffix = "Jr."
      }
    },
    ["container-title"] = "Pac. Rim L. & Pol’y J.",
    issued = {
      ["date-parts"] = {
        {
          2009,
          8
        }
      }
    },
    page = "463-509",
    title = "Getting property right: “Informal” mortgages in the Japanese courts",
    volume = "18"
  }
}

local citeproc_sys = {
  retrieveLocale = function (lang)
    if not lang then
      return nil
    end
    local locale_path = "./tests/locales/locales-" .. lang .. ".xml"
    local file = io.open(locale_path, "r")
    if not file then
      return nil
    end
    local content = file:read("*a")
    file:close()
    return content
  end,
  retrieveItem = function (id)
    return item_dict[id]
  end,
}

-- Load the CSL style string
local style_path = "./submodules/styles/apa.csl"
local file = io.open(style_path, "r")
if not file then
  error("Style file not found")
end
local csl_style = file:read("*a")
file:close()

-- Make a CSL engine instance.
local csl_engine = citeproc.new(citeproc_sys, csl_style, "en-US")
-- The default output format of citeproc-lua is LaTeX and we can switch to HTML.
csl_engine:set_output_format("html")

-- Make a citation
local citation1 = {
  citationID = "CITATION-1",
  citationItems = {
    {
      id = "ITEM-1",
      label = "page",
      locator = "42-44",
      prefix = "See",
      suffix = "and his other works."
    }
  },
  properties = {
    noteIndex = 0
  }
}
local citations_pre = {}
local citations_post = {}

local cite1 = csl_engine:processCitationCluster(citation1, citations_pre, citations_post)
print(inspect(cite1))

-- Generate the current bibliography
local bib_result = csl_engine:makeBibliography()
print(inspect(bib_result))

-- Append another citation
local citation2 = {
  citationID = "CITATION-2",
  citationItems = {
    {
      id = "ITEM-2",
    }
  },
  properties = {
    noteIndex = 0
  }
}
citations_pre = {
  {"CITATION-1", 0}
}
citations_post = {}

local cite2 = csl_engine:processCitationCluster(citation2, citations_pre, citations_post)
print(inspect(cite2))

bib_result = csl_engine:makeBibliography()
print(inspect(bib_result))

[jghauser]

Wow, this is great! I'll play around with this and let you know how it went and what improvements I'll come up with along the way. Thanks again for the detailed answer! 😊