Version: 0.3.2
Emacs package, provide block #+begin_cui in Org mode for communication with LLMs APIs. Inspired by Robert Krahn’s org-ai package
Protocol: “OpenAI like” HTTP REST API. (url.el)
Philosophy: stateless, one-direction for text, extensibility and comfort.
- Tags and links expansion of @ tags, Org [] links
- Noweb and tangling support as if it is src block of Org mode
- Parallel requests in the same or different buffer without blocking
- Highlighting of markdown big blocks, tags and links, headers, bold text, Org tables, LaTeX
- Request with image or audio like
@/a.jpgor[[/a.wav]]or[[file:/a.jpg]] - Xref jumping For Elisp Markdown blocks
- Transparency for inspecting raw data of request, powerful debugger
- Customization and autofilling, hooks
- Prompt engineering tools, there is
:chainof calls out-of-the-box - No external dependencies
- At left - optional debugging window
- At center - main window with agent answer.
- At right - optional help window with expanded link.
Emacs 29.1+, 30.2
Token API for remote LLMs.
Optional: org-links, available at MELA and at https://github.com/Anoncheg1/emacs-org-links
| File | Purpose |
|---|---|
| cui.el | Minor mode and C-c C-c activation of #+begin_cui |
| cui-restapi.el | Connection to REST API (OpenAI-compatible) |
| cui-block.el | Handling cui block |
| cui-block-msgs.el | for chat messages |
| cui-block-tags.el | Expand tags and links in block |
| cui-timers.el | Managing requests, buffers and notifications |
| cui-prompt.el | (optional) Working with prompt-engineering, :chain parameter |
| cui-async1.el | (optional) Solve “callback hell” for url.el, used by cui-prompt.el |
| cui-optional.el | (optional) Hooks functions for postprocessing |
cui.el
├─ cui-block.el
├─ cui-block-msgs.el (→ cui-block.el)
├─ cui-block-tags.el (→ cui-block.el, cui-block-msgs.el)
├─ cui-restapi.el (→ cui-block.el, cui-block-msgs.el, cui-async1.el, cui-timers.el)
└─ cui-prompt.el (→ cui-block.el, cui-block-msgs.el, cui-block-tags.el, cui-restapi.el, cui-async1.el, cui-timers.el)
- remote: any server that provides an OpenAI-compliant endpoint to LLM model or agent.
- local inference engines like llama.cpp, vLLM, huggingface, etc.
cui-restapi-con-service ./cui-restapi.el
[*] In practice all LLMs are not free, remote are considered as SaaSS (Service as a Software Substitute). These systems perform remote computation, not provide access to data, model weights, code, or licensing rights needed to reproduce the model.
- add to
.emacs
(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
(add-to-list 'package-archives '("melpa-stable" . "https://stable.melpa.org/packages/") t)
(package-initialize)- Install via
M-x package-install RET cui RETorM-x package-list-packages
cui-restapi-con-token - “token-string” or token per service: '(:together "token-string1" :local "token-string2") or nil for reading from “~/.authinfo” or “~/.authinfo.gpg”
cui-restapi-con-service - default service symbol if not specified in header of cui block like this: #+begin_cui :service local
(add-to-list 'load-path "path/to/cui")
(require 'cui)
(add-hook 'org-mode-hook #'cui-mode) ; cui.el
(setq cui-restapi-con-token "xxx") ; cui-restapi.el(plist-put cui-restapi-con-endpoints :local "http://localhost:8000/v1/chat/completions")
;; Optionally, set it as default
(setq cui-restapi-con-service 'local)Redefine:
cui-restapi--get-headers- If you need some special HTTP headers.cui-restapi--get-single-response-textandcui-restapi--normalize-response- if HTTP responses is not standard for"stream":falseand"stream":truerespectively.
By default fintified only major markdown elements: “#headers”, quotes, LaTeX, links&tags.
To enable all Org fintifications in cui blocks:
(setq org-protecting-blocks (delete "cui" org-protecting-blocks)) (setq org-protecting-blocks (delete "ai" org-protecting-blocks)) (org-restart-font-lock)
To disable all fontifications:
(setopt cui-fontification-flag nil)
or
(advice-add 'cui--add-cui-font-lock-to-org-keywords :override (lambda () t))
To set use custom function or disable auto-fill by setting it to nil:
M-x customize-variable RET cui-restapi-fill-function
- Simplies way to set in .emacs (same with “M-x customize”):
(setq cui-restapi-con-token "xxx")
- Next level is to keep token in separate Elisp file:
~/.emacs.d/cui-tokens.el
(setq cui-restapi-con-token "xxx")
.emacs
(add-to-list 'load-path "~/.emacs.d/cui-tokens.el")
(require 'cui-tokens)- Keep token in separate ”auth-source” file in plain text
To get more infomration use Shell commnd:
$ info auth-source
Secret should be stored in the format:
machine local password <your token>
or
machine local--0 password <your token> machine local--1 password <your token>
- Keep token in encrypted “auth-source” file. Full decription will be provided later…
This hooks add space “ “ before lines that in cui block that looks like Org header, that fix confustion for Org logic.
(add-hook 'cui-block-after-chat-insertion-hook #'cui-optional-remove-distant-empty-lines-hook-function)This hook remove empty lines if there is too much of them in response.
(add-hook 'cui-block-after-chat-insertion-hook #'cui-optional-remove-headers-hook-function)We provide cui-optional-markdown-cycle function same as org-cycle. It allow to fold and unfold markdown headers in cui block.
To set for Org mode, add those lines to your ~/.emacs file:
(require 'cui-optional)
(add-hook 'org-mode-hook #'cui-optional-markdown-folding-activation)
(add-hook 'org-tab-first-hook #'cui-optional-markdown-cycle) ; For TAB key on "# headers"—
If you use https://github.com/Anoncheg1/emacs-indent and indent-for-tab-command for TAB key instead of org-cycle:
(require 'cui-optional)
(add-hook 'org-mode-hook #'cui-optional-markdown-folding-activation)
(defun cui-org-mode-hook ()
(setq-local indent-for-tab-steps
(list
#'indent-for-tab-step-1-region-to-column
#'indent-for-tab-step-2-region-fill-prefix
#'indent-for-tab-step-3-region-indent-lines
#'cui-optional-markdown-cycle ; toggle Markdown header in cui block
#'indent-for-tab-step-4-insert-tab
#'indent-for-tab-step-5-indent-line
#'indent-for-tab-step-6-completion)))
(add-hook 'org-mode-hook 'cui-org-mode-hook)—
To cycle block use command M-x cui-optional-cycle-block or add this advice to make it with default “Shift-TAB” key:
(advice-add 'org-shifttab :around #'cui-optional-markdown-folding-shifttab-advice)- SYS-EVERYWHERE
- SYS - role: system
- AI - role: assistant
- AI_REASON - excluded
- ME - role: user
See cui-block-roles-prefixes in .cui-block.el for more info.
- :model - string, ex: :model “meta-llama/Llama-3.3-70B-Instruct-Turbo-Free”
- :service - label/string, ex: :service deepseek–1
- :max-tokens - int, ex: :max-tokens 300
- :stream - bool, ex: :stream nil
- :top-p - float, ex: :top-p 12.45
- :temperature - float
- :frequency-penalty - float
- :presence-penalty - float
- :sys-everywhere - string (persistant-sys-prompts) ./cui-block.el
- :chat :completion - bool
- :chain - bool ./cui-prompt.el
To get full request body and more information set:
(setopt cui-debug-buffer "*debug-cui*") (setq debug-on-error t) ; optional (ert-enabled t) ; debug output to stdout for ert tests
Built-in url.el have ability to output request and resonse headers, for that you need to set
(setq url-debug '(http))
Here we check that :content of the last user message is not just a string but a list, this happen when image link exist in body of ia block.
(defun my/cui-hook-multimodal-image-swith (plist)
(let* ((messages (plist-get plist :content ))
(idx (cui-block-msgs--find-last-user-index messages))
(last-user-mes (when messages (aref messages idx)))
(content (when last-user-mes (plist-get last-user-mes :content ))))
(when (and (stringp content)
(string-match cui-restapi--multimodal-internal-link-re content))
(plist-put plist :service "localmultimodal")
(plist-put plist :model nil)))
plist)
(add-hook 'cui-block-msgs-after-prepare-messages-hook #'my/cui-hook-multimodal-image-swith)Hook accpet two arguments (par1 par2)
- type
- simbol \=’role, \=’text or’end.
- role-text
- text or role name.
- pos
- position before text insertion.
- stream
- stream mode or a single insertion.
There is two implementations for hook in cui-optional.el:
- cui-optional-remove-distant-empty-lines-hook-function
- cui-optional-remove-headers-hook-function
Example to remove empty lines after LLM answer:
(require 'cui-optional) ; for `cui-optional-remove-distant-empty-lines'
(defun my/cui-postprocess (type _content _pos _stream)
(when (equal type 'end)
(save-excursion
(let* ((context (cui-block-p))
(con-beg (org-element-property :contents-begin context))
(con-end (org-element-property :contents-end context)))
(cui-optional-remove-distant-empty-lines con-beg con-end)))))
(add-hook 'cui-block-after-chat-insertion-hook #'my/cui-postprocess)There are two hooks for that:
- before any - raw text level: as `cui-block-parse-part-hook’ in cui-block.el
- after all - vector level: as `cui-restapi-after-prepare-messages-hook’ in cui-restapi.el
More info: doc.org in Main path of JSON decoding section.
There are cui-block-roles-prefixes variable that allow to customize roles at step of parsing cui block before sending.
For incoming RestAPI conversion to prefixes cui-block-roles-restapi variable used.
(cui-block-tags-get-content)
Or
(cui-block-tags--clear-properties
(cui-block-tags-replace (cui-block-get-content (cui-block-p) nil :export nil)
(list (cui-block-get-header-marker (cui-block-p)))))Here is example that print result in asynchronous way. Service name and model string define all connection information.
(setq debug-on-message "^#<buffer.*")
(let ((cui-restapi-show-error-function #'cui-restapi--show-error))
(cui-restapi--url-request 'github "openai/gpt-4.1"
(lambda (result) (print (cui-restapi--get-single-response-text result)))
:messages '((:role user :content "test")) ; chat
:stream nil))- org-in-src-block-p = cui-block-p
- cui-block-insert-result = cui-block-insert-result
- org-babel-where-is-src-block-result = cui-block-where-is-result
- org-fill-paragraph = cui-block-fill-paragraph
- org-babel-mark-block = cui-block-mark-at-point
- org-babel-find-named-block = cui-find-named-block
- org-src–contents-area = cui-block–region, cui-block–contents-region
- org-babel–normalize-body = cui-block–region, cui-block–contents-region
- org-babel-insert-header-arg - TODO!!
We dont activate noweb in subsequent block at expansion of links and tags.
“cui-noweb” (Org property for the entry at point-or-marker) is used to control noweb (org-entry-get) if :noweb block parameter is not set.
Example:
#+NAME: ina
#+begin_cui
test
#+end_cui
#+begin_cui
<<ina()>>
#+end_cui
Chain of noweb references is possible, referenced cui blocks evaluated with :tangle context.
“.cui” “.ai” files as links expanded as new chat messages, other files inserted in markdown block and with masked chat prefixes by adding “_” before chat role.
We add “cui” as a type of special block to variables org-protecting-blocks and org-structure-template-alist.
Org-mode intrernally use org-element-special-block-parser to process our cui block as special.
For highlighting we use hack to inject our font-lock staff to font-lock-defaults variable initialized by org-set-font-lock-defaults function. Source blocks only fontified with org-src-font-lock-fontify-block and not exposed as Org element for now, that is why it is not possible to use org-babel-do-in-edit-buffer to call functions inside of it with its major-mode of language.
For parallel requests we use cui-timers--element-marker-variable-dict variable that keeps pairs: url-buffer (key) -> Header-marker (variable). This allow for same block have many url-buffers.
For RESTAPI requesting LLMs we provide two functions:
cui-restapi-request-prepare- support streamingcui-restapi-request-llm-retries- retry request if it fails.
For make chain of calls we have cui-async1.el static framework that simplify making parallel and sequencial requests.
For keys remapping we bind keys to Org commands or key if pointer is not in cui block, we call original key or command with “hack” functions.
We do it in two steps, first we replace image link or tag with @image_jpeg:/path.jpg in cui-block-tags-replace and in cui-block-tags--compose-block-for-path-full as any other link, then cui-restapi--payload we encode image to base64 and split text around link and replace string in :context to a vector.
- Navigation in Dired, Packages, Buffers modes https://github.com/Anoncheg1/firstly-search
- Search with Chinese https://github.com/Anoncheg1/pinyin-isearch
- Ediff fix https://github.com/Anoncheg1/ediffnw
- Dired history https://github.com/Anoncheg1/dired-hist
- Selected window contrast https://github.com/Anoncheg1/selected-window-contrast
- Copy link to clipboard https://github.com/Anoncheg1/emacs-org-links
- Solution for “callback hell” https://github.com/Anoncheg1/emacs-async1
- Restore buffer state https://github.com/Anoncheg1/emacs-unmodified-buffer1
- outline.el usage https://github.com/Anoncheg1/emacs-outline-it
- hiding password in cafe https://github.com/Anoncheg1/emacs-hidepass
- Dates for Org-mode headers https://github.com/Anoncheg1/emacs-org-history
You can sponsor author crypto money directly with crypto currencies:
- BTC (Bitcoin) address: 1CcDWSQ2vgqv5LxZuWaHGW52B9fkT5io25
- USDT (Tether TRX-TRON) address: TVoXfYMkVYLnQZV3mGZ6GvmumuBfGsZzsN
- TON (Telegram) address: UQC8rjJFCHQkfdp7KmCkTZCb5dGzLFYe2TzsiZpfsnyTFt9D