Support visual learning of button sequences with mini-diagrams

[nrichards]

This PR introduces small animated reference images alongside many text commands for the Qun control surface:

• 
• PDF from images shows complete button sequences
• See a example below.
• And see the whole-file preview.
• And see all the new generated images in the manual_images/but folder (https://github.com/nrichards/Qun-mk2/tree/button-images/manual_images/but).

It uses a new tool mdpicgen, created to automate both creating these images from sets of PNGs, and updating the README.md.

• Supports GIF. Is cool.
• Uses image set, any image type.
• Can format Markdown - optional.
• Has diagnostics.

Additional changes

• Add Button "tips" table to Sequencer Live Recording
• Separates "Set BPM" to 3 entries

Usage

Set up

CLONE: Clone https://github.com/nrichards/mdpicgen -- generates images and writes updated Markdown.

Update Qun README.md, for new button commands

WRITE: For tables with headers "Button | ... "

• As normal, write a button sequence
  • E.g. "SHIFT + MODE PLAY | Dump the current preset ...."
• Also add a <br> after the button sequence
  • E.g. "SHIFT + MODE PLAY <br>| Dump the current preset ...."

GENERATE: Then generate the images + an updated Markdown file:

$ cd qun-mk2

$ python3 ../mdpicgen --md-file README.md --md-out-file README-merge_me.md \
--image-out-dir manual_images/but --image-height 56 --gif \
imageset --imageset-file ../mdpicgen/qunmk2_imageset.csv --imageset-dir ../mdpicgen/imageset

COPY:

• Inspect README-merge_me.md -- DIFF, and use Markdown preview tool.
• Inspect the images in manual_images/but.
• Copy the README-merge_me.md over the README.md.
• Git commit.

Customize it

• Change the height with --image-height param.
• Change the style
  • Base images (4 images for Qun) in folder imageset
  • Layout (x,y) in CSV qunmk2_imageset.csv
• Tweak and debug button sequences (regular expressions) in qunmk2.patset

Motivation

Qun offers several features through single-/multiple-button presses. Qun is a small device. The README documentation helps identify features by using text to declare the button commands to access these features, e.g. "SHIFT + LOOPER PLAY + B[1-3] or B[7-8]".

Visual thinkers like me can be challenged to learn button press sequence when presented as text. I find it challenging to translate the button sequence text into a mental visual or haptic experience while I simultaneously read and think about the functional description.

I find that when I have an image to look at, I can learn more effectively -- I can look at the description and then at the combination of buttons .. and I mentally feel my hand shape which is needed to execute the Qun command. I can repeat this thinking moment to teach myself more quickly and comfortably than when I also am visualizing text which declares a set of commands.

Limitation

• Reads / writes FIRST table column of Markdown, with "Button" header
• Writes AFTER new "<br>" tags inside table column, after button text

Examples

Diff - mdpicgen updates Markdown

New table for Sequencer Live Recording

qun-seq-live-rec-buttons.mov

PDF shows "poster images" with full sequence

nrichards_Qun-mk2 at button-images.pdf

[raspy135]

It looks good but I'm not sure that I can maintain this for long term.
I also prefer white boarder lines, I edited PSD locally

[raspy135]

I think Photoshop is not convenient enough for long term maintenance.
I need a Python script to generate the image, input is combination of buttons such as "S+B1", output is image. Format would be PNG, GIF or animated GIF. Are you familiar with Python? Otherwise I'll take a look when I have time.

[nrichards]

It looks good but I'm not sure that I can maintain this for long term.
I also prefer white boarder lines, I edited PSD locally

Very glad to hear you say that this looks good. I want to help understand Qun myself, and the images help me.

? Please would you share that file here in the PR discussion thread (drag & drop in a comment), for my reference? Thank you for the "white boarder lines" refinement idea.

Yes I agree, "cost" of maintaining this is concerning ... more time, mistakes, etc. Qun is fun after learning it, and this extra manual documentation would be extra tedium. I do think it would never become fun, making images, after any amount of time.

I think Photoshop is not convenient enough for long term maintenance. I need a Python script to generate the image, input is combination of buttons such as "S+B1", output is image. Format would be PNG, GIF or animated GIF. Are you familiar with Python? Otherwise I'll take a look when I have time.

I am and I will start working on PNG output. And I will update here as it develops.

? What is the idea for GIF (animated)? Is it, e.g. "S+B1 then S+B2" => two-frame GIF?

My steps for script:

• Focus on Photoshop PSD file, delete Affinity Designer file from PR.
• python should have a library to extract the layers from a PSD
• I would need to rasterize the layers from command line, and also composite them into a final image. python may have a rasterization library, or Image Magik.
• the script must generate images based upon the layers and the button names
• it would be nice if the script could automatically update all the images.
• I will explore extracting the button sequences from the markdown
• if that is possible, i might also be able to update the markdown file itself with ![](path_to_png) in the table column after the button sequence column

[raspy135]

I think it's simple enough to draw everything programmatically then we can remove Photoshop dependency.
I tried to upload PSD file here but psd was not supported. I'll send it to you on Discord.

[nrichards]

File

I'll send it to you on Discord.

I received it. The colors and design look fresh, thank you!

I tried to upload PSD file here but psd was not supported.

ZIP files are supported. They are not "unrolled" for inline preview in a discussion thread :(. So first compress it to share a PSD.

Progress

I am happy with the Python progress. Thank you for encouraging a programmatic approach.

The script now generates a single image from a PSD and a layer list of strings:

• Input == ( a_file.psd, [ "SHIFT - s", "B1 - 1" ] )
• Output == [ s_b1.png ]

Next, I plan to support normal button names: use "B1" instead of "B1 - 1" as above. Then to generate multiple small images. And finally to insert image links into the Markdown file. Soon. 👀

The free & open source app Krita can edit PSDs. It is a paint app with layers. Commercial apps are not needed, Adobe Photoshop nor Affinity Designer.

Draw Everything Programmatically

I think it's simple enough to draw everything programmatically then we can remove Photoshop dependency.

I understand. Krita is also an app dependency, and dependencies give incompatibility + bugs. And geometric drawing can be done in Python. So I could define a JSON, e.g define "B1" and "B2" with:

[
{"control":"B1","name":"1","shapes":
[{"shape":"rect","bounds":[238,120,268,180],"color":"0x000000"},{"shape":"rect","bounds":[242,124,264,176],"color":"0xff0000"}]},
{"control":"B2","name":"2","shapes":
[{"shape":"rect","bounds":[278,120,308,180],"color":"0x000000"},{"shape":"rect","bounds":[282,124,304,176],"color":"0xff0000"}]}
   ...
]

.. and then parse & execute in Python with Pillow https://note.nkmk.me/en/python-pillow-imagedraw/.

I plan to explore this as an optimization later, after prototyping the full script. I have a working image renderer now. And I can improve the app-dependency issue later.

[nrichards]

I agree more strongly with you @raspy135 about removing the PSD file from the script toolchain. I tested alternatives this morning, and found problem after problem. So I will support JSON file + Pillow before I release the script.

I will move the PR to "DRAFT" status, and continue working.

Details, for reference only - not important

I tested various apps:

• Photoshop & Affinity are $$ commercial. They do work okay, however the apps change over time, and there are incompatibilities between the two today.
• Krita, Gimp, SumpPaint - free-ish. They do NOT work reliably.
• Paint.NET is Windows only - untested

[nrichards]

I created a Python script to generate all the images. Input a Qun README.md and output is images: https://github.com/nrichards/psd-in-md

Next:

Next I plan to:

• migrate to compositing flat-images instead of using PSD, and I will
• add updating of the Markdown itself. I may also
• add a report generator, for what the image filenames are, to allow easier manual updating of the Markdown. And I would like to
• explore making GIFs.

Cool

It's cool to see the images appear in the output folder - when I run the script, it takes 5 minutes to generate 70 images. This should be faster with flat-images, in the future.

Python's Markdown libraries reformat tables. The table columns are widened to fit the text, in the Markdown text file, by the libraries. Currently the tables are compact, and easier to edit, IMO. Since that would be a big and probably painful change, I will

• explore alternative Markdown updating approaches.

[nrichards]

@raspy135 I have not explored GIFs and I am open to ideas.

I did finish everything else. It has been fun learning and programming.

Q: What do you think about the format of the README now? https://github.com/nrichards/Qun-mk2/blob/1b2180217875421f0bde1a11e891b6bf07058621/README.md

Unsure

E.g. What I have here is (1). I wanted to minimize the changes. However, I am unsure if this layout - [ button names + button image ] is difficult?
.. compared to (2) which uses mores space. However with (2) I compress the Function / Description.

I feel like (2) looks prettier. I will ask the group on Discord what other ideas to try.

Ideas - Flexible tags - may be fun

• Use unrecognized new raw HTML tags: <bp> and <bpc> and <bpr>. Web browser ignores them.
• <bp> could insert a button picture for any button sequence (SHIFT+turn dial) found anywhere on the line. No need for <br>
• Like (purple 1) but more automatic, <bpr> could do like <bp> and automatically insert a HTML break <br> immediately before. it would skip adding a new BR tag if one already exists
• Like (purple 2) but automatic, <bpc> could create a new column and insert the picture in the column. it would skip adding a new column if one was detected.

[nrichards]

I'm happy with what this gives, and will reopen the PR for review now.

FYI @raspy135 - for your consideration.

[raspy135]

Thank you for all your effort.
Where's the Python file and dependencies? This has to be manageable by me as well.

[nrichards]

Thank you for all your effort. Where's the Python file and dependencies? This has to be manageable by me as well.

It is my pleasure. Please let me know if does not feel manageable.

Here are the Python scripts: https://github.com/nrichards/mdpicgen
I made a demo video (6 min): https://youtu.be/P0uTpepmJNM
I attached the shell commands: demo_mdpicgen.txt

The demo clones Qun-mk2, and mdpicgen, prepares dependencies, then CD's into mdpicgen, and runs to make gifs then markdown:

python3 . --md-file ../Qun-mk2/README.md  --gif --image-out-dir ../Qun-mk2/manual_images/but imageset
python3 . --md-file ../Qun-mk2/README.md --md-out-file ../out.md --gif --image-out-dir manual_images/but

I am happy to walk through this