Add thor job graph command

.gitignore

@@ -17,3 +17,5 @@
 **/.DS_Store
 spec/fixtures/*.mrk
 .rubocop-https*
+*.mmd
+*.png

CHANGELOG.adoc

@@ -47,12 +47,12 @@ https://github.com/lyrasis/kiba-extend/pull/252[PR#252]
 
 * Initial job tests functionality, including `thor job_tests suite` command and `JobTest::CsvJob::Equal` test type https://github.com/lyrasis/kiba-extend/pull/246[PR#246]
 * `JobTest::CsvJob::Equal` job test type https://github.com/lyrasis/kiba-extend/pull/247[PR#247]
-* Add `mode` parameter to `Kiba::Extend::Job.output?` method.
+* `mode` parameter to `Kiba::Extend::Job.output?` method.
 The purpose of this is to turn off warnings if a job is not necessarily expected to exist, such as may be defined in dynamic code.
 https://github.com/lyrasis/kiba-extend/pull/249[PR#249]
-* Add `Kiba::Extend::Job.registered?` method, to check for whether a job exists, without verifying it has output, or reporting errors/warnings if it does not.
+* `Kiba::Extend::Job.registered?` method, to check for whether a job exists, without verifying it has output, or reporting errors/warnings if it does not.
 https://github.com/lyrasis/kiba-extend/pull/249[PR#249]
-* Add `report` parameter to `Delete::EmptyFields` transform, which causes deleted field names to be written to STDOUT.
+* `report` parameter to `Delete::EmptyFields` transform, which causes deleted field names to be written to STDOUT.
 This is useful if you want to mark deleted fields as omitted from a project, or otherwise keep track of what fields were empty at a given point in time
 https://github.com/lyrasis/kiba-extend/pull/250[PR#250]
 * `Normalize::FieldValues` transform - This is a wrapper around the improved `Utils::StringNormalizer`
@@ -67,8 +67,10 @@ https://github.com/lyrasis/kiba-extend/pull/259[PR#259]
 https://github.com/lyrasis/kiba-extend/pull/260[PR#260]
 * https://lyrasis.github.io/kiba-extend/Kiba/Extend/Transforms/Replace/GlobalReverse.html[`Replace::GlobalReverse` transform]
 https://github.com/lyrasis/kiba-extend/pull/260[PR#260]
-* Add handling of `omit_from_all_fields` param to https://lyrasis.github.io/kiba-extend/Kiba/Extend/Transforms/Allable.html[`Allable` transform mixin] and all transforms that use it
+* Handling of `omit_from_all_fields` param to https://lyrasis.github.io/kiba-extend/Kiba/Extend/Transforms/Allable.html[`Allable` transform mixin] and all transforms that use it
 https://github.com/lyrasis/kiba-extend/pull/260[PR#260]
+* `thor job graph` command, which will generate and open a .png dependency graph for the specified job key
+https://github.com/lyrasis/kiba-extend/pull/264[PR#264]
 
 === Bugfixes
 

Gemfile.lock

@@ -16,12 +16,14 @@ PATH
       amazing_print
       bigdecimal
       csv
+      diagram
       dry-configurable
       dry-container
       kiba
       kiba-common
       marc
       measured
+      mermaid
       psych
       thor
       xxhash (= 0.7.0)
@@ -52,6 +54,12 @@ GEM
     connection_pool (3.0.2)
     csv (3.3.5)
     date (3.5.1)
+    diagram (0.3.4)
+      dry-equalizer (>= 0.2)
+      dry-struct (>= 1.6.0)
+      dry-types (>= 1.0)
+      json
+      zeitwerk (>= 2.6)
     diff-lcs (1.6.2)
     docile (1.4.1)
     drb (2.2.3)
@@ -64,9 +72,29 @@ GEM
       concurrent-ruby (~> 1.0)
       logger
       zeitwerk (~> 2.6)
+    dry-equalizer (0.3.0)
+    dry-inflector (1.3.1)
+    dry-logic (1.6.0)
+      bigdecimal
+      concurrent-ruby (~> 1.0)
+      dry-core (~> 1.1)
+      zeitwerk (~> 2.6)
+    dry-struct (1.8.1)
+      dry-core (~> 1.1)
+      dry-types (~> 1.8, >= 1.8.2)
+      ice_nine (~> 0.11)
+      zeitwerk (~> 2.6)
+    dry-types (1.9.1)
+      bigdecimal (>= 3.0)
+      concurrent-ruby (~> 1.0)
+      dry-core (~> 1.0)
+      dry-inflector (~> 1.0)
+      dry-logic (~> 1.4)
+      zeitwerk (~> 2.6)
     erb (6.0.4)
     i18n (1.14.8)
       concurrent-ruby (~> 1.0)
+    ice_nine (0.11.2)
     io-console (0.8.2)
     irb (1.18.0)
       pp (>= 0.6.0)
@@ -87,6 +115,9 @@ GEM
       rexml
     measured (3.2.1)
       activesupport (>= 5.2)
+    mermaid (0.0.6)
+      diagram (>= 0.3.4)
+      zeitwerk
     method_source (1.1.0)
     minitest (6.0.6)
       drb (~> 2.0)

README.md

@@ -44,6 +44,30 @@ On the to-do list:
 
 - Wiki documentation for how to use the registry and job templating. In the meantime the best place to get an understanding of this is [kiba-extend-project](https://github.com/lyrasis/kiba-extend-project).
 
+## Non-Ruby/non-bundleable dependencies {#dep}
+
+### Rendering mermaid job dependency graphs {#mermaidrenderdep}
+
+If you wish to use the `thor job graph` command to render mermaid dependency graphs, you need to install the https://github.com/coolamit/mermaid-cli[the Go mmd-cli]. This is a replacement for the official mermaid.js/mermaid-cli, which is embedded deeply in the node/npm ecosystem, and which has some really annoying and problematic dependencies and Mac bugs.
+
+The preferred way to do this in the Lyrasis Data Migrations team environment is via https://mise.jdx.dev[mise] and then go's install command:
+
+Install go if you do not already have it:
+
+    mise use -g go@1.25.11 # or later version
+
+Install chromium. This is becoming more problematic, as the brew version doesn't work, and conflicts with ARM architecture exist. As of June 2026, https://github.com/ungoogled-software/ungoogled-chromium-macos[Ungoogled Chromium for the MacOS] works:
+
+    brew install --cask ungoogled-chromium
+
+Install mermaid-cli:
+
+    go install github.com/coolamit/mermaid-cli/cmd/mmd-cli@latest
+
+Alternate installation paths are documented at https://github.com/mermaid-js/mermaid-cli[the mermaid-cli GitHub repository].
+
+If you ever want to uninstall mmd-cli, do `which mmd-cli` and then delete the directory at that path.
+
 ## Documentation
 
 [API documentation](https://lyrasis.github.io/kiba-extend/)

kiba-extend.gemspec

@@ -43,12 +43,14 @@ Gem::Specification.new do |spec|
   spec.add_dependency "bigdecimal"
   spec.add_dependency "amazing_print"
   spec.add_dependency "csv"
+  spec.add_dependency "diagram"
   spec.add_dependency "dry-configurable"
   spec.add_dependency "dry-container"
   spec.add_dependency "kiba"
   spec.add_dependency "kiba-common"
   spec.add_dependency "marc"
   spec.add_dependency "measured"
+  spec.add_dependency "mermaid"
   spec.add_dependency "psych"
   spec.add_dependency "thor"
   # @todo can replace with stdlib?

lib/kiba/extend/project_config.rb

@@ -11,6 +11,11 @@ module ProjectConfig
 
       extend Dry::Configurable
 
+      # Path to directory in which Mermaid .mmd files and generated
+      #   image files will be stored. If not populated,
+      # @return [String, NilClass]
+      setting :graph_dir, reader: true, default: nil
+
       # Configuration for making and reversing global replacements in your
       #   project via the {Transforms::Replace::GlobalReversible} and
       #   {Transforms::Replace::GlobalReverse} transforms.

lib/kiba/extend/registry/ancestorable.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+require "diagram"
+
+module Kiba
+  module Extend
+    module Registry
+      # Mixin module for getting Node and Edge properties from RegisteredSource
+      #   and RegisteredLookup objects
+      module Ancestorable
+        include NodeLabelable
+
+        def node
+          Diagrams::Elements::Node.new(id: node_id, label: node_label)
+        end
+
+        def node_id = key.to_s
+
+        def edge
+          Diagrams::Elements::Edge.new(
+            source_id: edge_source_id,
+            target_id: edge_target_id,
+            label: edge_label
+          )
+        end
+
+        def edge_source_id = node_id
+
+        def edge_target_id = for_job.to_s
+
+        def edge_label
+          return "" if source?
+
+          "lookup on :#{lookup_on}"
+        end
+
+        def source? = is_a?(RegisteredSource)
+      end
+    end
+  end
+end

lib/kiba/extend/registry/creator.rb

@@ -19,8 +19,10 @@ def initialize(spec)
           set_vars
         end
 
+        def files = get_job.files
+
         def call
-          job = args ? mod.send(meth, **args) : mod.send(meth)
+          job = get_job
           job.run
           job
         end
@@ -38,6 +40,8 @@ def to_s
 
         attr_reader :spec
 
+        def get_job = args ? mod.send(meth, **args) : mod.send(meth)
+
         def args_type_ok?
           spec[:args].is_a?(Hash)
         end

lib/kiba/extend/registry/file_registry_entry.rb

@@ -13,6 +13,7 @@ module Registry
       class FileRegistryEntry
         include EntrySummarizable
         include EntryValidatable
+        include Treeable
 
         attr_reader :path, :key,
           :creator, :supplied, :dest_special_opts, :desc, :lookup_on, :tags,

lib/kiba/extend/registry/node_labelable.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+module Kiba
+  module Extend
+    module Registry
+      # Mixin module for generating Node label for dependency graph
+      module NodeLabelable
+        def node_label
+          return "#{node_id} (supplied)" if supplied
+          return node_id if desc.empty?
+
+          "#{node_id}\n#{desc}"
+        end
+      end
+    end
+  end
+end

lib/kiba/extend/registry/registered_lookup.rb

@@ -12,6 +12,7 @@ module Registry
       #
       # Assumes this file will be used to build a {Kiba::Extend::Lookup}
       class RegisteredLookup < RegisteredFile
+        include Ancestorable
         include RequirableFile
 
         # @param key [Symbol, Hash] file key from {FileRegistry} data

lib/kiba/extend/registry/registered_source.rb

@@ -19,6 +19,7 @@ def initialize(dest_class)
       # Value object representing a {Kiba::Extend::Registry::RegistryEntry}
       #   being used as a job source
       class RegisteredSource < RegisteredFile
+        include Ancestorable
         include RequirableFile
 
         # Arguments for calling Kiba Source class

lib/kiba/extend/registry/treeable.rb

@@ -0,0 +1,58 @@
+# frozen_string_literal: true
+
+require "diagram"
+require "mermaid"
+
+module Kiba
+  module Extend
+    module Registry
+      # Mixin module for generating dependency tree diagram for entry
+      module Treeable
+        include NodeLabelable
+
+        def parents
+          files = creator.files
+          [files[:source], files[:lookup]].compact
+            .flatten
+        rescue NoMethodError
+          []
+        end
+
+        def ancestors
+          result = [parents]
+          until result.last.empty?
+            result << traverse_up(result.last)
+          end
+          result.flatten.compact
+        end
+
+        def diagram
+          Diagrams::FlowchartDiagram.new(nodes: nodes, edges: edges)
+        end
+
+        def mermaid = diagram.to_mermaid
+
+        private
+
+        def traverse_up(elements)
+          elements.map do |member|
+            Kiba::Extend.registry
+              .resolve(member.key)
+              .parents
+          end.flatten
+            .compact
+        end
+
+        def node_id = key
+
+        def node = Diagrams::Elements::Node.new(id: node_id, label: node_label)
+
+        def nodes = ancestors.uniq { |anc| anc.key }
+          .map { |anc| anc.node } + [node]
+
+        def edges = ancestors.map { |anc| anc.edge }
+          .uniq
+      end
+    end
+  end
+end

lib/tasks/job.thor

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+require "thor"
+
+class Job < Thor
+  desc "graph JOB", "Render and save dependency graph for given job"
+  def graph(job)
+    unless mermaid_cli_installed?
+      puts "mmd-cli (Mermaid CLI) command not installed or not found on path. "\
+        "See README for installation instructions if you wish to use this "\
+        "command."
+      exit(1)
+    end
+
+    mermaid = Kiba::Extend.registry
+      .resolve(job.to_sym)
+      .mermaid
+
+    dir = Kiba::Extend::ProjectConfig.graph_dir
+    unless dir
+      puts "TIP: Add `Kiba::Extend::ProjectConfig.config.graph_dir = "\
+        "\"path_to_directory_for_your_project\" to your project config, "\
+        "so your graphs can be saved with your other project files."
+    end
+
+    mmd_dir = dir || File.join(Kiba::Extend.ke_dir, "data")
+    FileUtils.mkdir_p(mmd_dir) unless Dir.exist?(mmd_dir)
+
+    mmd_path = File.join(mmd_dir, "#{job}.mmd")
+    File.open(mmd_path, "w") { |f| f << mermaid }
+
+    png_path = File.join(mmd_dir, "#{job}.png")
+    `mmd-cli -i #{mmd_path} -o #{png_path}`
+
+    `open #{png_path}`
+    exit(0)
+  end
+
+  no_commands do
+    def mermaid_cli_installed?
+      result = `which mmd-cli`
+      true unless result.empty?
+    end
+  end
+end

spec/helpers.rb

@@ -103,6 +103,25 @@ def fake_creator_method
     FileUtils.touch(File.join(fixtures_dir, "base_job_missing.csv"))
   end
 
+  def dependency_graph_registry
+    Kiba::Extend.config.registry = Kiba::Extend::Registry::FileRegistry
+    extra_entries = {
+      b: {path: File.join(fixtures_dir, "b.csv"),
+          creator: B},
+      c: {path: File.join(fixtures_dir, "c.csv"),
+          creator: C},
+      d: {path: File.join(fixtures_dir, "d.csv"),
+          creator: D},
+      e: {path: File.join(fixtures_dir, "e.csv"),
+          creator: E},
+      f: {path: File.join(fixtures_dir, "f.csv"),
+          creator: F}
+
+    }
+    populate_registry(more_entries: extra_entries)
+    transform_registry
+  end
+
   class OutputJob
     attr_reader :outrows