|
8 | 8 |
|
9 | 9 | ## Glossary {#glossary} |
10 | 10 |
|
11 | | -Full job key |
12 | | -: A Ruby Symbol built from `FileRegistry` namespace (if used) and registry entry name, separated by the value of `Kiba::Extend.registry_namespace_separator` (defaults to `__` (two underscores)). Examples: With namespace: `:namespace__entry_name`; No namespace: `:unnamespaced_entry_name`. |
13 | 11 | File registry key |
14 | 12 | : Synonym for `Full job key`. |
15 | 13 |
|
| 14 | +Full job key |
| 15 | +: A Ruby Symbol built from `FileRegistry` namespace (if used) and registry entry name, separated by the value of `Kiba::Extend.registry_namespace_separator` (defaults to `__` (two underscores)). Examples: With namespace: `:namespace__entry_name`; No namespace: `:unnamespaced_entry_name`. |
| 16 | + |
| 17 | +Job definition module |
| 18 | +: Ruby Module in your project that defines: (1) source, destination, and optional lookups for the job; and (2) the data transformations for the job. See [Kiba::Extend::Jobs](https://lyrasis.github.io/kiba-extend/Kiba/Extend/Jobs.html). |
| 19 | + |
| 20 | +## Typology of kiba-extend projects {#projecttypes} |
| 21 | + |
| 22 | +### Oneoff/direct |
| 23 | + |
| 24 | +This project uses kiba-extend directly and is self-contained. [kiba-extend-project](https://github.com/lyrasis/kiba-extend-project) is an example of a oneoff/direct project. |
| 25 | + |
| 26 | +### Parent |
| 27 | + |
| 28 | +One powerful way of using kiba-extend is to create an "abstract" parent ETL project. |
| 29 | + |
| 30 | +A parent project handles the general logic of transforming data from a specific source system into the format required by a given target system. |
| 31 | +For example, if you frequently need to migrate data from OldSystem to NewSystem, you may create a parent OldSystem kiba-extend project that can handle the general structure of data out of OldSystem and its transformation, such as: |
| 32 | + |
| 33 | +- what the source data files are; |
| 34 | +- hardcoded enum values that need to be replaced in the data; |
| 35 | +- what standard data preprocessing needs to be done; |
| 36 | +- how to merge data from lookup tables; and |
| 37 | +- remapping the data into the "shape" you need it to be in for NewSystem. |
| 38 | + |
| 39 | +All the specifics that may change per specific instance of such a project are defined as configuration settings in the parent project. |
| 40 | +For instance one OldSystem user may only want to migrate records with `active=true` values to NewSystem, while another may wish to also migrate all records regardless of `active` status. |
| 41 | + |
| 42 | +In this case, you might define a `migrate_inactive` setting in the parent project. |
| 43 | + |
| 44 | +### Child |
| 45 | + |
| 46 | +A project for a specific client or data set, based on a Parent project |
| 47 | + |
| 48 | +You set the client-specific values for the configuration settings you defined in the parent project (such as `migrate_inactive`) for each client in the child projects. |
| 49 | + |
| 50 | +Client-specific jobs and transforms can also be defined in child projects as needed. |
| 51 | + |
| 52 | +Theoretically, you can also have grandchild projects and even deeper projects. |
| 53 | +However, those would get pretty difficult to understand and manage. |
| 54 | + |
16 | 55 | ## Assumptions and concepts used in libraries used by `kiba-extend` {#other-libraries} |
17 | 56 |
|
18 | 57 | ### Folder structure, file names, module/class name constants defined in files {#zeitwerk} |
|
0 commit comments