New SolrMarc plug-in for indexing MARC-based hierarchies

[Owen-Fitz]

This a program in use at National Library Ireland which was formerly a beanshell script and its purpose is to update the Collections (hierarchybrowse) index. It iterates through the 773 subfield to identify whether an imported bib record is a collection level or part of a collection itself.

[demiankatz]

@Owen-Fitz, thanks for sharing! A few suggestions that might help make this a little more general-purpose:

1.) It might be worth including a commented-out line in marc_local.properties demonstrating the use of the function here.

2.) It would probably be a good idea to include a longer comment at the top of the file explaining what MARC formatting is expected.

3.) Perhaps it's worth considering adding parameters to make the field assignments more flexible, if this is relying on Virtua-specific behavior rather than MARC21 standards -- that way, it might be easier to adjust for use with other systems.

[demiankatz]

Thanks for the progress on this! I've added a few more quick thoughts below. Also, in addition to those, I wonder if it would be helpful to include an example MARC file using these hierarchies for inclusion in the test suite -- maybe up to ten records that relate to each other hierarchically, so there's a tangible example people can see of what is expected.

[demiankatz]

Should the class be org.vufind.index.Collection, based on the package specified in the Java file?

[demiankatz]

If we are going to add a new setting here, we should add comments explaining its purpose... and maybe it needs a more specific name, like ils_id_prefix. But I wonder if we're better off passing it as a parameter to the custom functions instead of making it a config setting, since its usage is currently rather narrow, and it may cause confusion if we make it a high-visibility configuration setting.

[demiankatz]

Should that comment say "prefix" instead of "number"?

[demiankatz]

How would you feel about passing the ILS prefix as a parameter here (as alluded above), so that these end up looking like:

#hierarchy_top_id = custom(org.vufind.Collection), getHierarchyTopID(vtls)
#hierarchy_top_title = custom(org.vufind.Collection), getHierarchyTopTitle(vtls)
#hierarchy_parent_id = custom(org.vufind.Collection), getParentID(vtls)
#hierarchy_parent_title = custom(org.vufind.Collection), getParentTitle(vtls)
#hierarchy_sequence = custom(org.vufind.Collection), getSequence(vtls)
#is_hierarchy_id = custom(org.vufind.Collection), isHierarchyID(vtls)
#is_hierarchy_title = custom(org.vufind.Collection),isHierarchyTitle(vtls)
#hierarchy_browse = custom(org.vufind.Collection), getHierarchyBrowse(vtls)

I realize it's a bit redundant, but it's also more specific. Implementing this would just require adding a string argument to all the methods in the class, and passing it down to support methods as needed.

[Owen-Fitz]

Hi @demiankatz , thanks for the prompt feedback on the recent changes. After a discussion with some team members , we do recognize that we are in the minority of vufind users that work with Virtua and the NLI conventions on use of 773 Marc tag won't apply to others which may result in this plugin not being upstreamed. Having said that we are happy to keep this as our own local import modification script. What are your thoughts on this?

[demiankatz]

@Owen-Fitz, sorry for the slow response -- I was out of office last week. I think even if this code is somewhat dependent on local conventions, it may still be useful to upstream, especially if we can provide a useful example of how it works. Since you've been using this approach for some time, and it seems to be working for you, it might be inspirational for others looking to create MARC-based hierarchies. I think there's still potential to make it all a bit more flexible/configurable, but we can always start somewhere and grow from there as needed.

When I talk about having an example, I'm suggesting that you add some files to the tests/data directory. If you look at the existing collections.mrc and collections.mrc.properties files in there, you'll see how we currently set up hierarchies for integration testing. You could create a different pair of files to demonstrate your approach, and we could add a test to confirm that they load correctly -- then we'll have some way of identifying future breaks/regressions of the new code, and people wanting to adopt your approach can see exactly what mappings are needed and how the records look in real life.

Does that make any sense? If it seems like too much work, of course it's also fine if you prefer to keep this local for now. :-) Another option would be to upload the Java class to a JIRA ticket for future reference, if you want to share it but don't feel that it's ready for mainline code yet.

[demiankatz]

@Owen-Fitz, I'm just reviewing open pull requests and noticed that this conversation stalled a few months ago. Just touching base to check on the status. :-)

[demiankatz]

@Owen-Fitz, just checking in again: are you still interested in finishing up this work, or should I close this pull request? I'm happy to take either path -- just want to find out if there are any blockers I can help to clear for you. :-)

[danozard]

@demiankatz I'd like to try and finish this PR but at the moment i'm tied up with upgrading National Library VuFind from v4 to v8.1.2 which i hope to have completed within the next 2 weeks. Then i should be able to resume completion of the above PR.

[demiankatz]

Sounds good, @danozard! Good luck with the upgrade, and of course feel free to reach out if I can help with anything.

[demiankatz]

@danozard, do you have any updates? Just checking whether this work is still active or abandoned.