Skip to content

Add Stanford CoreNLP tool and data manager#8005

Open
ksuderman wants to merge 7 commits into
galaxyproject:mainfrom
ksuderman:corenlp-nlp
Open

Add Stanford CoreNLP tool and data manager#8005
ksuderman wants to merge 7 commits into
galaxyproject:mainfrom
ksuderman:corenlp-nlp

Conversation

@ksuderman

Copy link
Copy Markdown
Contributor

Summary

  • Adds Stanford CoreNLP annotation tool supporting 8 languages
  • Includes data manager for downloading CoreNLP language model JARs from Maven Central
  • Provides tokenization, POS tagging, lemmatization, dependency parsing, NER, coreference resolution
  • Multiple output formats: JSON, CoNLL, XML, tabular, text

CoreNLP Tool Features

  • Docker containerization with Java 21 runtime environment
  • Support for Arabic, Chinese, English, French, German, Hungarian, Italian, Spanish
  • Sentiment analysis with EJML dependencies included
  • Coreference resolution with common models JAR support
  • Flexible annotator pipeline configuration

Data Manager Features

  • Downloads language model JARs directly from Maven Central
  • Multi-select installation interface for language packages
  • Automatic registration in Galaxy data tables
  • Common models JAR for coreference resolution support

Test plan

  • Tool passes planemo lint validation
  • Data manager passes planemo lint validation
  • Comprehensive test data included
  • README documentation provided for both components
  • .shed.yml configured for IUC submission
  • Docker image updated and tested

🤖 Generated with Claude Code

ksuderman and others added 2 commits May 19, 2026 18:49
- Stanford CoreNLP tool wrapper supporting 8 languages
- Multi-language support: English, Chinese, Arabic, French, German, Spanish, Italian, Hungarian
- Multiple annotators: tokenization, POS, NER, parsing, coreference, sentiment
- Output formats: JSON, CoNLL, CoNLL-U, XML, text
- Docker containerization with Java 21
- Data manager for automated model downloads from Maven Central
- Comprehensive tests and documentation
- Fixed coreference resolution with common models JAR

Tool: stanford_corenlp (v4.5.10+galaxy4)
Data Manager: data_manager_corenlp_models
Categories: Text Manipulation, Natural Language Processing
Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where/When is this script used?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this file be an input/output of a tool test?

Comment thread tools/corenlp/.shed.yml Outdated
- Text Manipulation
- Natural Language Processing
homepage_url: https://stanfordnlp.github.io/CoreNLP/
remote_repository_url: https://github.com/galaxyproject/tools-iuc

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
remote_repository_url: https://github.com/galaxyproject/tools-iuc
remote_repository_url: https://github.com/galaxyproject/tools-iuc/tree/main/tools/corenlp

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be in a sub-directory called data_manager

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also goes to the sub-directory data_manager.
Is it possible to write some tests here too?

ksuderman and others added 5 commits May 21, 2026 23:46
- Fix remote_repository_url to point to specific tool directory
- Reorganize data manager files into data_manager/ subdirectory structure
- Both changes address reviewer feedback from RZ9082
Replace problematic has_json_property_with_text assertions with more robust has_text assertions to match successful pattern used in Stanza tool. This resolves JSON property path failures in Galaxy test framework.

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
ln -s "\$(dirname '${language_model.fields.models_path}')/stanford-corenlp-@TOOL_VERSION@-models.jar" ./common.jar &&
#end if
#if $language_model.fields.lang_code == "en"
echo 'pos.model = edu/stanford/nlp/models/pos-tagger/english-bidirectional-distsim.tagger' > corenlp.properties &&

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of creating the "corenlp.properties" file with echo you can use <configfile> in Galaxy tool XML, this should make things much more readable and matainable.

<inputs>
<param name="input" type="data" format="txt" label="Text"/>
<param name="language_model" type="select" label="Language Model">
<options from_data_table="corenlp_models">

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are not filtering by version here, do you? So later versions of this tool will show all models, even if they are not combat?

<command detect_errors="exit_code"><![CDATA[
ln -s '${language_model.fields.models_path}' ./model.jar &&
#if $annotators == "coref"
ln -s "\$(dirname '${language_model.fields.models_path}')/stanford-corenlp-@TOOL_VERSION@-models.jar" ./common.jar &&

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the language encoded here? This path does not exists, does it?

wouldn't it make more sense to have the following layout of the data table:

french French fr 4.5.10 /path/to/4.5.10/stanford-corenlp-models-french.jar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants