| copyright |
|
||
|---|---|---|---|
| lastupdated | 2025-08-22 | ||
| subcollection | natural-language-understanding |
{:shortdesc: .shortdesc} {:external: target="_blank" .external} {:tip: .tip} {:pre: .pre} {:codeblock: .codeblock} {:screen: .screen} {:javascript: .ph data-hd-programlang='javascript'} {:java: .ph data-hd-programlang='java'} {:python: .ph data-hd-programlang='python'}
{: #migrating-to-watson-nlp}
{: #useful-links}
- Watson NLU API documentation
- Watson NLP in watsonx.ai Studio in IBM Cloud documentation
- Watson NLP in watsonx.ai Studio in CloudPak for Data docs
{: #overview}
Watson Natural Language Understanding (NLU) consists of a set of APIs to analyze various features of text content at scale. You can submit plain text, raw HTML, or a public URL and receive results for the features you request through NLU. To analyze text, Watson NLU uses Watson NLP as its underlying processing framework.
In watsonx.ai Studio, all Watson NLP models are available as a Python library. You can access these models from notebooks or deploy them as Python functions or scripts. Both online and batch deployment options are available for use.
{: #migration-process}
To migrate from Watson NLU to watsonx.ai Studio with Watson NLP, see the following process.
Prepare your instance of watsonx.ai Studio. You can migrate to watsonx.ai Studio with Watson NLP in SaaS IBM Cloud environment, or using your own on-prem environment.
- Create an instance of watsonx.ai Studio.
- Create an instance of the Watson Machine Learning (WML) service.
- Configure using IBM Sales Configurator and ensure you have the following services selected (note these services are selected by default): Watson Studio and Watson Machine Learning
- Install the NLP models that you need for your application. For instructions, see Installing pre-trained NLP models for Python-based notebook runtimes.
To identify the corresponding model in watsonx.ai Studio with Watson NLP, see the section titled Model mapping.
Write a Python function that takes a string as input and returns the output of the NLP model applied to that text. See Model mapping on how to identify the code snippet to use in the Python function. Watson NLU and watsonx.ai Studio may not produce the same output schema.
{: #4-deploy-the-python-function} Deploy the Python function as an online deployment. Use the following documentation for deploying NLP models:
Every Watson NLP model has its own runtime performance characteristics. Therefore, every API call may take more or less compute, depending on the actual model that is executed. For example, syntax is much faster than a transformer-based entity extractor.
To estimate the necessary scale-out of deployed Python functions to meet a specific application workload using Watson NLP in watsonx.ai Studio, see the following guidance:
- Measure the runtime performance (throughput) of the model in their environment
- Use the results from step 1 to estimate the scale-up required to meet your application’s throughput needs.
Pricing in watsonx.ai Studio:
- In SaaS IBM Cloud, the pricing is based on CUH (Compute unit Hours). Once you estimate the scale out needed, you can estimate your price in CUH accordingly. To find the pricing for your specific plan, see watsonx.ai Studio plan. As of January 10, 2025, pricing is defined by the compute hours for the base environment based on the vCPU/RAM chosen plus an up-charge of 5 CUH for the NLP environment. For example, running a Notebook NLP environment with 16 vCPU/64 GB costs 13 (8+5) capacity unit per hour and price for one capacity unit is $1.02 in the professional plan.
- In on-prem in CPD, pricing is per VPC. Your throughput measurements can be used to estimate the number of VPCs needed.
For additional assistance with calculating costs, please contact IBM Sales.
{: #example-for-tone-classification}
This section exemplifies Steps 2 and 3 for the Tone Classification model.
Using Watson NLU Tone Classification involves calling this API:
curl \
-X POST \
-u "apikey:<apikey>" \
-H "Content-Type: application/json" \
-d '{
"text": "Yummy affordable bread that is great for parties! I would describe like a sweeter tiny pandesal bread. They heat up the bread post transaction so it always comes up nice and fresh. Expect about a 10 minute wait for that. The wait can be unpredictable though. I came on a Sunday, no line, but after I paid. There was a line out the door. Place is very tiny so expect to be a little cramped if it gets busy. So happy I dont have to drive all the way to Chula Vista to try starbread!",
"features": {
"classifications": {
"model": "tone-classifications-en-v1"
}
},
"return_analyzed_text": false
}' \
"https://api.us-south.natural-language-understanding.watson.cloud.ibm.com/v1/analyze?version=2025-04-07"Which produces this response:
{
"usage":
{
"text_units": 1,
"text_characters": 480,
"features": 1
},
"language": "en",
"classifications": [
{
"confidence": 0.167542,
"class_name": "frustrated"
},
{
"confidence": 0.157899,
"class_name": "sad"
},
{
"confidence": 0.154908,
"class_name": "excited"
},
{
"confidence": 0.141476,
"class_name": "polite"
},
{
"confidence": 0.053681,
"class_name": "satisfied"
},
{
"confidence": 0.0419,
"class_name": "sympathetic"
},
{
"confidence": 0.013666,
"class_name": "impolite"
}
]
}The corresponding Tone Classification models in watsonx.ai Studio models are documented here:
- ensemble_classification-workflow_en_tone-stock
- ensemble_classification-workflow_fr_tone-stock
The Python code snippet to invoke the model in a notebook is also documented in the same page:
import watson_nlp
# Load Tone Classification model for English
tone_workflow=watson_nlp.download_and_load("ensemble_classification-wf_v2-3-1_lang_en_tone-stock_2022-02-01-100000")
# Run the tone classification model on the input text
tone_workflow.run(breadReview)
# Print the classified tone
print(tone_ workflow)And produces this response:
{
"classes": [
{
"class_name": "frustrated",
"confidence": 0.1675415905898042
},
{
"class_name": "sad",
"confidence": 0.15789897138055303
},
{
"class_name": "excited",
"confidence": 0.15490848593639606
},
{
"class_name": "polite",
"confidence": 0.14147561901446545
},
{
"class_name": "satisfied",
"confidence": 0.053680803683219536
},
{
"class_name": "sympathetic",
"confidence": 0.04189955103069996
},
{
"class_name": "impolite",
"confidence": 0.013665584500951369
}
],
"producer_id": {
"name": "Voting based Ensemble",
"version": "0.0.1"
}
}Wrap the code snippet below in a Python function that takes a string as input and outputs the response of the model. Here is an example from the Sample Notebook:
def detect_tone():
import watson_nlp
tone_model = tone_classification_model = watson_nlp.load(' ensemble_classification-workflow_en_tone-stock ')
def score(input):
scoring_prediction_out = []
for input_data_row in input["input_data"][0]["values"]:
scoring_prediction_row = []
for input_data in input_data_row:
tone_classification = tone_classification_model.run(input_data)
scoring_prediction_row.append(tone_classification.to_dict())
scoring_prediction_out.append(scoring_prediction_row)
# Score using the pre-defined model
scoring_response = {
'predictions': [{'fields': ['nlp_prediction'],
'values': scoring_prediction_out
}]
}
return scoring_response
return scoreFollow the documentation to Deploy a Watson NLP model as a Python function.
{: #model-mapping}
This section describes the correspondence between Watson NLU features, and the corresponding models in watsonx.ai Studio with Watson NLP, as well as pointers to the code snippet necessary to implement the Python function necessary to expose NLP model as an online deployment in watsonx.ai.
Watson NLU documentation Note: The Categories feature in Watson NLP is only available in the English language.
Watsonx.ai Studio models:
- categories_esa_en_stock
Watsonx.ai Studio code snippet
def transform_categories(wnlp_response):
transformed_categories = []
for prediction in wnlp_response['predictions']:
categories = prediction['values'][0][0]['categories']
for category in categories:
labels = category['labels']
score = category['score']
explanation = category['explanation']
# Construct NLU response format
nlu_category = {
"score": score,
"label": '/'.join(f'/{label}' if i == 0 else f'{label}' for i, label in enumerate(labels)),
"explanation": {
"relevant_text":[
{ "text": explanation[0]['text'] }
]
}
}
transformed_categories.append(nlu_category)
return transformed_categoriesWatsonx.ai Studio models:
- ensemble_classification-workflow__tone-stock
Watsonx.ai Studio code snippet
def transform_tone_classification(wnlp_response):
transformed_predictions = []
for prediction in wnlp_response['predictions']:
classes = prediction['values'][0][0]['classes']
for classification in classes:
class_name = classification['class_name']
confidence = classification['confidence']
# Construct NLU response format
nlu_classifications = {
"confidence": round(confidence, 6),
"class_name": class_name,
}
transformed_predictions.append(nlu_classifications)
# Construct the final NLU response
nlu_response = {
"classifications": transformed_predictions
}
return nlu_responseWatsonx.ai Studio models:
- concepts_alchemy__stock
Watsonx.ai Studio code snippet
No transformation of Watson NLP response is required - the responses are equivalent.
Watsonx.ai Studio models:
- emotion_aggregated-workflow__stock
Watsonx.ai Studio code snippet
def transform_emotion(wnlp_response):
transformed_predictions = {}
# Transform @wnlp_response.json to @nlu_response.json
for prediction in wnlp_response['predictions']:
predictions = prediction['values'][0][0]['emotion_predictions']
for emotion_pred in predictions:
target = emotion_pred['target']
emotion_dict = emotion_pred['emotion']
target_emotion = {
"text": target,
"emotion": {}
}
# Add target-level emotion prediction (only if target is not empty)
if target:
target_emotion["emotion"] = sort_emotions(emotion_dict)
# create "targets" if none
if 'targets' in transformed_predictions:
transformed_predictions["targets"].append(target_emotion)
else:
transformed_predictions['targets'] = [target_emotion]
# Add document-level emotion prediction (only if target is empty)
if not target:
document_emotion = sort_emotions(emotion_dict)
if "document" in transformed_predictions:
transformed_predictions["document"]["emotion"] = document_emotion
else:
transformed_predictions["document"] = {"emotion": document_emotion}
# Construct the final NLU response
nlu_response = {
"emotion": transformed_predictions
}
return nlu_response
def sort_emotions(emotion_dict):
emotion_list = list(emotion_dict.items())
emotion_list.sort(key=lambda x: x[0], reverse=True)
return {emotion: round(probability, 6) for emotion, probability in emotion_list}Watson NLU documentation Note: Disambiguation is not available in Watson NLP in Watsonx.ai Studio.
| Watsonx.ai Studio models | Code Snippet Link |
|---|---|
| entity-mentions_transformer-workflow_multilingual_slate.153m.distilled entity-mentions_transformer-workflow_multilingual_slate.153m.distilled-cpu) |
Code snippet |
| entity-mentions_rbr__stock | Code snippet |
| entity-mentions_rbr_multi_pii | Code snippet |
def transform_entities(wnlp_response):
transform_entities = {'entities':[]}
for entities in wnlp_response['predictions'][0]['values'][0]['entities']:
entity_type = entities['type']
text = entities['text']
relevance = entities['relevance']
count = len(entities['mentions'])
confidence = entities['confidence']
disambiguation = entities['disambiguation']
transform_entities['entities'].append({
"type": entity_type,
"text": text,
"relevance": relevance,
"count": count,
"confidence": confidence
}
)
if disambiguation:
transform_entities['entities'][-1]["disambiguation"]=disambiguation
return transform_entitiesWatsonx.ai Studio models:
- keywords_embed-rank_multi_stock
Watsonx.ai Studio code snippet
def transform_keywords(wnlp_response):
transform_keywords = {'keywords':[]}
for keywords in wnlp_response['predictions'][0]['values'][0]['keywords']:
text = keywords['text']
relevance = keywords['relevance']
count = len(keywords['mentions'])
# Construct NLU response format
nlu_syntax = {
"text": text,
"relevance": relevance,
"count": count,
}
transform_keywords['keywords'].append(nlu_syntax)
return transform_keywordsWatsonx.ai Studio models: there is no corresponding feature in watsonx.ai Studio.
Watsonx.ai Studio models:
- relations_transformer-workflow_multilingual_slate.153m.distilled
Watsonx.ai Studio code snippet
def transform_relations(wnlp_response):
transform_relations = {'relations':[]}
for relations in wnlp_response['predictions'][0]['values'][0]['relations']:
relation_mentions = relations['relation_mentions']
for mention in relation_mentions:
relation_type = mention['type']
sentence = mention['text']
score = mention['confidence']
arguments = []
arguments.append(
{
"text": mention['mention1']['span']['text'],
"location": [mention['mention1']['span']['begin'], mention['mention1']['span']['end']],
"entities": [{"type": relations['entity1']['type'],
"text": relations['entity1']['text'],
"disambiguation": relations['entity1']['disambiguation']}]
}
)
arguments.append(
{
"text": mention['mention2']['span']['text'],
"location": [mention['mention2']['span']['begin'], mention['mention2']['span']['end']],
"entities": [{"type":relations['entity2']['type'],
"text":relations['entity2']['text'],
"disambiguation":relations['entity2']['disambiguation']}]
}
)
transform_relations['relations'].append({
"type": relation_type,
"sentence": sentence,
"score": score,
"arguments": arguments
}
)
return transform_relationsWatsonx.ai Studio models: there is no corresponding feature in watsonx.ai Studio.
- sentiment-aggregated_transformer-workflow_multilingual_slate.153m.distilled
- sentiment-aggregated_transformer-workflow_multilingual_slate.153m.distilled-cpu
- targets-sentiment_transformer-workflow_multilingual_slate.153m.distilled
- targets-sentiment_transformer-workflow_multilingual_slate.153m.distilled-cpu
Watsonx.ai Studio code snippet (aggregated)
Watsonx.ai Studio code snippet (targeted)
def transform_sentiment_document(wnlp_response):
transformed_syntax = {'sentiment':{'document':{}}}
sentiment_mapping = {'SENT_NEGATIVE':'negatve','SENT_POSITIVE':'positive', 'SENT_NEUTRAL':'neutral'}
sentiment_pred = wnlp_response['predictions'][0]['values'][0]['document_sentiment']
label = sentiment_mapping[sentiment_pred['label']]
mixed = int(sentiment_pred['mixed'])
score = sentiment_pred['score']
# Construct NLU response format
nlu_syntax = {
"score": score,
"mixed": mixed,
"label": label,
}
transformed_syntax['sentiment']['document']=nlu_syntax
return transformed_syntaxdef transform_sentiment_target(wnlp_response):
transformed_syntax = {'sentiment':{'document':{},'targets': []}}
sentiment_mapping = {'SENT_NEGATIVE':'negatve','SENT_POSITIVE':'positive', 'SENT_NEUTRAL':'neutral'}
sentiment_pred = wnlp_response['predictions'][0]['values'][0]['document_sentiment']
label = sentiment_mapping[sentiment_pred['label']]
mixed = int(sentiment_pred['mixed'])
score = sentiment_pred['score']
# Construct NLU response format
nlu_syntax = {
"score": score,
"mixed": mixed,
"label": label,
}
transformed_syntax['sentiment']['document']=nlu_syntax
for k, v in wnlp_response['predictions'][0]['values'][0]['targeted_sentiments']['targeted_sentiments'].items():
text = k
score = v['score']
label = sentiment_mapping[v['label']]
nlu_syntax = {
"text": text,
"score": score,
"label": label,
}
transformed_syntax['sentiment']['targets'].append(nlu_syntax)
return transformed_syntaxWatsonx.ai Studio models:
- syntax_izumo__stock if you do not need dependency parsing features, OR
- syntax_izumo__stock-dp if you need dependency parsing features
Watsonx.ai Studio code snippet
def transform_syntax_token(wnlp_response):
transformed_syntax = {'syntax':{'tokens':[]}}
for syntax in wnlp_response['predictions'][0]['values'][0]['tokens']:
lemma = syntax['lemma']
part_of_speech = syntax['part_of_speech'][4:]
location = [syntax['span']['begin'],syntax['span']['end']]
text = syntax['span']['text']
# Construct NLU response format
nlu_syntax = {
"text": text,
"part_of_speech": part_of_speech,
"location": location,
"lemma": lemma
}
transformed_syntax['syntax']['tokens'].append(nlu_syntax)
return transformed_syntaxdef transform_syntax_sentences(wnlp_response):
transformed_syntax = {'syntax':{'sentences':[]}}
for syntax in wnlp_response['predictions'][0]['values'][0]['sentences']:
location = [syntax['span']['begin'],syntax['span']['end']]
text = syntax['span']['text']
# Construct NLU response format
nlu_syntax = {
"text": text,
"location": location,
}
transformed_syntax['syntax']['sentences'].append(nlu_syntax)
return transformed_syntaxThis section describes customization features and corresponding functionality in watsonx.ai Studio with Watson NLP.