Skip to content

feat: grammar-correction-with-langchain #331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gabisponciano wants to merge 3 commits into
base: main
Choose a base branch
from
Open

feat: grammar-correction-with-langchain #331

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d6ae014
[test] notebook output and new requirements
gabisponciano Nov 11, 2025
b16ff84
[test] notebook output
gabisponciano Nov 18, 2025
54479fb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 56 additions & 47 deletions generative-ai/grammar-correction-with-langchain/notebooks/register-model.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 14:59:40 - INFO - ✅ Notebook execution started.\n",
" ✅ 2025-11-11 16:18:36 - INFO - ✅ Notebook execution started.\n",
" </div>\n",
" "
],
Expand Down Expand Up @@ -110,8 +110,8 @@
"output_type": "stream",
"text": [
"Note: you may need to restart the kernel to use updated packages.\n",
"CPU times: user 16.9 ms, sys: 7.02 ms, total: 23.9 ms\n",
"Wall time: 1.25 s\n"
"CPU times: user 107 ms, sys: 9.43 ms, total: 116 ms\n",
"Wall time: 2.88 s\n"
]
}
],
Expand Down Expand Up @@ -1612,20 +1612,13 @@
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2025/10/31 14:59:46 INFO mlflow.tracking.fluent: Experiment with name 'Markdown_Correction_Service' does not exist. Creating a new experiment.\n"
]
},
{
"data": {
"text/html": [
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 14:59:47 - INFO - Starting MLflow run with ID: 73fd2e421bf246d8bcde5c1982035172\n",
" ✅ 2025-11-11 16:18:50 - INFO - Starting MLflow run with ID: 2ab1a061ee4643b1b01a26000878adf4\n",
" </div>\n",
" "
],
Expand All @@ -1640,13 +1633,13 @@
"name": "stderr",
"output_type": "stream",
"text": [
"2025/10/31 14:59:47 INFO mlflow.pyfunc: Validating input example against model signature\n"
"2025/11/11 16:18:50 INFO mlflow.pyfunc: Validating input example against model signature\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "343ebcd297bf4855b096de206bbdd7fb",
"model_id": "c147c76022d84e10bec6e69bd92add69",
"version_major": 2,
"version_minor": 0
},
Expand All @@ -1660,7 +1653,7 @@
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "30bf2c4f7bb9480bbc12205eb288764e",
"model_id": "bda291918406471da4d22ec78ba01f5e",
"version_major": 2,
"version_minor": 0
},
Expand All @@ -1674,7 +1667,7 @@
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "4e6fcc4df6c14956aa5bfda508d21064",
"model_id": "d32a4e76064345ccb05f7abd9cec9a0b",
"version_major": 2,
"version_minor": 0
},
Expand All @@ -1685,13 +1678,27 @@
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "b97524895427493a918f20fec1d7a788",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Downloading artifacts: 0%| | 0/1 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
" <div style=\"background-color: #ffcc00; color: black;\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ⚠️ 2025-10-31 15:01:32 - WARNING - No GITHUB_ACCESS_TOKEN found in environment or secrets.yaml.\n",
" ✅ 2025-11-11 16:20:58 - INFO - Loaded secrets from /tmp/tmp0xqc02mp/model/artifacts/secrets.yaml.\n",
" </div>\n",
" "
],
Expand All @@ -1717,7 +1724,8 @@
"/opt/conda/lib/python3.12/site-packages/mlflow/pyfunc/model.py:1034: UserWarning: WARNING! num_threads is not default parameter.\n",
" num_threads was transferred to model_kwargs.\n",
" Please confirm that num_threads is what you intended.\n",
" python_model.load_context(context=context)\n"
" python_model.load_context(context=context)\n",
"llama_context: n_ctx_per_seq (32000) < n_ctx_train (131072) -- the full capacity of the model will not be utilized\n"
]
},
{
Expand All @@ -1726,7 +1734,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:01:46 - INFO - ✅ Model context loaded with only instantaneous evaluation metrics.\n",
" ✅ 2025-11-11 16:21:09 - INFO - ✅ Model context loaded with only instantaneous evaluation metrics.\n",
" </div>\n",
" "
],
Expand All @@ -1741,8 +1749,8 @@
"name": "stderr",
"output_type": "stream",
"text": [
"Successfully registered model 'markdown-corrector'.\n",
"Created version '1' of model 'markdown-corrector'.\n"
"Registered model 'markdown-corrector' already exists. Creating a new version of this model...\n",
"Created version '5' of model 'markdown-corrector'.\n"
]
},
{
Expand All @@ -1751,7 +1759,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:03:57 - INFO - ✅ Model 'markdown-corrector' logged successfully (fast eval version).\n",
" ✅ 2025-11-11 16:23:24 - INFO - ✅ Model 'markdown-corrector' logged successfully (fast eval version).\n",
" </div>\n",
" "
],
Expand All @@ -1766,8 +1774,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 15.7 s, sys: 20.6 s, total: 36.3 s\n",
"Wall time: 4min 11s\n"
"CPU times: user 3.68 s, sys: 34.7 s, total: 38.4 s\n",
"Wall time: 4min 35s\n"
]
}
],
Expand Down Expand Up @@ -1843,7 +1851,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:03:58 - INFO - Preparing data for evaluation…\n",
" ✅ 2025-11-11 16:23:24 - INFO - Preparing data for evaluation…\n",
" </div>\n",
" "
],
Expand All @@ -1858,9 +1866,9 @@
"data": {
"text/html": [
"\n",
" <div style=\"background-color: #ffcc00; color: black;\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ⚠️ 2025-10-31 15:03:58 - WARNING - No GITHUB_ACCESS_TOKEN found in environment or secrets.yaml.\n",
" ✅ 2025-11-11 16:23:25 - INFO - Loaded secrets from /phoenix/mlflow/164311871751671231/2ab1a061ee4643b1b01a26000878adf4/artifacts/markdown-corrector/artifacts/secrets.yaml.\n",
" </div>\n",
" "
],
Expand All @@ -1886,7 +1894,8 @@
"/opt/conda/lib/python3.12/site-packages/mlflow/pyfunc/model.py:1034: UserWarning: WARNING! num_threads is not default parameter.\n",
" num_threads was transferred to model_kwargs.\n",
" Please confirm that num_threads is what you intended.\n",
" python_model.load_context(context=context)\n"
" python_model.load_context(context=context)\n",
"llama_context: n_ctx_per_seq (32000) < n_ctx_train (131072) -- the full capacity of the model will not be utilized\n"
]
},
{
Expand All @@ -1895,7 +1904,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:05:14 - INFO - ✅ Model context loaded with only instantaneous evaluation metrics.\n",
" ✅ 2025-11-11 16:24:26 - INFO - ✅ Model context loaded with only instantaneous evaluation metrics.\n",
" </div>\n",
" "
],
Expand All @@ -1910,9 +1919,9 @@
"data": {
"text/html": [
"\n",
" <div style=\"background-color: #ffcc00; color: black;\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ⚠️ 2025-10-31 15:05:14 - WARNING - No GITHUB_ACCESS_TOKEN found in environment or secrets.yaml.\n",
" ✅ 2025-11-11 16:24:26 - INFO - Loaded secrets from ../configs/secrets.yaml.\n",
" </div>\n",
" "
],
Expand All @@ -1929,7 +1938,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:05:15 - INFO - Repository visibility: public\n",
" ✅ 2025-11-11 16:24:27 - INFO - Repository visibility: public\n",
" </div>\n",
" "
],
Expand All @@ -1946,7 +1955,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:05:16 - INFO - Raw markdown extraction complete.\n",
" ✅ 2025-11-11 16:24:28 - INFO - Raw markdown extraction complete.\n",
" </div>\n",
" "
],
Expand All @@ -1963,7 +1972,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:05:16 - INFO - Starting prediction for 2 files...\n",
" ✅ 2025-11-11 16:24:28 - INFO - Starting prediction for 2 files...\n",
" </div>\n",
" "
],
Expand All @@ -1980,7 +1989,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:05:16 - INFO - - Predicting for: README1.md\n",
" ✅ 2025-11-11 16:24:28 - INFO - - Predicting for: README1.md\n",
" </div>\n",
" "
],
Expand All @@ -1997,7 +2006,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:07:25 - INFO - - Predicting for: README2.md\n",
" ✅ 2025-11-11 16:24:33 - INFO - - Predicting for: README2.md\n",
" </div>\n",
" "
],
Expand All @@ -2014,7 +2023,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:12:43 - INFO - ✅ Prediction complete for all files.\n",
" ✅ 2025-11-11 16:24:51 - INFO - ✅ Prediction complete for all files.\n",
" </div>\n",
" "
],
Expand All @@ -2031,7 +2040,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:12:43 - INFO - ✅ Created evaluation DataFrame with 2 file(s).\n",
" ✅ 2025-11-11 16:24:51 - INFO - ✅ Created evaluation DataFrame with 2 file(s).\n",
" </div>\n",
" "
],
Expand Down Expand Up @@ -2099,8 +2108,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 2h 7min 33s, sys: 17.8 s, total: 2h 7min 51s\n",
"Wall time: 8min 45s\n"
"CPU times: user 27.2 s, sys: 2.79 s, total: 30 s\n",
"Wall time: 1min 27s\n"
]
}
],
Expand Down Expand Up @@ -2203,7 +2212,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:12:43 - INFO - Logging metrics for run ID: 73fd2e421bf246d8bcde5c1982035172\n",
" ✅ 2025-11-11 16:24:52 - INFO - Logging metrics for run ID: 2ab1a061ee4643b1b01a26000878adf4\n",
" </div>\n",
" "
],
Expand All @@ -2220,7 +2229,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:12:43 - INFO - ✅ Metrics logged successfully.\n",
" ✅ 2025-11-11 16:24:52 - INFO - ✅ Metrics logged successfully.\n",
" </div>\n",
" "
],
Expand All @@ -2237,7 +2246,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:12:43 - INFO - {\n",
" ✅ 2025-11-11 16:24:52 - INFO - {\n",
" \"semantic_similarity\": 9.96603608439988,\n",
" \"readability_improvement\": -0.3435385604734904,\n",
" \"grammar_quality_score\": 10.0\n",
Expand All @@ -2256,8 +2265,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 21.1 ms, sys: 16.1 ms, total: 37.2 ms\n",
"Wall time: 582 ms\n"
"CPU times: user 37.9 ms, sys: 17.6 ms, total: 55.5 ms\n",
"Wall time: 530 ms\n"
]
}
],
Expand Down Expand Up @@ -2300,7 +2309,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:12:43 - INFO - ⏱️ Total execution time: 13m 2.97s\n",
" ✅ 2025-11-11 16:24:52 - INFO - ⏱️ Total execution time: 6m 16.00s\n",
" </div>\n",
" "
],
Expand All @@ -2317,7 +2326,7 @@
"\n",
" <div style=\"background-color: #228B22; color: white;\n",
" padding: 4px 8px; font-family: monospace; border-radius: 4px;\">\n",
" ✅ 2025-10-31 15:12:43 - INFO - ✅ Notebook execution completed successfully.\n",
" ✅ 2025-11-11 16:24:52 - INFO - ✅ Notebook execution completed successfully.\n",
" </div>\n",
" "
],
Expand Down
Loading