Improve evaluation output by add more info : MaRefactor metrics computation with structured outputso & Micro scores & S… - #17
Conversation
Reviewer's GuideRefactors evaluation metrics computation to return detailed micro, macro, and per-label scores, adjusts division/warning logic, and adds a helper to pretty-print evaluation results as a DataFrame. Sequence diagram for updated evaluation and metrics reporting flowsequenceDiagram
actor User
participant Model
participant Evaluator
participant Metrics as compute_prf
participant Calc as _calculate_metrics
User->>Model: evaluate(dataset)
Model->>Model: run_inference_on_batches
Model->>Evaluator: __init__(all_trues, all_preds)
Model->>Evaluator: evaluate()
Evaluator->>Evaluator: transform_data()
Evaluator->>Metrics: compute_prf(all_true_typed, all_outs_typed)
Metrics->>Metrics: flatten_for_eval(y_true, y_pred)
Metrics->>Metrics: extract_tp_actual_correct
Metrics->>Calc: _calculate_metrics(tp_sum, pred_sum, true_sum, macro)
Calc-->>Metrics: per_class_array, macro_metrics
Metrics->>Calc: _calculate_metrics(tp_sum_micro, pred_sum_micro, true_sum_micro, micro)
Calc-->>Metrics: micro_metrics
Metrics-->>Evaluator: all_metrics
Evaluator-->>Model: all_metrics
Model-->>User: all_metrics
User->>Model: beautiful_df_print(all_metrics)
Model->>Model: build_micro_macro_dataframe
Model->>Model: build_per_class_dataframe
Model-->>User: dataframe
Class diagram for updated evaluation and metrics utilitiesclassDiagram
class Evaluator {
- all_true
- all_pred
+ transform_data()
+ evaluate()
}
class Model {
+ evaluate(dataset)
+ beautiful_df_print(eval_results)
+ encode_labels(labels, batch_size)
}
class MetricsFunctions {
+ flatten_for_eval(y_true, y_pred)
+ compute_prf(y_true, y_pred)
+ _calculate_metrics(tp, pred, true, average)
+ _prf_divide(numerator, denominator, metric, modifier, average, warn_for, zero_division)
}
Model --> Evaluator : uses
Evaluator --> MetricsFunctions : uses
Model --> MetricsFunctions : uses in beautiful_df_print
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Summary of ChangesHello @arthrod, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly upgrades the evaluation capabilities by introducing a more granular breakdown of performance metrics. Instead of a single F1-score, users will now receive micro-averaged, macro-averaged, and individual scores for each label, presented in a clear, tabular format. This enhancement aims to provide a richer understanding of model strengths and weaknesses across different classes. Highlights
Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe changes refactor evaluation metrics computation to produce structured outputs with per-class, micro, and macro metrics, while improving numeric stability in division operations. The model's evaluate method now returns the full results dictionary instead of a tuple, and a new method enables formatted display of these results in a pandas DataFrame. Changes
Estimated Code Review Effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Comment |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- Consider avoiding rounding inside
_prf_divideand_calculate_metrics(and instead rounding only at presentation time) so that low-level helpers remain numerically precise and reusable for other computations. - The evaluation logic is now split between
compute_prfand_calculate_metricswith mixed return types (tuple for macro, dict for micro); aligning these to a single, consistent return shape would make the API easier to reason about and less error-prone. - The
beautiful_df_printmethod is presentation/UI-focused and tightly coupled to pandas; you might want to move it out of the coreModelclass into a separate utility to keep the model implementation focused on modeling and evaluation logic.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Consider avoiding rounding inside `_prf_divide` and `_calculate_metrics` (and instead rounding only at presentation time) so that low-level helpers remain numerically precise and reusable for other computations.
- The evaluation logic is now split between `compute_prf` and `_calculate_metrics` with mixed return types (tuple for macro, dict for micro); aligning these to a single, consistent return shape would make the API easier to reason about and less error-prone.
- The `beautiful_df_print` method is presentation/UI-focused and tightly coupled to pandas; you might want to move it out of the core `Model` class into a separate utility to keep the model implementation focused on modeling and evaluation logic.
## Individual Comments
### Comment 1
<location> `gliner/evaluation/evaluator.py:27-30` </location>
<code_context>
result[denominator == 0] = 0.0 if zero_division in ["warn", 0] else 1.0
- if denominator == 0 and zero_division == "warn" and metric in warn_for:
+ if np.any(denominator == 0) and zero_division == "warn" and metric in warn_for:
msg_start = f"{metric.title()}"
if "f-score" in warn_for:
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Rounding in `_prf_divide` reduces numeric precision and pushes formatting concerns into a low‑level helper.
Previously this helper returned raw float/array values and left rounding to the presentation layer. Rounding to 2 decimals here means all downstream uses (including F-score, which already rounds) now operate on truncated values, increasing the risk of compounded rounding errors (e.g., micro scores derived from already-rounded per-class metrics) and reducing reusability for callers that need full precision. It would be safer to return the unrounded `result` and apply rounding only in the final user-facing layer (e.g., `_calculate_metrics` or printing).
Suggested implementation:
```python
return result
```
To fully follow your suggestion, you will also need to:
1. Identify the user-facing layer(s) that consume `_prf_divide` (e.g., `_calculate_metrics`, reporting/printing functions).
2. Apply rounding there, just before presenting values to the user (e.g., `np.round(metric_values, 2)` or appropriate formatting), ensuring that internal computations (micro/macro averaging, aggregations) continue to use full-precision results from `_prf_divide`.
</issue_to_address>
### Comment 2
<location> `gliner/evaluation/evaluator.py:117-126` </location>
<code_context>
+
+ return all_metrics
+
+def _calculate_metrics(tp,
+ pred,
+ true,
</code_context>
<issue_to_address>
**question:** Macro averages are simple means over all classes, which may be skewed by classes with zero support.
In the `average == "macro"` branch, precision/recall/F1 are averaged as `precision.sum() / len(precision)`, which gives equal weight to classes even when they have zero support (no true or predicted instances, depending on `_prf_divide`). That means unseen labels still affect the macro score. Please confirm this matches your intended definition of macro metrics; if not, consider excluding zero-support classes from the average or switching to a support-weighted scheme.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| if np.any(denominator == 0) and zero_division == "warn" and metric in warn_for: | ||
| msg_start = f"{metric.title()}" | ||
| if "f-score" in warn_for: | ||
| msg_start += " and F-score" if metric in warn_for else "F-score" |
There was a problem hiding this comment.
suggestion (bug_risk): Rounding in _prf_divide reduces numeric precision and pushes formatting concerns into a low‑level helper.
Previously this helper returned raw float/array values and left rounding to the presentation layer. Rounding to 2 decimals here means all downstream uses (including F-score, which already rounds) now operate on truncated values, increasing the risk of compounded rounding errors (e.g., micro scores derived from already-rounded per-class metrics) and reducing reusability for callers that need full precision. It would be safer to return the unrounded result and apply rounding only in the final user-facing layer (e.g., _calculate_metrics or printing).
Suggested implementation:
return resultTo fully follow your suggestion, you will also need to:
- Identify the user-facing layer(s) that consume
_prf_divide(e.g.,_calculate_metrics, reporting/printing functions). - Apply rounding there, just before presenting values to the user (e.g.,
np.round(metric_values, 2)or appropriate formatting), ensuring that internal computations (micro/macro averaging, aggregations) continue to use full-precision results from_prf_divide.
| def _calculate_metrics(tp, | ||
| pred, | ||
| true, | ||
| average : Literal["micro","macro"], | ||
| ): | ||
| precision = _prf_divide( | ||
| numerator=tp_sum, | ||
| denominator=pred_sum, | ||
| numerator=tp, | ||
| denominator=pred, | ||
| metric="precision", | ||
| modifier="predicted", |
There was a problem hiding this comment.
question: Macro averages are simple means over all classes, which may be skewed by classes with zero support.
In the average == "macro" branch, precision/recall/F1 are averaged as precision.sum() / len(precision), which gives equal weight to classes even when they have zero support (no true or predicted instances, depending on _prf_divide). That means unseen labels still affect the macro score. Please confirm this matches your intended definition of macro metrics; if not, consider excluding zero-support classes from the average or switching to a support-weighted scheme.
coderabbitai
Bot
changed the title
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request significantly enhances the evaluation output by providing macro, micro, and per-label scores, which is a great improvement for model analysis. The introduction of a helper method to display these metrics in a DataFrame is also a nice addition for usability. The fix for the division-by-zero warning is a crucial correction.
I've identified a critical issue that could lead to a crash when no entities are present in the evaluation data. I've also included a few medium-severity suggestions to improve code quality and documentation accuracy. Overall, these are valuable changes.
| macro_precision = np.round(precision.sum()/len(precision),decimals=2) | ||
| macro_recall = np.round(recall.sum()/len(recall),decimals=2) | ||
| macro_f_score = np.round(f_score.sum()/ len(f_score), decimals=2) |
There was a problem hiding this comment.
These calculations can lead to a ZeroDivisionError if precision, recall, or f_score are empty arrays. This would happen if there are no predictable entity types in the evaluation data, causing the program to crash. Using np.mean with np.nan_to_num provides a more robust way to calculate the macro average, gracefully handling the empty case by resulting in 0.0.
| macro_precision = np.round(precision.sum()/len(precision),decimals=2) | |
| macro_recall = np.round(recall.sum()/len(recall),decimals=2) | |
| macro_f_score = np.round(f_score.sum()/ len(f_score), decimals=2) | |
| macro_precision = np.round(np.nan_to_num(np.mean(precision)), decimals=2) | |
| macro_recall = np.round(np.nan_to_num(np.mean(recall)), decimals=2) | |
| macro_f_score = np.round(np.nan_to_num(np.mean(f_score)), decimals=2) |
| from collections import defaultdict | ||
| from typing import Union, List, Literal | ||
|
|
||
| import pandas as pd |
There was a problem hiding this comment.
The pandas library is imported here but it is not used within this file. It appears to be used in gliner/model.py for the beautiful_df_print function, where it is also correctly imported. To maintain clean code and avoid unnecessary dependencies in this module, this import should be removed.
| recall = _prf_divide( | ||
| numerator=tp_sum, | ||
| denominator=true_sum, | ||
| numerator=tp, # TODO check |
There was a problem hiding this comment.
There is a TODO check comment remaining in the code. The logic here, using tp as the numerator for recall, is correct. This leftover comment should be removed to clean up the code.
| numerator=tp, # TODO check | |
| numerator=tp, |
| """ | ||
| output : { | ||
|
|
||
| "per_class":{"tag1":{"precision":int, "recall":int,"f_score":int}, | ||
| "tag2":{}... | ||
| }, | ||
| "micro":{"precision":int, "recall":int,"f_score":int}, | ||
| "macro":{"precision":int, "recall":int,"f_score":int}, | ||
| } | ||
| """ |
There was a problem hiding this comment.
The docstring incorrectly specifies int as the data type for precision, recall, and f-score. These metrics are floating-point numbers. Updating the type to float will make the documentation accurate and more helpful for developers using this function.
| """ | |
| output : { | |
| "per_class":{"tag1":{"precision":int, "recall":int,"f_score":int}, | |
| "tag2":{}... | |
| }, | |
| "micro":{"precision":int, "recall":int,"f_score":int}, | |
| "macro":{"precision":int, "recall":int,"f_score":int}, | |
| } | |
| """ | |
| """ | |
| output : { | |
| "per_class":{"tag1":{"precision":float, "recall":float,"f_score":float}, | |
| "tag2":{}... | |
| }, | |
| "micro":{"precision":float, "recall":float,"f_score":float}, | |
| "macro":{"precision":float, "recall":float,"f_score":float}, | |
| } | |
| """ |
| """ | ||
| eval_results : | ||
| { | ||
|
|
||
| "per_class":{"tag1":{"precision":int, "recall":int,"f_score":int}, | ||
| "tag2":{}... | ||
| }, | ||
| "micro":{"precision":int, "recall":int,"f_score":int}, | ||
| "macro":{"precision":int, "recall":int,"f_score":int}, | ||
| } | ||
| """ |
There was a problem hiding this comment.
The docstring indicates that the metric values (precision, recall, f_score) are of type int. However, these are floating-point numbers. The type hint should be float to accurately describe the data structure.
| """ | |
| eval_results : | |
| { | |
| "per_class":{"tag1":{"precision":int, "recall":int,"f_score":int}, | |
| "tag2":{}... | |
| }, | |
| "micro":{"precision":int, "recall":int,"f_score":int}, | |
| "macro":{"precision":int, "recall":int,"f_score":int}, | |
| } | |
| """ | |
| """ | |
| eval_results : | |
| { | |
| "per_class":{"tag1":{"precision":float, "recall":float,"f_score":float}, | |
| "tag2":{}... | |
| }, | |
| "micro":{"precision":float, "recall":float,"f_score":float}, | |
| "macro":{"precision":float, "recall":float,"f_score":float}, | |
| } | |
| """ |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@gliner/evaluation/evaluator.py`:
- Around line 150-155: The current macro aggregation divides by len(precision)
which raises ZeroDivisionError when there are no labels; inside the same else
branch guard on an empty metrics array (e.g., if len(precision) == 0 or
precision.size == 0) and set macro_precision, macro_recall, macro_f_score to 0.0
(or np.nan if you prefer) instead of computing the division, then return
per_class_array (which may be empty) and the macro dict; reference the variables
precision, recall, f_score and the returned per_class_array to locate where to
add the guard.
- Around line 188-201: The docstring for the method currently claims integer
metrics but the method returns a dict of rounded floats (the variable
results_all returned from compute_prf after transform_data). Update the
docstring to state that the return value is a dict containing per_class, micro
and macro entries with float precision/recall/f_score (rounded floats), and show
the exact dict structure and example types (e.g. "precision": float) to match
compute_prf/results_all; mention transform_data and compute_prf in the
description so readers can trace the data flow.
In `@gliner/model.py`:
- Around line 656-661: The docstring for the evaluate() method no longer matches
its return type; update the evaluate() docstring (the one for the
Evaluator.evaluate / evaluate() method referenced by evaluator.evaluate and the
caller using all_results) to state that it returns a dict of metrics/results
(not a tuple with F1), describe the dict keys and value types (e.g., overall F1,
per-class metrics, any other entries), and remove any mention of returning a
tuple (out, f1) so callers and readers understand the new return structure.
- Line 5: The module-level "import pandas as pd" causes import errors for users
without pandas; either add "pandas" to project dependencies in
pyproject.toml/requirements.txt, or lazy-import it inside the function
beautiful_df_print in gliner.model: remove the top-level import, add "import
pandas as pd" at the start of beautiful_df_print, catch ImportError and raise a
clear RuntimeError (or ValueError) telling the user to install pandas, and
proceed to use pd as before—this localizes the dependency to callers of
beautiful_df_print and prevents breaking imports for other users.
| else : | ||
| per_class_array = {"precision": precision, "recall": recall, "f_score": f_score} | ||
| macro_precision = np.round(precision.sum()/len(precision),decimals=2) | ||
| macro_recall = np.round(recall.sum()/len(recall),decimals=2) | ||
| macro_f_score = np.round(f_score.sum()/ len(f_score), decimals=2) | ||
| return per_class_array,{"precision": macro_precision, "recall": macro_recall, "f_score": macro_f_score} |
There was a problem hiding this comment.
Handle empty label sets to avoid ZeroDivisionError.
If there are no labels (no true/pred entities), len(precision) is 0 and macro aggregation crashes. Please return zeros (or NaNs) in this edge case.
🛠️ Suggested guard for empty metrics
else :
per_class_array = {"precision": precision, "recall": recall, "f_score": f_score}
- macro_precision = np.round(precision.sum()/len(precision),decimals=2)
- macro_recall = np.round(recall.sum()/len(recall),decimals=2)
- macro_f_score = np.round(f_score.sum()/ len(f_score), decimals=2)
+ if len(precision) == 0:
+ return per_class_array, {"precision": 0.0, "recall": 0.0, "f_score": 0.0}
+ macro_precision = np.round(precision.sum()/len(precision),decimals=2)
+ macro_recall = np.round(recall.sum()/len(recall),decimals=2)
+ macro_f_score = np.round(f_score.sum()/ len(f_score), decimals=2)
return per_class_array,{"precision": macro_precision, "recall": macro_recall, "f_score": macro_f_score}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| else : | |
| per_class_array = {"precision": precision, "recall": recall, "f_score": f_score} | |
| macro_precision = np.round(precision.sum()/len(precision),decimals=2) | |
| macro_recall = np.round(recall.sum()/len(recall),decimals=2) | |
| macro_f_score = np.round(f_score.sum()/ len(f_score), decimals=2) | |
| return per_class_array,{"precision": macro_precision, "recall": macro_recall, "f_score": macro_f_score} | |
| else : | |
| per_class_array = {"precision": precision, "recall": recall, "f_score": f_score} | |
| if len(precision) == 0: | |
| return per_class_array, {"precision": 0.0, "recall": 0.0, "f_score": 0.0} | |
| macro_precision = np.round(precision.sum()/len(precision),decimals=2) | |
| macro_recall = np.round(recall.sum()/len(recall),decimals=2) | |
| macro_f_score = np.round(f_score.sum()/ len(f_score), decimals=2) | |
| return per_class_array,{"precision": macro_precision, "recall": macro_recall, "f_score": macro_f_score} |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@gliner/evaluation/evaluator.py` around lines 150 - 155, The current macro
aggregation divides by len(precision) which raises ZeroDivisionError when there
are no labels; inside the same else branch guard on an empty metrics array
(e.g., if len(precision) == 0 or precision.size == 0) and set macro_precision,
macro_recall, macro_f_score to 0.0 (or np.nan if you prefer) instead of
computing the division, then return per_class_array (which may be empty) and the
macro dict; reference the variables precision, recall, f_score and the returned
per_class_array to locate where to add the guard.
| """ | ||
| output : { | ||
|
|
||
| "per_class":{"tag1":{"precision":int, "recall":int,"f_score":int}, | ||
| "tag2":{}... | ||
| }, | ||
| "micro":{"precision":int, "recall":int,"f_score":int}, | ||
| "macro":{"precision":int, "recall":int,"f_score":int}, | ||
| } | ||
| """ | ||
| all_true_typed, all_outs_typed = self.transform_data() | ||
| precision, recall, f1 = compute_prf(all_true_typed, all_outs_typed).values() | ||
| output_str = f"P: {precision:.2%}\tR: {recall:.2%}\tF1: {f1:.2%}\n" | ||
| return output_str, f1 | ||
| results_all = compute_prf(all_true_typed, all_outs_typed) | ||
| # output_str = f"P: {precision:.2%}\tR: {recall:.2%}\tF1: {f1:.2%}\n" | ||
| return results_all |
There was a problem hiding this comment.
Update the docstring to reflect float metrics and dict output.
The new return value is a dict of rounded floats, but the docstring currently documents ints.
✏️ Docstring tweak
- "per_class":{"tag1":{"precision":int, "recall":int,"f_score":int},
+ "per_class":{"tag1":{"precision":float, "recall":float,"f_score":float},
"tag2":{}...
},
- "micro":{"precision":int, "recall":int,"f_score":int},
- "macro":{"precision":int, "recall":int,"f_score":int},
+ "micro":{"precision":float, "recall":float,"f_score":float},
+ "macro":{"precision":float, "recall":float,"f_score":float},📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| """ | |
| output : { | |
| "per_class":{"tag1":{"precision":int, "recall":int,"f_score":int}, | |
| "tag2":{}... | |
| }, | |
| "micro":{"precision":int, "recall":int,"f_score":int}, | |
| "macro":{"precision":int, "recall":int,"f_score":int}, | |
| } | |
| """ | |
| all_true_typed, all_outs_typed = self.transform_data() | |
| precision, recall, f1 = compute_prf(all_true_typed, all_outs_typed).values() | |
| output_str = f"P: {precision:.2%}\tR: {recall:.2%}\tF1: {f1:.2%}\n" | |
| return output_str, f1 | |
| results_all = compute_prf(all_true_typed, all_outs_typed) | |
| # output_str = f"P: {precision:.2%}\tR: {recall:.2%}\tF1: {f1:.2%}\n" | |
| return results_all | |
| """ | |
| output : { | |
| "per_class":{"tag1":{"precision":float, "recall":float,"f_score":float}, | |
| "tag2":{}... | |
| }, | |
| "micro":{"precision":float, "recall":float,"f_score":float}, | |
| "macro":{"precision":float, "recall":float,"f_score":float}, | |
| } | |
| """ | |
| all_true_typed, all_outs_typed = self.transform_data() | |
| results_all = compute_prf(all_true_typed, all_outs_typed) | |
| # output_str = f"P: {precision:.2%}\tR: {recall:.2%}\tF1: {f1:.2%}\n" | |
| return results_all |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@gliner/evaluation/evaluator.py` around lines 188 - 201, The docstring for the
method currently claims integer metrics but the method returns a dict of rounded
floats (the variable results_all returned from compute_prf after
transform_data). Update the docstring to state that the return value is a dict
containing per_class, micro and macro entries with float
precision/recall/f_score (rounded floats), and show the exact dict structure and
example types (e.g. "precision": float) to match compute_prf/results_all;
mention transform_data and compute_prf in the description so readers can trace
the data flow.
| import os | ||
| import re | ||
| import warnings | ||
| import pandas as pd |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Locate common dependency manifests and check for pandas.
echo "=== Searching for dependency manifests and pandas ==="
files=$(find . -maxdepth 3 -type f \( -name 'requirements*.txt' -o -name 'pyproject.toml' -o -name 'setup.cfg' -o -name 'setup.py' -o -name 'Pipfile' -o -name 'environment.yml' \) 2>/dev/null)
if [ -n "$files" ]; then
echo "Found manifest files:"
echo "$files"
echo ""
echo "=== Checking for pandas in these files ==="
rg -n 'pandas' $files
else
echo "No dependency manifest files found."
fiRepository: arthrod/GLiNER
Length of output: 213
🏁 Script executed:
#!/bin/bash
# Check the actual import in gliner/model.py and the beautiful_df_print function
echo "=== First 20 lines of gliner/model.py ==="
head -20 gliner/model.py
echo ""
echo "=== Lines 663-676 of gliner/model.py (beautiful_df_print function) ==="
sed -n '663,676p' gliner/model.py
echo ""
echo "=== Check for all imports of pandas in gliner/model.py ==="
rg -n 'import.*pandas\|from.*pandas' gliner/model.pyRepository: arthrod/GLiNER
Length of output: 1318
🏁 Script executed:
#!/bin/bash
echo "=== pyproject.toml ==="
cat pyproject.toml
echo ""
echo "=== requirements.txt ==="
cat requirements.txtRepository: arthrod/GLiNER
Length of output: 1533
Add pandas to dependencies or move the import inside beautiful_df_print.
Pandas is imported at module scope (line 5) but not declared in dependencies in pyproject.toml or requirements.txt. This breaks module loading for any user without pandas installed, even if they never call beautiful_df_print. Either add "pandas" to the dependencies list in pyproject.toml, or move the import inside the method with a helpful error message if pandas is not available.
Option: lazy-import inside the method
def beautiful_df_print(self, eval_results:Dict):
+ try:
+ import pandas as pd
+ except ModuleNotFoundError as exc:
+ raise ModuleNotFoundError(
+ "pandas is required for beautiful_df_print. Please install it (e.g., pip install pandas)."
+ ) from exc
"""🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@gliner/model.py` at line 5, The module-level "import pandas as pd" causes
import errors for users without pandas; either add "pandas" to project
dependencies in pyproject.toml/requirements.txt, or lazy-import it inside the
function beautiful_df_print in gliner.model: remove the top-level import, add
"import pandas as pd" at the start of beautiful_df_print, catch ImportError and
raise a clear RuntimeError (or ValueError) telling the user to install pandas,
and proceed to use pd as before—this localizes the dependency to callers of
beautiful_df_print and prevents breaking imports for other users.
| # out, f1 = evaluator.evaluate() | ||
|
|
||
| return out, f1 | ||
| # return out, f1 | ||
| all_results = evaluator.evaluate() | ||
|
|
||
| return all_results |
There was a problem hiding this comment.
Update evaluate() docs to match the new return type.
The function now returns a dict (not a tuple with F1), so the docstring should be updated accordingly.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@gliner/model.py` around lines 656 - 661, The docstring for the evaluate()
method no longer matches its return type; update the evaluate() docstring (the
one for the Evaluator.evaluate / evaluate() method referenced by
evaluator.evaluate and the caller using all_results) to state that it returns a
dict of metrics/results (not a tuple with F1), describe the dict keys and value
types (e.g., overall F1, per-class metrics, any other entries), and remove any
mention of returning a tuple (out, f1) so callers and readers understand the new
return structure.
1 participant
…cores per label
Summary by Sourcery
Add richer evaluation metrics output including macro, micro, and per-label scores, and expose a tabular view helper for inspection.
New Features:
Bug Fixes:
Enhancements: