Skip to content

Allow using the projects.yaml file to identify nonbillable projects#241

Merged
QuanMPhm merged 2 commits into
CCI-MOC:mainfrom
QuanMPhm:232/migrate_nonbillable
Feb 2, 2026
Merged

Allow using the projects.yaml file to identify nonbillable projects#241
QuanMPhm merged 2 commits into
CCI-MOC:mainfrom
QuanMPhm:232/migrate_nonbillable

Conversation

@QuanMPhm

Copy link
Copy Markdown
Contributor

Closes #232. Related: https://github.com/CCI-MOC/non-billable-projects/pull/49

More details in the commit message. This is a draft since I have some questions below.

Comment thread process_report/loader.py Outdated

mask = (dataframe["Start Date"] <= invoice_settings.invoice_month) & (
invoice_settings.invoice_month <= dataframe["End Date"]
# TODO (Quan): Do we also want to specify the clusters for these timed projects?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably best to specify clusters whenever possible.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Comment on lines +112 to +115
# TODO (Quan): If two Coldfront projects have the same name
# (maybe because they exist on different clusters),
# the `allocation_data` dict will only accurately store info for one of the projects
# Would this be a concern?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a way to merge it with cluster info?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would storing the cluster name solve this?

@QuanMPhm QuanMPhm force-pushed the 232/migrate_nonbillable branch 2 times, most recently from 1b0248c to b3a3219 Compare October 30, 2025 17:33
@QuanMPhm QuanMPhm force-pushed the 232/migrate_nonbillable branch from b3a3219 to 047671c Compare January 5, 2026 22:46
@QuanMPhm QuanMPhm marked this pull request as ready for review January 5, 2026 22:50
@QuanMPhm

QuanMPhm commented Jan 5, 2026

Copy link
Copy Markdown
Contributor Author

@knikolla @naved001 The PR is now considered complete and ready for review. It has 2 commits, the first allows the projects.yaml list to be used. The second adds a small improvement to the validation check made by the Coldfront processor (more details in the commit message). If you guys think the PR is too big, I'm fine just merging the first commit, instead of both.

@naved001 naved001 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@QuanMPhm I may bug you for a code walk-through at some point.

Comment thread process_report/loader.py Outdated
3. Is Timed: Boolean indicating if the nonbillable status is time-bound
"""

def _check_time_range(timed_object) -> bool:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I have a function that returns a boolean I tend to stick to a name that tells me it'll return a boolean. Something like is_within_time_range is_in_time_range.

Comment thread process_report/loader.py Outdated
# Leveraging inherent lexicographical order of YYYY-MM strings
return (
timed_object["start"] <= invoice_settings.invoice_month
and invoice_settings.invoice_month < timed_object["end"]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the older implementation has

invoice_settings.invoice_month <= dataframe["End Date"]

is the change to < from <= intentional?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mistake. That should not have changed.

@QuanMPhm

QuanMPhm commented Jan 6, 2026

Copy link
Copy Markdown
Contributor Author

@naved001 @knikolla Do you guys want to meet after the 2:30PM meeting for a code walk-through? It says you're both free at that time on your calendars

@QuanMPhm QuanMPhm force-pushed the 232/migrate_nonbillable branch from 047671c to 92c4a2f Compare January 6, 2026 18:44
@naved001

naved001 commented Jan 6, 2026

Copy link
Copy Markdown
Collaborator

@QuanMPhm Works for me

@QuanMPhm QuanMPhm force-pushed the 232/migrate_nonbillable branch from 92c4a2f to 328efa1 Compare January 26, 2026 22:21
@QuanMPhm

Copy link
Copy Markdown
Contributor Author

Just renamed is_in_time_range, following Naved's suggestion

Related: CCI-MOC/invoicing-private-data#49

Implementation required significant code changes:
- `get_nonbillable_projects()` in `loader.py` was
modified to read the `projects.yaml` file
- Logic to filter for nonbillable projects have been refactored
into a new function `find_nonbillable_projects()` in
`validate_billable_pi_processor.py`
- After some discussion, it was formally agreed that `projects.yaml`
will identify projects by name, but may use IDs for certain cases.
This has implications explained in `find_nonbillable_projects()`
- `ColdfrontFetchProcessor`, `ValidateBillablePIsProcessor`,
and test cases changed accordingly
`_validate_allocation_data` will now consider both
project and cluster names when validating nonbillable projects,
as opposed to just project names.

This ensures that projects with identical names but from different clusters
are accurately validated against the nonbillable list.
@QuanMPhm QuanMPhm force-pushed the 232/migrate_nonbillable branch from 328efa1 to 06de027 Compare January 30, 2026 20:13
@QuanMPhm QuanMPhm merged commit e4fb9d7 into CCI-MOC:main Feb 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrate to using the new project.yaml in the non-billable-projects repo

4 participants