Skip to content

Cold storage interface and service #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Cold storage interface and service #153

wants to merge 3 commits into from

Conversation

psaiz
Copy link
Contributor

@psaiz psaiz commented Apr 29, 2025

No description provided.

@psaiz psaiz marked this pull request as draft April 29, 2025 07:01
@psaiz psaiz force-pushed the cold2 branch 3 times, most recently from b4b1244 to 65bd08a Compare May 5, 2025 07:43
@psaiz psaiz force-pushed the cold2 branch 9 times, most recently from 2aa4716 to cb92e84 Compare May 15, 2025 15:07
return rb

@staticmethod
def start(req, num_files, size):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def start(req, num_files, size):
def start_processing(req, num_files, size):

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't it be better 'started'?

):
self._transfer.append(entry_point.load()())
else:
self._cold_path = [
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep this in the database instead of the entrypoints/code

@psaiz psaiz force-pushed the cold2 branch 5 times, most recently from 886f413 to 5e4caab Compare May 16, 2025 14:37
@psaiz psaiz marked this pull request as ready for review May 16, 2025 14:52
@psaiz psaiz force-pushed the cold2 branch 2 times, most recently from 183d001 to 2279ec0 Compare May 16, 2025 15:17
Copy link
Contributor

@zzacharo zzacharo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some comments but nothing major so that cannot be addressed or ticketized for a next iteration on the relevant code.

if tag:
info["tags"][tagName] = tag.value
if "availability" in self.data:
del self.data["availability"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe worth checking before merging?

avl = "ready"
for t in self.obj.tags:
if t.key == "hot_deleted":
avl = "needs request"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
avl = "needs request"
avl = "needs_request"

minor: You can add these in the next iteration but I would try to avoid these hardcoded values and create enums instead.

return transfer

@staticmethod
def _get_ongoing_transfers(last_check):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems contradictory to have a staticmethod that is private....

Suggested change
def _get_ongoing_transfers(last_check):
def get_ongoing_transfers(last_check):

)

@staticmethod
def _load_class(full_class_path):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above...

return current_app.config["COLD_ACTIVE_ARCHIVING_TRANSFERS_THRESHOLD"]

@staticmethod
def _send_email(req, emails):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def _send_email(req, emails):
def send_email(req, emails):

db.session.add(req)

@staticmethod
def complete(req):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def complete(req):
def mark_as_completed(req):

return True

@staticmethod
def subscribe(transfer_id, email):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: move to Transfer class or utility?


By default, it prints the urls for all the files of the entry.
"""
m = ColdStorageManager(current_app)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you plan to pass different Flask application to the class? If not, I think current_app can be accessed directly from the class it needs it...

RequestService._check_running()

@staticmethod
def _check_submitted():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def _check_submitted():
def check_submitted():

db.session.commit()

@staticmethod
def _check_running():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def _check_running():
def check_running():

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants