-
Notifications
You must be signed in to change notification settings - Fork 60
Update command.py fix #267
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,8 @@ | |
| from ckanext.xloader.jobs import xloader_data_into_datastore_ | ||
| from ckanext.xloader.utils import XLoaderFormats, get_xloader_user_apitoken | ||
|
|
||
| log = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| class XloaderCmd: | ||
| def __init__(self, dry_run=False): | ||
|
|
@@ -133,17 +135,37 @@ def _submit_resource(self, resource, user, indent=0, sync=False, queue=None): | |
|
|
||
| def print_status(self): | ||
| import ckan.lib.jobs as rq_jobs | ||
|
|
||
| jobs = rq_jobs.get_queue().jobs | ||
| if not jobs: | ||
| print('No jobs currently queued') | ||
| return | ||
|
|
||
| for job in jobs: | ||
| job_params = eval(job.description.replace( | ||
| 'ckanext.xloader.jobs.xloader_data_into_datastore', '')) | ||
| job_metadata = job_params['metadata'] | ||
| # FIX DEFINITIVO: | ||
| # Non usare job.description (non è un canale dati stabile e può essere troncato). | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How does this affect backward compatibility? |
||
| # I parametri veri stanno in job.args / job.kwargs. | ||
| metadata = {} | ||
|
|
||
| try: | ||
| if getattr(job, 'args', None) and len(job.args) >= 1: | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Doesn't the truthiness check already ensure it's not empty? |
||
| payload = job.args[0] or {} | ||
| if isinstance(payload, dict): | ||
| metadata = payload.get('metadata') or {} | ||
| elif getattr(job, 'kwargs', None) and isinstance(job.kwargs, dict): | ||
| payload = job.kwargs.get('data') or job.kwargs | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why would it be "either the 'data' field or the whole thing"? |
||
| if isinstance(payload, dict): | ||
| metadata = payload.get('metadata') or {} | ||
| except Exception: | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we use a narrower exception type? Surely retrieving keys has a limited range of types it can raise. |
||
| metadata = {} | ||
|
|
||
| res_id = metadata.get('resource_id', 'N/A') | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm dubious about using "N/A" as the default value. |
||
| url = metadata.get('original_url') or metadata.get('url') or 'N/A' | ||
|
|
||
| print('{id} Enqueued={enqueued:%Y-%m-%d %H:%M} res_id={res_id} ' | ||
| 'url={url}'.format( | ||
| id=job._id, | ||
| id=getattr(job, '_id', None), | ||
| enqueued=job.enqueued_at, | ||
| res_id=job_metadata['resource_id'], | ||
| url=job_metadata['original_url'], | ||
| res_id=res_id, | ||
| url=url, | ||
| )) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use English.