TaskAdmin +readonly_fields "created_at", "updated_at"; fix setup name #118

ikvk · 2022-01-11T11:51:57Z

TaskAdmin +readonly_fields "created_at", "updated_at"
Ignore pycharm files
Let github.com to show "Used by" block at main repo page (setup.name, on my own experience):
ikvk/imap_tools@f1b2614#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7R16
Try in models.Task.__str__

…; Task.__str__ try

ikvk · 2023-03-17T10:51:47Z

"used by" block example:

ikvk · 2024-04-23T07:04:27Z

Dear @amureki , is project support are stopped/dead?

amureki

Thank you for the contribution!

I feel like, it'd be good to clarify some things here before we proceed; hence, I left some comments.

.gitignore

amureki · 2024-04-26T08:02:44Z

django_dramatiq/admin.py

-            instance.message.options.get("eta", instance.message.message_timestamp) / 1000
-        )
-
+        """Estimated time of arrival"""


What would be the value of this docstring? It just deciphers a common-known abbreviation, which the function named after.

First time I took a few moments to decipher it, as I didn't meet her very often. I think it definitely won't hurt

Suggested change

"""Estimated time of arrival"""

I would agree with @amureki on this. Lets remove it before merging.

Are you serious? How can the documentation line interfere?

I conducted a survey among 10 of my colleagues: "I know exactly what the abbreviation ETA stands for." Most of them are engineers. The result: 40% know for sure.

I didn't think I'd have to defend the docstring either.

One more argument:
Explicit is better than implicit.
:D

By the way, is "Estimated time of arrival" the correct decoding of ETA?

amureki · 2024-04-26T08:40:59Z

django_dramatiq/models.py

+        try:
+            return str(self.message)
+        except Exception as e:
+            return f'Failed to display Task: {e}'


Oh, this is interesting.

Could you, please, provide some information regarding this?
What kind of exception did you get here and on which conditions?
If this is a valid case, and we'll decide to handle it - then we'd need to document this more and update a test suite to cover it.

Unfortunately, it was a long time ago, the log has already been cleared. I remember that it was for various reasons:

When I interfered with the models data.Task during migration

Without my participation, maybe it was a bug in dramatiq.

On incorrect using dramatiq actor.send, I have not saved example. I think it's somewhere in https://github.com/Bogdanp/dramatiq/issues

If I can find the info, I will definitely let you know. (or the dramatiq project)

if you'd happen to find it, I am curious to see. 👍
And yeah, then we can check what'd be the best way to proceed - either in this library, or directly in dramatiq. But without this knowledge, I am afraid, I would not proceed with these lines here.

If the broken data gets into the database, no matter how, then it actually breaks the admin web page. (we are dealing with bytes in BinaryField)

This edit will avoid 500 errors if the data was suddenly broken. I don't understand your skepticism, security doesn't suffer, performance doesn't suffer either.

Please, if you felt scepticism in my comments - that was definitely not the intention.

This code might not suffer from security or performance issues, but this is not the point.
We start here, then we end up wrapping every __str__ or __repr__ functions for premature "safety".
But we don't want that, right?
We want to have every line of code meaning something and solving something. And we intend to ensure its function by adding a test case to make sure it won't happen again, and we won't forget about it.

Plus, I think, if we just catch all exceptions here - we'd start ignoring them (as it is kinda handled already, right?), we could accidentally ignore important symptoms leading to this error. So if there is something wrong with bytes in message property - we should handle that bit and not try-catch every place that uses message.

And this is what I am missing here - I do not know the real case that this code change will solve, but it certainly introduces a different behaviour (even though it is a small one).

I hope you could follow me here, and I am sorry, if you still would not agree with my thinking process here, but I truly believe this is only for the good of the project.

Again, I am looking forward to understanding the underlying issue and let's try to solve it in its roots!

we could accidentally ignore important symptoms leading to this error

I absolutely agree, I added a message to the log.

The key feature here is that the database cannot guarantee us the encodability of a BinaryField data.
Any byte error will result in 500 at admin web page, no matter how it got into the database.

And I finally found the trace!
I then changed some dramatic settings in the project, but I didn’t log which ones :D

2022-03-10T16:01:44.770354+05:00 job1.stage.int.<mywork>.ru django: Failed to decode message using encoder . Traceback (most recent call last): File "/home/int/.pyenv/versions/3.9.7/envs/int-3.9.7/lib/python3.9/site-packages/dramatiq/message.py", line 102, in decode return cls(**global_encoder.decode(data)) _pickle.UnpicklingError: invalid load key, '_'. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/int/.pyenv/versions/3.9.7/envs/int-3.9.7/lib/python3.9/site-packages/dramatiq/brokers/rabbitmq.py", line 511, in __next__ message = Message.decode(body) File "/home/int/.pyenv/versions/3.9.7/envs/int-3.9.7/lib/python3.9/site-packages/dramatiq/message.py", line 104, in decode raise DecodeError("Failed to decode message.", data, e) from e dramatiq.errors.DecodeError: Failed to decode message.

amureki · 2024-04-26T08:46:50Z

Dear @amureki , is project support are stopped/dead?

hey @ikvk !
The actual status of the project is explained here:
#122

It remains minimally maintained, as continuous support of upcoming Python and Django versions, but the reviews are limited (as I'm sadly struggling with finding the time and strength for it).

We are still grateful for the interest and opened PRs, I'll try to deal with them, but please do not expect instant feedback 😅.

ikvk · 2024-04-26T10:23:14Z

Also I suggest to implement white and black lists:
DJANGO_DRAMATIQ_TASKS_NOT_WRITES = ['task_name',]
DJANGO_DRAMATIQ_TASKS_WRITES_ONLY = ['task_name',]
In TaskManager.create_or_update_from_message by "if not: return None"
It is really useful on practice.

Do you agree to this?

I cat put it here or at next MR.

amureki · 2024-04-26T11:14:42Z

Also I suggest to implement white and black lists: DJANGO_DRAMATIQ_TASKS_NOT_WRITES = ['task_name',] DJANGO_DRAMATIQ_TASKS_WRITES_ONLY = ['task_name',] In TaskManager.create_or_update_from_message by "if not: return None" It is really useful on practice.

Do you agree to this?

I cat put it here or at next MR.

I'd suggest opening an issue here with a bit more details, as from this description I am not sure, I did follow what exactly you are proposing here.
Does this mean you want to stop certain tasks from being stored in the admin interface?
Any way, let's proceed in a new issue regarding this topic.

…Task; logger var lo lower case

andrewgy8

Hi @ikvk,

Thank you for the contribution, and apologies for the delay in reviewing this!

To help move this PR forward, it would be great if you could provide a clearer description of the specific issue you’re addressing. It also looks like this PR might be tackling multiple issues instead of focusing on one.

Would the following description be more accurate for this PR?
Add "created_at" and "updated_at" fields to the Task Admin

If so, could we limit this PR to just the changes related to that and remove any unrelated code? It also seems like a linting issue might have renamed a couple of variables—could you confirm?

Thanks again for your efforts!

django_dramatiq/admin.py

andrewgy8 · 2024-12-27T07:52:27Z

django_dramatiq/admin.py

-            instance.message.options.get("eta", instance.message.message_timestamp) / 1000
-        )
-
+        """Estimated time of arrival"""


Suggested change

"""Estimated time of arrival"""

I would agree with @amureki on this. Lets remove it before merging.

andrewgy8 · 2024-12-27T07:54:21Z

django_dramatiq/middleware.py

@@ -3,7 +3,7 @@
 from django import db
 from dramatiq.middleware import Middleware

-LOGGER = logging.getLogger("django_dramatiq.AdminMiddleware")
+logger = logging.getLogger(__name__)


issue: Any reason for this change? ie. this is potentially a breaking change for people relying on the namespacing of this logger.

Logger naming for this place is not ideomatic, there is no reasons to do this like here.
https://docs.python.org/3/library/logging.html

Anyway the log path will be the same here.

A potential problem may look like this: they changed the name of the class and forgot to change it in the logger.

This approach would be logical if you specified a single name for the entire package, for example: "django_dramatiq_all_pain" however, it is obvious from this place that there was no such goal.

andrewgy8 · 2024-12-27T07:54:44Z

django_dramatiq/models.py

@@ -10,6 +11,8 @@
 #: The database label to use when storing task metadata.
 DATABASE_LABEL = DjangoDramatiqConfig.tasks_database()

+logger = logging.getLogger(__name__)


issue: Same issue as above. 😉

andrewgy8 · 2024-12-27T07:55:33Z

setup.py

@@ -19,7 +19,7 @@ def rel(*xs):


 setup(
-    name="django_dramatiq",
+    name="django-dramatiq",


issue: Any reason to change the name of the project?? 🤔

Yes,
Let github.com to show "Used by" block at main repo page (setup.name, on my own experience):
For my lib it works, I found this decision somewhere at web.
ikvk/imap_tools@f1b2614#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7R16

Here is rules:
https://packaging.python.org/en/latest/specifications/name-normalization/

This is not killer fature, but useful.

If I saw such a MR for my library, I would also double-check everything 100 times.

Let github.com to show "Used by" block at main repo page

I dont think this is the case, and its most likely due to the number of public repo dependents being less than 100..

Maybe so, I suggest you try, and if anything, you can roll back the changes in the minor release immediately after the release.
In my old screenshot, the number of uses was 500, there is a chance of success.
*The block is not always visible immediately after opening the repository page, sometimes you need to wait a few seconds and refresh the page.

Co-authored-by: Andrew Graham-Yooll <[email protected]>

andrewgy8 · 2024-12-29T19:59:16Z

django_dramatiq/models.py

+        try:
+            return str(self.message)
+        except Exception as e:
+            logger.exception(f'Failed to display Task {self.id}')
+            return f'Failed to display Task: {e}'


Under what condition would this raise?

As I told above

The key feature here is that the database cannot guarantee us the encodability of a BinaryField data.
Any byte error will result in 500 at admin web page, no matter how it got into the database.

And I finally found the trace!
I then changed some dramatic settings in the project, but I didn’t log which ones :D

2022-03-10T16:01:44.770354+05:00 job1.stage.int.<mywork>.ru django: Failed to decode message using encoder . Traceback (most recent call last): File "/home/int/.pyenv/versions/3.9.7/envs/int-3.9.7/lib/python3.9/site-packages/dramatiq/message.py", line 102, in decode return cls(**global_encoder.decode(data)) _pickle.UnpicklingError: invalid load key, '_'. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/int/.pyenv/versions/3.9.7/envs/int-3.9.7/lib/python3.9/site-packages/dramatiq/brokers/rabbitmq.py", line 511, in __next__ message = Message.decode(body) File "/home/int/.pyenv/versions/3.9.7/envs/int-3.9.7/lib/python3.9/site-packages/dramatiq/message.py", line 104, in decode raise DecodeError("Failed to decode message.", data, e) from e dramatiq.errors.DecodeError: Failed to decode message.

ikvk · 2025-02-03T04:17:14Z

@andrewgy8 , please summarize this PR to close it, what do I need to do?

andrewgy8 · 2025-02-14T08:38:54Z

Apologies for the delay. I appreciate your work on this! I noticed that the suggestions from myself and @amureki haven't been addressed yet. Once those are incorporated, I'll be happy to review your changes again.

Thanks!

ikvk · 2025-02-14T09:48:31Z

I would like to discuss your suggestions:

Comment
"""Estimated time of arrival"""
I have given my arguments, do you disagree with them?
Honestly, I am confused by this suggestion, why is it important?
logger
logger = logging.getLogger(__name__)
Please tell me why it is important to use a non-ideomatic version of the logger here.
What problem is being solved in this case?
Setup name
name="django_dramatiq",
This is a really sensitive change, and it cannot but arouse suspicion.
I suggest trying to issue a release with a special mark, instead of v0.12.0 put v0.12.0rc1 or v0.12.0_usedby_test.
If this doesn't work (used by doesn't appear in a couple of days) - you can always remove the release from PyPI and github and return the name back.
Note that even the library's README.md uses the wording:
pip install django-dramatiq
What do you think about this?

Thank you for your attention and answers.

ikvk added 3 commits January 11, 2022 16:38

TaskAdmin +readonly_fields "created_at", "updated_at"; fix setup name

30261b0

TaskAdmin +readonly_fields "created_at", "updated_at"; fix setup name…

8831f1d

…; Task.__str__ try

TaskAdmin +readonly_fields "created_at", "updated_at"; fix setup name…

d402b4d

…; Task.__str__ try

amureki reviewed Apr 26, 2024

View reviewed changes

del /.idea/

0e7a745

ikvk added 2 commits April 26, 2024 17:17

flake

01fb783

logging to idiomatic usage; add logger.exception - Failed to display …

69c482b

…Task; logger var lo lower case

andrewgy8 requested changes Dec 27, 2024

View reviewed changes

Update django_dramatiq/admin.py

e56726c

Co-authored-by: Andrew Graham-Yooll <[email protected]>

andrewgy8 reviewed Dec 29, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TaskAdmin +readonly_fields "created_at", "updated_at"; fix setup name #118

TaskAdmin +readonly_fields "created_at", "updated_at"; fix setup name #118

ikvk commented Jan 11, 2022 •

edited

Loading

ikvk commented Mar 17, 2023

ikvk commented Apr 23, 2024

amureki left a comment

amureki Apr 26, 2024

ikvk Apr 26, 2024 •

edited

Loading

andrewgy8 Dec 27, 2024

ikvk Dec 28, 2024

ikvk Dec 28, 2024

ikvk Jan 19, 2025

amureki Apr 26, 2024

ikvk Apr 26, 2024 •

edited

Loading

amureki Apr 26, 2024

ikvk Apr 26, 2024

amureki Apr 26, 2024

ikvk Apr 27, 2024 •

edited

Loading

amureki commented Apr 26, 2024

ikvk commented Apr 26, 2024

amureki commented Apr 26, 2024

andrewgy8 left a comment •

edited

Loading

andrewgy8 Dec 27, 2024

andrewgy8 Dec 27, 2024

ikvk Dec 28, 2024

andrewgy8 Dec 27, 2024

ikvk Dec 28, 2024

andrewgy8 Dec 27, 2024

ikvk Dec 28, 2024

andrewgy8 Jan 19, 2025

ikvk Jan 19, 2025

andrewgy8 Dec 29, 2024

ikvk Jan 9, 2025

ikvk commented Feb 3, 2025

andrewgy8 commented Feb 14, 2025

ikvk commented Feb 14, 2025

TaskAdmin +readonly_fields "created_at", "updated_at"; fix setup name #118

Are you sure you want to change the base?

TaskAdmin +readonly_fields "created_at", "updated_at"; fix setup name #118

Conversation

ikvk commented Jan 11, 2022 • edited Loading

ikvk commented Mar 17, 2023

ikvk commented Apr 23, 2024

amureki left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ikvk Apr 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ikvk Apr 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ikvk Apr 27, 2024 • edited Loading

Choose a reason for hiding this comment

amureki commented Apr 26, 2024

ikvk commented Apr 26, 2024

amureki commented Apr 26, 2024

andrewgy8 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ikvk commented Feb 3, 2025

andrewgy8 commented Feb 14, 2025

ikvk commented Feb 14, 2025

ikvk commented Jan 11, 2022 •

edited

Loading

ikvk Apr 26, 2024 •

edited

Loading

ikvk Apr 26, 2024 •

edited

Loading

ikvk Apr 27, 2024 •

edited

Loading

andrewgy8 left a comment •

edited

Loading