Skip to content

pypi - Failed to process PriorityResourceURI #601

Open
@ghsa-retrieval

Description

@ghsa-retrieval

The pull request #600 allows to use the api/collect/index_packages/ API endpoint for pypi packages. I have encountered an error while doing so, which seems to indicate that some data is longer than the database column permits.

INFO:minecode.management.commands.priority_queue:Processing pkg:pypi/[email protected]
/opt/purldb/venv/lib/python3.12/site-packages/django/db/models/fields/__init__.py:1665: RuntimeWarning: DateTimeField Package.release_date received a naive datetime (2025-03-28 00:00:00) while time zone support is active.
 warnings.warn(
ERROR:minecode.management.commands.priority_queue:Error: Failed to process PriorityResourceURI: <PriorityResourceURI: pkg:pypi/[email protected]>
DataError DataError('value too long for type character varying(255)')
Traceback (most recent call last):
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
   return self.cursor.execute(sql, params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/psycopg/cursor.py", line 97, in execute
   raise ex.with_traceback(None)
psycopg.errors.StringDataRightTruncation: value too long for type character varying(255)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "/opt/purldb/minecode/management/commands/priority_queue.py", line 86, in handle
   errors = process_request(priority_resource_uri)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/minecode/management/commands/priority_queue.py", line 118, in process_request
   errors = _priority_router.process(purl_to_visit, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/minecode/route.py", line 159, in process
   return endpoint(string, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/minecode/collectors/pypi.py", line 127, in process_request
   error_msg = map_pypi_package(package_url, pipelines, priority)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/minecode/collectors/pypi.py", line 83, in map_pypi_package
   db_package, _, _, error = merge_or_create_package(package, visit_level=0)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/minecode/model_utils.py", line 428, in merge_or_create_package
   Party.objects.create(
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
   return getattr(self.get_queryset(), name)(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/models/query.py", line 679, in create
   obj.save(force_insert=True, using=self.db)
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/models/base.py", line 892, in save
   self.save_base(
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/models/base.py", line 998, in save_base
   updated = self._save_table(
             ^^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/models/base.py", line 1161, in _save_table
   results = self._do_insert(
             ^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/models/base.py", line 1202, in _do_insert
   return manager._insert(
          ^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
   return getattr(self.get_queryset(), name)(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/models/query.py", line 1847, in _insert
   return query.get_compiler(using=using).execute_sql(returning_fields)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1836, in execute_sql
   cursor.execute(sql, params)
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 79, in execute
   return self._execute_with_wrappers(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
   return executor(sql, params, many, context)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 100, in _execute
   with self.db.wrap_database_errors:
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
   raise dj_exc_value.with_traceback(traceback) from exc_value
 File "/opt/purldb/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
   return self.cursor.execute(sql, params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/purldb/venv/lib/python3.12/site-packages/psycopg/cursor.py", line 97, in execute
   raise ex.with_traceback(None)
django.db.utils.DataError: value too long for type character varying(255)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions