Skip to content

Regions cache (sqlite3) access fails from a forked process #641

@randomir

Description

@randomir

When accessing regions cache (dwave.cloud.regions.get_regions) from multiple processes created by forking, the following error has been reported:

  File "/home/wls/.conda/.../lib/python3.9/site-packages/dwave/system/samplers/dwave_sampler.py", line 30, in <module>
    from dwave.cloud.client import Client
  File "/home/wls/.conda/.../lib/python3.9/site-packages/dwave/cloud/__init__.py", line 19, in <module>
    from dwave.cloud.client import Client
  File "/home/wls/.conda/.../lib/python3.9/site-packages/dwave/cloud/client/__init__.py", line 17, in <module>
    from dwave.cloud.client.base import Client
  File "/home/wls/.conda/.../lib/python3.9/site-packages/dwave/cloud/client/base.py", line 72, in <module>
    from dwave.cloud.regions import get_regions, resolve_endpoints
  File "/home/wls/.conda/.../lib/python3.9/site-packages/dwave/cloud/regions.py", line 37, in <module>
    @cached.ondisk(maxage=_REGIONS_CACHE_MAXAGE, key='cache_key', bucket='regions')
  File "/home/wls/.conda/.../lib/python3.9/site-packages/dwave/cloud/utils/decorators.py", line 291, in ondisk
    cache = diskcache.Cache(disk=diskcache.JSONDisk, directory=directory,
  File "/home/wls/.conda/.../lib/python3.9/site-packages/diskcache/core.py", line 478, in __init__
    self.reset(key, value, update=False)
  File "/home/wls/.conda/.../lib/python3.9/site-packages/diskcache/core.py", line 2431, in reset
    ((old_value,),) = sql(
sqlite3.OperationalError: locking protocol

Possible causes:

  • "Under Unix, you should not carry an open SQLite database across a fork() system call into the child process." (source here)
  • Windows accessing sqlite3 file over NFS (see here)
  • DiskCache accessing a file on WSL1
  • this SO answer

See also: #621.

Possibly related: grantjenks/python-diskcache#325, grantjenks/python-diskcache#172.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions