|
| 1 | +Metadata-Version: 2.1 |
| 2 | +Name: dill |
| 3 | +Version: 0.4.0 |
| 4 | +Summary: serialize all of Python |
| 5 | +Home-page: https://github.com/uqfoundation/dill |
| 6 | +Download-URL: https://pypi.org/project/dill/#files |
| 7 | +Author: Mike McKerns |
| 8 | +Author-email: mmckerns@uqfoundation.org |
| 9 | +Maintainer: Mike McKerns |
| 10 | +Maintainer-email: mmckerns@uqfoundation.org |
| 11 | +License: BSD-3-Clause |
| 12 | +Project-URL: Documentation, http://dill.rtfd.io |
| 13 | +Project-URL: Source Code, https://github.com/uqfoundation/dill |
| 14 | +Project-URL: Bug Tracker, https://github.com/uqfoundation/dill/issues |
| 15 | +Platform: Linux |
| 16 | +Platform: Windows |
| 17 | +Platform: Mac |
| 18 | +Classifier: Development Status :: 5 - Production/Stable |
| 19 | +Classifier: Intended Audience :: Developers |
| 20 | +Classifier: Intended Audience :: Science/Research |
| 21 | +Classifier: License :: OSI Approved :: BSD License |
| 22 | +Classifier: Programming Language :: Python :: 3 |
| 23 | +Classifier: Programming Language :: Python :: 3.8 |
| 24 | +Classifier: Programming Language :: Python :: 3.9 |
| 25 | +Classifier: Programming Language :: Python :: 3.10 |
| 26 | +Classifier: Programming Language :: Python :: 3.11 |
| 27 | +Classifier: Programming Language :: Python :: 3.12 |
| 28 | +Classifier: Programming Language :: Python :: 3.13 |
| 29 | +Classifier: Programming Language :: Python :: Implementation :: CPython |
| 30 | +Classifier: Programming Language :: Python :: Implementation :: PyPy |
| 31 | +Classifier: Topic :: Scientific/Engineering |
| 32 | +Classifier: Topic :: Software Development |
| 33 | +Requires-Python: >=3.8 |
| 34 | +License-File: LICENSE |
| 35 | +Provides-Extra: graph |
| 36 | +Requires-Dist: objgraph >=1.7.2 ; extra == 'graph' |
| 37 | +Provides-Extra: profile |
| 38 | +Requires-Dist: gprof2dot >=2022.7.29 ; extra == 'profile' |
| 39 | +Provides-Extra: readline |
| 40 | + |
| 41 | +----------------------------- |
| 42 | +dill: serialize all of Python |
| 43 | +----------------------------- |
| 44 | + |
| 45 | +About Dill |
| 46 | +========== |
| 47 | + |
| 48 | +``dill`` extends Python's ``pickle`` module for serializing and de-serializing |
| 49 | +Python objects to the majority of the built-in Python types. Serialization |
| 50 | +is the process of converting an object to a byte stream, and the inverse |
| 51 | +of which is converting a byte stream back to a Python object hierarchy. |
| 52 | + |
| 53 | +``dill`` provides the user the same interface as the ``pickle`` module, and |
| 54 | +also includes some additional features. In addition to pickling Python |
| 55 | +objects, ``dill`` provides the ability to save the state of an interpreter |
| 56 | +session in a single command. Hence, it would be feasible to save an |
| 57 | +interpreter session, close the interpreter, ship the pickled file to |
| 58 | +another computer, open a new interpreter, unpickle the session and |
| 59 | +thus continue from the 'saved' state of the original interpreter |
| 60 | +session. |
| 61 | + |
| 62 | +``dill`` can be used to store Python objects to a file, but the primary |
| 63 | +usage is to send Python objects across the network as a byte stream. |
| 64 | +``dill`` is quite flexible, and allows arbitrary user defined classes |
| 65 | +and functions to be serialized. Thus ``dill`` is not intended to be |
| 66 | +secure against erroneously or maliciously constructed data. It is |
| 67 | +left to the user to decide whether the data they unpickle is from |
| 68 | +a trustworthy source. |
| 69 | + |
| 70 | +``dill`` is part of ``pathos``, a Python framework for heterogeneous computing. |
| 71 | +``dill`` is in active development, so any user feedback, bug reports, comments, |
| 72 | +or suggestions are highly appreciated. A list of issues is located at |
| 73 | +https://github.com/uqfoundation/dill/issues, with a legacy list maintained at |
| 74 | +https://uqfoundation.github.io/project/pathos/query. |
| 75 | + |
| 76 | + |
| 77 | +Major Features |
| 78 | +============== |
| 79 | + |
| 80 | +``dill`` can pickle the following standard types: |
| 81 | + |
| 82 | + - none, type, bool, int, float, complex, bytes, str, |
| 83 | + - tuple, list, dict, file, buffer, builtin, |
| 84 | + - Python classes, namedtuples, dataclasses, metaclasses, |
| 85 | + - instances of classes, |
| 86 | + - set, frozenset, array, functions, exceptions |
| 87 | + |
| 88 | +``dill`` can also pickle more 'exotic' standard types: |
| 89 | + |
| 90 | + - functions with yields, nested functions, lambdas, |
| 91 | + - cell, method, unboundmethod, module, code, methodwrapper, |
| 92 | + - methoddescriptor, getsetdescriptor, memberdescriptor, wrapperdescriptor, |
| 93 | + - dictproxy, slice, notimplemented, ellipsis, quit |
| 94 | + |
| 95 | +``dill`` cannot yet pickle these standard types: |
| 96 | + |
| 97 | + - frame, generator, traceback |
| 98 | + |
| 99 | +``dill`` also provides the capability to: |
| 100 | + |
| 101 | + - save and load Python interpreter sessions |
| 102 | + - save and extract the source code from functions and classes |
| 103 | + - interactively diagnose pickling errors |
| 104 | + |
| 105 | + |
| 106 | +Current Release |
| 107 | +=============== |
| 108 | + |
| 109 | +The latest released version of ``dill`` is available from: |
| 110 | + |
| 111 | + https://pypi.org/project/dill |
| 112 | + |
| 113 | +``dill`` is distributed under a 3-clause BSD license. |
| 114 | + |
| 115 | + |
| 116 | +Development Version |
| 117 | +=================== |
| 118 | + |
| 119 | +You can get the latest development version with all the shiny new features at: |
| 120 | + |
| 121 | + https://github.com/uqfoundation |
| 122 | + |
| 123 | +If you have a new contribution, please submit a pull request. |
| 124 | + |
| 125 | + |
| 126 | +Installation |
| 127 | +============ |
| 128 | + |
| 129 | +``dill`` can be installed with ``pip``:: |
| 130 | + |
| 131 | + $ pip install dill |
| 132 | + |
| 133 | +To optionally include the ``objgraph`` diagnostic tool in the install:: |
| 134 | + |
| 135 | + $ pip install dill[graph] |
| 136 | + |
| 137 | +To optionally include the ``gprof2dot`` diagnostic tool in the install:: |
| 138 | + |
| 139 | + $ pip install dill[profile] |
| 140 | + |
| 141 | +For windows users, to optionally install session history tools:: |
| 142 | + |
| 143 | + $ pip install dill[readline] |
| 144 | + |
| 145 | + |
| 146 | +Requirements |
| 147 | +============ |
| 148 | + |
| 149 | +``dill`` requires: |
| 150 | + |
| 151 | + - ``python`` (or ``pypy``), **>=3.8** |
| 152 | + - ``setuptools``, **>=42** |
| 153 | + |
| 154 | +Optional requirements: |
| 155 | + |
| 156 | + - ``objgraph``, **>=1.7.2** |
| 157 | + - ``gprof2dot``, **>=2022.7.29** |
| 158 | + - ``pyreadline``, **>=1.7.1** (on windows) |
| 159 | + |
| 160 | + |
| 161 | +Basic Usage |
| 162 | +=========== |
| 163 | + |
| 164 | +``dill`` is a drop-in replacement for ``pickle``. Existing code can be |
| 165 | +updated to allow complete pickling using:: |
| 166 | + |
| 167 | + >>> import dill as pickle |
| 168 | + |
| 169 | +or:: |
| 170 | + |
| 171 | + >>> from dill import dumps, loads |
| 172 | + |
| 173 | +``dumps`` converts the object to a unique byte string, and ``loads`` performs |
| 174 | +the inverse operation:: |
| 175 | + |
| 176 | + >>> squared = lambda x: x**2 |
| 177 | + >>> loads(dumps(squared))(3) |
| 178 | + 9 |
| 179 | + |
| 180 | +There are a number of options to control serialization which are provided |
| 181 | +as keyword arguments to several ``dill`` functions: |
| 182 | + |
| 183 | +* with *protocol*, the pickle protocol level can be set. This uses the |
| 184 | + same value as the ``pickle`` module, *DEFAULT_PROTOCOL*. |
| 185 | +* with *byref=True*, ``dill`` to behave a lot more like pickle with |
| 186 | + certain objects (like modules) pickled by reference as opposed to |
| 187 | + attempting to pickle the object itself. |
| 188 | +* with *recurse=True*, objects referred to in the global dictionary are |
| 189 | + recursively traced and pickled, instead of the default behavior of |
| 190 | + attempting to store the entire global dictionary. |
| 191 | +* with *fmode*, the contents of the file can be pickled along with the file |
| 192 | + handle, which is useful if the object is being sent over the wire to a |
| 193 | + remote system which does not have the original file on disk. Options are |
| 194 | + *HANDLE_FMODE* for just the handle, *CONTENTS_FMODE* for the file content |
| 195 | + and *FILE_FMODE* for content and handle. |
| 196 | +* with *ignore=False*, objects reconstructed with types defined in the |
| 197 | + top-level script environment use the existing type in the environment |
| 198 | + rather than a possibly different reconstructed type. |
| 199 | + |
| 200 | +The default serialization can also be set globally in *dill.settings*. |
| 201 | +Thus, we can modify how ``dill`` handles references to the global dictionary |
| 202 | +locally or globally:: |
| 203 | + |
| 204 | + >>> import dill.settings |
| 205 | + >>> dumps(absolute) == dumps(absolute, recurse=True) |
| 206 | + False |
| 207 | + >>> dill.settings['recurse'] = True |
| 208 | + >>> dumps(absolute) == dumps(absolute, recurse=True) |
| 209 | + True |
| 210 | + |
| 211 | +``dill`` also includes source code inspection, as an alternate to pickling:: |
| 212 | + |
| 213 | + >>> import dill.source |
| 214 | + >>> print(dill.source.getsource(squared)) |
| 215 | + squared = lambda x:x**2 |
| 216 | + |
| 217 | +To aid in debugging pickling issues, use *dill.detect* which provides |
| 218 | +tools like pickle tracing:: |
| 219 | + |
| 220 | + >>> import dill.detect |
| 221 | + >>> with dill.detect.trace(): |
| 222 | + >>> dumps(squared) |
| 223 | + ┬ F1: <function <lambda> at 0x7fe074f8c280> |
| 224 | + ├┬ F2: <function _create_function at 0x7fe074c49c10> |
| 225 | + │└ # F2 [34 B] |
| 226 | + ├┬ Co: <code object <lambda> at 0x7fe07501eb30, file "<stdin>", line 1> |
| 227 | + │├┬ F2: <function _create_code at 0x7fe074c49ca0> |
| 228 | + ││└ # F2 [19 B] |
| 229 | + │└ # Co [87 B] |
| 230 | + ├┬ D1: <dict object at 0x7fe0750d4680> |
| 231 | + │└ # D1 [22 B] |
| 232 | + ├┬ D2: <dict object at 0x7fe074c5a1c0> |
| 233 | + │└ # D2 [2 B] |
| 234 | + ├┬ D2: <dict object at 0x7fe074f903c0> |
| 235 | + │├┬ D2: <dict object at 0x7fe074f8ebc0> |
| 236 | + ││└ # D2 [2 B] |
| 237 | + │└ # D2 [23 B] |
| 238 | + └ # F1 [180 B] |
| 239 | + |
| 240 | +With trace, we see how ``dill`` stored the lambda (``F1``) by first storing |
| 241 | +``_create_function``, the underlying code object (``Co``) and ``_create_code`` |
| 242 | +(which is used to handle code objects), then we handle the reference to |
| 243 | +the global dict (``D2``) plus other dictionaries (``D1`` and ``D2``) that |
| 244 | +save the lambda object's state. A ``#`` marks when the object is actually stored. |
| 245 | + |
| 246 | + |
| 247 | +More Information |
| 248 | +================ |
| 249 | + |
| 250 | +Probably the best way to get started is to look at the documentation at |
| 251 | +http://dill.rtfd.io. Also see ``dill.tests`` for a set of scripts that |
| 252 | +demonstrate how ``dill`` can serialize different Python objects. You can |
| 253 | +run the test suite with ``python -m dill.tests``. The contents of any |
| 254 | +pickle file can be examined with ``undill``. As ``dill`` conforms to |
| 255 | +the ``pickle`` interface, the examples and documentation found at |
| 256 | +http://docs.python.org/library/pickle.html also apply to ``dill`` |
| 257 | +if one will ``import dill as pickle``. The source code is also generally |
| 258 | +well documented, so further questions may be resolved by inspecting the |
| 259 | +code itself. Please feel free to submit a ticket on github, or ask a |
| 260 | +question on stackoverflow (**@Mike McKerns**). |
| 261 | +If you would like to share how you use ``dill`` in your work, please send |
| 262 | +an email (to **mmckerns at uqfoundation dot org**). |
| 263 | + |
| 264 | + |
| 265 | +Citation |
| 266 | +======== |
| 267 | + |
| 268 | +If you use ``dill`` to do research that leads to publication, we ask that you |
| 269 | +acknowledge use of ``dill`` by citing the following in your publication:: |
| 270 | + |
| 271 | + M.M. McKerns, L. Strand, T. Sullivan, A. Fang, M.A.G. Aivazis, |
| 272 | + "Building a framework for predictive science", Proceedings of |
| 273 | + the 10th Python in Science Conference, 2011; |
| 274 | + http://arxiv.org/pdf/1202.1056 |
| 275 | + |
| 276 | + Michael McKerns and Michael Aivazis, |
| 277 | + "pathos: a framework for heterogeneous computing", 2010- ; |
| 278 | + https://uqfoundation.github.io/project/pathos |
| 279 | + |
| 280 | +Please see https://uqfoundation.github.io/project/pathos or |
| 281 | +http://arxiv.org/pdf/1202.1056 for further information. |
0 commit comments