diff --git a/.gitignore b/.gitignore index 677acf6d..f6e395a3 100644 --- a/.gitignore +++ b/.gitignore @@ -190,3 +190,6 @@ venv # Pixi lock file (because it changes with every upstream commit) pixi.lock + +# AI-assisted code development: https://github.com/ezyang/codemcp +codemcp* diff --git a/docs/faq.md b/docs/faq.md index f0eb3bc7..c8c1ede5 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -92,6 +92,10 @@ You can also use this approach to write a reader that starts from a kerchunk-for Currently if you want to call your new reader from `virtualizarr.open_virtual_dataset` you would need to open a PR to this repository, but we plan to generalize this system to allow 3rd party libraries to plug in via an entrypoint (see [issue #245](https://github.com/zarr-developers/VirtualiZarr/issues/245)). +### What ML/AI model formats are supported? + +VirtualiZarr has built-in support for [SafeTensors](safetensors.md) files, which are commonly used for storing ML model weights in a safe, efficient format. + ## How does this actually work? I'm glad you asked! We can think of the problem of providing virtualized zarr-like access to a set of archival files in some other format as a series of steps: diff --git a/docs/index.md b/docs/index.md index 3326fb43..cef1118f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -15,7 +15,7 @@ VirtualiZarr aims to make the creation of cloud-optimized virtualized zarr data ## Features * Create virtual references pointing to bytes inside a archival file with [`open_virtual_dataset`](https://virtualizarr.readthedocs.io/en/latest/usage.html#opening-files-as-virtual-datasets), -* Supports a [range of archival file formats](https://virtualizarr.readthedocs.io/en/latest/faq.html#how-do-virtualizarr-and-kerchunk-compare), including netCDF4 and HDF5, +* Supports a [range of archival file formats](https://virtualizarr.readthedocs.io/en/latest/faq.html#how-do-virtualizarr-and-kerchunk-compare), including netCDF4, HDF5, and [SafeTensors](safetensors.md), * [Combine data from multiple files](https://virtualizarr.readthedocs.io/en/latest/usage.html#combining-virtual-datasets) into one larger store using [xarray's combining functions](https://docs.xarray.dev/en/stable/user-guide/combining.html), such as [`xarray.concat`](https://docs.xarray.dev/en/stable/generated/xarray.concat.html), * Commit the virtual references to storage either using the [Kerchunk references](https://fsspec.github.io/kerchunk/spec.html) specification or the [Icechunk](https://icechunk.io/) transactional storage engine. * Users access the virtual dataset using [`xarray.open_dataset`](https://docs.xarray.dev/en/stable/generated/xarray.open_dataset.html#xarray.open_dataset). @@ -79,6 +79,7 @@ self installation usage examples +safetensors faq api releases diff --git a/docs/safetensors.md b/docs/safetensors.md new file mode 100644 index 00000000..d3855148 --- /dev/null +++ b/docs/safetensors.md @@ -0,0 +1,134 @@ +# SafeTensors Reader User Guide + +The SafeTensors reader in VirtualiZarr allows you to reference tensors stored in SafeTensors files. This guide explains how to use the reader effectively. + +## What is SafeTensors Format? + +SafeTensors is a file format developed by HuggingFace for storing tensors (multidimensional arrays) +that offers several advantages: +- Safe: No use of pickle, eliminating security concerns +- Efficient: Zero-copy access for fast loading +- Simple: Straightforward binary format with JSON header +- Language-agnostic: Available across Python, Rust, C++, and JavaScript + +The format consists of: +- 8 bytes (header size): little-endian uint64 containing the size of the header +- JSON header: Contains metadata for all tensors (shapes, dtypes, offsets) +- Binary data: Contiguous tensor data + +## How VirtualiZarr's SafeTensors Reader Works + +VirtualiZarr's SafeTensors reader allows you to: +- Create "virtual" Zarr stores pointing to chunks of data inside SafeTensors files +- Open the virtual zarr stores as xarray DataArrays with named dimensions +- Access specific slices of tensors from cloud storage +- Preserve metadata from the original SafeTensors file + +## Basic Usage + +Opening a SafeTensors file is straightforward: + +```python +import virtualizarr as vz + +# Open a SafeTensors file +vds = vz.open_virtual_dataset("model.safetensors") + +# Access tensors as xarray variables +weight = vds["weight"] +bias = vds["bias"] +``` + +## Custom Dimension Names + +By default, dimensions are named generically (e.g., "weight_dim_0", "weight_dim_1"). You can provide custom dimension names for better semantics: + +```python +# Define custom dimension names +custom_dims = { + "weight": ["input_dims", "output_dims"], + "bias": ["output_dims"] +} + +# Open with custom dimension names +vds = vz.open_virtual_dataset( + "model.safetensors", + virtual_backend_kwargs={"dimension_names": custom_dims} +) + +# Now dimensions have meaningful names +print(vds["weight"].dims) # ('input_dims', 'output_dims') +print(vds["bias"].dims) # ('output_dims',) +``` + +## Loading Specific Variables + +You can specify which variables to load as eager arrays instead of virtual references: + +```python +# Load specific variables as eager arrays +vds = vz.open_virtual_dataset( + "model_weights.safetensors", + loadable_variables=["small_tensor1", "small_tensor2"] +) + +# These will be loaded as regular numpy arrays +small_tensor1 = vds["small_tensor1"] +# Large tensors remain virtual references +large_tensor = vds["large_tensor"] +``` + +## Working with Remote Files + +The SafeTensors reader supports reading from the HuggingFace Hub: +```python +# HuggingFace Hub +vds = vz.open_virtual_dataset( + "https://huggingface.co/openai-community/gpt2/model.safetensors", + virtual_backend_kwargs={"revision": "main"} +) +``` + +It supports reading from object storage: + +```python +# S3 +vds = vz.open_virtual_dataset( + "s3://my-bucket/model.safetensors", + reader_options={ + "storage_options": { + "key": "ACCESS_KEY", + "secret": "SECRET_KEY", + "region_name": "us-west-2" + } + } +) +``` + +## Accessing Metadata + +SafeTensors files can contain metadata at the file level and tensor level: + +```python +# Access file-level metadata +print(vds.attrs) # File-level metadata + +# Access tensor-specific metadata +print(vds["weight"].attrs) # Tensor-specific metadata + +# Access original SafeTensors dtype information +original_dtype = vds["weight"].attrs["original_safetensors_dtype"] +print(f"Original dtype: {original_dtype}") +``` + +## Known Limitations + +### Performance Considerations +- Very large tensors (>1GB) are treated as a single chunk, which may impact memory usage when accessing small slices +- Files with thousands of tiny tensors may have overhead due to metadata handling + +## Best Practices + +- **For large tensors**: Use slicing to access only the portions you need +- **For remote files**: Use appropriate credentials and optimize access patterns +- **For many small tensors**: Consider loading them eagerly using `loadable_variables` diff --git a/docs/usage.md b/docs/usage.md index b2a59743..26aa8a69 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -89,6 +89,28 @@ aws_credentials = {"key": ..., "secret": ...} vds = open_virtual_dataset("s3://some-bucket/file.nc", reader_options={'storage_options': aws_credentials}) ``` +### Opening different file formats + +VirtualiZarr automatically detects the file format based on the file extension or content. Currently supported formats include: + +- **NetCDF/HDF5**: Scientific data formats (NetCDF3, NetCDF4/HDF5) +- **DMRPP**: OPeNDAP Data Access Protocol responses +- **FITS**: Astronomical data in Flexible Image Transport System format +- **TIFF**: Tagged Image File Format for geospatial and scientific imagery +- **SafeTensors**: ML model weights format (`*.safetensors`), see the [SafeTensors guide](safetensors.md) for details +- **Kerchunk references**: Previously created virtualized references + +Each format has specific readers optimized for its structure. For SafeTensors files, additional options like custom dimension naming are available: + +```python +# Open a SafeTensors file with custom dimension names +custom_dims = {"weight": ["input_features", "output_features"]} +vds = open_virtual_dataset( + "model.safetensors", + virtual_backend_kwargs={"dimension_names": custom_dims} +) +``` + ## Chunk Manifests In the Zarr model N-dimensional arrays are stored as a series of compressed chunks, each labelled by a chunk key which indicates its position in the array. Whilst conventionally each of these Zarr chunks are a separate compressed binary file stored within a Zarr Store, there is no reason why these chunks could not actually already exist as part of another file (e.g. a netCDF file), and be loaded by reading a specific byte range from this pre-existing file. diff --git a/pyproject.toml b/pyproject.toml index fce23f0c..ce31243e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -52,6 +52,11 @@ hdf = [ "imagecodecs-numcodecs==2024.6.1", "obstore>=0.5.1", ] +safetensors = [ + "safetensors", + "ml-dtypes", + "obstore>=0.5.1", +] # kerchunk-based readers hdf5 = [ @@ -71,6 +76,7 @@ fits = [ ] all_readers = [ "virtualizarr[hdf]", + "virtualizarr[safetensors]", "virtualizarr[hdf5]", "virtualizarr[netcdf3]", "virtualizarr[fits]", @@ -176,7 +182,7 @@ rust = "*" run-mypy = { cmd = "mypy virtualizarr" } # Using '--dist loadscope' (rather than default of '--dist load' when '-n auto' # is used), reduces test hangs that appear to be macOS-related. -run-tests = { cmd = "pytest -n auto --dist loadscope --run-network-tests --verbose --durations=10" } +run-tests = { cmd = "pytest -n auto --dist loadscope --run-network-tests --verbose --durations=10" } run-tests-no-network = { cmd = "pytest -n auto --verbose" } run-tests-cov = { cmd = "pytest -n auto --run-network-tests --verbose --cov=virtualizarr --cov=term-missing" } run-tests-xml-cov = { cmd = "pytest -n auto --run-network-tests --verbose --cov=virtualizarr --cov-report=xml" } @@ -186,12 +192,12 @@ run-tests-html-cov = { cmd = "pytest -n auto --run-network-tests --verbose --cov [tool.pixi.environments] min-deps = ["dev", "test", "hdf", "hdf5", "hdf5-lib"] # VirtualiZarr/conftest.py using h5py, so the minimum set of dependencies for testing still includes hdf libs # Inherit from min-deps to get all the test commands, along with optional dependencies -test = ["dev", "test", "remote", "hdf", "hdf5", "netcdf3", "fits", "icechunk", "kerchunk", "hdf5-lib", "obstore"] -test-py311 = ["dev", "test", "remote", "hdf", "hdf5", "netcdf3", "fits", "icechunk", "kerchunk", "hdf5-lib", "obstore", "py311"] # test against python 3.11 -test-py312 = ["dev", "test", "remote", "hdf", "hdf5", "netcdf3", "fits", "icechunk", "kerchunk", "hdf5-lib", "obstore", "py312"] # test against python 3.12 -minio = ["dev", "remote", "hdf", "hdf5", "netcdf3", "fits", "icechunk", "kerchunk", "hdf5-lib", "obstore", "py312", "minio"] -upstream = ["dev", "test", "hdf", "hdf5", "hdf5-lib", "netcdf3", "upstream", "icechunk-dev"] -all = ["dev", "test", "remote", "hdf", "hdf5", "netcdf3", "fits", "icechunk", "kerchunk", "hdf5-lib", "obstore", "all_readers", "all_writers"] +test = ["dev", "test", "remote", "hdf", "safetensors", "hdf5", "netcdf3", "fits", "icechunk", "kerchunk", "hdf5-lib", "obstore"] +test-py311 = ["dev", "test", "remote", "hdf", "safetensors", "hdf5", "netcdf3", "fits", "icechunk", "kerchunk", "hdf5-lib", "obstore", "py311"] # test against python 3.11 +test-py312 = ["dev", "test", "remote", "hdf", "safetensors", "hdf5", "netcdf3", "fits", "icechunk", "kerchunk", "hdf5-lib", "obstore", "py312"] # test against python 3.12 +minio = ["dev", "remote", "hdf", "safetensors", "hdf5", "netcdf3", "fits", "icechunk", "kerchunk", "hdf5-lib", "obstore", "py312", "minio"] +upstream = ["dev", "test", "hdf", "safetensors", "hdf5", "hdf5-lib", "netcdf3", "upstream", "icechunk-dev"] +all = ["dev", "test", "remote", "hdf", "safetensors", "hdf5", "netcdf3", "fits", "icechunk", "kerchunk", "hdf5-lib", "obstore", "all_readers", "all_writers"] docs = ["docs"] # Define commands to run within the docs environment diff --git a/safetensors_examples.ipynb b/safetensors_examples.ipynb new file mode 100644 index 00000000..498a3c2a --- /dev/null +++ b/safetensors_examples.ipynb @@ -0,0 +1,358 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Using VirtualiZarr with SafeTensors Files\n", + "\n", + "This notebook demonstrates how to use VirtualiZarr to work with SafeTensors files efficiently." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup and Dependencies\n", + "\n", + "First, let's import the necessary libraries:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import torch\n", + "from safetensors.torch import save_file\n", + "\n", + "import virtualizarr as vz\n", + "\n", + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create a Sample SafeTensors File\n", + "\n", + "Let's create a sample SafeTensors file with different types of tensors:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Created sample file at sample_model.safetensors\n" + ] + } + ], + "source": [ + "# Create a directory for our files\n", + "os.makedirs(\"sample_data\", exist_ok=True)\n", + "\n", + "# Create tensors of different shapes and types\n", + "tensors = {\n", + " \"embedding\": torch.randn((10000, 7680), dtype=torch.float32),\n", + " \"weights\": torch.randn((768, 768), dtype=torch.float32),\n", + " \"bias\": torch.randn(768, dtype=torch.float32),\n", + " \"int_values\": torch.randint(-100, 100, (50, 50), dtype=torch.int32),\n", + " \"bool_mask\": torch.rand((100, 100)) > 0.5,\n", + "}\n", + "\n", + "# Add metadata\n", + "metadata = {\n", + " \"model_type\": \"transformer\",\n", + " \"created_by\": \"virtualizarr_example\",\n", + " \"version\": \"1.0\",\n", + "}\n", + "\n", + "# Save to file\n", + "filepath = \"sample_model.safetensors\"\n", + "save_file(tensors, filepath, metadata=metadata)\n", + "print(f\"Created sample file at {filepath}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Open with VirtualiZarr\n", + "\n", + "Now let's open the file using VirtualiZarr:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Available tensors:\n", + "- bias: shape=(768,), dtype=float32\n", + "- embedding: shape=(10000, 7680), dtype=float32\n", + "- weights: shape=(768, 768), dtype=float32\n", + "- int_values: shape=(50, 50), dtype=int32\n", + "- bool_mask: shape=(100, 100), dtype=bool\n", + "\n", + "Metadata:\n", + "- version: 1.0\n", + "- model_type: transformer\n", + "- created_by: virtualizarr_example\n" + ] + } + ], + "source": [ + "# Open the file\n", + "vds = vz.open_virtual_dataset(filepath)\n", + "\n", + "# Print available tensors\n", + "print(\"Available tensors:\")\n", + "for name, var in vds.data_vars.items():\n", + " print(f\"- {name}: shape={var.shape}, dtype={var.dtype}\")\n", + "\n", + "# Print metadata\n", + "print(\"\\nMetadata:\")\n", + "for key, value in vds.attrs.items():\n", + " print(f\"- {key}: {value}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using Custom Dimension Names\n", + "\n", + "Let's open the file again, but with custom dimension names:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Dimensions with custom names:\n", + "- bias: dims=('output',)\n", + "- embedding: dims=('token', 'feature')\n", + "- weights: dims=('input', 'output')\n", + "- int_values: dims=('int_values_dim_0', 'int_values_dim_1')\n", + "- bool_mask: dims=('bool_mask_dim_0', 'bool_mask_dim_1')\n" + ] + } + ], + "source": [ + "# Define custom dimension names\n", + "custom_dims = {\n", + " \"embedding\": [\"token\", \"feature\"],\n", + " \"weights\": [\"input\", \"output\"],\n", + " \"bias\": [\"output\"],\n", + "}\n", + "\n", + "# Open with custom dimension names\n", + "vds_named = vz.open_virtual_dataset(\n", + " filepath, virtual_backend_kwargs={\"dimension_names\": custom_dims}\n", + ")\n", + "\n", + "# Print dimensions\n", + "print(\"Dimensions with custom names:\")\n", + "for name, var in vds_named.data_vars.items():\n", + " print(f\"- {name}: dims={var.dims}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Accessing Parts of Tensors Efficiently\n", + "\n", + "One of the main benefits of VirtualiZarr is the ability to efficiently access parts of tensors:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Shape of extracted slice: (10, 100)\n", + "Shape of single token vector: (7680,)\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1gAAAE8CAYAAADdZRDNAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvc2/+5QAAAAlwSFlzAAAPYQAAD2EBqD+naQAAjtNJREFUeJztnQWY1Mb7x989wTncOdzdtUBb/GhLS12hQv91oQY1So32V1da2kLdhbYUd3e3w9398JP8n3eO7GWzyW6yG1n5fp5n4TYbmUwmM/POax5JkiQCAAAAAAAAABA2CeGfAgAAAAAAAAAAAwELAAAAAAAAACwCAhYAAAAAAAAAWAQELAAAAAAAAACwCAhYAAAAAAAAAGARELAAAAAAAAAAwCIgYAEAAAAAAACARUDAAgAAAAAAAACLgIAFAAAAAAAAABYBAQsAAKIEj8dDDz30EMUTM2bMEPf9+++/236tl156SVzLCLwf7y/z9ddfi23bt2+3sYQAAACiAQhYAABgIzzpNvJhQSKauPTSS3XvpV69em4XL25o0qQJValShSRJ0t2nY8eOVK5cOcrKyrLsumfOnBECZrS1WwAAcIIkR64CAABxynfffefz/dtvv6XJkyf7ba9fvz5FG5UrV6bhw4f7bS9WrBjFG7fffjvddNNNlD9/fkeve+utt9LgwYNp9uzZ1LlzZ7/fWaM2f/58oflMSkqyVMAaNmyYV9gGAACQBwQsAACwkdtuu83n+4IFC4SApd4ejbAgFQv3YQWJiYni4zS33HILDRkyhH788UdNAeunn34S2i0WxKKB06dPU+HChd0uBgAAhAVMBAEAIAImlU888QSlpqYKDUjdunXp7bffDmj2JfPqq69SQkICffTRR95t48ePp06dOomJatGiRalPnz60du1an+MGDBhARYoUoT179tDVV18t/i5Tpgw9+eSTlJ2dbblf08aNG4UwxkIZX+eFF14Q97dr1y7q27cvpaSkUPny5emdd97RPA+X6dlnnxX78H1dddVV4lg1CxcupF69eonrFCpUiLp06UJz587122/OnDnUunVrKlCgANWsWZM+//xzzeueP3+eHn/8cVFmrku+7u7du/320/LBqlatGl1xxRXiWm3atBHXqlGjhtBiqlm1apUoa8GCBYVmkJ/r6NGjg/p1cZthwYp91DIzM/1+Z8GL769t27biOz/vu+66S5gMcltr2LAhjRo1yu+4c+fOiWdXp04dUe4KFSpQv379aMuWLaI8XB8Ma7Fk01ClT9q0adO8bbB48eLiGa9fv16zbaxbt04IiiVKlKBLLrlE914BACBagAYLAABchIUMnrRPnz6d7r77bmrWrBlNnDiRnnrqKTEZfu+993SPff755+n1118XwsHAgQPFNjY97N+/P/Xs2ZPefPNNYco1YsQIMXFdvny5mPQrhRbejyffLNBNmTJFCDg8Ib///vuDlp2PP3z4sN92FhLUWogbb7xRmEG+8cYb9N9//wkBomTJkqLsl19+uSjrDz/8IAQ8FnzU2pjXXntNTMafeeYZOnjwIL3//vvUrVs3WrFihbiePKnv3bs3tWzZkoYOHSoETxZS+PxsQsdCDrN69Wrq0aOHEBJ4ks++Sbw/Cx1q7rnnHvr++++FANChQwdxDRZYjbJ582a67rrrxLPl58LCDAu3XEYWbhh+zpdddpm4P9ZGcd19+eWXhs0NWTt17733inbDAp0M3+eaNWvoxRdfFN8PHDhA7dq18wZL4ftnYZzLdvLkSXrssce8z5XPM3XqVGH2+Oijj1JGRobQvPL5uN65TXEbueaaa4TgJfuDMdyO+DmwMMn1e/bsWbEAwL5gy5Yt82mDzPXXX0+1a9cWbdnIogIAAEQ8EgAAAMd48MEHeQbp/T5mzBjx/dVXX/XZ77rrrpM8Ho+0efNm7zbej49nnnjiCSkhIUH6+uuvvb9nZGRIxYsXlwYOHOhzrv3790vFihXz2d6/f39xvpdfftln3+bNm0stW7YMeh9dunQRx2t9/u///s+739ChQ8W2e++917stKytLqly5sri/N954w7v92LFjUsGCBUXZZKZPny6Or1SpknTy5Env9l9//VVs/+CDD8T3nJwcqXbt2lLPnj3F3zJnzpyRqlevLnXv3t277eqrr5YKFCgg7dixw7tt3bp1UmJios+zWbFihfj+wAMP+Nz7LbfcIrbzvcmMHj1abNu2bZt3W9WqVcW2WbNmebcdPHhQyp8/v3h+Mg8//LCoi+XLl3u3HTlyRCpZsqTfObU4evSoOOfNN9/ss33w4MHi+PT0dPH97rvvlipUqCAdPnzYZ7+bbrpJtA+uK2bUqFHiuHfffdfvWnLdHjp0yK8OZJo1ayaVLVtW3IPMypUrRXu94447/NqGutwAABDtwEQQAABcZNy4ccJ355FHHvHZziaDLFOxhkEJb2PtwwcffCA0K6wVkWENw/Hjx+nmm28WmiX5w+dnLRVrydTcd999Pt/ZrGvr1q2Gys6aCL6m+iNrQtSaIBkuT6tWrcS9sPZEhk3J2DxS6/p33HGHMNGTYa0Qm61x/TGsydq0aZPQNB05csR772x+2bVrV5o1axbl5OQI7QxretgskqPvybB2jbV5SuRzq5+N1v3p0aBBA1GnMqw1Ut/jhAkTqH379kJ7KcPaPaN+U2xal5aWRv/884+4X4br9ueffxb1zGZ+/P2PP/6gK6+8UvytbB983ydOnBDaJYb3K126ND388MN+1woWxn7fvn3iWbCWju9BhrVb3bt399ZpoDYIAADRDkwEAQDARXbs2EEVK1b0ER6UUQX5dyXsv3Pq1ClhosWClBIWMBg2idOC/ZyUsG+N7EujnKwfO3bMUNnZlI3NxYygFGYY9pHi6/NEXr2dBSQ1bEKmnujXqlXL658k37tS4FTDQgT7VLHJmvp8DAs+SgGA657NDNlkUr2fUdT3rVXHfB0WsNTw/RmFhbG//vqL/v77byFkzps3T9QNm/cxhw4dEsL3yJEjxUcLNr1k2M+K7zGUqINye9WqI27TLNyqA1lUr17d9HUAACCSgYAFAABRBPuxsIbg448/phtuuMFHS8AaGtkPi4NBqFFPmJ2Meqd1Lb3rh+KHI9/7W2+95aMJUsKBPFjAchIr7zEQ7DPFwikHtWABi//na7MPlbJ+ONCInhAq+1A5jexDBwAAsQIELAAAcJGqVauKoAAcRECpxdqwYYP3d7VW43//+5/IPcTR8jgQgXycrGkpW7asYc1StCBrqJQCCgeQkIUC+d5ZSxfo3lljxxN69fmY9PR0n+9c9yyYyBodvf3Cha/D96JGa5seHBCDzSZZw8nBLH777TehyZQFbTkKIptIBmsbXJccjZGjEiYnJ2vuo2cqKLdXrTriNs0aS4RhBwDEOvDBAgAAF2HfGZ70skZKCUcP5EksR2NTw0IFm7Jx2Gv2qWGTN4Z9aVjA4GhsWiG72UwsWmHBgYVQGQ5Lzv4+cv1wVD4WDDgaIptQ6t07a3W4nsaMGUM7d+70/s51yeZrSuRzf/jhhz7bOYKhlXB5OBkwayZljh49KqIqmoHNBPm5/9///Z+4X6UPF9/3tddeK/yrOBJgoLbB+7FvlrpNKjVvHAKfYbNDJewXxxrEb775xuc3vuakSZNEewcAgFgHGiwAAHARFpA4RPdzzz0nfGaaNm0qJqLsS8PBFNT+PzIcbpv34Qkray5YYGDhin2zbr/9dmrRooUwD2PNBQsSHBqdzQu1Js2hwj5NHGhDC6sTELMpJIeav/POO4WGhoUc1ubJ4enZV4pDm7NQxOHPeb9KlSqJEOgc3IPr5t9///XmbuLAEhx84oEHHhBh2jmMOB/H+ahkWFBgP7dPP/1U3CuHaWeNoRnNkhGefvppUY8cBIIDS8hh2tl/iwWtYIElZDiPFufQ4nbBWjo5fLoMh8jnuuCAJ1xvHICDz8/BLViLyn/LAUVYoB00aBAtWrRI1BP7TfE+XF+c04rPz8f/8ssvIogGP59GjRqJD5tp8nNgvzIOYiKHaWcTRmWuLAAAiFncDmMIAADxHKZdDq/++OOPSxUrVpSSk5NFuPG33nrLJ9y4Oky7zN9//y0lJSVJN954o5Sdne0Nbc7hyjn0Nocjr1mzpjRgwABpyZIl3uM4FHrhwoX9yieHzg4nTLvyePl8HNZbid71+bwNGzb0C9P+008/SUOGDBHhvzmUe58+fXzCrMtwqPN+/fpJpUqVEqHLOVT6DTfcIE2dOtVnv5kzZ4pw9Pny5ZNq1KghffbZZ5r3fvbsWemRRx4R5+PyXnnlldKuXbsMh2nncmrdI3/U5e7UqZMoM4ewHz58uPThhx+Kc3KYfaM89dRT4hi+Zy0OHDgg2lBqaqpoa+XLl5e6du0qjRw50mc/Dtn+3HPPiRD38n6cOmDLli3efebNm+etQ3V9TJkyRerYsaN4VikpKaLeOBS+Er22AQAA0Y6H/3FbyAMAAACAL6zB5ETMbPLoZEASAAAA4QEfLAAAAMBlZD86GQ5Vz9Eg2SwSwhUAAEQX8MECAAAAXIb9lTgyJOeKYh+zr776ik6ePEkvvPCC20UDAABgEghYAAAAgMtwsBKOjMhJgDmoBQcpYSGrc+fObhcNAACASeCDBQAAAAAAAAAWAR8sAAAAAAAAALAICFgAAAAAAAAAYBFx5YOVk5NDe/fupaJFixpO3AgAAAAAAACIPSRJooyMDKpYsaJIWG8VcSVgsXCVmprqdjEAAAAAAAAAEcKuXbuocuXKlp0vrgQs1lzJlZiSkuJqWTIzM2nSpEnUo0cPSk5OdrUs8QDq2zlQ186C+nYW1LezoL6dA3XtLKjvyKhvTofByhdZRrCKuBKwZLNAFq4iQcAqVKiQKAdeLPtBfTsH6tpZUN/Ogvp2FtS3c6CunQX1HVn1bbXrEIJcAAAAAAAAAIBFQMACAAAAAAAAAIuAgAUAAAAAAAAAFgEBCwAAAAAAAAAsAgIWAAAAAAAAAFgEBCwAAAAAAAAAsAgIWCAmOHDyHA0YvYimbTjgdlEAAAAAAEAcAwELxAQv/r2GZqQforu+XuJ2UQAAAAAAQBwDAQvEBAczzrtdBACiEkmS3C4CAAAAEFNAwAIxAeaIAJhn7Kq91PyVyTRvy2G3iwJAzHDyXCZd+dEc+mzmFreLAgBwCQhYAAAQpzz043I6fiaT7vhqkdtFASGw+eApaj98Kn23YIfbRQEKRs3ZRqv3nKA3xm9wuygAAJeAgAViAiiwAADxxvNjVtO+E+fohTFr3C4KUHA+K8ftIgAAXAYCFgAAABCFZOdgaQkAED3k5Ehx4/cLAQsAAOKc+BjuYg8PedwuAriIctIYJ/NHAEwvCPV8fxbdOHIBxQMQsAAAAAAAQuTDqZuo1atTaNfRM24XBWhwKOM8rd930u1ixD2bDmbQpoOnaNG2oxQPQMACsQGWDAEIGehBQKyy+WAG/bF0t61mSe9O3khHTl+gtyeli+8evFARRevXplDvD2bT1kOn3C4KuEg8mAkmuV0AAKwg9l9VAOwD70+Ugol8ULq9O0v8n5yUQFc1rWjrteJgzhjVrNh1nGqUKeJ2MeIWT5x1WNBggbiFzTlGzNgicpYAAACIXVbtOu7YtSJR0LqQlUNvTthAC7YeoXglEp9LvCLFwbOAgAXigknrDtD7axJp17E8G/krP54jBhyEOAZqzlzIosd+Xk4T1+53uygAABA2nCuNFxRvipMAAwC4DQQs4IpN7O9Ld4sEmev2OuN4+uBPK2lbhoee/WutdxsnWGXmbYmcFT1eXRz69xoxwQfu8dnMrTRmxV76v++WUjxyLjNbOCIjDDiIFaQovsbOI2dowOhFND+MsWr74dMUq2Rl59DSHceElg5EBxLFPhCwYpizF7JpzPI9dOz0haD7Lt5+lJoOmyScgZ3gyd9WigSZg35dYcn5jMqGJ85mRbSqmlcXv5m/gz6attntosQ1B0+eo3iGBcsbPp9PH0d4Ozx6+gJNXndATLDikfjyaIgerA5y8cjPy2lG+iG6+QtrtU/Ldx6j35bsomhn+PgNdO2IeTTkz9VuFwUYfC+kSJp42QQErBjm5bFr6bFfVtAdoxYF3Xfgt0vo5LkseuK3leQkmQ5PjKIlutOOI7G72gjs49+Ve+nVsetEMsdwmLnxkPj/uwXbKZK58qM5ou8aPTeyywncx8n5nNXX2nfibNjnkDR0Btd8Oo+e+n1V1PtlfTVnm/j/j2XOLBCD0PBQfAEBK4b5Z8Ve8f/qPSeC7hvtiwlag0c0E+3PAwSHV/DemZQutMxW8fBPy+nLOdtoyvoDFA/sOZ478ZwAXzkAQmZbDJsPAvN96tO/r6QN++1135Ao9oGABUAEvuoQsCLTL8lK2MeJTUFZy2w1nJMHxD7RopGPl7qKZLOneAuRHSlEcpvQ4oEfltGvS3ZTnw/nuF2UqAcCFnAVddfDwR3i1Z8CRO5k7JPpm6neCxNoqoWaIfYfinwwKYvVxQL2vTmYcc7w/hxxdemOoxStODHPja6pNLCb5/5aTd3fmyX84aOFDftyNVd2BziS4uBlgYAFIoYTZzKpwYsTqdcHsx1dxYyHF91pDpwlen/qZjpx1r4cY4P/WEUv/ZMXFdJO3pqYnnvNGHWijrZVVhAeLCyx7w0HBjDC5zO3ihDf146Yb3vZgD3va6yZ0UcDPyzcSZsPnqJxq/e5XZTIC3JBsd8eIWABVydYSnln/kVHW+6QzIL5YWQxfEUifTJjqwg5bwfs9P3z4l309bztlpvugfhh97Ez9MWsrXTqfHSmRQjV7IsjLzK7jhoLnrD5kPk+OdKIlQkd+8h0fGMafTZzi2tlYI2+UxGHY4FgLe+/1ftF2poVDibD1gNmx9YBAQuACCTaJwPSxYmfXQNGVnZ010+k4bFhVI0GrRinRXht3Hp60YaFAM7J8+XsrbTxQAZFO9HwLOOFtyZsoL0nztEb4ze44oPFwQ9Yo68VcZgTs3f637SIEBSiicd+XSXS1tzzzRKKF6Q46FIgYAFXUb5j4czxjL6skeLom74/I6D/Qzx0PuFgf/1ERjuJ1gn0/hPnqN3wqfTu5I0Uyew+lqvBGb96vy2ho1/9bz31eG8WRRroX+zhv1X7bPFdUT4vt9eWjp/JDJg/j7Wid3292NEyxQpZOc74n3NuVKP+l9bioXgCAlYMwrmleMJkx6q0UThYxX3fLTUUgloOauFWaSUXclz1fH8WtXltqu2d6JA/V0W1Y3qkcepcFv26ZJfwF4wHQu1CPpy2iQ6cPE8fTt1E0YAdk+KVDqzix7M5z8lzmREZKGbu5sMUbVYRRpsRj9UcuCEY52G2HbHw3LD5K5PF/IPnacA+IGDFGOyP0ua1KXTD5/NNrUpbPb0YNWebyE0TLAT11kOnqe4LE2jCGmecQCNhQuKU+cQrY9fRT4t2wTHdQs5mZtPTv6+i//ve35SDBdm+H8+hZTuPUbwTbRoSK0xy2S/l18W7vN+zbI7CFe80eWkStXhlMp222X+Ox1E281QL4WwCqjXG2qGFOJhxniKBf1bupS2HgufMkmJ0bhULAomyGbNZopNWPp4ImH85CQSsKNFIGWXx9qN07EwmLd7u7iTv6GnjK/w8cN33/TJbyxOPbFElj4wlPwq3fdQWbPXXCrIgu3L3Cbr+s/m2BfZYusPYe+32OBZvA+nmgxnCL+XpP1Z5t83ZfMjVMsULO46cMbRfqN3f57O2CjPPZxTPloU6Fu5u/HxBxJqh28GRU5GnMXSCnBxJCPQc5ZgFayNBZD5XBCFxokWwvycHP9l11Nj74DZS7ExHdIGAFeGwZqf2c+Ppz2WxH7EnHJPGcN5VpwUPI/dpRYmUV3lr4gbq/NZ0YTYI7BUQ7Mof0n74NBFWe82eE0H3lUvAK6693p9Fr45dR04SbVPMcCfFJ876r2wjEIsvHHXuio9miyh40bTY8sGUXDPX3xVR89gMkCNPLtruv9CSI0lC4I4mrF4QMTqkBtMI7T1+VpiCus2F7BzxYQ6cDO67NPDbJTRcEYQklBZ65NR5OnzKuOaS/T353Xp93Pqw5jtOLY5JMann9AUCVoQja3YG/eofsUcNJ7P7YcHO0ASWCGjrVr/XbNbx8bRNPkn+wrkGd0zDx68XPjjRxifTtwjn41Fzt1Gkwytx3y3YQbGO+vVkE0OOwDXlYghtPZabMDH9a/ke2rA/g76c4+xzjzYNVriD/TKDmsV4rmeOOrdmz0ka5lDuOjvbRqCxlc3ip6w/aHOpoh9e9GGN0Lwt2j5rLMh0eGOa0BxZhRXTHCfWY9nXreWrU6jVq1PofFa26wt8B0+eo76fzBXJycPBQ/EFBKwYgk1U2O8pWk3C1GMWdzLhBBNgs463J22kdyfnJonVw2gtsdklJ9xkH5xohVdXncTs1TiyEa/EvTBmjemBJdpQP4o7vlokhOB7vg0SqtfAM/RczCdnVGOpd8ZDGeejrh9xAw71brXww/1fOD4fHMHx8V9WRNzzOx0DfiyBHm3GuSy/hT6eoEaDGRznYzriUFHlRZ83J2iPz8t3xk6od3V74eBfg35d4WduKO93+nze2Gd2DhSs35F0gowFgrVxHLSHk5NbxWKX3VicAAJWDDF7k77N/3aVP040rKxe8+k8avrypLBtitkvxgpOnHXOVEE9J+JVKV7pi9aEqEZRahsdiljrA09GX/pnLf24ME8T7GQADSNsOnhKRIfkJLl6sNaq27szxQJDuHBum0gnlO7m7xXBI5y6aXbY+4PZYoU/1EUmjuDIGkwzGk8niDB5z3Z4oa/N61NpxIwtNPiPVREn8MpwW+F8TC8vT3L0uvGm1ZC1nH8u20N/GIiyrAfPCUJpS8pDeJyr9dx4avXq5IAROdWLBlbw/pTITuFhBRCwoohg0ecSVFKN0ozhjlGLHJlQZZzLpC2HTpEVrL7oazJudfAIg4Y7mjB68wSLRgJPiGZzt3yxkG77cqHx62hcyGkHbLNX0ysfTzKVwpddLNt5nL6et93SyFN6Nvvq52N0qPx2/g4RHfLeb5fq7jMrwGKLFvz6TE8/qBmRbdE28yuNTrezQHXHCxNfz93m10c8+nPgCKdhE2YVsCDNzN96JMAlgl/EiFO+HuwDwpoNK7FavnBDXgllAfHNCRvo58W7RB8TiSwI0M7MmtKaMbnVr8vIFESt5JhOMLBg9cfjSuf/Tae7NRITm+l7OVcfc/jUBRF12ElTZCn2Hy8ErGgiWPK+hAASwM4gWiBJlSwwVDoMn0Zd35lJ6/ae1I1yyJFuzLzEkfIeOuvv4HvXst+XmRDv0bIyyBO40XO36QZvYK0dazKbDJtoe1m0NIThTDC7vDWd2r4+VVODHO4As26f9jsWKneOXkz3/7As5nyDeGHipX/X0bwtRxwd7COlCkK9NxZM2Qfk/h+WutYHs5mw1up5pGqBjMCTY2DuPeGFWx4fIuG5WxmcQQqxgnjRhQNaTNvg7+vHwTgG/bKC/l25V+ea2leVLSLs7Lc8ioEhEp6l3UDAiiKCreCr5Ss3GnDGxQmq3qora2D0okg5seotX4M7bBnDyq9ImjWGiNO3YKRq/1i2m4b9u46u+GiO6tjco9P350bkynQpKlvDoaELdpxsl5mRHtzp3dJHE2JVzdpoTWjxSHxTAplURjbWt3sjY8MXs7aK/yeuDRx0xU44GWqL16aRDRZKYREDQ0FIOKuZzrtWr/dni/FhRvohb8oKK5JMbz10ypDpfzQ9bxa6/ly+hx7+aTlFGh7F37EvXkHAiikSrbJhs5HzAcxVvpyTO6A7AavXlZMNduYPhtoEMxDvTEoXTqzBJjJs/sgh+D+dsdlx4W7TgQz6fsEOy6MOBbPXnrR2v0+kvPX78kIaK28zr+q0knlKEecbFQit0irvlScLsTTgOL0YYeRqjpvHhni5F/9eQzeNNJZLzVSgWEmKCB9OMxoAefK7xwLZONQFx2hcaH9j/AYR9S0atGX6fUVexcuLsv+u2ivMxTllBechCwcOpX/5OzOF/1EstgGz9xLoHq3oOedtOUwfTM1NeRAvRI2ANXz4cGrdujUVLVqUypYtS1dffTWlpweODhdrBJvoqTsqOyc5HJ1MGRnp6d9X0vWfzQvrnHM3Hwk40LKD8EIDduJG4GTMMifPZVHr16YIp3AZjpjDyQKVmKnNj6ZtFk6sa3VMJWUe+GGZCMH/vwnptGG/tSZfwej+3ix6fswa+nmxtQEdRs7epltnrDm897ulIlKerJENpZl+NmNLVA1+wcrIkwVL78MTfflK/lm5l274fH5ERlzjwCdP/hY8VUYggY7bOwfWCBa0gn3stBJZhwtHP200dKKfmfHMjYfoo6mb/AQQu8aPUNq5mUP4PoL5nfGiEjv48yJTKER6MuHPZm4RYxi/U5E1iZd8rEcYMzXJYyqbi1vB7E2HXbWM0EJ+5yJtTAulKzh1Pou6vztTpLaRTbU5kIpMpN1jXAtYM2fOpAcffJAWLFhAkydPpszMTOrRowedPu1sdLxIjhToZJff/JXJIjKSvCL665LdumE32YfFSChQPfhF5BU5dhC+ceQCum7EPFq207jjvcdgR8FhjWV49Y+TBSoTRobSyZgJNX5cMfGyovMxOkHigTic5xNIeA1k5qo1CTKaWPE3RdLPWCczO4cWbbN+wh1pPPLTcnGfHKY/0ELLJ9M3+0UWVb8uvOLN5o5GkjJrweHROZQyC0P8bnDgE04063ddgy8qJ3uu/+IEEVjjzq/1Aw7ZIVzKArL8zlz9yVyf3/uPWkTvTN7omCmg3bmIuI7rPD8+YEJY9ml99q/VYpFJz1/YLpwcp8OxTuDASv0+nesnEIXDkD9XU+OXJvn0Z/rDVOQIsY4l39Vo2M5r3q253q+Ld4lAPZzaRgskGo4gJkyYQAMGDKCGDRtS06ZN6euvv6adO3fS0qWhB2SIRvQcF5lw3otQzSeM+DVcM2KeMMkLJ5qVMkjAkh3HqN+nvtoyo8VnjVKwYCF6wUHMmAgGKpduMA/FvnZ1PZ6LQowyPxILxzwh4czxZgfvvRr+dD5mfkZM5RR/syYv0L7xyMv/rhOaHbcwM8Czqa1WJEIzBPKJ4Ekx5/vjBZBAcCAdjpzq49cX4DbUbY01uxxK+Z5vF/v8pjRN5Yln57em07B/tRPnKt8DDpsvEyiK3HtT/E1ojPZtLAzO23w45KAsLyvug4VKOYqrk/CCnXJRy4uJW5K1NhxcRs/flyf6Mmkfzg64cBnhc/+AGA+u639DvNDBbXXUnO0hTYrZT0o9r+BFUkZpLcLXVi5ECnNpcZw7vofBz2H+GO4L+Nqc59EKzDY/fg/kKJEhlT+EBp/toBl/pOJswgMLOXEit/MvWbKk7j7nz58XH5mTJ3NXqlj7xR83ka9vthz8kuodox5Y1Z3hyp1HqEGFFFPlC0ZWZlbQfVlDwqzdbVzrpDxndnY25Ug5AfdRdpzq8vj8li1pRt7RO/b0uby2kpOdrbnfou1HadmO43Rvp+p+kRyzsvzrJ1tHq5WdnTcxzcnJ8T1OcQ9rdx8VX+uWL6p7H7mH+Hdwp89nishgarjp/LRwB/1f5+pklIHfLaMZGw/TyNua02V1y/iUXVkG5X1cUPydlcV167v/sTN5gp94TxMkylLUl1Zbu5B5gZI8/l3ZkdMXaNbGw9SrYTkqmC/Rx0esaAHtri87y7hwEKjdq3/Lys723xZAu6nc97sFOwLup9WXSBqDm9F32q+95viXXQuu73ZvzBC+oGmNypm+rpcAfRwLEPIkTP1+BLsO9yNGnxlrrxjWyu86kqFqs7nv+E+LdorE0KPnbtc8T6ApifJan83cSkt2HKcRtzaj85n+7U+r7Wi9430+mk27j52l1/o2oBtaVc677yz/47WmPntPnPOOEcPGrvfxS5UU72i4Y6df36bgsrdm0KFTF+jXe9tQ89TiPr8pj1GOdVp9rMyJU+cMXfe7+dupXTXf62khH6/sJ8zWR1Z28DEzEMr+0kybD3Qct7Fsnd9Pn897114ft8HnOL3rs681+0nd26kaPdWjjv89KMZzHj/rPj+Bvr+rFe0/eZ6e/H01DbykGjWtXCzgveld/+0J62n6xkP0w12tqXD+JN364L+D3YfMG+PTDbU3Pfhaj/+8nMas3Ecjbmnms13rnjKz1OOkry7EaNnl3+TIzb8ObEN1yxfR3DdHow/1th3F8zJ679k5gcvI77DT83C9ebdd5YhKAYs7iscee4w6duxIjRo1Cui3NWzYML/tkyZNokKFClEkwOaOZh7Rrl27aNw47QnX8eOJPsP6WbGSnPf9nlHz6fnm2pO6zCzfY8eNG2eoXLNnz6Ythf3LqcWcuXMMN7nc6+fum56+gY4cT/CbsijLeDIjr/x523OPP3qUzRGMrcCoj31n7ArK2ZFbZ5tO8DlyJ+r//TfOu0L96PzcfQ9uT6dWZeSBP3fbvPnz6YBqgXv54bzzKFm4YIH3uIMHD/rcX8apvPvr83GuNuOtNlmkkBt82H+GaMMe32fKrEjfpqu4Hrsonb6bs5FurJFNNQ3I4TM25pb17X+X0tkteR3wnt0J3mucOXPG5z6OizlbkrftF0wi2rY9b//cRRCP9z0tkEi0LSPvGPXzYSZOnET5NerhjRWJtO+sh/6eu4puqJFbvql7PPTPzkS6uWY2tSvrP81cf1z72Wjh/47klUldzvXr19G4474NYeUR/Wv5nlv/nVHul9uX5O6bazqt/74EOrfy3WO2b9tO48YFD0Cz7lju/fDK5b+r9pvoS3zLcuiQb9tXcuGC+j2/+I4fOxa0zlatWkWF9q/U3IcFCJ/jpbzr9Pt4tvfvGTNmUtmCubukp2/UfH7yebKy/d8/9T7MOxf7j+HfT6Tdoo/xfT+XLVtGOTu0V4MPH8p7d1i4Yr6bsYaKHFzlvb+FCxfS0Q2Sz/2ePXtWs2x9P11ADzTIpu/X+d7Xvn37vNcx/jzV5F7/yNFj9N2f46hUAf89Dp3K3Wfk2AV0ZdUcnzIrx8pc+Sr3tzlz5tAOnzlj3jGzZs/yft++fQeNG7dNs33s37/f777Y4OJkJk9m/ceW9Rfbet4241OphQsX0THxPMyQd/59e/d6n8U7P46n+sUl3f1Xr1lNKYe4LRDt2JHXVtT90+rVq2j3yby2p+xLtmzZSuPGyUGYfO/ziS/GU9dKyuv7/j5y9nZqmLXZ7/fDhw/7tfOnflpMJ8Q810NfzNlOd9XJDtoXa40HH8/I7auGfT+ZLq3gOx5zH1DwYh+wbp/6Geoz+uI7ysyYMYPKXOwD9PGth5UrV9KYzbnXev3v5d72tGXLZqqVtxZCFy5cEGXJTY+Ve44pU6ZSSj7fs/u3P/9rav32w6T5lCrma/71euxiH5qlmAvKx+/fr9V2ArN+b+B35MSJk2H0JdbOu3meYgdRKWCxL9aaNWtExxqIIUOG0KBBg7zfefKWmpoqfLdSUoxpcuyCJWZ+yN27d6fk5GTd/R6d7+vQWSU1ldLSGmruO2rXQtpxKs+sI0c1gCblL0hpaZ01jx2ydCpdUPjHpKWlBSy/XK5OnToJTYq6nFqwQPzOamOJcvn68jnr1q1HB9gh9aSvBkxZxo+3zKX9Z097t4+ex0Joep6WU3VsoOsq72//+URKS+sp/l647Sh9vC43sV/v3r292ip53+KptSnt8lo+29q3b0+tqpbwvcjq/fT1ptxBT0nbdu3oo4vn50AuaWktvL9p1W+HS7tS2aL5/bafz8ymRi9P1by/ShUr0dLD2ombN5zI7UA/XJtEm17poblarrTPlstUunRpSktr5d0+b8waogO5Zjq8kJGW1sn72/6T52joMp70EPXo0Z2KFkimVRPSafq+3EWDlKJFae+Z3ASr3bv3EJomNlN5f80izefD9OzZgwrl8+/K5H3WZeSntLTLcre9kLvtpy2J9PIA/3ssuukwfbbeWC4o9TuiLJO6nPXrN6C0DlV99k9ce4BGbVwZ9NyB3i1uhw/8uJzOHTtII/+vK9H83OiYhQsXJjrnO2hovdNa51a+e0y16tUoLa0eBaPQxkP0+Qb/0MDB+hJ1Wcqo2r6SYaum0+mLq7vKcpYsUYLS0toEvK+mTZpQWotKmvt4EhK87zkzaOFkeRZPxy/ktflLL+1ClVLyib67dp06RDu36N7vc8um0XmFVlprH2U56jdqTKd3HKdFh3zNwFu0aCG0sFr8dmgp0QnfwD+lSpehtLSW3vO2aduG2tco5XO/BQoUILqgbRL8qUq4YipUqEDLjxzwvb8xa2nV7hP0+33tKH9ScG8D+frbMjz08vIk+vimptRTdV/yPjVr1qS0HrW93yXyUPfu3bxjJQvxjy/InSRdcskl1LBi3niuvE8en95YmbsgVaVqVUpLq++3D1OuXHlKS8vTLDDXfr6AVu329c+S7130ExuWab4vwWh78XmYQXn+ChUrEh3JXcCYe7IEPXFLO939eQE6rXWq+Hvhv+to7oHdmv3TdqkMLTqU5xfF8xK5L6lZswalXdRCqe+TF6vu6N3Wq23S60/U5SpdqjRtPOHrV8oBzM5mnKczF99vbvd6/aP63FrXrVWnHqV1qu7zexNFH3Bo/g76c3u6qfkOc+mll1LVUoEX6NXlYbeW7zevEX8XL16Mdp3ObVdlKvKYkKcBz5cvnxirRAj6JTPEtm7dulLpIr7jfBFV+9OrA/VvDerXp+ET8nzNlZS42Idyv0UX+y35+HEnVtCqowdN9ef75m6nv3ds1H1HiqakUFpae4qEebds3UbxLmA99NBDNHbsWJo1axZVrqwQ/TXInz+/+Kjhig0k1DjFSX6Hdp2kTnXKGXYsTEhI0C97kHOwD5HR+9baT55gK81SkpKTDJ8zKcl4c1OeMyExQbN+lPsof+ftrytU+macNrXuRd6WT/FbUnKyMIVShsH1ePyfDd+z1jYtEhOTfMqccUESAQ7KpWgs9Ypyadf9yfP6vm6JicbcLtXnnb/lCD3ww1J69erG1KdJBZ/fElT3zW1UeR/K35KS8iaciYnJ9NLYDTRx7X7NNpyYlETpB8/w7Fe3XMz5HA8VU22fsGa/bhmU59p2+LSIbNQstRgdzrhA5Ypp17UWgdq9+rfExESNdqC/Omv0ndp8+BxN2ZC7Gqz3Ppg9p3q/gH2OgiRF+w3lujKJAa6n9AVQ7rN053HKpgQqkKxfp1rPQIb7NEP3mJQ3dijbuZJlu05SclJCwJDoWtdKSEgUfYiZcmuVgRd+fN65RP9+wqwjOwug6rL/ujTXjHLOlqPUq5Fvn2CEbxbspCuaaY/h3Oery6wctxMUJoLcb+rVDz+vvHvQH//UdcaohSu5DLnnTQq5fWs9DzME6l/9981rO8rj/Pp3VdRK3/5cv/0xx89mm+oLRbk1UsqINukx1j8GOneg90a5je/LyHn8zqsxpgc9JlF5L3k3+c3C3dRCIWPIzzM5WQo4X1Wez+w4pId8beWTMdJ29Eg0UL9uzcPVdWpXOaJGwOIB8OGHH6a//vpLqGirVzfuKxKpvLwskTKXLqWRt7ekHg3LGzom0LioDARhNRyRiaNP3dg6lR7tWpucxi13SaUbk7Lu2V45kTz01Zxtho83dd2LobuZNcPyVtbDxWMi0ECxgsk+kcY4Q/yDPy6jPk36+Ow7Z/NhER1Q9nMyOnebsHYf/bBQP0T8+1M2Cv+WGmWETUPAZKQf3NSM+jar5O0r7vveWPCbtA9m+6Q/ePCymmQHdiX9dsKR2GjRrYoKZbSNsnCshN/FBy+r5WgZtOBIp1bnCIxk5/J4CLdsKRYGyIiSWBtRBy9sJhtYjORxkoOq9GlcgUoWVtnx6eBG8BiZdyZpa698QKOKvyiCbBb4/fff048//ihUyWwzzZ9cW/LoJFPyePOQGEe/9QfLSB5OlEGO+rPvxDl6fwrnSyFHMRJQyK1BXhlFb5xCa2LlxForUp+8mm/6/AbbwCFVtKMshQO0FkZyBLFTK+cHkzmaa2jug/J25OABWw8FXzh46jd/k8tQcsttP2yNLbY61xDDIcOtiiIls+No5Kep0Ms19PbEdOr9weyQk99e9nauCY2MOoS6FsfPXBBJfH9bkhvNTEujs/ngKUOJrMPpT19gM1oVQ/9Z66vNDZF5W47Qu5MszhHpQv8aaPHK6gULDlHf9Z0Zon0YweE82q4WZO3ewAKB0SexdMdRVwSlP5cZS+nB8ycex24euYDqvTCBVu8+IcYsZfoWNU/8ukK8y3d/YzwycSTnUjXSzDjiI6ew+Xb+dsPHxCtRI2CNGDFCRA5k+1e2B5c/v/zyC8UqkywYbJWwE7QyPLcZIjHi5kM/LqP1+4LbzrLvlNXI47uyE+GJmXowkixMiKhm5Kwt1Pq1qbTjyGnD5j+h5tRQ34c6b9Z/q/cFvcbY1ftEck9lLho7CGvuZdGgwNpe5SSQw3RzyHDWtlmJXiJrS3MWe8Krd841tHznMVEf/I7I9fLx9M3i/f150U7D1zNSFr2JOb8XvEDESXyf+t1XIJfLxKGlu707U//6PsdQyOhFhjQrbOpVx4eKhQwpCrVOI2ZsoVfGrvPbzhPejQcyDLdvpVY12D1vOXSavpgdPJiL3zUivTJDQDlm6o0/MjzZliMF68GpQa4d4XyqiS9nb6NBv2ov/kkaCz6c627+1iNCC8zpF3jBQxlSXs2U9bl+ScsDpF4IhsL1PWJRjumc7HnyugP04t9r/eYE4t28+D5IigrmiKtqYvC1iV4Bix+a1odzY0U78qSBzfB4gJc7pHu/sz7H18iLA8j0DQfp8ndmiIkP53PRa+xbDp3y8TNi3HgvtMyPxq7aJ1bA1WWycsCTdCYzsrZQLUwY0YDoTRK5Y9cy89GaaDBfzN4m2snr43wTs3osmCx3e3eWbpLWp39fKZJFmmWnShBUm3iFA5sv8sSaE8SqYYfhcHKwhYqyGS7e7i/kr9h1wpKJqN1Y8TpxaoT3Jm8Uwou6vXLqBKPvrJHd9N4X5mSQpKnBFmxc6fukyF8lFib8Py2nwX+sCrn/fTXAc5PP+Mp/66jHe7Oo9nPjzZfRwD7cFo2g7PednigGe+R69W+mnLeNyg22ZJSHfgocGGj/CV/NvVN1Fii/mTqtzS5VTk/WfmnlUdOrX/Wio89vAVaoj2rEmlFe4+5vzD0Lu1EmnlbmAGTtH7+bo1RpK9R552SQaBg4Br+cnBiRc0ewQKNnqhDugCpP3O/8erEwvbrm03lisqxWHfPE/bm/VlPXd2b6JfV96vfg5mBahNOpmjnWic679WtTdAWQUFGulAVbNXTifv9PIeArr8GJibVMDazQuLJvW6jwxPqN8Rs0u+0rlQlnDU62w0Vv0sEJcLceOkWfzbRWOBo5O7A/oN0E077IWhVeGFAPtMP+1Z9cWwWbBfLqKwgPTiQ9b8thP+uIf1fuFYlkWWMZaLKpJaTwAsiXQfxZGXXOMTMY6VpCEQ6dniYGu95Ohaks5/zjCTFrEERkOrvKFKRQRuctx8+Y65O/DKJxDFSubxf4tiV1ES8EEbZZG68kUALxp1UaczNoaQfNBqkxwtIdx8SYpHlmj/77KOfLG7MiN/rpp9OVYfn1gQYLOMY5xQo7J+zUa3zKdv7J9M00fLzvarBVXPHhHG8AgnWqVV0nJimsWVNi5l0M571lwfLxX1Zo/qZehGKTLy2U/g9OdCKT1uWGTzaCx0Z7bVnjqtf32zEoqJm18ZDmJCldww+oiYYWzsoSjlu9X7eMd31tvc3+W5M22aIdNJbAOose/XlFiOcj+nqecrITwMQ1jAdkxFQ4Egf9QPcs575yktu+Wki3fOGbbkOpceeJp+yDyD6H93yzmKan6yd3D3c1mxdl2Exaq13+fXHSJ1/FquevfCZumgh6goxTb07YILQKrEEYr+EjHEnsvWi9Y4ZX/wt9/sNJwtVRltUaLD3YFC6QKXGoeM3rDO5nNbxAaVVQHU+EaNfdJGqiCMYyvJJnti2yevutibkT+dvbVaXKJaxNnMz5itx8WVizFirhdD7ssKo1EHEncsPn/jbkWvWi9H+INMw8x1Dr0WOz3506J4iSc5nhmQI6IQQy24/Yk9jQCGw/XyA5gTrVLmP62PT9GXTzFwvokctr0YCO1X3C+Jsh0Gq6GwOzdHFCFcwcx405g95ryFFj1avoZo4PFS1/E71LsGaS/VRkXxX/skl08mwWFcgXeK030C28+PcaWrn7BL14RQO665LqtpvQvjVxA63ek7foaLZ6d/PEPkCwUtb+fTN/B3WoWYrqVwg/XycHp7IbFvSNBgjRQy3QhNtuTcd/8hizpvh2/g7VgpD28SAwEbiWZTnQYEUA6hcz0MRW3leycFIZCyjrTLJy4iv5m12YKpdGaezuh63q6CULIx5adU32u9Ma3JS27k533EdOnQ86kYsUzQhraNkp/favFvn5IOixXeE7N+TPVUI4eilMk77uitXfSAkSMDPdTDTX8DF636xJVppfyivGnB7BqMZpnSoYit1vCWuzh49b7+d7oy4DC2BNX54kfIJDhYUr5m0LIicGq5UaQ/6jT6ZvEVpo7zEmq/LpP1bpLnz8sninsBxhc2fZv9gqHyw3oz4aWUJO0skrF4yhf/tH5NRr51LA+vEto16JA40/4XLsTCYt2X40aG05tRDI/rIs8HtsCYokUawDAStC8DU50N/vnxV7hb26bxuO/YZq5mU8eNEmOBSMRCVUr2YFJMIejRldqdEJuJKOb04LS8ILt9N1YxWR75lNcQIRKaubylVmozV95ny25jFmfBDVGis2g5ZRN7NQqyrcOjYSmt2NFXZ2Hm80dKLwzeXIofw3130+g0nD5XPYifq95ef9+azgEfnkyeqbEywOKx+yD1bg37WaCAf2sAJe+Hjmj9U0dpXSrNGeIWWeQeHczvfHqveZtX2BNO5qXyj2gdUaZ/wWuskc/63a5/VHCofrPptPa3SiwzqdF2/krK0isqISddWF2u9KFPtAwIrQia9e4zt5Los+mrZJFTXP3AvjRkS1cDHzMi4waa5kJxyK3OnEn4Ews0gotyszztFabXHM8j0iZ4aRjjjclTlPmBPftSEELokmDXIodaNcBfbo+CAGO+3xIDn6lKibwKrdx0XAndx26LExiI6z76nZq7EW/fVxG4RG6/kxaygp0cRiSZjSYDCNl96vAa0xFM8ykL9LwAv4XCu833MvY76e/lxurU/y4u2+/sfB3hUWvK8bMU+EGc/dFrxd3PLlQppjIpCSEX5bYizflFGsUHCwRkgNB5DS6rPDXQPjxQIOfsUBjKz2QSdVBONxq/LSopwIEhjErOmmuv1MDaJd5vanjN57+NQFY/n8JIp54IMVAYhJoYnWxqYEj3WrE/L16jxvPsStFQvw4bxPbmmTwzWl4cg67WuWohtbVxHfufPVMxFxBuNPUr7zl/7xzXdhlscuBg25rV0VRzRY4TyzrTYGh3Cbv5bvpkYVi0Vcqwv2yK/6eK5pIS0U9PyElMxIP0g3t6pkicaMhZ5EE+8jm3cq35MkExosNQdOhr/SLvPSv2vph3vahXUOK4J1sOAZKOgT9wscqTUxwR4h3S3YbHrJjmPi82TPuoaPm7/VnIDFpmtG22dwNwhyFXV5OMqiVpCNUMrJEZrt5N5vl/gE7GET2yaVi1m2aCmnoFHON5Woq6TLW74J3+UIxIN71wt4nSh81UwDDVYEwCpupXrd7c4n0li24xht0ogCJ8M2wpwkUsbN6vtGI7v5H4qoixwggEPnWs0DPywVkz9eCQ5kBmg2yAWf75+VgU1W/K6hs93Ifdv57NSmDk4SCRaCj/+yUiQ8thIejD+etilkH0VGLRDrtd9AfYA4zkNBo9WF2z7Y9+yXi/s57YT/miJiGg8XkdCmgglrVr3PRnXEn8/UN0s8cuoCvTt5ozc4lBazAuRNshKOuhkqHA5fiVr7pzaLs4LX/lsnTNesQpnzUY9wFzDMLLRxlMU1isAlMmNWmNdO7rAggFGgMVwrGuqqi36IkUR2EMudePDBggbLJdSNi32rfH83cS6KbYKpqO2cOJuN77hIo/NTbrNy5VgdEpw/qSUL0tiHOllyTm5XoST01Htehtp0mI2ZQ+3qXcduP5RogJPBmkUZIEG9GsoO5nL+k0AEWkUNZNOvTFDJ+wVaJWfuHG3v6jEzed1Bus58AEY/OPn0BBOhs5X+Haz9ihS/vlwkwxEHQ+G7TQn0hAXJyIPBuSGdoOHQibThlV5UIDkxrPN4NIS1+77Py19oFercdWY4eyE7pL7X7fm3U21BC713OxShxK1uQnL7AUYA0GC5xLQNh3TVskZWXqKx8a7ebc1gq2amIqKT1fCzWLbzmK1JGq1EndsjHELtmPccDz0scLitOqLmnAoi8W3V60PUK+KnL2SHlVtKa9XdpxwBBDs23ZHZZCAkeTQ9y1u/XEjfLQgSJEeBckE4V4MVqa3dek5nxd691nthghA+wiWcXFBOMGruNlprYdAGo+w47F4qDLsIlkYikpCC+URS7AMNlkusD6DGD9YwT6psZJ2QtQJFxzEaOeeFv8Pz5dHDzgSKmdkS9QsjJ5cbBBLQzYVbtXZSY6SZhpsg16nwtcye42epUvGCFGs88av+arMttavTgXGS2kgk3IARVgjEoUT4DIesbMnRAAda2GFarYcVgo8R2Ky7d+MKFMuEGlnPTLAPLX5ZsiukkPJO893m4FpMDhqRWqIQTQshnYHVQ+I8i4KISXEgYUHAcolgE8FAk+RM1eDqROafSSpHR5lfFu8SUWRAdKDOVh9rWlInufWLBTTjqcso1jDrc2c13lx/kdr8JPcFO6dNBPX6fxkjIdn9MFn+Fq9No3Aw054e+8W8KW0oyL7XK3Ydj/gFpVAJNW8Ua77sIpAfntPsPh38GXLQiFCxWtOtzAEXDlIc6LAgYLmEJ4xEuRzU4Z5vfVXFT//ujn/J6Ln2Jd0LFa6feCbQRMLMeGzF4O3GJNmpa3JSaqNtLRKnQREzvOm0sy9mm5u0O5UOwa16k1QT80hsU9HZAP2ZuDawQGm10Hz1J7mRMs0S9W0AxOxzzbEgbUK0Ax8slwg0dw2mOeDcDjPSfVcRfnXARCNaGKxwjI9H0gNEWztw8pxrnbwTGrFww7SbpcUrk205r9ncJW71W/tOWG/GJ6+4mjVFcaoPdEuzqzQLZFNaN6NixjLKoC5um17aSaT58LllehvzRNZjjisgYMVAm0e/BJTcNHKB6yuzMlZktjeD0xYznPjbDvqNsN/vb3OEBI1QP7L/Vu+jHUciNx+ZW3nD1f18sOiqEU+ETvzSPpzt2LWyMXh7OeOQ31u84ZRm3yxSZBbLUiBguYTa/CqcxhYPDRVEJ5z802l/gUh8H8zeuxMhgnt/4NxEMhBaVaOVvDJSWLT9mCtCVjgr/FbnP7MCrZQWdrLKpii2bk5+o8AFC4RAQoBk2Ga5aaR1+cvMIEWyDbBDQMCKQLhZ5phwI0JDBnaQEUYyTLOhumPV7CVWAoaMuhh1K1Lr12lWHXW+HgKFywfuaZvDgfO8BQsS9bMiVYEdSW1BbLPxgDuWCsfP+Ea7jrUx0QgQsCIEZVPjdmfWwRsAQCJfWex3287z8th1bhchoriQkxumH4Bw+XZ+4EBRb0/Sj3h3MAwTbCzMAjejR0oU+0DAcom3J2/y+a4U5h/6cRn9tXyP84UCtjHkz1VuFyEuOHU+i7JddBwH8QHrr35ajCATIHxOqPJaqjGz0B8pfpUgNv2mrESK/VuEgBUpcCK5UO3l46GhRjs/LdpFJ4KozIE13P3NYoo0OKR7tJPO/ZINlnEwOwTxTLD2f+S08YiifUwE6MC8IXJ5d/JGinWkONBhIQ9WhLBun3OhYYE7NH15kttFiAuW7DjmdhFikp7vz3K7CBHDjlMQCoH9mPVTOZ9l3Hk79qe34KuLvrORiBQHDRACFgAARCCcxPh4EPOhWGD2Jt+cftHAmciLlwCilECh2uNhEgrs4xX4zroKBKwYYOvhyM0ZAwAIjes+m08rdkVeaGmrgcYRxDMjZmzR/c1O+QqJfYGbSBT7wAcrBnjkp+VuFwEAYDHxIFxFK0sPY+gE9nPFR3NsO/fJONCOg8hFigMJC6MEAAAAAECEsd5G32wOvASAe0gU60DAAgAAAAAAADiCFPvyFQQsAAAAAAAAALAKCFgAAAAAAAAAR5Ao9oGABQAAAAAAAIjIHG/RCAQsAAAAAAAAgCNIFPtAwAIAAAAAAAA4ghQHEhYELAAAAAAAAIAjSHEgYUHAAgAAAAAAADiCRLEPBCwAAAAAAAAAsAgIWAAAAAAAAABnkCjmgYAFAAAAAAAAcASJYh8IWAAAAAAAAABHkBDkAgAAAAAAAACsQaLYBwIWAAAAAAAAAFgEBCwAAAAAAACAI0hxoMKCgOUCGecy3S4CAAAAAAAAjiPFgZEgBCwX2HfinNtFAAAAAAAAwHGk2JevIGC5QYLH7RIAAAAAAADgPBLFPhCwXAESFgAAAAAAiEMkinmiTsD65JNPqFq1alSgQAFq27YtLVq0iKINaLAAAAAAAEA8ciE7J+ZzYYUkYGVlZdGUKVPo888/p4yMDLFt7969dOrUKbKTX375hQYNGkRDhw6lZcuWUdOmTalnz5508OBBiiY8HkhYAAAAAAAgPvl1yS6KZUwLWDt27KDGjRtT37596cEHH6RDhw6J7W+++SY9+eSTZCfvvvsuDRw4kO68805q0KABffbZZ1SoUCEaNWoURRPQYAEAAAAAgHjlj6V7KJZJMnvAo48+Sq1ataKVK1dSqVKlvNuvueYaIfzYxYULF2jp0qU0ZMgQ77aEhATq1q0bzZ8/X/OY8+fPi4/MyZMnxf+ZmZni4xasAQQAAAAAACAeycrJcXQuLl9LfU27ymBawJo9ezbNmzeP8uXL57Od/aL27LFPGj18+DBlZ2dTuXLlfLbz9w0bNmgeM3z4cBo2bJjf9kmTJgnNl1scORdS1QMAAAAAABD1HDl6jMaNG+f4dSdPnuzz/cyZM7Zcx/QsPycnRwg6anbv3k1FixalSIK1XeyzpdRgpaamUo8ePSglJcW1cu05fpZeXj7btesDAAAAAADgFinFilFaWjvHrseaKhauunfvTsnJyX7Wba4LWCycvP/++zRy5EhvwAYObsGBJ9LS0sguSpcuTYmJiXTgwAGf7fy9fPnymsfkz59ffNRwxSor12mSk2EiCAAAAAAA4pMcKXc+7jRqGcCuMpgOcvHOO+/Q3LlzRZCJc+fO0S233OI1D+RAF3bBJoktW7akqVOn+mjT+Hv79u0pmkCQCwAAAAAAEK9ks4QVw5jWYFWuXFkEuPj5559p1apVQnt1991306233koFCxYkO2Fzv/79+4sgG23atBGatNOnT4uogtGEB4mGAQAAAABAnJI/KepS8ZoipEgLSUlJdNttt5HT3HjjjSIs/Isvvkj79++nZs2a0YQJE/wCX0Q6SIMFAAAAAADilbsuqU6xjGkB69tvvw34+x133EF28tBDD4lPNAMBCwAAAAAAgNgkpDxY6qgcHOKQfaQ49LndAlYsABNBAAAAAAAQr3hiXNtg2gDy2LFjPh/2wUpPT6dLLrmEfvrpJ3tKGWMgyAUAAAAAAACxiSUeZrVr16Y33njDT7sF4lNqBwAAAAAAQI9YnwlbFsKDA1/s3bvXqtPFNNBgAQAAAACAeMUT43Nh0z5Y//zzj893SZJo37599PHHH1PHjh2tLFvMAh8sAAAAAAAQr2zcn0HUhGIW0wLW1Vdf7WfuVqZMGbr88stFEmJgAMhXAAAAAAAgTjmfnUOxjGkBKycntivECWAiCAAAAAAA4hVPjGsbYjuNcoSCIBcAAAAAACBe8cT4VNiQBmvQoEGGT/juu++GU564ABosAAAAAAAQr3gotjEkYC1fvtzQyaCZMUasq0UBAAAAAACIVwwJWNOnT7e/JHEE5FAAAAAAAABiE/hguQAELAAAAAAAEK94YnwubDqKILNkyRL69ddfaefOnXThwgWf3/7880+ryhazwEQQAAAAAACA2MS0Buvnn3+mDh060Pr16+mvv/6izMxMWrt2LU2bNo2KFStmTyljDAS5AAAAAAAAIDYxLWC9/vrr9N5779G///5L+fLlow8++IA2bNhAN9xwA1WpUsWeUsYYCAYCAAAAAABAbGJawNqyZQv16dNH/M0C1unTp4XA8Pjjj9PIkSPtKGPMAQ0WMErtskXcLgJQ8cjltfy25UuCO6vTlC6Sz+0iAAAACBFPjLvLmJ4VlChRgjIyMsTflSpVojVr1oi/jx8/TmfOnLG+hDEINFgARC9VShX223Znh2qulAUAAAAAUSxgyYJU586dafLkyeLv66+/nh599FEaOHAg3XzzzdS1a1f7SgpAHAJZPEo00HhOAAAAADAbRbBJkybUunVruvrqq4VgxTz33HOUnJxM8+bNo2uvvZaef/55o6cDABigQHKi20UAKiD0AgAAAOHhifGx1LAGa+bMmdSwYUMaPnw41a9fn/r3709z586lwYMH0z///EPvvPOOMB8EAFjHpXXLul0EoCJBY1SIdVtyEJwGFVLcLgIAAIBoE7A6depEo0aNon379tFHH31E27dvpy5dulCdOnXozTffpP3799tb0hijS53SbhcBuEz5lAJB90FAFAAAAADEGuezciiWMR3konDhwnTnnXcKjdbGjRuFueAnn3wiQrRfddVV9pQyBvnituZuFwFYSLmU/KaP+evBDkH3kSSKKW5vV5ViUYMVLVQrVcjtIsQsMfaqAgCArXStF9sWOmHFFq5VqxY9++yzwveqaNGi9N9//1lXshgHkQRji5QCyaaPYbOyvs0qUjzxytWNKBr556GOgU0EPfEtBFzuwkAZaYsPUqQVCABgOwXhJx0yzavEtltRyALWrFmzaMCAAVS+fHl66qmnqF+/fsInCwBgnLrli7pdBGByENUy24wS+co2Cuc3HC/JMiDOACeZ8Fgnt4sAIpCXrmrg2LUqFS9IsYQnxgdOUwLW3r176fXXXxd+V5deeilt3ryZPvzwQ7H9iy++oHbt2tlXUgAimJwQVq8DdS5JCR76+d52mERGCFKQ56bc9kyveraXp3W10Fb+7FKyJLowUEJjBJzi41uaU73y7gUx6deiErnNxMc6u10E4CKJGiuLdcuFt0DsodjGsIDVu3dvqlq1qghwcc0119D69etpzpw5wh+L/bIAiGcuZOdY2rkM6lGH2tUoRU5Tuoh5XzKj1C5bxPC+xQuZN7mMFPPe7g3sN5f75d729OmtLUwfJ9kksicmhGVtHhMEk/fcMKME1uC2LN+9fjnXrj3mwY40b/DlsLawiMe61Q752DJF7RufA3Fbuyqa20sWzhfWeT0xrsIyPCpyvqvff/+ddu/eLaIG1q1b196SAVfoVLs0zR9yOQ27qqHbRYkqzmeGEA3HE9rI/tltLckuiuS3z54804QQGmmrpcqBJPhjs3fQqFG6MCUkeKhoAfNmeRWL2WNikhiGfPW/a5uEdFyk6a9KFw082biiSQXHyhJr5Esw/7RDeT+cbmtsqbDixe5B93NzHlo4XyJVjDHTNDcJJ6XHhzc1p0tqlabv725LTjH0ygb0Sl9/3+k21UrS/64Lre+WiW3xyoSAxbmu+vbtS4mJcOiLZZITE6hCsYIRb+v7+jWNKZK4pa32Ck+oK6OBVkx7NSpP0UjREAKBRAqs2Rs1oBX9OLCt5qqbk3mwioWh3Xv3xmbklPmIUW5onRqVWgU1b4YoKIbDVU3jK0iOGSqXiPyIme1rlqJiBZNjuu8E1gnLVUoVou/vaUuX1HYuzU+NMkU0x7xf72tPqSXDe8c8MS5hwa4DRCWhCjR28dBltUwfY2RSbuUc8uHLa9F3d7cht3jvxqYUzVxerxx1qFnawEAhWdampj3RRff3UIQ6uxZOwhGwYsEHi5MMB5vQW13cJpWLiRVkPZ8/J3wBw6V3o/IB23ik4GZb69++KnWo6Zy5eKFEib68HWlkjNKxljlhJ9pkik4m788MnhiXsCBguci11bKDmutFAn8+EDxfU7yTpLKRKpQvuKbXSN/SqFIxsoonetSlTrXLkFvUKqttw68l9EVyt+tU2RpWTBGrh9FAn8bhaVJCkc+snPI+l1afopECyYm6k+/CNpr7WsWI21oaauOhPOtgTYr7nSmDOtO1LSpTm+olyQ14ghlskjmsbyPHJqLfDGhJr7fOFmZowBe9R2BWUxpNMsUDl9YU5uggNCBguUjnChJVKam/onxjiKYzVtCiam6UsqL5k8KOFBOPvHtDcG0Nd1td6uQKPPmSEjRXTHs0cM+52Sm0hD4rJs8Fkt3q3jyGV6bDukoEjXs1y4YX6OivBzqad+C2UMIa2LmG7Rp1u3QgdreDWF1gYw0wL/q8c0NTGhFCwJhYhIX1SOpXIgmPRf6V6sXYaCGSLAaiheh80jFEIDMfrYSmZripdSpdrZPIdsGQrvTH/e0DOvUve6E7LX6+W8Q5k0c6vCLKK8vB4FVJ1lBxQIfFz3bz+a134wquqdA/uEnfT6disQIULfxf55q2nFczTLvq+zd3tYkK8wjWkqmpUy4ErVmYnUTT1OL008C2YV/yw5uNmTaFdI8BuNWAgBVKKodABFv4sqp1tYjRZKDK169UkfzU2EJrATtgLWuz1OJhnyfZjZwKMYBkQUAV9uEtoFpMjZZ3JO3inAQYJ3qedIwSaI4Vbjf4xrVN6P2btCcc5YsVoJZVA5tFsJDFgoIVKxd6gl4s8eUdrahb/XL0bJo53wcOf6sMXHBv5xpU0yXTMLUmTcnS57uF7dSqNbHWwmPzvdj9IsuaSauTHPtc0oJzK1/tWmWLUOUSBam8TZEGzZTF2P7+B6SWMFb2S2r5P5+v+reiUHFDWH7+CucSnLpOCEOQkeARhi8f4hD46tW+0dcaVfJd0PCY1LI+3bNu2MLUTwPbUR8Tk2W7m/aS530XF92kSojjm5H20a95JeHDGwkLa3rUr6Cf6+2xbnUcLUssAAHLZQK9ai1DTCaqR9PKoa3QWbHu+lyf2J8MdGtQjr7s30qshkphPPtSYeaWCIcPdARyhu/LKGmpgf0LZX68x7lws1Zh1fho9Dz3X6qjibN4nJ70WGea+dRlISUNtlPLPaBDNcuvqZUPLNDkwhIsriQrBYhY5Mme9k4ISwSJ5tmtflm6vlVln21/3N/BJ6iH2b4kx2QbGj3AX5PevEoJ+iSISaTS7aZM0QIRmXuRF4Reu8Y/fHg4BAoC5QlXAL94gj4hpmswMy948LKallxDaWGVz4BpY7T6stoFBKwIpqzNHZvRToRzdYR9Xk9sRpYLh0hcyLJ9kqmicP6ksIU5I/X7ZI/IW30LFgVQLr968O5+0S/P6tDw7MwcajRAO83zq5YqZOk1eWKmeT6KLSLpfn6+t50t96LXNh7vVodKFAo8IVW39OwA0ouWQN4hhEAQ+ZMSAwb1+CHIglN2mC/a6Dtbe9/xQBN91rJMf/JSmvBYp4gV5KcM6kK3tg3ux1ouJb/hBeeqpfR9SaUwE7jL/TULlKH0s0bMzsOloCo4l9k5CltAgDwgYMXTJDvEixXKlySSzdnBNc19V/jMwMmQ/6+Lr3P65fXKUrQ8u1AmyKEIuxyNMtRkrmpCHd7Lp/gvFrSoUtyxsN8PXV7bsnOt35fhty2/jeaIysGbnfEHdgovIEPw62kTKOCKkQlGqGi1hMvqlgnpmpyqwMkknVbWkd6qtF4/Ekk+6e1qlPIzl/rLguAZvRtV0O2bJQsToYdal0EXUUz68pj141O2OfbZu6xu3vj4/o3NaOzDl+geW710YapX3tkFNzvQqzLO+anWOIaKEc2icr4Qih+WGUHXyOK81kJToOiR4c53nugeeYucdgMBK0qoUSa8KF1MOO/HnR2rh3Qc24DPHXy54Un0XSau079DNRrS21clHTFKIUWHy35ZVjGsb0PTx3x3d1tDyVzZh8wq1M9hcG9/v7RHFTbddkRLtCv57/4T53yixzWvUpxuaVPFkQkCBz9RTwzMwr5GnEbAbOS0mjqaH6dZM6wnjRrQWnPiFGyec1+XmsL/1Mg8tUbp8PtcNwWeQIsUbqcA4fxdbKpmFMkGoVUdfjqQgOUUQduIhW2I+xEr04BYBQt+TrRPycJnwws3wXBibsILzDe3qSI+wdCal/RrUcm2Mney0C85WoCA5Tr2vHYf39I8IkzS3ujXxFRy06d7mXPijQazO/bL0iSEsoYbWTKYD5ldaBVbOb95yqTzthHs1CopkwFziHG1aYUjGtAQj+9avxyteamnN1JluLCwZqfwoD51kfxJwoTJqkuyNkxcJ5JUPiaoV8E3mmC98rnfr2nuO1lSsuPIGXIaNk+LFNpUK0mtq/kGeZK/a/UboTQNO5pTIDPGYItMkTg2atG3WUVD/j7W4wlZgxTItNCqsdtIe+KFyuH9Gocc4MkvAIfJMtsZtC0agYAVJdQorb16/IuObfsVTSqG1MCt7ITZ7l5eEYm0l+vRrrVtFfoM2WSHEkxAyotOVVzlZH3pxYkik9a4vPmTe8vlX7A7wszZFArP9KoXsqkgm1NVLF4gsoNcGNxPryWFUwx59V6yKAqYXaLJyNtb6v6WY9bjX+fZsTbMCSQbTZ141V9mwmOdadvwNF3/RjvC1BuhsomFtkAYyZfmCTIh/fW+9n59ywtXNhALPeMf7WRZwIFgfYXZiHJGw4FrjUN69cECTSThVJQ9swsqqSULmnaVUEYmdeK28pvI/WikLZkvssevzuIZCFguE+ylY3MYzkel9zLICYEjEba7N9tZht8JWd+LPXBpLVr4bFfa/kYf6t2ovGVXCuf429pV9evsSyqcuoOF4Dcz8FQrVYheurJhWM8xlLDlHD0vmNO3Hk/1rBfyCvLGV3s70l6DH6sT5cJm9C7nCeCjacl1NbaVLppf97pZKgGL00qEgl4fxaaEoWrD1FrOt65rYsljvL5lrs/q5yrBU23uFqzf1fNdCoUhGua/dk6iFz3bNe88Fvb3KQWS6cHLamlqI9SPTl3foeIxaZrapnpJYUZvpcbfzu5FyzTcaoJZyFh5e2ZdJdhawEkBq3yKMYGGzdq1orNyEZ/vE3okwPYqX8sCSYlRp0G1EghYEQ6bw/DEQTmxD5VwfTfcpkBygrDNfv2axo5fu5xGkIZANK2cG8DBSVMHu8bJuy+pnjehCPEiWuYVdvlIhQubV5jJE2M3uhosm0Ysreuxg3zAKFo6s7RwBmszrBzag+YP0fb11GpzWuVVbuI237Oh+T6XEx2zNonDccs82bMuXd8qNewgF3z0W9c3FdopddnMrMZ/emsLSrCwW6poUDNlXXqDyOw31JgpZsrFBdTa5YoENDfme3/xygaGfGzCKY9VhJoYWald5EiGC4Z0DTmacWGd+pRM1k+wVyxQgIiLV/D+1VM1n7u1Tao3OqxRlNohDiDBC5LtagRfWGV/sc9va6mp4eY6UM4TzbYZzue57uWe3u9SFIz3dhLdM+4YwGiT45ePoy6pJ35ax39wU565iNoBsmj+JB9TskiDX0I935nC+ZJEwAYOLBAKyR7nNAEcZpxNp1YM7e6zvb/C1E5voiCFIYy7MYHhaFNOM2VQZ5qqyCcTKlfohCrmibIR5EEjlMmL0XqWV0HlSVje8eQYwa7FUSI5HDIP+sqJ0D2danh9guyEBXgOgW0VT/aoa6h+OdT1IwoH96IFkunq5pWoZdUSIqzyxMc6W64pCFfA8Njsy2kHksn+J/f2QqtwrZoJxT+PDzFay4uf7yasVVgbbNWjYR88qye1XW2K0vvZbb4a2ef7NKAKxQqIBRqOZBiKNlnm01v1zYy1mPx43jurRKsJKNMPBAtApXyur/RtRDe2ygs89VxacI2keoHm34fyIkDyAg6b1BvpG57oUZfK6iwYqxfhjbQfOUCVPJZaZdEQC0DAihL4xeGoSykFAzdentT3babt3Myd1JIXutGo/s47G2u9+HqDlp62zsjAc2OAaHmhDLe82mu2DDKc70Ld2SiTKoYy9AWyqb+sXpmwErfqEuSm2Q+QQ+abJZyJRK2yRYXAbRe8gsrBG2w1ETSxCsyD/hydaJxOEai8SYkJNPeZy2nGk5dZKviZPZXTcSo+uaUFDeqhPTlis9i6YQiXoSaGdxonBOimJSXHruUk8rvCiwPyAlmpwuHnAJTdB6xOn/DxLeYij6p5WUcIKavKVVWlVCGaN/hysUATLg0qppjqK2qXM9bGShfJ55N+oGaA/Gbqvoy1Rze1SfURbMz2dXZohIxaOSkD6Lx3YzMRkZYjg6rxKP+OrjUdS4CA5TLhNDpetVULLsGyonNHHth+3BPB9RG4bKyZM+IAzRgddtJUGkMrJ3BWdzjKthDo1Oy7xYOXVZMVXg3jkPk+ZeGoYf3zVg6DraxZOQ1481rjJqShXNeNN4QHffYRcQKthQ8jUftYyNIKSmL0nalSspCfSU+lAIkreYWbKZhsXHMlR9fSKlJSor0R18y2tV4W+knZaWbHbfOnge0CBiS5eFWNv4xxU40cevmq+vS9hk+mx+L70zpU69ldo7OQafJqflte79c45KAWSixOJSgIJVqqkjva+44Tt7WrErWmoFppBgIFPzFzO6wB9zte1VaUwrOli1qKc+m1v1eubuQjLHJEWmiuolTA2r59O919991UvXp1KliwINWsWZOGDh1KFy5coGiHTUpC5bf72pPVdAnDfDDUJL+jLYriFQ0+ZsrOK9wVKL+Jq8EZHA9c7DfRIwQfEzMroiULh9a2q5YqFFZZbmxtX04qI4KsU/MCKy5j1PTJqlti52o9CiQnCvv9ZS92F+affz/YMWDCzO/ubiO03X/qJKxl0xV1NE1ZANS67QrFCooJ3z2XVBdlcTtquzrUsqWLO0F+NxswpH3NUlQjyAq+ErO3wvO8m1unBl1AtKu+tN6TrgYS04YiJHDQhvRXewmNwCRdc7XgNxcJZqCBiskmz8OuamT4WZlJtGskqJJdUT31CHu8V5VYeX0rn7TyXKG6Y8hIir/NpOuJFSJ/RkpEGzZsoJycHPr8889p7dq19N5779Fnn31Gzz77LEU7b/Yzb1olU8egKtsMRhOmPn4xSSxHNZJhH4xQUDprBtStGehF7B5SImDM0uTaFpV9OuArm+aG3g2UsJGjmxlxlg50yw0qaJtfGEHvvN/f3daxCZRVj1M5eBouV5gXjyQfLKMES7rNq6CsZWdzm6ZBHOTZTHTEbS2pvk4bHHlHK9P+F69e3Ziev8JcKGbDGGwYHCGuRZXidHObVBG91I3ned3FaIVWntMqH1G70RKKtDICBBOewnln+B24oVWqEPqNoBVVUK3BYj8dq/n2rjamj7n14qT9iztamUrDwT6NwULLcwTYra+nUeUAmm89gpVEIyyOufN7bOxjNY61QqDhhSarxoASIUZ5jWaiosfr1auX+MjUqFGD0tPTacSIEfT2229TNFOtVGExkVi563hIxyvb/O3tws9VZLTD4zxDraqVEP4h707eSD8u3CnCmf+0aJf4XZ2jSdNxWP7fqObFyD4BdpIsnp2LAdbkeXxWnSycIL99fROann7Q+51NJTe80itgsl1eIb+yaQX6adFOw9dRBxIY+/AlPtqJ5Ttz27EnjCo2OjgGW802cvmwW0FYPljhNgDjx7M/mVbycS20zEK4rXoMhiUPREjBAsgdrBZgjd4HhwrnD6M0u7HWty3wydjk94vZW6lfc+OCVrDisQ/ws2n1xD2Fmt/OrXZjNsGvuL6BQ6x6plraKrUAeKdZ31sDdA4h/QbncBySVt+0wN2oUrGgApaZBLtF8lsXFEdG0lms+H3pbhGZNBztptpEXLKrbekcqCyv05YP0UpUCFhanDhxgkqWDByS8vz58+Ijc/LkSfF/Zmam+LiJfH3+X5JyAu6jhDV5yt+VyTb7Ni0f8L6ys7OD3rfRepFysqlNVe7wJBrcszY92a2m8MH4+Z7W9PbkTfR8Wj2fc13QOa94FllZ3u9ZWVmUrbhHM+WTcnLE8VrMeLwDdX1/rve73jX+vK8t9ftsoe719JKbGq035XWz+N4px9RzUv6Wle1bb1lZ2T7l4eEjK0u/LsX5VMfkbsvSvebQK+pS9/cPK37LouyLp/jp7tZUb+hknzKpz617L5m++8qdud7zFPuwg+31jenx31b7/Wbk+nKb0Tp3oG2+zzDrYhtWbMvK3RaMnJzswL8HeV/Vz0mLTrVKUd+mFejKJhWE76Xf+RQDpfzb4J51aPexM3RH2yr05B9rLu7Hz8u/rhpWKBqwjKKPUvRvCRqRPIPVFbcTpZbcaJ0Eeo7ZirrXO5/yfdLDzDjSs34ZevlfD3WsWYpmbjps+pyB+gX1ux+IrOzsoJPbsoWTaM2L3cSE1eiYwX1BsH3ubJ+rvQh0zkcuq0kfTt+iew71O6QZcp/HAkWfYuRcgfB7pyUp6Dm43av3UX8vlBy8frVQH6N8R2RS8iX4tAtlXx1oTqF1fqPlUKPsg9lnUt4/f4LvsaUK5Qk7Rq7N9yXvp/QOlbdp3ZOal6+sT13fm+P9zu1IvS+Xed+Jc7m/5+T4jQPq/dVtkX8ffnUDeumKemKx0+d4VV+lnFsoy8/HfXBjEyqSz6M7p5LHIVVhNO9bud9rfRvQc3+v8/nt0tq58+q65Yr47FumUCK1rlaCCiQlUP6E4O1fXR/BxlUn0GvfdpUjKgWszZs300cffRRUezV8+HAaNmyY3/ZJkyZRoULG/DzsZvLkyXT8WKKmrD9u3Di/bTt38gpNgvf33Pab+xjnzZ1LezStBnN/X7FiBSXuXq75W6Brau0baL/bKxLtWDGHdqzI23YqU6O5SZI4z/4zeb+NHz+e9u7Nu0cl58+d07lu7rEHDhygeXP3aTbrlQtmsX7Opw1pXWPj0rkB73P/vryy5U7OPQbqLa+MGzeme8sxceJE8l1wS/KaxI7LWO+3nVm9eg0VO5QrUKw66PGei6+/9pjvdyNsOuF/zIrDeduYtWvX0LjDqw22g7zfFi5c4P3uu1/utvTlC+johtwt+1RtQGbzCd9zqq+755T/7977OKK+N//z7N23z68dyMdnZeW9l8ry79ie1wamTp1KKfk46W1eOWbNmknpBYkaFE+gdcdz98uXINGFHN93fMuWrQGttJcuW0qZ2/VXCrdn+N/7/fWzacT6vGd3XZkDRHsP0IS9ipdRQY18HtpAiVSmQO67KHM3WxPtPeo9Py9obdly3K+8mzZvpnHnN/psy87xrbeMjLzvpY6tp0qFEmnPmby6CNZW1+/Le453VT8dcP89p/PqJG8///a6Y4dvP6rFWsV176mbTV+m+654P1A/2/B7JvNma6JEz346cyyBtp/y0KFzRvr9JG97GTeO+y1/VhzSe/f92/zyZcuoSUmJOpRNoHkHc+sg0SNRthTomehPFeR9D53V389o/9iwRA7VPJeueR4eK9X7b92ylYqId8+3XW5I30BJB9eHXR6Z1WvW0rgja7zbeeFWr0+ROXjw4MW+zL/93VLTQ3MPJFDLpF00blyu1YcZMlTjaXo6d6S+7fPs1iW0UqN/V6Mcb+U69q1rNer70a+DxYvyxoDM82cD1vtddTxUONm3H1JTp1gC7T3joRMbF9O4izL4mbP+/bR6nqRVxjULZvhsP3Bgv1+fUSHpDO27eJ4TGRk+5+Lnqy7rhQt5ZamVkhPwXnYoxi6u7wMH8sp87Nhx73lqF82i81uX0DgeLnTagDwOKTmrqBclyjLlek36t8/hrYnyJx73K/+tF11alWO0NrnnzDh1SvVs8q5VVjXmOIm6fZ85IyYgsSVgDR48mN58882A+6xfv57q1cvLBr5nzx5hLnj99dfTwIEDAx47ZMgQGjRokI8GKzU1lXr06EEpKaH7jlgBS8z8kLt3705f7VxKO0/nateUpKWl+W2b9/damn9wj/d3XiF4bEFuY+nQsaNmaN9H508S/zdr1ozSVHl/5N8CXVNr30D7aXHk9AV6bgl3aAo8HnGeTQdP0fCV83LP27s3TftjNS09vN/vHAUKFqC0tC665SpfvjxdckkNens1d+q+cD2nrp5G20/lJiyuVasGTdy91d8xvkd3Grx4uu59jj+5kujogdzic7bOi6tOwepDLmPdOnVp3K7cSVLPXr4mfN596tajtM7VNeu9UaNGlHYxvOu55Xvohy1rvdcvkH6IaMNyU89nwdaj9PG6JT7H5KzaR99sWu17TUX4+0DtQP6NFVBt27YjWuV7bqZCo+N09PQFHyfxHUfO0Bsr5/jtu3DbUfroYvnYX/GZP/Pu10v5bTR5/UFaufuEz2+eNftp9MZV3m3qti7KUr4CrTiS+zzV9zR4yRReSvS73tL/NtCs/blmld27dRU5zzKzc+iJhVPEts6du1DNMoXp8u7Z1PjlqUJjcfzsBVq7V0hEXmrWrEFT924nPVq2aEndG+g70q/YdZzeW7PIZ9tjN/WiiR/Npa2HzxhqB92zc+jKjYepZdXiVKKQv428XGclihejmjVK0ZS923x+r1O7FqUp8kAxTyycTDkXVy75+h9tnkt0Vkg+1O/KNOp3JVHtF4z3JYcX7KQ/tudK4nddF3hf7g/3FNggNF1pXWrottcF/6yjuQd2B7z+EcV1n7mtN32pKDPTqX1ralU9WHJRbfpcLGudF/0nsnrvFLeXtB65vq9qMlfuo+82r/Y7XqvNt2jRgno2LCfKID+HhIQEys6WgpZBC3nfHUfP0Ksr8jQCge5JjXz+iuXLU1paM83rcR+enJysqpOa9GqHKvTe1M2i/X42K7d91qtbjzrVLU1vrJwfVnlk6jVoQGntq9KMc2vor+V7aVCvBqIfDlQvZcuWpbTezenxi+Oz8rrmRk9/jpw6T88vmen9zmPG0zdWoE5v8UIi0d0dq9IVverSnM1HaMT6pT7XVjP51CpadmS/t47leYlc12rU71OgOnjwht704UWrhsKFClNaWp5JuRojddK7tyTMNdlaRuat9bPo6PlzAd9vrTKqt5crl9v2lPdYNbUSLTvCi3BEKUWLUlpaB+9vZcqUpbQ037D1Q1dMZ3WS+Pvvx3sENFlctfsEvbs611qG63vs8bVER3PN/EuUKE7bT+WOZ+XKlaO0NH/z7sOKNiCPQ0re2jCb6LxY9RDc26kaNU8tTt1UwVnCmdfpIZ+zSJEidOBi36+sb9ac9Wte0ec5Oj3vVrZv2botpgSsJ554ggYMGBBwH/a3ktm7dy9ddtll1KFDBxo5cmTQ8+fPn1981HDF6nUeTiPKoWPzqlVGHgiVvytVsMlJSQHvKzExMeh9G60Xs/WXnORvXsRFF88iKa8Z8nflPar9BgJdl02gkhTneqVvQ3rh77U0pHc9cdyddbJpQ2I1uuuSGvTvyr2G7kv9XS/EvdH6UN5bPnHvCUGf0/UtK9NvS3f7/ZaQkLdiydsSVd+NkJjkf0yiog61yqMk0HUK5EvW3K9NTX+7/VrliwkfQo4UpdxX+TzTmlSiNyduojbVSvrs81DXOlQof7JXwPLeR6Jvu9JCq63l7evRPF55TJLclyTk+JRZ7mO2v8HTWKIrP/KfeHK9Bns2geq3aZVSwpH55LlMyjiXa26SL18yvXV9M7p55AJ6ulddA+87Ue8mwcNN82JCgsZg6EkIXEbRR6m+a+0TCHWfF4zX+vnnY1Ef36FWGfpp8e6A51Q+H71y2zGO6LdV/bpOClJWn32DtCsj59DaV9mPh3q+JpWLB+xr1L81Ti1O5UsUoTevayZMrGQBi59dUlLgdmkGjydBHPPO9c3osW51qGqp4MnVEy4eE8519UhK9h1P+X5TSxX1eWf4Wvysg11bLBSq9gnUtjmE+NIdx0RiWeU+yYke8fz4N5n8+fP5BDmw433xKMbkvLExeJ+hNbartynrhscD5e9sxu53bsX0oHDBwD7CSX7zHmWaCI/fs1RTtliSNy1NmWKF/fwa1dPKZ/sED6hm9fPxKAqhPHdKofxUsIA1+d5CQd2+7ZIHXBWwypQpIz5GYM0VC1ctW7ak0aNH607Co5221UuKVXt953H1SxQdboSGA6sFuB2zftG3t68moukVL5RPrFwUz0/0cloDv5eJQ0J3fSdvNdApjD46DtcrC1huEEowhoKJHOWyiFgpN5qbTJlbQy+lweLnutmS3yVUwgnTHu5t8OrorKcvo3V7jtGVn8z3mQBxuHMrVwe5rOcyg/skMbe1q0pfz9su8tIxboc710JMDhMTqFEl45YMH9zUjB77ZYX3fsoabNdWYXXSWP93XLKsn3i0a236YOomw8eOe6STCNJzTyffQAB6THiskwgM1UeRpzBwfsfwkCevfA2lcMW5BDfs99VMK7FreA72Ttk5L/iqfyuatO6AX47IDjVLi0h/1Qb/57N91IBW9Mn0LfT29U3JKYwG0Pjj/vZ07Yj5hsa5wgaCYoTT1/lEovXZrt8mOZ+l/DeIPKLCB4uFq0svvZSqVq0q/K4OHTrk/Y3NwmKJz29vSeNW76c+KlM+a8KAuvcSBup4jAtfwcuv3oWFq2DXVJrpmakjp2LA+UTvUWwPMQ2Wqgzm6zQQr13TiGalH6TWRfeKcn9+eyuyEqsHklAmraFEwrPr1eP60Dq11aYXXH6jgvKzafWFcCWncAhXLAi36niRhTXWykUrbpu9GpkbO/o2qyTq+6Efl/skOrYKTlJ6X5eatrRjTuPwx7LAizT/u7YJPf3HKvr0Vl+zp1Da+L2da9DBjPOGI5Q2qJgiPkapVz5FfJwQSFkI55DpWnx4c3Pq8V6uWZ6bqFOktLqYqDbJwEK02feLx1S9+tDi8nrlxMdJOKIxR7Tt21w72qBMy6p5gdJaVPVPC8G96ye3tKCPp2/2ExCtXu5oXLkYTVi739TYESj3Z/hRakFcCFhsM8lBCfhTuXLlsCc7kYbyFrjzCje5W7QgdxzKELPhCIHcoZRPCW/Sw1cf2Kk6fTF7m1ixdpKKxQrQ3hPnAvrdBMKqd0H9BC6pZdzP5Na2VemGFhVp3DhtE0yzNL4YmjfYc3Vy7cBndVHjwm52SXYtovBZ7+xQnfYcOysSVPcf5ev7pdasXVo3rw3L/lhu8ea1jalXw/JhJVF3gtnPXOYTll1NSZ3FIiMYETZuaJ1KVzevZCrUtf71YoNaZYvQ2Ic76f6uTkfiFJwImq99/Ewm3dmxmhD+mdlPX0br952k7g1yBRpe5OBciHwfsYqWIFGsUDL9el97Q8ezBcu8LUfoJoWfsRJe7NZa8Nbq1syMwequmrW3b01M9xuDY0lQ6lKnDC3beYwuqxfaHCfaiAoBi/20gvlqxQsdapbSXRWMthdR7os4IEDXemV1tU1mOix29Pzj/g5UUJEgT3PfAL8916cBDepelwrmsz5Phk8ZVD3stCcvFcEfKgZIECivTDLtqpcS/ydd1OpYlchPOURwotNyYQqt4VA4f5Iwdwu0Uqf3PDmkLBMoF5ia/x7Jc8J+87om9MhPy+lJncACete1ikh6m7mt8vvw2jWNTR+rJWBxkw0htVBIsNASikWA1mTJjT6WF3qmrD8oclPZTTjCFWv0WMuZLzGBCgXpf+0mhBSFNpXDnvbCmtRFz3bze2apJQuJj3K/74IkbreqmtwykLmxdaoQTNg0OhQ4qTl/jN4Tz8FYIGNT6HDqUt2XKHNMRkDTtYWv72xNWTlS0PE8VogKAQv4+g7whFGZdC8aCLSCyoPQVwNae7+3q1GK/l6xN+QO3EhHKwVJ/muXcBXItpqzpusJV4ue7Ur7T56j+hXyzGKqlCokVix5tU4Wvh64tCbV0BksQsFN4Uom0Kq+DGtVXvp3nfCJkCmbUkAIiMHs8Xs0KCd8Cp7qWZcaVsx7r65qWpEur1fW7/hgzdBoO9Xy4Zn2RBe63AV/wGB4AviMBiNAWjtDKM14Yhk94Y21E7KGQg89YWJQ9zr03YId9ESPuvTnstzos1ai1G6zWSr7hHhs9ocygtkktlbCibq5L3/+iga2XscKTWMs8H+da4joeE1S/U38wkWrFbOf2d7jZw0FOglEySL5bBVcI9E938OBQRIjsGA2AQErAjBjK84NlCeTVsEhy89lhjkDMoKJJRm27y6UL5FeGbtehCKN1A4l3OuZOZ6FBf6oUa5Wctt4ule9sMsQjd0fC6Yrh/agwirBOJiAyBPTj29pIcxqZHPEYBM1XxNBY+VT7vbZbS1oRvohuqVtVSEUKrFSOLYCebX29va+q7U8kd566DR1NGA+qpkI1qRvAmulKxZ3VtjXNP+MsrXlR7rWpocvr2WpJmXk7S3p3u9yQ3+nFPR9P9xemX65b0OatfEQXd+qsmif4RKK2fW9nWrQQxbXeTTglsaQBfsOJszYw4XbeLjCFcMRYN++thFtXLvSR+PIYeh5kZl9yJg4a0YxBZZAIoDmqaGpttWE8iJOesw/r5TbcCfDK7aVS/hqcyKhownXRMjHtjoCbkirBEqfuGiCQ7wbCe6gDpTBK8FNU4tbtuquNdG4tmWu72iDCinUq1EFeuPaJlGxAv31nW1oyqDOfhoUFmgvqW1sUmPFvIu10hWK6ZvOxjulVTlwyKZ+hs3/eIGPJ4dM70bmTS/NUseE/9Ad7avRl/1bC3Mr5XvIC4lGg7QoKR9iIJNI6NuBC5js7Po2q0gNS+QdNP2JS2nYVQ1FFM5w5xyXKfxggTtAgxUBPNO7nhgg+zRxPiIim5mxr8ri7Xm5KyK1r7qhZap15w5x1hdtK9hGJq4NK6ZQ9dJ5K3KcAJhN7ZpX0Rb8OZm1nHMqHvExKTU4+N3WtirVLVeUGtpg2iubiNoBC4G1yuaZXYaC20EuQsVpH6xw5uQcyODBy2r6mBAHuJKhTYEY92gn2nwwg1ro9BFWcXObVBrapx5NnDA+rPOsGtqTen0wiw5lGLeIYO7tHDiqo1Z7iFfZKhbv2+w9SRbMx/x8LUOs12d61RPpMoJRrVQh2n4kNzE9sBYIWBEAmyE92i1vxSJUIlnzEG7H8/3dbaldjcC+GFZMfux2Yk8pGBkJrmVY4zP24Ut8VlzZf2DCY511j7E7+Ifd2B15VOs1ZO1Y2xqlTJ7HWFvkCIu31cqmjm1aUiTCZoTs/1Na4XMQpTJXxC6wcFt5qmc9By6Upy12wjeONVGhpmaoUSZ30YjNhkPVFgcLliQHGDICJ/4G7sJ+r0UKJIlk0XYQSVGteZw2kp6hceXiELBsAgJWDMDhRTnnSP0Koa00P3BZLbpz9GLh1B9MEDx1Piuka4Tb7xg1R7KbcAUwNq3ihLopBSLn1Ytnc5awx0MLq07tP2aG1mUk6lY/Mk1C2OSlfvkUSlNE8uPQ0qPnbheJqIGzqE2vTRE588eg8ELR2mGccNu+/o0jt97RvqroBr6ZvyPgvu1NLrAA67mmeSUaklbf8P6REJnZ7hK80reh8G/t19w3BRIIn8iZ5YGQYX+OcF5UttVd8nw3kegyEOw/smj7UYpY3O8LDXG7RnhX4Ey0qc9nbaXn+zSgl8f6BpcIFVk29YS4qh0PFC2QTAM71/BLRtytfrmQQyu7RSRMuELl1/9rT3uOn3EsAi37l3BKEaX5sRtpHsLByNrTy30bif9lAUtv0SZSF7LYRYATcVuFnNg7khj/aCeatPaASIAdbYTTboxo3Dk9zpDexoVOM3StV5Y2Hzzl9dmMNyBgxRGcnDAUJ2nXiCB1u/5ky1wZI3OINR+ZbMHWhcL8IJoY3LuesG9nLaIsYLHzuxWDFpta8kp2xrksqqKI7Aj0I3EZiUAIrIOTzhKFadZnogPjSGicxLViFAcnsXIIiiTzMSW3tKlCBZISqbWBlAtGiMQo3OybaMw/MTzseMJhVafLTa5yyUK0+LluftFG44X4vOs44/0bm9GG/Rki5LJb/gehHBuZw1EexQslC9NMM1QrHf2T7w41S9PKF3tEXafJK4FynrHh/RrTd/N30OAQVu70otnJK9lm+PDm5iKZcTS0d8AmYZHlQxnp6CVwjWUiVFGlCy8O3dA6N4BUZmam28WJ7iAXNnTiUW0QIUkhRe+MFSI/TjAIm6ubVxKr926aKDixeOexQOAzU0Wj72xNTSoXEwE4gsGBJEYNaBV2RLZIgSPXRarJixFublNFREILJQxzMF9Ft84F7If9aO7rXJ1ur5Vt+bndfJ2MTuKi943HCgaw/50MZCVkFg6KUi4lPz3ZE8FRohUIWMARtBJQBovsdPXF3DscMjwSaVixGP3z0CWGAnCw38Pl9eDQHwuEGtUMRBftNDT+vKjwRPfa1KpMbMzWP7q5OZUolEzf3NkmaCAlxopotwDEKl8NaEXNUosbWnQNxgOX1qIFQ7pS5RKhW73ERi8VvUSXjQ+IWlhN/MClNYWgxR3Qi/+soXdvaBbwGPaXqVu+KDWubMwpO1SNijJ0OibPAACmXvkU4RzPoZ1jFQ5IcEWTCkH7zteuaUz3dKoedyZ/FUJMNByPeGJyUu8x3WeMebCjdVePYisRAAELOMjTvfLytMyud3nQ/VnYccIRnnO6/HBPWyH8aWnaAHDDjBZDq/s44RjvdoRCI5M47otjxbzZCDxJPnbmAqWGELCmZOHYFcgDEVuCFQDhAwELGGZgpxq0ePtSEV451jAsyGHWC2ymYcUUWr/vJLUNklgbAGAPbGVhlg9uakZzNx+m61tFV3RVELtEauTKeAECFjBMj4blae7gy6l8SmSaTTgi+6C/AhaTPymBzmflUKNKudqSfx+6hC5k54hEqSC2Yf9SjvAK7OedG5rSrV8upGcUlhRW0rdZJfHRA0NH9BHtFnpoc+4CeyhgCk4YFwt+SljYAZECR5jkHFof3dxCfE9I8EC4ihM+vTX3mcfCZC7SaV6lBK15qafw7QU2+mDF0NhaNMxE1fGORPENBCwQMzgyQcEkCFhM7XJFRQ6tUELGg+gGTuzOwosXAATj9Wsai8TcHMkPgFCBeA4AAAAAAAAR3dK2ivhEO25rE6uEECQmloCABQAAAAAATNO2eklauO0o3dK2qttFARHCL/e2E4GautQpQ/EMBCwQMzhh/FG/fFH6b9U+B64EIpXihfLyphVIgq8UACB++f6etrT3+FmqWqqw20UBKtxSYLWtUUp84h0IWACYYGDnGpQjEV1Wt6zbRQEuUShfEk1+vLPwn8mXBDdWAEDkm2vZBeeOhHAVmVxSqxT9u3Kv28WIWyBggbh0GA91rMuflEiPdK0d4tEglgJTAAAA8CVG5cio5PqWqVQkfzI1r2I+rxsIHyy/gqhHTnyM8LsAgGglVjUcoVDkYnjsy+vBUiDaQJzGyIqa2adJBapYvKDbRYlLoMECUc/I21vSyXOZVLxQPreLAgAAhqlashA1TS0u8u0kJ2JqKjP76cto25HT1KJKCcrMzKRYoEByfKxnP9WzLs3bcoT6t0fQCxDfQMACMbFKA+EKABCNfdeYBzqIv5ETK48ShfOJTyzwdK+6tO/4OWpQIYXigdSShWjxc13RnkHcAwELAAAAcAlMRGObeExWizYNAHywQJwCfwcAAAAAAGAHELAAAAAAAAAAwCIgYAEAAAAAAACARUDAAnGJhGwdAAAAAADABiBgAQAAAAAAAIBFQMACcYkH6RABAAAAAIANQMACcQlMBAEAAAAAgB1AwAIAAAAAAAAAi4CABQAAAAAAAAAWAQELxCewEAQAAAAAADYAAQsAAAAAAAAALAICFgAAAAAAAABYBAQsAAAAAAAAALAICFgAAAAAAAAAYBEQsAAAAAAAAADAIiBggbgEQQQBAAAAAIAdQMACAAAAAAAAAIuAgAUAAAAAAAAA8SpgnT9/npo1a0Yej4dWrFjhdnEAAAAAAAAAIHoFrKeffpoqVqzodjEAAAAAAAAAILoFrPHjx9OkSZPo7bffdrsoAAAAAAAAAOBHEkUJBw4coIEDB9KYMWOoUKFChs0J+SNz8uRJ8X9mZqb4uIl8fbfLES+o6zs7O9vvN2ANaNvOgvp2FtS3s6C+nQN17Syo78iob7vq3yNJUsRHrOYipqWlUceOHen555+n7du3U/Xq1Wn58uXCH0uPl156iYYNG+a3/ccffzQspIHYZMz2BJq+L1eB+0H7LLeLAwAAAAAAHObMmTN0yy230IkTJyglJSU2NFiDBw+mN998M+A+69evF2aBGRkZNGTIEFPn5/0HDRrko8FKTU2lHj16WFqJocAS8+TJk6l79+6UnJzsalniAXV9r5qQTtP37RC/sfAOrANt21lQ386C+nYW1LdzoK6dBfUdGfUtW7dZjasC1hNPPEEDBgwIuE+NGjVo2rRpNH/+fMqfP7/Pb61ataJbb72VvvnmG81jeX/1MQxXbKQ05kgqSzwg13dCQp77IerfHtC2nQX17Syob2dBfTsH6tpZUN/u1rddde+qgFWmTBnxCcaHH35Ir776qvf73r17qWfPnvTLL79Q27ZtbS4lAAAAAAAAAMRQkIsqVar4fC9SpIj4v2bNmlS5cmWXSgUAAAAAAAAAURymHQCriPzQLgAAAAAAIBqJCg2WmmrVqonIggAAAAAAAAAQSUCDBQAAAAAAAAAWAQELxCV9mlQQ/1cpiXxoAAAAAAAgzk0EAQiX5lVK0MynLqVyKQXcLgoAAAAAAIghIGCBuKVqqcJuFwEAAAAAAMQYMBEEAAAAAAAAAIuAgAUAAAAAAAAAFgEBCwAAAAAAAAAsAgIWAAAAAAAAAFgEBCwAAAAAAAAAsAgIWAAAAAAAAABgERCwAAAAAAAAAMAi4ioPliRJ4v+TJ0+6XRTKzMykM2fOiLIkJye7XZyYB/XtHKhrZ0F9Owvq21lQ386BunYW1Hdk1LcsE8gyglXElYCVkZEh/k9NTXW7KAAAAAAAAIAIkRGKFStm2fk8ktUiWwSTk5NDe/fupaJFi5LH43G1LCwxs6C3a9cuSklJcbUs8QDq2zlQ186C+nYW1LezoL6dA3XtLKjvyKhvFoNYuKpYsSIlJFjnORVXGiyuuMqVK1MkwQ8ZL5ZzoL6dA3XtLKhvZ0F9Owvq2zlQ186C+na/vq3UXMkgyAUAAAAAAAAAWAQELAAAAAAAAACwCAhYLpE/f34aOnSo+B/YD+rbOVDXzoL6dhbUt7Ogvp0Dde0sqO/Yru+4CnIBAAAAAAAAAHYCDRYAAAAAAAAAWAQELAAAAAAAAACwCAhYAAAAAAAAAGARELAAAAAAAAAAwCIgYLnAJ598QtWqVaMCBQpQ27ZtadGiRW4XKSqYNWsWXXnllSLbtsfjoTFjxvj8zvFaXnzxRapQoQIVLFiQunXrRps2bfLZ5+jRo3TrrbeKJHPFixenu+++m06dOuWzz6pVq6hTp07i+XDW7//9738UbwwfPpxat25NRYsWpbJly9LVV19N6enpPvucO3eOHnzwQSpVqhQVKVKErr32Wjpw4IDPPjt37qQ+ffpQoUKFxHmeeuopysrK8tlnxowZ1KJFCxHZp1atWvT1119TvDFixAhq0qSJNwFi+/btafz48d7fUdf28cYbb4j+5LHHHvNuQ31bx0svvSTqV/mpV6+e93fUtfXs2bOHbrvtNlGnPBY2btyYlixZ4v0dY6V18FxO3b75w22aQfu2juzsbHrhhReoevXqot3WrFmTXnnlFdGeI7JtcxRB4Bw///yzlC9fPmnUqFHS2rVrpYEDB0rFixeXDhw44HbRIp5x48ZJzz33nPTnn3/y2yT99ddfPr+/8cYbUrFixaQxY8ZIK1eulK666iqpevXq0tmzZ7379OrVS2ratKm0YMECafbs2VKtWrWkm2++2fv7iRMnpHLlykm33nqrtGbNGumnn36SChYsKH3++edSPNGzZ09p9OjRog5WrFghpaWlSVWqVJFOnTrl3ee+++6TUlNTpalTp0pLliyR2rVrJ3Xo0MH7e1ZWltSoUSOpW7du0vLly8XzK126tDRkyBDvPlu3bpUKFSokDRo0SFq3bp300UcfSYmJidKECROkeOKff/6R/vvvP2njxo1Senq69Oyzz0rJycmi/hnUtT0sWrRIqlatmtSkSRPp0Ucf9W5HfVvH0KFDpYYNG0r79u3zfg4dOuT9HXVtLUePHpWqVq0qDRgwQFq4cKGom4kTJ0qbN2/27oOx0joOHjzo07YnT54s5ifTp08Xv6N9W8drr70mlSpVSho7dqy0bds26bfffpOKFCkiffDBBxHZtiFgOUybNm2kBx980Ps9OztbqlixojR8+HBXyxVtqAWsnJwcqXz58tJbb73l3Xb8+HEpf/784uVguGPi4xYvXuzdZ/z48ZLH45H27Nkjvn/66adSiRIlpPPnz3v3eeaZZ6S6detK8QwPIlx3M2fO9NYtCwDcwcmsX79e7DN//nzxnQeKhIQEaf/+/d59RowYIaWkpHjr9+mnnxaTLyU33nijEPDiHW6HX375JeraJjIyMqTatWuLCVGXLl28Ahbq23oBiyczWqCurYfHq0suuUT3d4yV9sL9SM2aNUU9o31bS58+faS77rrLZ1u/fv2EIBSJbRsmgg5y4cIFWrp0qVBZyiQkJIjv8+fPd7Vs0c62bdto//79PnVbrFgxYYIp1y3/z+rgVq1aeffh/fkZLFy40LtP586dKV++fN59evbsKczjjh07RvHKiRMnxP8lS5YU/3M7zszM9KlvNvupUqWKT32zaUq5cuV86vLkyZO0du1a7z7Kc8j7xPP7wGYQP//8M50+fVqYCqKu7YHNdtgsR10nqG/rYRMdNu2uUaOGMM1hkygGdW09//zzjxjjrr/+emFu1rx5c/riiy+8v2OstHeO9/3339Ndd90lzATRvq2lQ4cONHXqVNq4caP4vnLlSpozZw717t07Its2BCwHOXz4sJg8KV8khr9zowChI9dfoLrl/3nAUZKUlCSEBuU+WudQXiPeyMnJEf4pHTt2pEaNGnnrgjsf7qgC1XewutTbhweXs2fPUjyxevVqYaPPNvb33Xcf/fXXX9SgQQPUtQ2wALts2TLha6gG9W0tPLlhf5EJEyYIX0OeBLFvQ0ZGBuraBrZu3SrquXbt2jRx4kS6//776ZFHHqFvvvlG/I6x0j7YL/z48eM0YMAA8R3t21oGDx5MN910kxBSk5OTxeIBz0140SYS23ZSSHcJAIirlf41a9aIlSJgH3Xr1qUVK1YIbeHvv/9O/fv3p5kzZ7pdrJhj165d9Oijj9LkyZOFAzOwF3l1meFALixwVa1alX799VfhhA6sXxDj1fnXX39dfOdJKPffn332mehTgH189dVXor2zthZYD/cZP/zwA/3444/UsGFDMV6ygMX1HYltGxosByldujQlJib6RZDh7+XLl3etXLGAXH+B6pb/P3jwoM/vHKmHI8oo99E6h/Ia8cRDDz1EY8eOpenTp1PlypW927ku2ByCV+sC1XewutTbh6P7xNvki1c6OTpUy5YthWaladOm9MEHH6CuLYbNdrgf4IhcvHLJHxZkP/zwQ/E3r1Sivu2DV/Pr1KlDmzdvRtu2AY6exppvJfXr1/eaZWKstIcdO3bQlClT6J577vFuQ/u2Fo6uKGux2Kzy9ttvp8cff9xriRBpbRsClsMTKJ48sQ2pcrWJv7OvBQgdDtvJDV9Zt6w+Z5tauW75f+7oeIIlM23aNPEMeFVV3ofDwbPdtAyvdLN2oUSJEhQvcBwRFq7YTI3riOtXCbdjVtEr65vtk3kQV9Y3m70pOzOuSx4U5AkA76M8h7wP3ofcvuH8+fOoa4vp2rWrqCte/ZQ/vOLPZiby36hv++BwyFu2bBGCANq29bAptzqlBvussNaQwVhpD6NHjxamZ+zXKYP2bS1nzpwRvlJKWGnB7TIi23aIwTxAGGHaOaLJ119/LaKZ3HvvvSJMuzKCDNCP+sVhTPnDTffdd98Vf+/YscMbnpPr8u+//5ZWrVol9e3bVzM8Z/PmzUX42jlz5ogoYsrwnBxxhsNz3n777SI8Jz8vDo8ab6Fn77//fhHqdMaMGT4haM+cOePdh8PPcuj2adOmifCz7du3Fx91+NkePXqIUO8cUrZMmTKa4WefeuopEV3pk08+icvws4MHDxYRGjn0LLdd/s5RjSZNmiR+R13bizKKIIP6to4nnnhC9CPctufOnSvCUXMYao5MyqCurU89kJSUJEJab9q0Sfrhhx9E3Xz//ffefTBWWgtHg+Y2zJHm1KB9W0f//v2lSpUqecO0c8oe7ks4ymIktm0IWC7AOQz4heN8WBy2nWPxg+BwXgkWrNQffunkEJ0vvPCCeDFYiO3atavIKaTkyJEj4kXi3AkcBvXOO+8UgpsSzp3AYW75HPwy8wsbb2jVM384N5YMd1gPPPCACGfKnc8111wjhDAl27dvl3r37i1ySHBHyJOtzMxMv+farFkz8T7UqFHD5xrxAoee5dw1XAc8uHLblYUrBnXtrICF+rYODiddoUIFUQfcn/J3ZU4m1LX1/Pvvv2LSzmNYvXr1pJEjR/r8jrHSWjjPGI+P6jpk0L6t4+TJk6Kf5vlzgQIFRD1wblRlOPVIatse/id0hR0AAAAAAAAAABn4YAEAAAAAAACARUDAAgAAAAAAAACLgIAFAAAAAAAAABYBAQsAAAAAAAAALAICFgAAAAAAAABYBAQsAAAAAAAAALAICFgAAAAAAAAAYBEQsAAAAAAAAADAIiBgAQAAAA7g8XhozJgxbhcDAACAzUDAAgAAEDIDBgwQgoP6s3nzZkvO//XXX1Px4sXJ7Xu8+uqrXS0DAACA6CHJ7QIAAACIbnr16kWjR4/22VamTBmKNDIzMyk5OdntYgAAAIhxoMECAAAQFvnz56fy5cv7fBITE8Vvf//9N7Vo0YIKFChANWrUoGHDhlFWVpb32HfffZcaN25MhQsXptTUVHrggQfo1KlT4rcZM2bQnXfeSSdOnPBqxl566SVdczvWdLHGi9m+fbvY55dffqEuXbqI6//www/ity+//JLq168vttWrV48+/fRTU/d76aWX0iOPPEJPP/00lSxZUtyvXC6ZTZs2UefOncU1GjRoQJMnT/Y7z65du+iGG24Q5ebz9O3bV5Sb2bBhAxUqVIh+/PFH7/6//vorFSxYkNatW2eqvAAAAJwFAhYAAABbmD17Nt1xxx306KOPCqHg888/FwLQa6+95t0nISGBPvzwQ1q7di198803NG3aNCG4MB06dKD333+fUlJSaN++feLz5JNPmirD4MGDxfXXr19PPXv2FELWiy++KMrA215//XV64YUXxLXNwPuzULhw4UL63//+Ry+//LJXiMrJyaF+/fpRvnz5xO+fffYZPfPMM37aNC5P0aJFRT3NnTuXihQpIrSBFy5cEILf22+/LQTOnTt30u7du+m+++6jN998UwhsAAAAIhgJAAAACJH+/ftLiYmJUuHChb2f6667TvzWtWtX6fXXX/fZ/7vvvpMqVKige77ffvtNKlWqlPf76NGjpWLFivntx8PXX3/95bON9+P9mW3btol93n//fZ99atasKf34448+21555RWpffv2Ae+xb9++3u9dunSRLrnkEp99WrduLT3zzDPi74kTJ0pJSUnSnj17vL+PHz/ep8xcD3Xr1pVycnK8+5w/f14qWLCgOF6mT58+UqdOnURd9ujRw2d/AAAAkQl8sAAAAITFZZddRiNGjPB+Z80Os3LlSqGZUWqssrOz6dy5c3TmzBlhAjdlyhQaPny4MIk7efKkMB9U/h4urVq18v59+vRp2rJlC9199900cOBA73a+ZrFixUydt0mTJj7fK1SoQAcPHhR/s2aMzR0rVqzo/b19+/Y++3PdcCAQ1mAp4XvnMsqMGjWK6tSpIzR9rOVjs0cAAACRDQQsAAAAYcECVa1atfy2sy8V+1yxuZwa9k1if6MrrriC7r//fiGEsR/SnDlzhADEZnKBBCwWNHIVWb5md1plU5aH+eKLL6ht27Y++8k+Y0ZRB8vg8rBpoFG4LC1btvT6hekFCGFBjAVDFrDYRJIFOQAAAJENBCwAAAC2wMEt0tPTNYUvZunSpUIoeeedd4QAIQdyUMJ+TKz10hJCWOBQBpVgrVcgypUrJ7RKW7dupVtvvZXsggNocAALpUC0YMECv7rhABxly5YVPmZaHD16VISIf+6558S5uMzLli0TgS4AAABELghyAQAAwBY4mMS3334rtFhs3samcz///DM9//zz4ncWvFjr9NFHHwmh57vvvhMBIZRUq1ZNaHumTp1Khw8f9gpRl19+OX388ce0fPlyWrJkiQgAYSQEO5eFTRI5sMbGjRtp9erVIsQ8RzO0im7dugmzvv79+wsNFAexYCFJCQtLpUuXFpED+fdt27aJqIkcnZADWjB8T2xqyPXF5WNB02yQDwAAAM4DAQsAAIAtcJS8sWPH0qRJk6h169bUrl07eu+996hq1ari96ZNmwrBgSPjNWrUSJjLsfCjhCMJsqBx4403Cq0VR+xjWOvFwkenTp3olltuEYKHEZ+te+65R4RpZ6GKw8NzCHeObFi9enXL7pu1cX/99RedPXuW2rRpI66p9ENjuKyzZs2iKlWqCBNK1nqxaST7YLFGiwXTcePGCaEzKSlJmDp+//33wrxx/PjxlpUVAACA9Xg40oUN5wUAAAAAAACAuAMaLAAAAAAAAACwCAhYAAAAAAAAAGARELAAAAAAAAAAwCIgYAEAAAAAAACARUDAAgAAAAAAAACLgIAFAAAAAAAAABYBAQsAAAAAAAAALAICFgAAAAAAAABYBAQsAAAAAAAAALAICFgAAAAAAAAAYBEQsAAAAAAAAACArOH/AVS9ojGj97AHAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "vds = vz.open_virtual_dataset(filepath, loadable_variables=[\"embedding\"])\n", + "\n", + "# Get a slice of the embedding tensor\n", + "token_slice = vds[\"embedding\"][100:110, 100:200].values\n", + "print(f\"Shape of extracted slice: {token_slice.shape}\")\n", + "\n", + "# Get a single token embedding\n", + "single_token = vds[\"embedding\"][42].values\n", + "print(f\"Shape of single token vector: {single_token.shape}\")\n", + "\n", + "# Visualize the token embedding\n", + "plt.figure(figsize=(10, 3))\n", + "plt.plot(single_token)\n", + "plt.title(\"Token Embedding Vector\")\n", + "plt.xlabel(\"Feature Index\")\n", + "plt.ylabel(\"Value\")\n", + "plt.grid(True)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Memory usage before: 913.12 MB\n", + "Memory usage after loading slice: 916.11 MB (+ 2.98 MB)\n", + "Memory usage after loading full tensor: 1209.08 MB (+ 292.97 MB)\n" + ] + }, + { + "data": { + "text/plain": [ + "3219" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import gc\n", + "\n", + "import psutil\n", + "\n", + "\n", + "def get_memory_usage():\n", + " \"\"\"Get current memory usage in MB\"\"\"\n", + " return psutil.Process().memory_info().rss / (1024 * 1024)\n", + "\n", + "\n", + "# Memory before loading\n", + "mem_before = get_memory_usage()\n", + "\n", + "# Access a small slice\n", + "small_slice = vds[\"embedding\"][0:100].values\n", + "\n", + "# Memory after loading a slice\n", + "mem_after_slice = get_memory_usage()\n", + "\n", + "# Access the whole tensor\n", + "full_tensor = vds[\"embedding\"].values\n", + "\n", + "# Memory after loading the whole tensor\n", + "mem_after_full = get_memory_usage()\n", + "\n", + "# Print results\n", + "print(f\"Memory usage before: {mem_before:.2f} MB\")\n", + "print(\n", + " f\"Memory usage after loading slice: {mem_after_slice:.2f} MB (+ {mem_after_slice - mem_before:.2f} MB)\"\n", + ")\n", + "print(\n", + " f\"Memory usage after loading full tensor: {mem_after_full:.2f} MB (+ {mem_after_full - mem_after_slice:.2f} MB)\"\n", + ")\n", + "\n", + "# Clean up\n", + "del small_slice, full_tensor\n", + "gc.collect()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Note:** I'm not entirely sure how this works under-the-hood as we seem to only be retrieving a partial range of the chunk (great! but I didn't do anything to support this...). Should be confirmed by a core dev." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Not shown\n", + "\n", + "Similar results for remote files on HuggingFace Hub e.g. \"https://huggingface.co/openai-community/gpt2/resolve/main/model.safetensors\"." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "This notebook demonstrated how to use VirtualiZarr with SafeTensors files, including:\n", + "\n", + "1. Opening SafeTensors files as virtual datasets\n", + "2. Using custom dimension names for better semantics\n", + "3. Efficiently accessing parts of tensors\n", + "4. Comparing memory usage\n", + "\n", + "VirtualiZarr provides an efficient way to work with large tensor data, particularly when you only need to access specific parts of the tensors or when working with remote files." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.7" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/virtualizarr/backend.py b/virtualizarr/backend.py index 16901c9c..f39d61c4 100644 --- a/virtualizarr/backend.py +++ b/virtualizarr/backend.py @@ -27,6 +27,7 @@ HDFVirtualBackend, KerchunkVirtualBackend, NetCDF3VirtualBackend, + SafeTensorsVirtualBackend, TIFFVirtualBackend, ) from virtualizarr.readers.api import VirtualBackend @@ -45,6 +46,7 @@ "kerchunk": KerchunkVirtualBackend, "dmrpp": DMRPPVirtualBackend, "hdf5": HDFVirtualBackend, + "safetensors": SafeTensorsVirtualBackend, "netcdf4": HDFVirtualBackend, # note this is the same as for hdf5 # all the below call one of the kerchunk backends internally (https://fsspec.github.io/kerchunk/reference.html#file-format-backends) "netcdf3": NetCDF3VirtualBackend, @@ -70,6 +72,7 @@ class FileType(AutoName): fits = auto() dmrpp = auto() kerchunk = auto() + safetensors = auto() def automatically_determine_filetype( @@ -89,6 +92,8 @@ def automatically_determine_filetype( if Path(filepath).suffix == ".zarr": # TODO we could imagine opening an existing zarr store, concatenating it, and writing a new virtual one... raise NotImplementedError() + elif Path(filepath).suffix.lower() == ".safetensors": + return FileType.safetensors # Read magic bytes from local or remote file fpath = _FsspecFSFromFilepath( diff --git a/virtualizarr/manifests/store.py b/virtualizarr/manifests/store.py index a9d7e6d9..749958ff 100644 --- a/virtualizarr/manifests/store.py +++ b/virtualizarr/manifests/store.py @@ -152,7 +152,14 @@ def default_object_store(filepath: str) -> ObjectStore: ) if parsed.scheme in ["http", "https"]: base_url = f"{parsed.scheme}://{parsed.netloc}" - return obs.store.HTTPStore.from_url(base_url) + client_options = {} + # TODO: add support and update docs for default_headers when released on obstore + # if token := os.environ.get("HF_TOKEN"): + # client_options = {"default_headers": {"authorization": f"Bearer {token}"}} + # else: + # client_options = {} + return obs.store.HTTPStore.from_url(base_url, client_options=client_options) + raise NotImplementedError(f"{parsed.scheme} is not yet supported") diff --git a/virtualizarr/readers/__init__.py b/virtualizarr/readers/__init__.py index aa3e4e64..86fcd14f 100644 --- a/virtualizarr/readers/__init__.py +++ b/virtualizarr/readers/__init__.py @@ -4,6 +4,7 @@ from virtualizarr.readers.hdf5 import HDF5VirtualBackend from virtualizarr.readers.kerchunk import KerchunkVirtualBackend from virtualizarr.readers.netcdf3 import NetCDF3VirtualBackend +from virtualizarr.readers.safetensors import SafeTensorsVirtualBackend from virtualizarr.readers.tiff import TIFFVirtualBackend __all__ = [ @@ -13,5 +14,6 @@ "HDF5VirtualBackend", "KerchunkVirtualBackend", "NetCDF3VirtualBackend", + "SafeTensorsVirtualBackend", "TIFFVirtualBackend", ] diff --git a/virtualizarr/readers/safetensors.py b/virtualizarr/readers/safetensors.py new file mode 100644 index 00000000..6b4ea00e --- /dev/null +++ b/virtualizarr/readers/safetensors.py @@ -0,0 +1,671 @@ +import json +import struct +from collections.abc import Iterable, Mapping +from pathlib import Path +from typing import TYPE_CHECKING, Any, Dict, Optional +from urllib.parse import urlparse + +import numpy as np +from xarray import Dataset, Index + +from virtualizarr.manifests import ( + ChunkEntry, + ChunkManifest, + ManifestArray, + ManifestGroup, + ManifestStore, + ObjectStoreRegistry, +) +from virtualizarr.manifests.store import default_object_store +from virtualizarr.manifests.utils import create_v3_array_metadata +from virtualizarr.readers.api import VirtualBackend +from virtualizarr.types import ChunkKey + +if TYPE_CHECKING: + from obstore.store import ( + ObjectStore, # type: ignore[import-not-found] + ) + + +class SafeTensorsVirtualBackend(VirtualBackend): + """ + Backend for reading SafeTensors files as virtual datasets. + + SafeTensors is a format for safely storing tensors (multidimensional arrays), + without using pickle, and with zero-copy access. It is commonly used for storing + model weights in the fields of ML and AI. + + The format consists of: + - 8 bytes (header size): unsigned little-endian 64-bit integer containing the size of the header + - N bytes (header): a JSON UTF-8 string containing tensor metadata + - Rest of the file: byte-buffer containing tensor data + + Examples + -------- + Open a local SafeTensors file with default settings: + + >>> vds = open_virtual_dataset("model_weights.safetensors") + + Open a SafeTensors file with custom dimension names: + + >>> custom_dims = {"weight": ["input_dims", "output_dims"], "bias": ["output_dims"]} + >>> vds = open_virtual_dataset( + ... "model_weights.safetensors", + ... virtual_backend_kwargs={"dimension_names": custom_dims} + ... ) + + Open a GPT-2 model from Hugging Face Hub: + + >>> vds = open_virtual_dataset( + ... "https://huggingface.co/openai-community/gpt2/resolve/main/model.safetensors" + ... ) + >>> # Access various GPT-2 tensors + >>> print(vds["wte.weight"].shape) # Word Token Embeddings: (50257, 768) + >>> print(vds["wpe.weight"].shape) # Word Position Embeddings: (1024, 768) + >>> print(vds["ln_f.weight"].shape) # Final layer norm weight + """ + + @staticmethod + def _parse_safetensors_header( + filepath: str, store: "ObjectStore" + ) -> tuple[dict[str, Any], int]: + """ + Parse the header of a SafeTensors file to extract metadata. + + This method reads the header of a SafeTensors file which contains: + 1. Header size (8 bytes): uint64 little-endian indicating header length + 2. Header content (variable length): JSON-encoded metadata describing tensors + + The header metadata includes tensor names, data types, shapes, and byte offsets. + + Parameters + ---------- + filepath : str + Path to the SafeTensors file. Can be a local path or a URL. + store : ObjectStore + Object store to use for reading the file. Should be compatible with + the filepath type (LocalStore for local files, HTTPStore for URLs, etc.). + + Returns + ------- + tuple[dict[str, Any], int] + A tuple containing: + - header (dict): Parsed JSON header containing tensor metadata + including names, dtypes, shapes, and data_offsets + - header_size (int): Size of the header in bytes + + Examples + -------- + The returned header might look like: + { + "weight": { + "dtype": "F32", + "shape": [10, 20], + "data_offsets": [0, 800] + }, + "bias": { + "dtype": "F32", + "shape": [20], + "data_offsets": [800, 880] + }, + "__metadata__": { + "framework": "pytorch", + "version": "2.0" + } + } + """ + from virtualizarr.utils import ObstoreReader + + reader = ObstoreReader(store, filepath) + + # 8 bytes, uint64 little-endian + header_size_bytes = reader.read(8) + header_size = struct.unpack(" ManifestGroup: + """ + Create a ManifestGroup from a SafeTensors file. + + This method reads the SafeTensors header, parses tensor metadata, and creates + ManifestArrays for each tensor. Each tensor is treated as a single chunk for + efficient memory-mapped access. + + Parameters + ---------- + filepath : str + Path to the SafeTensors file. Can be a local path or a URL. + drop_variables : list + List of tensor names to exclude from the dataset. + store: ObjectStore + Object store used for reading the file. Should match the filepath type. + dimension_names : Dict[str, list[str]], optional + Custom dimension names for specific tensors. The keys should be tensor names, + and the values should be lists of dimension names matching the tensor's shape. + If not provided, default names are generated as "{tensor_name}_dim_{i}". + + Returns + ------- + ManifestGroup + A group containing ManifestArrays for each tensor, with appropriate + metadata and attributes. + + Notes + ----- + - Each tensor is represented as a single chunk for direct memory access + - The __metadata__ field from the SafeTensors header is preserved as attributes + - Tensor metadata includes: + - Original SafeTensors dtype (e.g., "F32", "I64", "BF16") + - Storage information indicating contiguous layout + - Any additional tensor-specific metadata from the header + + Examples + -------- + >>> store = default_object_store("model.safetensors") + >>> dimension_names = { + ... "weight": ["input_channels", "output_channels"], + ... "bias": ["features"] + ... } + >>> manifest_group = _create_manifest_group( + ... "model.safetensors", + ... drop_variables=[], + ... store=store, + ... dimension_names=dimension_names + ... ) + """ + header, header_size = SafeTensorsVirtualBackend._parse_safetensors_header( + filepath, store + ) + + manifest_dict = {} + + attrs = {} + if "__metadata__" in header: + metadata_content = header["__metadata__"] + if isinstance(metadata_content, dict): + for key, value in metadata_content.items(): + # safetensors spec only allows text-to-text map in __metadata__ + if not isinstance(value, str): + value = json.dumps(value) + attrs[key] = value + else: + attrs["__metadata__"] = ( + metadata_content + if isinstance(metadata_content, str) + else json.dumps(metadata_content) + ) + + data_start = 8 + header_size + + def should_skip_tensor(tensor_name: str, drop_variables: list) -> bool: + return tensor_name == "__metadata__" or tensor_name in drop_variables + + for tensor_name, tensor_info in header.items(): + if should_skip_tensor(tensor_name, drop_variables): + continue + + dtype_str = tensor_info["dtype"] + shape = tuple(tensor_info["shape"]) + # data offsets relative to end of header + start_offset, end_offset = tensor_info["data_offsets"] + + dtype = SafeTensorsVirtualBackend._map_dtype(dtype_str) + + abs_start = data_start + start_offset + abs_end = data_start + end_offset + + chunk_manifest = SafeTensorsVirtualBackend._create_chunk_manifest( + filepath=filepath, + offset=abs_start, + length=abs_end - abs_start, + shape=shape, + ) + + if dimension_names and tensor_name in dimension_names: + custom_names = dimension_names[tensor_name] + if len(custom_names) != len(shape): + raise ValueError( + f"Provided dimension names for '{tensor_name}' has {len(custom_names)} " + f"names, but tensor has {len(shape)} dimensions." + ) + dim_names = custom_names + else: + dim_names = [f"{tensor_name}_dim_{i}" for i in range(len(shape))] + + tensor_attrs = {} + # not clear to me from the spec if additional keys allowed, parse just in case + for key, value in tensor_info.items(): + if key not in {"dtype", "shape", "data_offsets"}: + tensor_attrs[key] = ( + value if isinstance(value, str) else json.dumps(value) + ) + + tensor_attrs["original_safetensors_dtype"] = dtype_str + + tensor_attrs["safetensors_storage_info"] = json.dumps( + { + "chunked": False, + "contiguous": True, + } + ) + + metadata = create_v3_array_metadata( + shape=shape, + data_type=dtype, + chunk_shape=shape, # Treat the whole tensor as a single chunk + dimension_names=dim_names, + attributes=tensor_attrs, + ) + + manifest_array = ManifestArray( + metadata=metadata, + chunkmanifest=chunk_manifest, + ) + + manifest_dict[tensor_name] = manifest_array + + return ManifestGroup(arrays=manifest_dict, attributes=attrs) + + @staticmethod + def _create_manifest_store( + filepath: str, + drop_variables: list, + dimension_names: Optional[Dict[str, list[str]]] = None, + revision: Optional[str] = None, + ) -> ManifestStore: + """ + Create a ManifestStore for a SafeTensors file. + + This method handles the complete workflow of reading a SafeTensors file and + creating a virtual store. It automatically determines the appropriate store type + based on the filepath and handles Hugging Face Hub URLs specially. + + Parameters + ---------- + filepath : str + Path to the SafeTensors file. Can be: + - Local filesystem path (e.g., "/path/to/file.safetensors") + - HTTP/HTTPS URL (e.g., "https://huggingface.co/.../file.safetensors") + drop_variables : list + List of tensor names to exclude from the dataset. + dimension_names : Dict[str, list[str]], optional + Custom dimension names for specific tensors. Keys are tensor names, + values are lists of dimension names matching the tensor's shape. + revision : str, optional + Repository revision (branch, tag, or commit hash) for Hugging Face Hub. + Defaults to "main" if not specified. + + Returns + ------- + ManifestStore + A store containing virtual references to tensors with metadata, + optimized for the detected storage backend. + + Notes + ----- + For Hugging Face Hub URLs, this method: + - Automatically inserts the correct API format with "/resolve/{revision}/" + + Examples + -------- + Local file: + >>> store = _create_manifest_store( + ... "model.safetensors", + ... drop_variables=["optimizer_state"] + ... ) + + Hugging Face Hub file: + >>> store = _create_manifest_store( + ... "https://huggingface.co/openai-community/gpt2/model.safetensors", + ... drop_variables=[], + ... revision="v2.0" + ... ) + """ + from obstore.store import ( + HTTPStore, + LocalStore, + ) + + store_registry = ObjectStoreRegistry() + store = default_object_store(filepath) + + if not revision: + revision = "main" + + # file is on the hub and not local + if isinstance(store, HTTPStore): + path = urlparse(filepath).path + + # Check if path already contains '/resolve/' - if so, don't modify it + if "/resolve/" not in path: + # HF API requires insertion of 'resolve' + '{revision}' after repo name + filepath = "/".join( + path.split("/")[0:3] + + ["resolve", f"{revision}"] + + path.split("/")[3:] + ) + + # obstore and virtualizarr require absolute paths + if isinstance(store, LocalStore): + filepath = str(Path(urlparse(filepath).path).resolve()) + + store_registry.register_store(filepath, store) + + manifest_group = SafeTensorsVirtualBackend._create_manifest_group( + filepath=filepath, + drop_variables=drop_variables, + dimension_names=dimension_names, + store=store, + ) + + return ManifestStore(group=manifest_group, store_registry=store_registry) + + @staticmethod + def open_virtual_dataset( + filepath: str, + group: str | None = None, + drop_variables: Iterable[str] | None = None, + loadable_variables: Iterable[str] | None = None, + decode_times: bool | None = None, + indexes: Mapping[str, Index] | None = None, + virtual_backend_kwargs: Optional[dict] = None, + reader_options: Optional[dict] = None, + ) -> Dataset: + """ + Open a SafeTensors file as a virtual dataset. + + SafeTensors is a format used primarily for storing ML and AI model weights + and parameters. This method creates a virtual xarray Dataset where each tensor + becomes a separate variable, without loading the data into memory. + + Parameters + ---------- + filepath : str + Path to the SafeTensors file. Can be a local path or a Hugging Face Hub URL. + For Hugging Face Hub, use format: + "https://huggingface.co/{username}/{repo_name}/{filename}" + group : str, optional + Not used for SafeTensors files as they don't have hierarchical structure. + drop_variables : Iterable[str], optional + Names of tensors to exclude from the dataset. Useful for skipping large + or unnecessary tensors. + loadable_variables : Iterable[str], optional + Variables to load as lazy numpy/dask arrays instead of ManifestArrays. + These will be loaded on-demand when accessed. + decode_times : bool, optional + Not applicable for SafeTensors files (no time encoding). + indexes : Mapping[str, Index], optional + Custom indexes to attach to the returned Dataset. + virtual_backend_kwargs : dict, optional + Additional keyword arguments for the SafeTensors backend: + + - dimension_names : Dict[str, list[str]], optional + Custom dimension names for specific tensors. The keys should be tensor names, + and the values should be lists of dimension names matching the tensor's shape. + Example: {"weight": ["input_dims", "output_dims"]} for a 2D weight tensor. + If not provided, defaults to "{tensor_name}_dim_{i}". + + - revision : str, optional + Repository revision for Hugging Face Hub (branch/tag/commit). + Defaults to "main" if not specified. + + reader_options : dict, optional + Not supported for SafeTensors files. + + Returns + ------- + xr.Dataset + A virtual dataset where: + - Each tensor becomes a separate variable + - Metadata from the SafeTensors header becomes dataset attributes + + Raises + ------ + ValueError + If group parameter is provided (not supported for SafeTensors) + NotImplementedError + If unsupported virtual_backend_kwargs or reader_options are provided + + Examples + -------- + Open a local SafeTensors file: + >>> vds = open_virtual_dataset("model.safetensors") + >>> print(vds.variables.keys()) + ['weight', 'bias', 'embedding.weight'] + + Open with custom dimension names: + >>> dims = { + ... "weight": ["hidden_size", "output_size"], + ... "bias": ["output_size"] + ... } + >>> vds = open_virtual_dataset( + ... "model.safetensors", + ... virtual_backend_kwargs={"dimension_names": dims} + ... ) + >>> print(vds["weight"].dims) + ('hidden_size', 'output_size') + + Open from Hugging Face Hub: + >>> vds = open_virtual_dataset( + ... "https://huggingface.co/openai-community/gpt2/model.safetensors" + ... ) + >>> print(vds["wte.weight"].shape) # Word token embeddings + (50257, 768) + + Load specific tensors as lazy arrays: + >>> vds = open_virtual_dataset( + ... "model_weights.safetensors", + ... loadable_variables=["small_tensor"], + ... drop_variables=["large_optimizer_state"] + ... ) + + Notes + ----- + - Each tensor is treated as a single chunk for optimal access patterns + """ + if group is not None: + raise ValueError("group parameter is not supported for SafeTensors files") + + dimension_names = None + revision = "main" # Default to main branch + + if virtual_backend_kwargs: + if "dimension_names" in virtual_backend_kwargs: + dimension_names = virtual_backend_kwargs.pop("dimension_names") + + if "revision" in virtual_backend_kwargs: + revision = virtual_backend_kwargs.pop("revision") + + if virtual_backend_kwargs: + raise NotImplementedError( + f"SafeTensors reader does not support the following virtual_backend_kwargs: {list(virtual_backend_kwargs.keys())}" + ) + + if reader_options: + raise NotImplementedError( + "SafeTensors reader does not support non-empty reader_options." + ) + + _drop_vars = [] if drop_variables is None else list(drop_variables) + + manifest_store = SafeTensorsVirtualBackend._create_manifest_store( + filepath=filepath, + drop_variables=_drop_vars, + dimension_names=dimension_names, + revision=revision, + ) + + ds = manifest_store.to_virtual_dataset( + loadable_variables=loadable_variables, + decode_times=decode_times, + indexes=indexes, + ) + return ds + + @staticmethod + def _create_chunk_manifest( + filepath: str, + offset: int, + length: int, + shape: tuple[int, ...], + ) -> ChunkManifest: + """ + Create a ChunkManifest for a tensor in a SafeTensors file. + + SafeTensors files store tensors as contiguous binary data. This method creates + a chunk manifest that points to the exact location of a tensor within the file, + treating the entire tensor as a single chunk for efficient memory mapping. + + The structure of the variable names within a Safetensors file often reflects a + hierarchical organization, commonly represented using a dot separator (e.g., + 'a.b.c'). While this structure could naturally map to a nested format like Zarr + groups (e.g., a/b/c), the dominant framework for using these models, PyTorch, + utilizes a flattened dictionary structure (a 'state dict') where these dot-separated + names serve as keys. + + To ease integration with PyTorch's expected format, ChunkManifests are currently a + flattened dictionary where the keys are the dot-separated variable names. + + Further consideration could be given to optionally returning the data as an + xarray.DataTree to better represent the inherent hierarchical structure, but + this has been deferred to prioritize compatibility with PyTorch workflows. + + Parameters + ---------- + filepath : str + Path to the SafeTensors file where the tensor data is stored. + offset : int + Byte offset from the start of the file where the tensor data begins. + This is calculated as: header_size + 8 bytes + tensor_data_offset. + length : int + Length of the tensor data in bytes. Calculated from data_offsets + as: end_offset - start_offset. + shape : tuple[int, ...] + Shape of the tensor. Used to determine the dimensionality for + creating chunk keys. + + Returns + ------- + ChunkManifest + A ChunkManifest object containing a single chunk entry that references + the tensor's location within the file. + + Notes + ----- + - Each tensor is represented as a single chunk, regardless of size + - Chunk keys are generated as: "0" for 1D, "0.0" for 2D, "0.0.0" for 3D, etc. + + Examples + -------- + For a 3D tensor (shape=(10, 20, 30)) starting at byte 1024 with length 24000: + >>> manifest = _create_chunk_manifest("model.safetensors", 1024, 24000, (10, 20, 30)) + >>> print(manifest.entries.keys()) + dict_keys(['0.0.0']) + """ + # Create a single chunk key (e.g., "0" for a 1D tensor, "0.0" for a 2D tensor) + key_parts = ["0"] * (len(shape) or 1) + chunk_key = ChunkKey(".".join(key_parts)) + + chunk_entry = ChunkEntry.with_validation( # type: ignore[attr-defined] + path=filepath, + offset=offset, + length=length, + ) + + chunk_entries = {chunk_key: chunk_entry} + + return ChunkManifest(entries=chunk_entries) + + @staticmethod + def _map_dtype(dtype_str: str) -> np.dtype: + """ + Map SafeTensors dtype string to NumPy dtype. + + SafeTensors uses its own dtype naming convention that needs to be mapped + to NumPy dtypes for use in xarray. This method performs that mapping, + supporting all standard dtypes including ML-specific types like BF16. + + Parameters + ---------- + dtype_str : str + SafeTensors dtype string. Valid values are: + - Integer types: "I8", "I16", "I32", "I64" + - Unsigned types: "U8", "U16", "U32", "U64" + - Float types: "F16", "F32", "F64" + - ML types: "BF16" (bfloat16), "F8_E5M2", "F8_E4M3" (float8 variants) + - Boolean: "BOOL" + + Returns + ------- + np.dtype + Corresponding NumPy dtype object that can be used with xarray. + + Raises + ------ + ValueError + If the provided dtype_str is not recognized or supported. + + Notes + ----- + - BF16, F8_E5M2, and F8_E4M3 require the ml_dtypes package + + Examples + -------- + >>> dtype = _map_dtype("F32") + >>> print(dtype) + float32 + + >>> dtype = _map_dtype("BOOL") + >>> print(dtype) + bool + + >>> dtype = _map_dtype("BF16") + >>> print(dtype) + bfloat16 + """ + try: + # this import will register new numpy dtypes as a side-effect e.g. bfloat16 + import ml_dtypes # noqa: F401 + except ImportError: + raise ImportError( + "The ml_dtypes package is required to read safetensors files. Please install it with pip install virtualizarr[safetensors]." + ) + + dtype_map = { + "BOOL": np.dtype("bool"), + "U8": np.dtype("uint8"), + "I8": np.dtype("int8"), + "I16": np.dtype("int16"), + "U16": np.dtype("uint16"), + "I32": np.dtype("int32"), + "U32": np.dtype("uint32"), + "I64": np.dtype("int64"), + "U64": np.dtype("uint64"), + "F16": np.dtype("float16"), + "BF16": np.dtype( + "bfloat16" + ), # TO DO: broken until zarr supports dtype extensions + "F32": np.dtype("float32"), + "F64": np.dtype("float64"), + "F8_E5M2": np.dtype( + "float8_e5m2" + ), # TO DO: broken until zarr supports dtype extensions + "F8_E4M3": np.dtype( + "float8_e4m3" + ), # TO DO: broken until zarr supports dtype extensions + } + + if dtype_str not in dtype_map: + raise ValueError(f"Unsupported SafeTensors dtype: {dtype_str}") + + return dtype_map[dtype_str] diff --git a/virtualizarr/tests/test_readers/test_safetensors.py b/virtualizarr/tests/test_readers/test_safetensors.py new file mode 100644 index 00000000..7aa9a397 --- /dev/null +++ b/virtualizarr/tests/test_readers/test_safetensors.py @@ -0,0 +1,348 @@ +""" +Tests for the SafeTensors reader in VirtualiZarr. +""" + +import json +import os +import tempfile + +import numpy as np +import pytest + +pytest.importorskip("safetensors") + +from safetensors.numpy import save_file + +from virtualizarr import open_virtual_dataset +from virtualizarr.backend import FileType +from virtualizarr.readers.safetensors import SafeTensorsVirtualBackend +from virtualizarr.tests import requires_network + + +@pytest.fixture +def sample_safetensors_file(tmp_path): + """Create a sample SafeTensors file for testing.""" + + filepath = str(tmp_path / "test.safetensors") + + # TO DO: after zarr supports dtype extensions, test bfloat16 here + tensors = { + "tensor1": np.ones((10, 10), dtype=np.float32), + "tensor2": np.zeros((5, 5), dtype=np.float32), + "tensor3": np.arange(100, dtype=np.float32).reshape(10, 10), + } + + metadata = { + "framework": "numpy", + "version": "1.0", + "created_by": "virtualizarr_test", + } + + save_file(tensors, filepath, metadata=metadata) + + yield filepath + + +@pytest.fixture +def sample_safetensors_file_with_complex_metadata(tmp_path): + """Create a sample SafeTensors file with complex metadata for testing.""" + + filepath = str(tmp_path / "test.safetensors") + + tensors = { + "weight": np.random.randn(10, 20).astype(np.float32), + "bias": np.random.randn(20).astype(np.float32), + } + + # Convert complex nested metadata to strings as required by safetensors + model_info = { + "name": "test_model", + "version": "1.0", + "parameters": 220, + "layers": [ + {"name": "layer1", "type": "linear"}, + {"name": "layer2", "type": "activation"}, + ], + } + training = {"epochs": 10, "optimizer": "adam", "learning_rate": 0.001} + + metadata = { + "model_info": json.dumps(model_info), + "training": json.dumps(training), + } + + save_file(tensors, filepath, metadata=metadata) + + yield filepath + + +def test_reader_registration(): + """Test that the SafeTensors reader is properly registered in VirtualiZarr.""" + from virtualizarr.backend import VIRTUAL_BACKENDS + + assert "safetensors" in VIRTUAL_BACKENDS + assert VIRTUAL_BACKENDS["safetensors"] == SafeTensorsVirtualBackend + + +def test_file_detection(sample_safetensors_file): + """Test that VirtualiZarr correctly identifies the file as a SafeTensors file.""" + from virtualizarr.backend import automatically_determine_filetype + + filetype = automatically_determine_filetype(filepath=sample_safetensors_file) + assert filetype == FileType.safetensors + + +def test_open_virtual_dataset(sample_safetensors_file): + """Test opening a SafeTensors file as a virtual dataset.""" + vds = open_virtual_dataset(sample_safetensors_file) + + assert set(vds.variables) == {"tensor1", "tensor2", "tensor3"} + + assert vds["tensor1"].shape == (10, 10) + assert vds["tensor2"].shape == (5, 5) + assert vds["tensor3"].shape == (10, 10) + + +def test_tensor_values(sample_safetensors_file): + """Test that the tensor values are correctly read.""" + + loadable_variables = ["tensor1", "tensor2", "tensor3"] + vds = open_virtual_dataset( + sample_safetensors_file, loadable_variables=loadable_variables + ) + + tensor1 = vds["tensor1"].values + tensor2 = vds["tensor2"].values + tensor3 = vds["tensor3"].values + + expected_tensor1 = np.ones((10, 10)) + expected_tensor2 = np.zeros((5, 5)) + expected_tensor3 = np.arange(100).reshape(10, 10) + + np.testing.assert_array_equal(tensor1, expected_tensor1) + np.testing.assert_array_equal(tensor2, expected_tensor2) + np.testing.assert_array_equal(tensor3, expected_tensor3) + + +def test_custom_dimension_names(sample_safetensors_file): + """Test specifying custom dimension names when opening a SafeTensors file.""" + + custom_dims = {"tensor1": ["dim_x", "dim_y"], "tensor2": ["height", "width"]} + + vds = open_virtual_dataset( + sample_safetensors_file, virtual_backend_kwargs={"dimension_names": custom_dims} + ) + + assert vds["tensor1"].dims == ("dim_x", "dim_y") + assert vds["tensor2"].dims == ("height", "width") + # tensor3 should use default dimension names + assert vds["tensor3"].dims == ("tensor3_dim_0", "tensor3_dim_1") + + +def test_invalid_dimension_names(sample_safetensors_file): + """Test error handling for invalid dimension names.""" + + invalid_dims = { + "tensor1": ["dim_x", "dim_y", "dim_z"] # tensor1 is 2D but we provide 3 names + } + + with pytest.raises(ValueError, match="has 3 names, but tensor has 2 dimensions"): + _ = open_virtual_dataset( + sample_safetensors_file, + virtual_backend_kwargs={"dimension_names": invalid_dims}, + ) + + +def test_metadata_preservation(sample_safetensors_file): + """Test preservation of metadata from SafeTensors file.""" + + vds = open_virtual_dataset(sample_safetensors_file) + + assert vds.attrs["framework"] == "numpy" + assert vds.attrs["version"] == "1.0" + assert vds.attrs["created_by"] == "virtualizarr_test" + + +def test_complex_metadata(sample_safetensors_file_with_complex_metadata): + """Test handling of complex nested metadata.""" + + vds = open_virtual_dataset(sample_safetensors_file_with_complex_metadata) + + assert "model_info" in vds.attrs + assert "training" in vds.attrs + + model_info = json.loads(vds.attrs["model_info"]) + training = json.loads(vds.attrs["training"]) + + assert model_info["name"] == "test_model" + assert model_info["parameters"] == 220 + assert training["epochs"] == 10 + assert training["optimizer"] == "adam" + + assert vds["weight"].shape == (10, 20) + assert vds["bias"].shape == (20,) + + assert vds["weight"].attrs["original_safetensors_dtype"] == "F32" + assert vds["bias"].attrs["original_safetensors_dtype"] == "F32" + + +def test_enhanced_metadata(sample_safetensors_file): + """Test that enhanced metadata is correctly created and preserved.""" + + vds = open_virtual_dataset(sample_safetensors_file) + + assert vds["tensor1"].attrs["original_safetensors_dtype"] == "F32" + assert vds["tensor2"].attrs["original_safetensors_dtype"] == "F32" + assert vds["tensor3"].attrs["original_safetensors_dtype"] == "F32" + + +def test_relative_path_handling(sample_safetensors_file): + """Test that relative paths to SafeTensors files are handled correctly.""" + import os + + # Get the full path and then compute a relative path from current directory + full_path = sample_safetensors_file + current_dir = os.getcwd() + + # Make the relative path by removing the common prefix + relative_path = os.path.relpath(full_path, current_dir) + + # First test without loading data + vds_metadata = open_virtual_dataset(relative_path) + + # Verify that the content is accessible + assert set(vds_metadata.variables) == {"tensor1", "tensor2", "tensor3"} + assert vds_metadata["tensor1"].shape == (10, 10) + assert vds_metadata["tensor2"].shape == (5, 5) + assert vds_metadata["tensor3"].shape == (10, 10) + + # Now test with loadable_variables to actually access the data + vds = open_virtual_dataset(relative_path, loadable_variables=["tensor1"]) + + # Verify we can access the data + tensor1 = vds["tensor1"].values + np.testing.assert_array_equal(tensor1, np.ones((10, 10))) + + +@pytest.fixture +def sample_safetensors_file_with_many_small_tensors(): + """Create a sample SafeTensors file with many small tensors for testing.""" + + with tempfile.NamedTemporaryFile(suffix=".safetensors", delete=False) as tmpfile: + filepath = tmpfile.name + + # Create 1000 small tensors + tensors = { + f"tensor_{i}": np.random.randn(10, 10).astype(np.float32) + for i in range(1000) + } + + metadata = { + "description": "many_small_tensors", + "tensor_count": "1000", + "shape": "(10, 10)", + "dtype": "float32", + } + + save_file(tensors, filepath, metadata=metadata) + + yield filepath + + os.unlink(filepath) + + +def test_many_small_tensors(sample_safetensors_file_with_many_small_tensors): + """Test handling a SafeTensors file with many small tensors.""" + + vds = open_virtual_dataset(sample_safetensors_file_with_many_small_tensors) + + # Check that all 1000 tensors are present + assert len(vds.variables) == 1000 + + # Check that all tensors have the expected names + expected_names = {f"tensor_{i}" for i in range(1000)} + assert set(vds.variables) == expected_names + + # Check metadata preservation + assert vds.attrs["description"] == "many_small_tensors" + assert vds.attrs["tensor_count"] == "1000" + assert vds.attrs["shape"] == "(10, 10)" + assert vds.attrs["dtype"] == "float32" + + # Check shape of several tensors + for i in [0, 500, 999]: # First, middle, and last tensor + tensor_name = f"tensor_{i}" + assert vds[tensor_name].shape == (10, 10) + assert vds[tensor_name].attrs["original_safetensors_dtype"] == "F32" + + # Load a few tensors to check values + loadable_variables = [f"tensor_{i}" for i in range(3)] # Load first 3 tensors + vds_with_data = open_virtual_dataset( + sample_safetensors_file_with_many_small_tensors, + loadable_variables=loadable_variables, + ) + + for i in range(3): + tensor_name = f"tensor_{i}" + tensor_data = vds_with_data[tensor_name].values + assert tensor_data.shape == (10, 10) + assert tensor_data.dtype == np.float32 + # Check that data is not all zeros + assert not np.all(tensor_data == 0) + + +@requires_network +def test_open_huggingface_safetensors(): + """Test opening a SafeTensors file from Hugging Face Hub with metadata inspection only.""" + + # This is a widely-downloaded GPT-2 model (500mb) file from Hugging Face Hub with standard dtypes + vds = open_virtual_dataset( + "https://huggingface.co/openai-community/gpt2/resolve/main/model.safetensors" + ) + + assert len(vds.variables) > 0 + + # Check that some expected GPT-2 parameter tensors are present + # These are common parameter names in GPT-2 models + expected_tensors = [ + "wte.weight", # Word Token Embeddings + "wpe.weight", # Word Position Embeddings + "ln_f.weight", # Final layer norm weight + ] + + for tensor_name in expected_tensors: + assert tensor_name in vds.variables + + for tensor_name in expected_tensors: + var = vds[tensor_name] + assert hasattr(var, "shape") + assert hasattr(var, "dtype") + assert "original_safetensors_dtype" in var.attrs + + # Check wte.weight shape - should be (vocab_size, embedding_dim) + wte_shape = vds["wte.weight"].shape + assert len(wte_shape) == 2 + assert wte_shape[0] == 50257 # GPT-2 vocabulary size + assert wte_shape[1] == 768 # GPT-2 embedding dimension + + # Check wpe.weight shape - should be (context_length, embedding_dim) + wpe_shape = vds["wpe.weight"].shape + assert len(wpe_shape) == 2 + assert wpe_shape[0] == 1024 # GPT-2 context length + assert wpe_shape[1] == 768 # GPT-2 embedding dimension + + # Test loading a small subset of data from a large tensor + # Only load a small slice to avoid downloading the entire 500MB file + loadable_vds = open_virtual_dataset( + "https://huggingface.co/openai-community/gpt2/model.safetensors", + loadable_variables=["wpe.weight"], # shape (1024, 768) + ) + + # Get a small subset from the word positin embeddings: rows 1000-1100, columns 100-200 + subset = loadable_vds["wpe.weight"][1000:1100, 100:200].values + assert subset.shape == (24, 100) + assert subset.dtype == np.float32 + + # Verify that we actually got data (not all zeros or NaNs) + assert not np.all(subset == 0) + assert not np.any(np.isnan(subset))