httpx.ConnectError: [Errno 111] Connection refused

[prjemian]

When running the new saving tutorial in bluesky-tiled-plugins, received the httpx:ConnectError.

Installation notes

conda create -y -n test python=3.13
conda activate test
pip install bluesky-tiled-plugins "tiled[server]" bluesky ophyd

Exception

(test) jemian@wow ~/.../Bluesky/hklpy2 $ python ./dev_i145.py 
Tiled version 0.2.2
Scheduled retry in 0.62 seconds due to ConnectError('[Errno 111] Connection refused') (attempt 1)
Scheduled retry in 0.52 seconds due to ConnectError('[Errno 111] Connection refused') (attempt 2)
Scheduled retry in 0.43 seconds due to ConnectError('[Errno 111] Connection refused') (attempt 3)
Scheduled retry in 1.4 seconds due to ConnectError('[Errno 111] Connection refused') (attempt 4)
Scheduled retry in 2.4 seconds due to ConnectError('[Errno 111] Connection refused') (attempt 5)
Scheduled retry in 4.1 seconds due to ConnectError('[Errno 111] Connection refused') (attempt 6)
Scheduled retry in 5.0 seconds due to ConnectError('[Errno 111] Connection refused') (attempt 7)
Scheduled retry in 5.0 seconds due to ConnectError('[Errno 111] Connection refused') (attempt 8)
Scheduled retry in 5.0 seconds due to ConnectError('[Errno 111] Connection refused') (attempt 9)
Traceback (most recent call last):
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
    yield
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpx/_transports/default.py", line 250, in handle_request
    resp = self._pool.handle_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 256, in handle_request
    raise exc from None
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 236, in handle_request
    response = connection.handle_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 101, in handle_request
    raise exc
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 78, in handle_request
    stream = self._connect(request)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 124, in _connect
    stream = self._network_backend.connect_tcp(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpcore/_backends/sync.py", line 207, in connect_tcp
    with map_exceptions(exc_map):
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/beams1/JEMIAN/Documents/projects/Bluesky/hklpy2/./dev_i145.py", line 48, in <module>
    demo(save_path)
  File "/home/beams1/JEMIAN/Documents/projects/Bluesky/hklpy2/./dev_i145.py", line 33, in demo
    tw = TiledWriter(tiled_client, batch_size=1)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/bluesky_tiled_plugins/writing/tiled_writer.py", line 1013, in __init__
    self.client = client.include_data_sources()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/tiled/client/base.py", line 313, in include_data_sources
    return self.new_variation(include_data_sources=True).refresh()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/tiled/client/base.py", line 211, in refresh
    for attempt in retry_context():
                   ^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/stamina/_core.py", line 525, in __iter__
    for r in _t.Retrying(
             ^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/tenacity/__init__.py", line 445, in __iter__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/tenacity/__init__.py", line 378, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/tenacity/__init__.py", line 420, in exc_check
    raise retry_exc.reraise()
          ^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/tenacity/__init__.py", line 187, in reraise
    raise self.last_attempt.result()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/tiled/client/base.py", line 214, in refresh
    self.context.http_client.get(
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpx/_client.py", line 1053, in get
    return self.request(
           ^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpx/_client.py", line 825, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpx/_client.py", line 914, in send
    response = self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpx/_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpx/_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpx/_client.py", line 1014, in _send_single_request
    response = transport.handle_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/tiled/client/transport.py", line 91, in handle_request
    response = self.transport.handle_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpx/_transports/default.py", line 249, in handle_request
    with map_httpcore_exceptions():
         ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 111] Connection refused

Demo code

"""
Move to tiled writer.  hklpy2 issue 145

conda create -y -n test python=3.13
conda activate test
pip install bluesky-tiled-plugins "tiled[server]" bluesky ophyd

"""

save_path = "/tmp/tiled_data"


def get_client(save_path):
    """Initialize the Tiled server and client."""
    from tiled.server import SimpleTiledServer
    from tiled.client import from_uri

    tiled_server = SimpleTiledServer(readable_storage=[save_path])
    return from_uri(tiled_server.uri)


def demo(save_path):
    """Initialize the RunEngine and subscribe TiledWriter"""
    from bluesky import RunEngine
    import bluesky.plans as bp
    from bluesky_tiled_plugins import TiledWriter
    from ophyd.sim import det
    from ophyd.sim import hw

    tiled_client = get_client(save_path)

    RE = RunEngine()
    tw = TiledWriter(tiled_client, batch_size=1)
    RE.subscribe(tw)

    # Run an experiment collecting internal data
    (uid,) = RE(bp.count([det], 3))
    data = tiled_client[uid]["primary/det"].read()
    print(f"{data=}")

    # Run an experiment collecting external data
    (uid,) = RE(bp.count([hw(save_path=save_path).img], 2))
    data = tiled_client[uid]["primary/img"].read()
    print(f"{data=}")


if __name__ == "__main__":
    demo(save_path)

Conda environment

(test) jemian@wow ~/.../Bluesky/hklpy2 $ conda list
# packages in environment at /home/beams/JEMIAN/.conda/envs/test:
#
# Name                       Version          Build                 Channel
_libgcc_mutex                0.1              conda_forge           conda-forge
_openmp_mutex                4.5              2_gnu                 conda-forge
adbc-driver-manager          1.9.0            pypi_0                pypi
adbc-driver-postgresql       1.9.0            pypi_0                pypi
adbc-driver-sqlite           1.9.0            pypi_0                pypi
aiofiles                     25.1.0           pypi_0                pypi
aiosqlite                    0.21.0           pypi_0                pypi
alembic                      1.17.2           pypi_0                pypi
annotated-doc                0.0.4            pypi_0                pypi
annotated-types              0.7.0            pypi_0                pypi
anyio                        4.12.0           pypi_0                pypi
argon2-cffi                  25.1.0           pypi_0                pypi
argon2-cffi-bindings         25.1.0           pypi_0                pypi
asgi-correlation-id          4.3.4            pypi_0                pypi
asyncpg                      0.31.0           pypi_0                pypi
attrs                        25.4.0           pypi_0                pypi
awkward                      2.8.10           pypi_0                pypi
awkward-cpp                  50               pypi_0                pypi
blosc2                       3.11.1           pypi_0                pypi
bluesky                      1.14.6           pypi_0                pypi
bluesky-tiled-plugins        2.0.0rc3         pypi_0                pypi
bzip2                        1.0.8            hda65f42_8            conda-forge
ca-certificates              2025.11.12       hbd8a1cb_0            conda-forge
cachetools                   6.2.2            pypi_0                pypi
canonicaljson                2.0.0            pypi_0                pypi
certifi                      2025.11.12       pypi_0                pypi
cffi                         2.0.0            pypi_0                pypi
charset-normalizer           3.4.4            pypi_0                pypi
click                        8.3.1            pypi_0                pypi
cloudpickle                  3.1.2            pypi_0                pypi
cryptography                 46.0.3           pypi_0                pypi
cycler                       0.12.1           pypi_0                pypi
dask                         2025.11.0        pypi_0                pypi
donfig                       0.8.1.post1      pypi_0                pypi
duckdb                       1.3.2            pypi_0                pypi
ecdsa                        0.19.1           pypi_0                pypi
entrypoints                  0.4              pypi_0                pypi
et-xmlfile                   2.0.0            pypi_0                pypi
event-model                  1.23.1           pypi_0                pypi
fastapi                      0.123.0          pypi_0                pypi
flexcache                    0.3              pypi_0                pypi
flexparser                   0.4              pypi_0                pypi
fsspec                       2025.10.0        pypi_0                pypi
google-crc32c                1.7.1            pypi_0                pypi
greenlet                     3.2.4            pypi_0                pypi
h11                          0.16.0           pypi_0                pypi
h5netcdf                     1.7.3            pypi_0                pypi
h5py                         3.15.1           pypi_0                pypi
hdf5plugin                   6.0.0            pypi_0                pypi
historydict                  1.2.6            pypi_0                pypi
httpcore                     1.0.9            pypi_0                pypi
httptools                    0.7.1            pypi_0                pypi
httpx                        0.28.1           pypi_0                pypi
idna                         3.11             pypi_0                pypi
importlib-metadata           8.7.0            pypi_0                pypi
importlib-resources          6.5.2            pypi_0                pypi
jinja2                       3.1.6            pypi_0                pypi
jmespath                     1.0.1            pypi_0                pypi
json-merge-patch             0.3.0            pypi_0                pypi
jsonpatch                    1.33             pypi_0                pypi
jsonpointer                  3.0.0            pypi_0                pypi
jsonschema                   4.25.1           pypi_0                pypi
jsonschema-specifications    2025.9.1         pypi_0                pypi
ld_impl_linux-64             2.45             default_hbd61a6d_104  conda-forge
libexpat                     2.7.3            hecca717_0            conda-forge
libffi                       3.5.2            h9ec8514_0            conda-forge
libgcc                       15.2.0           he0feb66_14           conda-forge
libgcc-ng                    15.2.0           h69a702a_14           conda-forge
libgomp                      15.2.0           he0feb66_14           conda-forge
liblzma                      5.8.1            hb9d3cd8_2            conda-forge
libnsl                       2.0.1            hb9d3cd8_1            conda-forge
libsqlite                    3.51.1           h0c1763c_0            conda-forge
libuuid                      2.41.2           he9a06e4_0            conda-forge
libxcrypt                    4.4.36           hd590300_1            conda-forge
libzlib                      1.3.1            hb9d3cd8_2            conda-forge
llvmlite                     0.45.1           pypi_0                pypi
locket                       1.0.0            pypi_0                pypi
lz4                          4.4.5            pypi_0                pypi
mako                         1.3.10           pypi_0                pypi
markdown-it-py               4.0.0            pypi_0                pypi
markupsafe                   3.0.3            pypi_0                pypi
mdurl                        0.1.2            pypi_0                pypi
minio                        7.2.20           pypi_0                pypi
mongoquery                   1.4.3            pypi_0                pypi
msgpack                      1.1.2            pypi_0                pypi
msgpack-numpy                0.4.8            pypi_0                pypi
ncurses                      6.5              h2d0b736_3            conda-forge
ndindex                      1.10.1           pypi_0                pypi
networkx                     3.6              pypi_0                pypi
numba                        0.62.1           pypi_0                pypi
numcodecs                    0.16.5           pypi_0                pypi
numexpr                      2.14.1           pypi_0                pypi
numpy                        2.3.5            pypi_0                pypi
obstore                      0.8.2            pypi_0                pypi
openpyxl                     3.1.5            pypi_0                pypi
openssl                      3.6.0            h26f9b46_0            conda-forge
opentelemetry-api            1.38.0           pypi_0                pypi
ophyd                        1.11.0           pypi_0                pypi
orjson                       3.11.4           pypi_0                pypi
packaging                    25.0             pypi_0                pypi
pandas                       2.3.3            pypi_0                pypi
partd                        1.4.2            pypi_0                pypi
pillow                       12.0.0           pypi_0                pypi
pint                         0.25.2           pypi_0                pypi
pip                          25.3             pyh8b19718_0          conda-forge
platformdirs                 4.5.0            pypi_0                pypi
prometheus-client            0.23.1           pypi_0                pypi
py-cpuinfo                   9.0.0            pypi_0                pypi
pyarrow                      22.0.0           pypi_0                pypi
pyasn1                       0.6.1            pypi_0                pypi
pycparser                    2.23             pypi_0                pypi
pycryptodome                 3.23.0           pypi_0                pypi
pydantic                     2.12.5           pypi_0                pypi
pydantic-core                2.41.5           pypi_0                pypi
pydantic-settings            2.11.0           pypi_0                pypi
pygments                     2.19.2           pypi_0                pypi
python                       3.12.12          hd63d673_1_cpython    conda-forge
python-dateutil              2.9.0.post0      pypi_0                pypi
python-dotenv                1.2.1            pypi_0                pypi
python-jose                  3.5.0            pypi_0                pypi
python-multipart             0.0.20           pypi_0                pypi
pytz                         2025.2           pypi_0                pypi
pyyaml                       6.0.3            pypi_0                pypi
readline                     8.2              h8c095d6_2            conda-forge
redis                        7.1.0            pypi_0                pypi
referencing                  0.37.0           pypi_0                pypi
requests                     2.32.5           pypi_0                pypi
rich                         14.2.0           pypi_0                pypi
rpds-py                      0.30.0           pypi_0                pypi
rsa                          4.9.1            pypi_0                pypi
setuptools                   80.9.0           pyhff2d567_0          conda-forge
shellingham                  1.5.4            pypi_0                pypi
six                          1.17.0           pypi_0                pypi
sparse                       0.17.0           pypi_0                pypi
sqlalchemy                   2.0.44           pypi_0                pypi
stamina                      25.1.0           pypi_0                pypi
starlette                    0.50.0           pypi_0                pypi
tenacity                     9.1.2            pypi_0                pypi
tifffile                     2025.10.16       pypi_0                pypi
tiled                        0.2.2            pypi_0                pypi
tk                           8.6.13           noxft_ha0e22de_103    conda-forge
toolz                        1.1.0            pypi_0                pypi
tqdm                         4.67.1           pypi_0                pypi
typer                        0.20.0           pypi_0                pypi
typing-extensions            4.15.0           pypi_0                pypi
typing-inspection            0.4.2            pypi_0                pypi
tzdata                       2025.2           pypi_0                pypi
tzlocal                      5.3.1            pypi_0                pypi
urllib3                      2.5.0            pypi_0                pypi
uvicorn                      0.38.0           pypi_0                pypi
uvloop                       0.22.1           pypi_0                pypi
watchfiles                   1.1.1            pypi_0                pypi
websockets                   15.0.1           pypi_0                pypi
wheel                        0.45.1           pyhd8ed1ab_1          conda-forge
xarray                       2025.11.0        pypi_0                pypi
zarr                         3.1.5            pypi_0                pypi
zipp                         3.23.0           pypi_0                pypi
zstandard                    0.25.0           pypi_0                pypi
zstd                         1.5.7            h3691f8a_5            conda-forge

[prjemian]

(test) jemian@wow ~/.../Bluesky/hklpy2 $ cat /tmp/tmpoxrx7ptl/error.log
[-] INFO:     Started server process [1370802]
[-] INFO:     Waiting for application startup.
[-] INFO:     Application startup complete.
[-] INFO:     Uvicorn running on http://127.0.0.1:34687 (Press CTRL+C to quit)
[-] INFO:     Shutting down
[-] INFO:     Waiting for application shutdown.
[-] INFO:     Application shutdown complete.
[-] INFO:     Finished server process [1370802]

[prjemian]

(test) jemian@wow /tmp/tmpoxrx7ptl $ ll
total 136K
-rw-rw-r-- 1 jemian bcda  331 Dec  1 14:04 access.log
-rw-r--r-- 1 jemian bcda 128K Dec  1 14:04 catalog.db
drwxrwxr-x 2 jemian bcda    6 Dec  1 14:04 data/
-rw-rw-r-- 1 jemian bcda  484 Dec  1 14:04 error.log
(test) jemian@wow /tmp/tmpoxrx7ptl $ cat access.log 
[af4e315b4c46fadb] 127.0.0.1:45854 (anon) - "GET /api/v1?api_key=32ce9085c70d98b0 HTTP/1.1" 307 Temporary Redirect
[78e3138b8aa5d9d5] 127.0.0.1:45864 (anon) - "GET /api/v1/ HTTP/1.1" 200 OK
[5d39d6277201dbb2] 127.0.0.1:45864 (singleuser) - "GET /api/v1/metadata/ HTTP/1.1" 200 OK

[danielballan]

Thanks for the clear report.

In the tutorial, the SimpleClientServer instance is a global scope. In yours, it is a local variable in a function. I think the garbage collector is garbage-collecting your server and shutting it down. Logs show a couple successful requests processed, followed by a clean shutdown, consistent with this hypothesis.

If you can confirm, we can add a note about this.

[prjemian]

Thanks for the diagnosis. I concur. I've revised the code to avoid a global definition, typical of how I expect to use it. This works in a single, continuous process. It still has timeouts in a (VSCode) source-code debugger with breakpoints at the RE() calls and single-stepping by a human.

"""hklpy2 Issue 145: Switch from databroker to tiled

Conda environment::

    conda create -y -n test -c conda-forge python=3.13
    conda activate test
    pip install bluesky-tiled-plugins tiled[server] bluesky ophyd
"""

save_path = "/tmp/tiled_data"


def run_server(save_path):
    """Run a SimpleTiledServer and return a client."""
    from tiled.client import from_uri
    from tiled.server import SimpleTiledServer

    with SimpleTiledServer(readable_storage=[save_path]) as server:
        client = from_uri(server.uri)
        yield client


def simple_demo(tiled_client):
    """conda install bluesky ophyd"""
    from bluesky import RunEngine
    import bluesky.plans as bp
    from bluesky_tiled_plugins import TiledWriter
    from ophyd.sim import det
    from ophyd.sim import hw

    # Initialize the RunEngine and subscribe TiledWriter
    RE = RunEngine()
    tw = TiledWriter(tiled_client, batch_size=1)
    RE.subscribe(tw)

    # Run an experiment collecting internal data
    (uid,) = RE(bp.count([det], 3))
    data = tiled_client[uid]["primary/det"].read()
    print(f"{uid=} {data=}")

    # Run an experiment collecting external data
    (uid,) = RE(bp.count([hw(save_path=save_path).img], 2))
    data = tiled_client[uid]["primary/img"].read()
    print(f"{uid=} {data=}")
    
    print(f"{type(tiled_client)=}  {tiled_client=}")


if __name__ == "__main__":
    for tiled_client in run_server(save_path):
        simple_demo(tiled_client)

[prjemian]

This variation (defining both server & client at global level) also generates ReadTimeout retries in the debugger. Works fine when run straight through. Really want that debugger use case ...

"""hklpy2 Issue 145: Switch from databroker to tiled

Conda environment::

    conda create -y -n test -c conda-forge python=3.13
    conda activate test
    pip install bluesky-tiled-plugins tiled[server] bluesky ophyd
"""
from tiled.client import from_uri
from tiled.server import SimpleTiledServer

save_path = "/tmp/tiled_data"
server = SimpleTiledServer(readable_storage=[save_path])
client = from_uri(server.uri)


def run_server(save_path):
    """Run a SimpleTiledServer and return a client."""
    from tiled.client import from_uri
    from tiled.server import SimpleTiledServer

    with SimpleTiledServer(readable_storage=[save_path]) as server:
        client = from_uri(server.uri)
        yield client


def simple_demo(tiled_client):
    """conda install bluesky ophyd"""
    from bluesky import RunEngine
    import bluesky.plans as bp
    from bluesky_tiled_plugins import TiledWriter
    from ophyd.sim import det
    from ophyd.sim import hw

    # Initialize the RunEngine and subscribe TiledWriter
    RE = RunEngine()
    tw = TiledWriter(tiled_client, batch_size=1)
    RE.subscribe(tw)

    # Run an experiment collecting internal data
    (uid,) = RE(bp.count([det], 3))
    data = tiled_client[uid]["primary/det"].read()
    print(f"{uid=} {data=}")

    # Run an experiment collecting external data
    (uid,) = RE(bp.count([hw(save_path=save_path).img], 2))
    data = tiled_client[uid]["primary/img"].read()
    print(f"{uid=} {data=}")
    
    print(f"{type(tiled_client)=}  {tiled_client=}")


if __name__ == "__main__":
    # for tiled_client in run_server(save_path):
    #     simple_demo(tiled_client)
    simple_demo(client)

[prjemian]

The Python process running the most recent variation (without the context-in-generator) stays running. Adding del client; del server at the end allows it to quit gracefully. An alternative to, say, ^C or import sys; sys.exit()...

The ^C choice ends with:

^CException ignored in: <module 'threading' from '/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/threading.py'>
Traceback (most recent call last):
  File "/home/beams/JEMIAN/.conda/envs/test/lib/python3.12/threading.py", line 1624, in _shutdown
    lock.acquire()
KeyboardInterrupt: 

[prjemian]

This version works in single-process and exits properly. I'm concerned how it might fare for a long-running single-process.

"""hklpy2 Issue 145: Switch from databroker to tiled

Conda environment::

    conda create -y -n test -c conda-forge python=3.13
    conda activate test
    pip install bluesky-tiled-plugins tiled[server] bluesky ophyd
"""

from typing import Iterator, Optional
from tiled.client.container import Container

save_path = "/tmp/tiled_data"


def run_server(save_path: Optional[str] = None) -> Iterator[Container]:
    """Run a SimpleTiledServer and return a client."""
    from tiled.client import from_uri
    from tiled.server import SimpleTiledServer

    kwargs = {}
    if save_path is not None:
        kwargs["readable_storage"] = [save_path]

    with SimpleTiledServer(**kwargs) as server:
        client = from_uri(server.uri)
        yield client


def simple_demo(tiled_client: Container):
    """conda install bluesky ophyd"""
    from bluesky import RunEngine
    import bluesky.plans as bp
    from bluesky_tiled_plugins import TiledWriter
    from ophyd.sim import det
    from ophyd.sim import hw

    # Initialize the RunEngine and subscribe TiledWriter
    RE = RunEngine()
    tw = TiledWriter(tiled_client, batch_size=1)
    RE.subscribe(tw)

    # Run an experiment collecting internal data
    (uid,) = RE(bp.count([det], 3))
    data = tiled_client[uid]["primary/det"].read()
    print(f"{uid=} {data=}")

    # Run an experiment collecting external data
    (uid,) = RE(bp.count([hw(save_path=save_path).img], 2))
    data = tiled_client[uid]["primary/img"].read()
    print(f"{uid=} {data=}")

    print(f"{type(tiled_client)=}  {tiled_client=}")


if __name__ == "__main__":
    for tiled_client in run_server(save_path):
        simple_demo(tiled_client)

[prjemian]

New advice is working for me in hklpy2. These two fixtures:

[danielballan]

The fixture looks great.

For a straight-through example, where you don't want to use a context manager, server.close() is also available.