salt-sproxy works with 3004

[network-shark]

Describe the bug
Can't run salt-sproxy

Steps To Reproduce
salt-sproxy -c . -L 'csr1000v' net.cli 'show version'

Expected behavior
Command should return show version

Error

[INFO    ] Runner completed: 20220310220153300412
[DEBUG   ] []
[DEBUG   ] Using importlib_metadata to load entry points
[DEBUG   ] LazyLoaded proxy.execute
[DEBUG   ] Reading configuration from /home/sharky/salt/etc/salt/master
[DEBUG   ] Guessing ID. The id can be explicitly set in /etc/salt/minion
[DEBUG   ] Found minion id from generate_minion_id(): debian-dev.home.lan
[DEBUG   ] Grains refresh requested. Refreshing grains.
[DEBUG   ] Reading configuration from /home/sharky/salt/etc/salt/master
[DEBUG   ] Override  __utils__: <module 'salt.loaded.int.grains.zfs' from '/home/sharky/venv/salt/lib/python3.9/site-packages/salt/grains/zfs.py'>
[DEBUG   ] /etc/resolv.conf: The domain and search keywords are mutually exclusive.
[DEBUG   ] Unable to derive osmajorrelease from osrelease_info '('proxy',)'. The osmajorrelease grain will not be set.
[DEBUG   ] Closing IPCMessageClient instance
[DEBUG   ] Closing IPCMessageClient instance
[DEBUG   ] Closing IPCMessageClient instance
[DEBUG   ] LazyLoaded zfs.is_supported
[DEBUG   ] MasterEvent PUB socket URI: /home/sharky/salt/var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /home/sharky/salt/var/run/salt/master/master_event_pull.ipc
[ERROR   ] Unable to connect pusher: Stream is closed
Traceback (most recent call last):
  File "/home/sharky/venv/salt/lib/python3.9/site-packages/salt/utils/event.py", line 423, in connect_pull
    self.pusher.connect(timeout=timeout)
  File "/home/sharky/venv/salt/lib/python3.9/site-packages/salt/utils/asynchronous.py", line 125, in wrap
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/sharky/venv/salt/lib/python3.9/site-packages/salt/utils/asynchronous.py", line 131, in _target
    result = io_loop.run_sync(lambda: getattr(self.obj, key)(*args, **kwargs))
  File "/home/sharky/venv/salt/lib/python3.9/site-packages/salt/ext/tornado/ioloop.py", line 459, in run_sync
    return future_cell[0].result()
  File "/home/sharky/venv/salt/lib/python3.9/site-packages/salt/ext/tornado/concurrent.py", line 249, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "/home/sharky/venv/salt/lib/python3.9/site-packages/salt/transport/ipc.py", line 342, in _connect
    yield self.stream.connect(sock_addr)
  File "/home/sharky/venv/salt/lib/python3.9/site-packages/salt/ext/tornado/gen.py", line 1056, in run
    value = future.result()
  File "/home/sharky/venv/salt/lib/python3.9/site-packages/salt/ext/tornado/concurrent.py", line 249, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
salt.ext.tornado.iostream.StreamClosedError: Stream is closed
[DEBUG   ] Using importlib_metadata to load entry points
[DEBUG   ] LazyLoaded roots.envs
[DEBUG   ] Could not LazyLoad roots.init: 'roots.init' is not available.
[DEBUG   ] Updating roots fileserver cache
[INFO    ] Syncing roster for environment 'base'
[INFO    ] Loading cache from salt://_roster, for base
[INFO    ] Caching directory '_roster/' for environment 'base'

Versions Report
Print the output from salt-sproxy -V inside the backticks below:

(salt) sharky@debian-dev:~/salt$ salt-sproxy -V
Salt Version:
           Salt: 3004
    Salt SProxy: 2021.6.1

Dependency Versions:
        Ansible: Not Installed
           cffi: 1.15.0
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 3.0.3
     junos-eznc: 2.6.3
       jxmlease: Not Installed
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 1.0.3
         NAPALM: 3.3.1
       ncclient: 0.6.9
        Netmiko: 3.4.0
       paramiko: 2.9.2
      pycparser: 2.21
       pycrypto: Not Installed
   pycryptodome: 3.14.1
         pyeapi: 0.8.4
         pygit2: Not Installed
       PyNetBox: Not Installed
          PyNSO: Not Installed
         Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
   python-gnupg: Not Installed
         PyYAML: 6.0
          PyZMQ: 21.0.2
            scp: 0.14.4
          smmap: Not Installed
        textfsm: 1.1.2
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.3

System Versions:
           dist: debian 11 bullseye
         locale: utf-8
        machine: x86_64
        release: 5.10.0-12-amd64
         system: Linux
        version: Debian GNU/Linux 11 bullseye

Additional context
Add any other context about the problem here.

[network-shark]

Digged a bit further . It works with 3004 if I start the master and than use salt-sproxy to execute command against my sproxy minions.

For me it looks like that something it wrong with the ipc, maybe they changed the order of the master startup procedure ? The connect method

File "/home/sharky/venv/salt/lib/python3.9/site-packages/salt/utils/event.py", line 423, in connect_pull
self.pusher.connect(timeout=timeout)

is not successful , because the file is not there...

[DEBUG ] MasterEvent PULL socket URI: /home/sharky/salt/var/run/salt/master/master_event_pull.ipc

maybe....

[network-shark]

The error indicates that connecting to the ipc socket /home/sharky/salt/var/run/salt/master/master_event_pull.ipc is not working. That is true . There is no socket under var/run/master/master.... . From my understanding this has nothing todo with salt-sproxy . salt-sproxy only calls or loads the master , but I can't find it how this is done? Just the import of import salt.loader ?

[network-shark]

I just found this commit on the salt project.

saltstack/salt#60781

Good news , if I just ignore the errors connecting to devices is not a problem.

I think this error always occurred , but was never visible.

[mirceaulinic]

Hey @network-shark - thanks for digging into this. I'll reopen this issue, as I'll try to work around that error.. it may not bother you, but it will surely bother me. 🙃