Cannot get target from remote resource in python shell

[nvincent-vossloh]

Hello,

After being away from labgrid I came back recently to integrate some labgrid features into our test environment.

I used to debug with a python shell open and test various commands with the following setup:
(from a python shell)

from labgrid import Environment
e = Environment('/home/nicolas/work/siema/be/VOG/src/vog-zephyr/vog-zephyr-nodes/test/labgrid/remote-firefly.yaml')
t = e.get_target('main')

Nothing fancy, it comes from the online documentation https://labgrid.readthedocs.io/en/stable/usage.html#environments

the environment file looks like this

  main:
    resources:
      RemotePlace:
        name: nvi-m211
    drivers:
      SerialDriver:
        name: shell_zephyr_driver
        timeout: 1.0
        txdelay: 0.001
      StLinkV3Driver: {}
      NetworkInterfaceDriver: {}

(StLinkV3Driver is a custom labgrid driver I implemented when I did not how to use openocd, it works fine for my use case and is already integrated in our pytest environment.)

The issue I am having is when I hit t = e.get_target('main') I have the following error:

---------------------------------------------------------------------------                                                                                                                                                        [25/1078]
RuntimeError                              Traceback (most recent call last)                                                                                                                                                                 
Cell In[5], line 1                                                                                                                                                                                                                          
----> 1 t = e.get_target('main')                                                                                                                                                                                                            
                                                                                                                                                                                                                                            
File ~/.virtualenvs/vog_zephyr_3.4/lib/python3.10/site-packages/labgrid/environment.py:49, in Environment.get_target(self, role)                                                                                                            
     47     if not config:                                                                                                                                                                                                                  
     48         return None                                                                                                                                                                                                                 
---> 49     target = target_factory.make_target(role, config, env=self)                                                                                                                                                                     
     50     self.targets[role] = target                                                                                                                                                                                                     
     52 return self.targets[role]                                                                                                                                                                                                           
                                                                                                                                                                                                                                            
File ~/.virtualenvs/vog_zephyr_3.4/lib/python3.10/site-packages/labgrid/factory.py:152, in TargetFactory.make_target(self, name, config, env)                                                                                               
    150     name = item.pop('name', None)                                                                                                                                                                                                   
    151     args = item # remaining args                                                                                                                                                                                                    
--> 152     self.make_resource(target, resource, name, args)                                                                                                                                                                                
    153 for item in TargetFactory._convert_to_named_list(config.get('drivers', {})):                                                                                                                                                        
    154     driver = item.pop('cls')                                                                                                                                                                                                        
                                                                                                                                                                                                                                            
File ~/.virtualenvs/vog_zephyr_3.4/lib/python3.10/site-packages/labgrid/factory.py:124, in TargetFactory.make_resource(self, target, resource, name, args)                                                                                  
    122     cls = self.resources[resource]                                                                                                                                                                                                  
    123     args = filter_dict(args, cls, warn=True)                                                                                                                                                                                        
--> 124     r = cls(target, name, **args)                                                                                                                                                                                                   
    125 except TypeError as e:                                                                                                                                                                                                              
    126     raise InvalidConfigError(                                                                                                                                                                                                       
    127         f"failed to create {resource} for target '{target}' using {args} "                                                                                                                                                          
    128     ) from e                                       

File <attrs generated init labgrid.resource.remote.RemotePlace>:11, in __init__(self, target, name)                   
      9     __attr_validator_avail(self, __attr_avail, self.avail)                                                    
     10     __attr_validator_timeout(self, __attr_timeout, self.timeout)                                              
---> 11 self.__attrs_post_init__()                         

File ~/.virtualenvs/vog_zephyr_3.4/lib/python3.10/site-packages/labgrid/resource/remote.py:103, in RemotePlace.__attrs_post_init__(self)
    101 self.timeout = 10.0                                
    102 self.tags = {}                                     
--> 103 super().__attrs_post_init__()                      

File ~/.virtualenvs/vog_zephyr_3.4/lib/python3.10/site-packages/labgrid/resource/common.py:148, in ManagedResource.__attrs_post_init__(self)
    146 self.avail = False                                 
    147 self.manager = self.manager_cls.get()              
--> 148 self.manager._add_resource(self)                   

File ~/.virtualenvs/vog_zephyr_3.4/lib/python3.10/site-packages/labgrid/resource/common.py:124, in ResourceManager._add_resource(self, resource)
    122 def _add_resource(self, resource: 'ManagedResource'):                                                         
    123     self.resources.append(resource)                
--> 124     self.on_resource_added(resource)               

File ~/.virtualenvs/vog_zephyr_3.4/lib/python3.10/site-packages/labgrid/resource/remote.py:50, in RemotePlaceManager.on_resource_added(self, resource)
     48         self.url = config.get_option('crossbar_url', self.url)                                                
     49         self.realm = config.get_option('crossbar_realm', self.realm)                                          
---> 50     self._start()                                  
     51 place = self.session.get_place(remote_place.name)  # pylint: disable=no-member                                
     52 resource_entries = self.session.get_target_resources(place)  # pylint: disable=no-member                      

File ~/.virtualenvs/vog_zephyr_3.4/lib/python3.10/site-packages/labgrid/resource/remote.py:26, in RemotePlaceManager._start(self)
     23 if self.session:                                   
     24     return                                         
---> 26 from ..remote.client import start_session          
     27 try:                                               
     28     self.session = start_session(self.url, self.realm, {'env': self.env})                                     

File ~/.virtualenvs/vog_zephyr_3.4/lib/python3.10/site-packages/labgrid/remote/client.py:36                           
     33 from ..driver import Mode, ExecutionError          
     34 from ..logging import basicConfig, StepLogger      
---> 36 txaio.config.loop = asyncio.get_event_loop()  # pylint: disable=no-member                                     
     39 class Error(Exception):                            
     40     pass                                           

File /usr/lib/python3.10/asyncio/events.py:656, in BaseDefaultEventLoopPolicy.get_event_loop(self)                    
    653     self.set_event_loop(self.new_event_loop())     
    655 if self._local._loop is None:                      
--> 656     raise RuntimeError('There is no current event loop in thread %r.'                                         
    657                        % threading.current_thread().name)                                                     
    659 return self._local._loop                           

RuntimeError: There is no current event loop in thread 'MainThread'.

I have defined and exported an LG_CROSSBAR environement var.
I have defined and exported no_proxy/NO_PROXY var to set the ip of the coordinator and the exporter.

When I provide a 'local' environment file (one where resources are in the file instead of a RemotePlace resource) the command succeeds. But as you can imagine, I need to test with the coordinator/exporter in the loop.

Looking at my ipython history, it looks like I had managed to get the get_target to return something that I could use.

It is probably something really silly that I forgot, but I cannot see what 🙁
Any help would be greatly appreciated 🙂

[Emantor]

Which version of labgrid are you using?

[Emantor]

This looks to be stable, could very well be that txaio now requires a running event loop in accordance with the get_event_loop changes within python. I would've expected our CI to fail in this case, please post the output for pip freeze.

[nvincent-vossloh]

Hello,

I am using custom version available at https://github.com/SiemaApplications-attic/labgrid/tree/dda69a2515d18396816a97f63ebcc63bcae8afe4

$ labgrid-client version
23.1a3.dev195

pip freeze output:

aiohappyeyeballs==2.4.0
aiohttp==3.10.5
aiosignal==1.3.1
alabaster==0.7.16
ansicolors==1.1.8
anytree==2.12.1
appdirs==1.4.4
arrow==1.2.3
astroid==3.2.2
asttokens==2.4.1
async-timeout==4.0.3
attrs==23.2.0
autobahn==23.6.2
Automat==22.10.0
Babel==2.15.0
bcrypt==4.2.0
breathe==4.35.0
canopen==2.2.0
capstone==4.0.2
cbor==1.0.0
cbor2==5.6.4
certifi==2024.6.2
cffi==1.16.0
charset-normalizer==3.3.2
clang-format==18.1.6
click==8.1.3
cmd2==2.4.3
cmsis-pack-manager==0.5.3
colorama==0.4.6
colorlog==6.8.2
constantly==23.10.4
coverage==7.5.3
crcmod==1.7
cryptography==42.0.8
decorator==5.1.1
Deprecated==1.2.14
dill==0.3.8
docker==7.1.0
docopt==0.6.2
docutils==0.18.1
exceptiongroup==1.2.1
executing==2.0.1
frozenlist==1.4.1
gcovr==7.2
gitlint==0.19.1
gitlint-core==0.19.1
graphviz==0.20.3
grpcio==1.64.1
grpcio-tools==1.64.1
hyperlink==21.0.0
idna==3.7
imagesize==1.4.1
importlib_metadata==7.1.0
importlib_resources==6.4.0
incremental==22.10.0
iniconfig==2.0.0
intelhex==2.3.0
intervaltree==3.1.0
ipython==8.26.0
isort==5.13.2
jedi==0.19.1
Jinja2==3.1.4
junit2html==31.0.2
junitparser==3.1.2
labgrid @ file:///home/nicolas/work/siema/be/labgrid/srcs/labgrid
lark==1.1.9
libusb-package==1.0.26.2
lpc-checksum==3.0.0
lxml==5.2.2
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib-inline==0.1.7
mccabe==0.7.0
mdurl==0.1.2
mock==5.1.0
msgpack==1.0.8
multidict==6.0.5
mypy==1.10.0
mypy-extensions==1.0.0
natsort==8.4.0
packaging==24.1
paho-mqtt==1.6.1
paramiko==3.4.0
parso==0.8.4
pathspec==0.12.1
pexpect==4.9.0
pillow==10.3.0
platformdirs==4.2.2
pluggy==1.5.0
ply==3.11
prettytable==3.10.0
progress==1.6
prompt_toolkit==3.0.47
protobuf==3.19.4
psutil==5.9.8
ptyprocess==0.7.0
pure-eval==0.2.2
py==1.11.0
pycairo==1.26.1
pycodestyle==2.11.1
pycparser==2.22
pyelftools==0.31
PyGithub==2.3.0
Pygments==2.18.0
PyGObject==3.48.2
PyJWT==2.8.0
pykwalify==1.8.0
pylink-square==1.2.0
pylint==3.2.3
pymodbus==3.5.4
pymodbus_repl==2.0.4
PyNaCl==1.5.0
pyocd==0.36.0
pyperclip==1.8.2
pyserial==3.5
pyserial-labgrid @ file:///tmp/tmp.zjr1GG9ZML/pyserial
pytest==7.1.1
pytest-repeat==0.9.3
python-can==4.2.2
python-dateutil==2.9.0.post0
python-magic==0.4.27
pyudev==0.24.3
pyusb==1.2.1
PyYAML==6.0.1
regex==2024.5.15
requests==2.32.3
rich==13.8.0
ruamel.yaml==0.18.6
ruamel.yaml.clib==0.2.8
sh==1.14.3
shellingham==1.5.4
six==1.16.0
snowballstemmer==2.2.0
sortedcontainers==2.4.0
Sphinx==5.3.0
sphinx-copybutton==0.5.2
sphinx-notfound-page==1.0.2
sphinx-rtd-theme==1.3.0
sphinx-tabs==3.4.5
sphinxcontrib-applehelp==1.0.8
sphinxcontrib-devhelp==1.0.6
sphinxcontrib-htmlhelp==2.0.5
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.7
sphinxcontrib-serializinghtml==1.1.10
sphinxcontrib-svg2pdfconverter==1.2.2
stack-data==0.6.3
tabulate==0.9.0
tomli==2.0.1
tomlkit==0.12.5
traitlets==5.14.3
Twisted==23.10.0
txaio==23.1.1
typer==0.9.4
typing_extensions==4.12.2
uritools==4.0.1
urllib3==1.26.18
wcwidth==0.2.13
west==1.2.0
wrapt==1.16.0
xmodem==0.4.7
yamllint==1.35.1
yarl==1.9.11
zcbor==0.8.1
zipp==3.19.2
zope.interface==6.4.post2

[Emantor]

Could you please rebase on top of the latest release and test there?

[nvincent-vossloh]

Hi, I merged my branch on top of v24.0.2 and I have the same error. Which is weird, as you mentionned you would have caught it in CI.
I will try to drop my changes, maybe there is something wrong with the change I made to the codebase.

I will come back to you, thank you for your support.

[nvincent-vossloh]

Unfortunately it does not work either with the vanilla version of labgrid (pip install labgrid). I tested inside a fresh ubuntu container in order to make sure that I do not have any python packages pollution...

I am wondering if the version of the coordinator and/or the labgrid-exporter could be the reason behind the failures above. For my previous tests I only changed the version of labgrid for the client side.