Error when trying to get ids of workflow outputs

[stvlar]

Hello,
I'm using bioblends to get results from workflows I've developed and my code worked well until recently.
I'm trying to get the ids of the outputs to then download them as files through a web platform I'm developing with django.

This is the code I'm using:

  media_path = ".."
   result_path = ".."
   max_timeout = 1*24*60*60 #days*hours*mins*secs

   if request.method == 'POST':
       form = SnippetForm(request.POST, request.FILES)
       if form.is_valid():
           form = Snippet(file=request.FILES['file'])
           form.save()

           file_name_final = form.file.name[10:]
           file_path = media_path + file_name_final

          server = 'https://usegalaxy.eu/'
          api_key = ''
          gi = bioblend.galaxy.GalaxyInstance(url=server, key=api_key)

           test = gi.tools.upload_file(file_path, 'upload history id')

           hda = test['outputs'][0].get('id')
           datamap = dict()
           datamap = {0: {'id': hda['id'], 'src': 'hda'}}

           results_run = gi.workflows.invoke_workflow('workflow id', inputs=datamap, history_name='script results')
           results_id = results_run['outputs']
           id_str = ''.join(results_id)
           #download output to user's pc
           gi.datasets.download_dataset(id_str, file_path=result_path, maxwait=max_timeout)

[simonbray]

Hi @stvlar,

It seems the result returned by the Galaxy API has changed, I wasn't aware of that.

Probably the best solution is to change the line after the invocation to something like the following:

results_id = gi.invocations.show_invocation(results_run['id'])['outputs']

[stvlar]

Thank you for your response that worked!
Also I'm getting an error releated to
gi.datasets.download_dataset(id_str, file_path=result_path, maxwait=max_timeout)

This is the error message

TypeError at /tool/
list indices must be integers or slices, not str

This is the traceback

Traceback (most recent call last):
  File "/home/stella/.local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/stella/.local/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/stella/pasteur/app/views.py", line 198, in tool
    gi.datasets.download_dataset(id_str, file_path=result_path, maxwait=max_timeout)
  File "/home/stella/.local/lib/python3.8/site-packages/bioblend/galaxy/datasets/__init__.py", line 92, in download_dataset
    dataset = self.wait_for_dataset(dataset_id, maxwait=maxwait, check=False)
  File "/home/stella/.local/lib/python3.8/site-packages/bioblend/galaxy/datasets/__init__.py", line 385, in wait_for_dataset
    state = dataset['state']

Exception Type: TypeError at /tool/
Exception Value: list indices must be integers or slices, not str
```
`

Again this is the first time this has been an issue for me so I'm a bit lost

[simonbray]

That is quite strange. The error is on this line: https://github.com/galaxyproject/bioblend/blob/v0.16.0/bioblend/galaxy/datasets/__init__.py#L385

So it would suggest that the show_dataset() method (used on line 384 above) is returning a list, rather than a dict as expected.

Can you check if that is really the case?

[stvlar]

As far as I can see it is returning a dictionary

`

dataset | [{'create_time': ...',   
'dataset_id': '...',  
'deleted': False,   
'extension': 'auto',   
'hid': 187,   
'history_content_type': 'dataset',   
'history_id': '..',   
'id': '...',   
'name': '...',   
'purged': False,   
'state': 'queued',   
'tags': [],   
'type': 'file',   
'type_id': 'dataset',   
'update_time': '....',   
'url': '....',   
'visible': True},
-- | --

`

[simonbray]

[{

Looks like a list of dicts to me?

@davelopez maybe you have some idea here? 😕

[simonbray]

@stvlar are you using a particular Galaxy version?

[stvlar]

I'm using the most recent release v21.09

[stvlar]

[{

Looks like a list of dicts to me?

@davelopez maybe you have some idea here? confused

yes there were more dicts there I just left them out since the only difference between them was the input file and the time

[davelopez]

I quickly checked and /api/datasets/{dataset_id} should return a dictionary not a list. Otherwise, https://github.com/galaxyproject/bioblend/blob/v0.16.0/bioblend/_tests/TestGalaxyDatasets.py#L84 should be failing too right? I'm confused too...

[mvdbeek]

It seems the result returned by the Galaxy API has changed, I wasn't aware of that.

We should not be changing the API, which endpoint do you think has changed the response ? I'd also assume both the bioblend tests and Galaxy tests would be failing if that was the case. Does the invocation in question maybe not have any outputs ? What was the original exception ?

[stvlar]

@mvdbeek

What was the original exception ?

The original exception was a KeyError with 'outputs' being the exception value

the original problem was this line
results_id = results_run['outputs']

[mvdbeek]

Can you paste it please together with the response ? And ideally a reproducible example

[stvlar]

@mvdbeek
this is the original traceback

Traceback (most recent call last):
  File "/home/stella/.local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/stella/.local/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/stella/project/app/views.py", line 195, in tool
    results_id = results_run['outputs']

Exception Type: KeyError at /tool/
Exception Value: 'outputs'

Request Method: | POST
Request URL : http://127.0.0.1:8000/tool/
Django Version : 3.2.11
Exception Type : KeyError
Exception Value : 'outputs'
Exception Location : /home/stella/project/app/views.py, line 195, in tool
Python Executable : /usr/bin/python3
Python Version : 3.8.10
Python Path : ['/home/stella/project',  '/usr/lib/python38.zip',  '/usr/lib/python3.8',  '/usr/lib/python3.8/lib-dynload',  '/home/stella/.local/lib/python3.8/site-packages',  '/usr/local/lib/python3.8/dist-packages',  '/usr/lib/python3/dist-packages',  '/usr/lib/python3.8/dist-packages',  '/home/stella/project']
Server time : Fri, 21 Jan 2022 12:40:56 +0000

the code is in my original question

Unfortunately I can't send a reproducible example for confidentiality reasons