-
Notifications
You must be signed in to change notification settings - Fork 8
[WIP] Add payload download action. #33
[WIP] Add payload download action. #33
Conversation
97cae93
to
d6d5a52
Compare
Checked commits martinpovolny/cfme-migration_analytics@d6d5a52~...67dbee5 with ruby 2.5.5, rubocop 0.69.0, haml-lint 0.20.0, and yamllint 1.10.0 **
|
Thanks @martinpovolny, I'll test it out. @bthurber, @fdupont-redhat, is it acceptable for us to require that the API and UI workers are on the same appliance in order to download a payload? I assume this will be true in the majority of cases, but I wonder if it's ok to make it a hard requirement. @martinpovolny maybe if we do have that requirement, the UI can somehow detect if the workers are on the same appliance, and if not it can fall back to the current SCP command display instead of a download button. |
I'll defer to @pemcg for the best practices. I would say that it's a valid requirement, but he's the master :) |
@martinpovolny I just tested this and the download works on my end. Once it's grabbing the actual payload file do you think we can preserve the filename (e.g. cfme_inventory-20200120-38317-hdpx95.tar.gz)? |
Also do you know of any way I can make an API call to check whether the API and UI workers are on the same appliance (i.e. check if the download will work)? |
I'd say that in a multi-appliance setup you cannot guarantee that the API and UI workers that you're talking to are necessarily on the same appliance, without somehow being able to examine the URI path (i.e. IP address) to each endpoint. The workers are grouped as region resources, and workers are often allocated using the following region methods: https://github.com/ManageIQ/manageiq/blob/master/app/models/miq_region.rb#L214 If you have 2 appliances you might have both configured with active UI and API workers, but depending how workers (i.e. region resources) have been allocated to your inbound request, you may well be served by different appliances. For example this is the reason we have to store the temporary API access token in the database rather than appliance memcache when using embedded Ansible, if the Ansible playbook needs to make an API call back to Automate using the token. We can't guarantee that Ansible and the Automation engine will be on the same appliance. Could we use the same mechanism? Perhaps an ignorant question, but how is the authentication handled for other parts of the CloudForms UI that allow for downloading, such as exporting policies or automation domains? |
This is not about general CF. This is about V2V. Therefor I was asking about the guarantee of running on a single appliance. In the context of V2V. @pemcg :
In other places we are serving data from the database. Which is shared by the workers across the region. Therefor we do not have this problem. In this particular situation we are downloading a payload that is generated by a worker on the filesystem. AFAIK there's a flag on that task that says that it should be picked by a worker running on the same appliance as the API. (Correct me if I am wrong, I have limited experience with V2V.) But it's not guarantee that also the UI worker would be running there. |
I can probably also do a simpler implementation of C) w/o introducing new entity to store the tokens in the database by encrypting information about the task_id into a token and then creating an API endpoint that is not authenticated by the API auth token but instead only decrypts this new token, and takes the task_id from there w/o using the database. |
@martinpovolny thanks for #35, we might just be able to go with that if the single-appliance requirement is acceptable. It would be nice to get it working in a multi-appliance setup, but I'm not sure it's worth all the effort of a custom-auth solution. I think I'm still missing something about why we can't just use a normal API endpoint and have it return the payload directly in a GET. Could we do option (a) (make an API endpoint that would return the data) but then instead of calling it with AJAX, just put an |
If that's not possible, I think we could have the download button appear only if you're in a single-appliance setup, and otherwise fall back to the SCP command we display today. That way it won't be broken on a multi-appliance setup, and we'll still get the download button in the majority use case (a single appliance has been installed at a new customer specifically for running MA). |
The API is authenticated by a custom header. How do you envision adding a custom header to a request that is fired from an |
Ahh.. I forgot it was a custom header and not a cookie or some other means of implicit auth... I figured it would work since I can drop an API URL in my browser address bar (which I now realize uses a saved basic auth password). Fair enough, thanks for clarifying. Looking into other options.. maybe there is a less ugly way to do it with AJAX (stream to the localStorage API or something). |
the '<a ...>' would work if the data was served from the UI worker which would need the assumption that both run on the same appliance. Another variant is to have 2 API actions, one that generates a one-time token, and second one (w/o auth) that based on the on-time token returns the data. But for that I would need to store the token somewhere. Storing it in the database would require a migration and therefor would be a problem with Ivanchuk. Maybe I could just keep it in the memory. Or I could encrypt the information about the task_id in the token and some timestamp so that the validity of the token is limited.... |
I'm happy with whatever solution works for you, but I think I figured out a way to handle this via a regular AJAX API call if you want to keep the backend simple. I just realized we are already using https://github.com/eligrey/FileSaver.js in v2v to download migration logs via the API. Under the hood it does what you were suggesting (a Alternatively, there is https://github.com/jimmywarting/StreamSaver.js which has polyfills and doesn't have that RAM restriction. I could make it work via AJAX if we want to. |
Thank you for this information. I did not know that. If it has been already decided to use this approach in the project I am not going to oppose that. I'll do it the same. Can you, please, point me to the place? I am not familiar with the codebase and |
The npm package name is |
67dbee5
to
bfc2122
Compare
This is not going to work as simple. Because:
What could work:
a) I could make an API endpoint that would return the data. Then an create a
data:
link with base64 data and click it. This will blow the browser if the file is large. And it's ugly.b) Or I could create an API endpoint that is NOT authenticated using the headers. It could work like this:
I am against a).
b) is not very nice, but if it can be guaranteed that the workers are on the same appliance in the V2V scenario, then this will work.
c) is the IMO the best and for example Github does that. But the tokens need to be stored somewhere and it would be wise to implement this in a generic way on the API level.
Trying the C) variant
@mturley : in the second commit I am adding an UI side implementation of the action.
You can call it from the UI (javascript) like this:
Please, take a look.
The first commit can be ignored.