Where are the docs for the API frontends?

[JWaters02]

Hi all,

Where exactly are the docs for what is shown on the API Catalog/Swagger API frontends (given there are both for the APIML)? This repo is too massive to effectively search in it for what I am looking for. I have some issues with the API that runs on the mainframe that I want to look into but can't actually find any docs on it. Surely they must be generated by whatever is in the API, but what does that and how can I look at it? I'm still learning about Zowe and this APIML thing so sorry if I am missing tons of context... please ask me if necessary!

Thanks!

[balhar-jakub]

I am not sure what is the nature of the issues, but generally we keep documentation available within Zowe Docs:

On usage under - https://docs.zowe.org/stable/user-guide/api-mediation/using-api-mediation-layer
On Troubleshooting under - https://docs.zowe.org/stable/troubleshoot/troubleshooting/ and https://docs.zowe.org/stable/troubleshoot/troubleshoot-apiml

Is it possible that you are talking about Open API Documentation for the API Catalog itself?

[JWaters02]

Hi, I think this is where my confusion lies. I am not asking about the setting up of the API ML or API Catalog or Swagger frontends, but rather about what is actually on them, i.e. the endpoint URLs, the expected response codes etc. I have come across some issues with these, for example the get unix files from path endpoint URL or expected response codes being wrong. I can go into more detail if required, but again, do not know where these are actually defined. Thanks

[balhar-jakub]

Can you go into more detail on what you are trying to do and where do you fail, please?

[JWaters02]

Firstly, I would like to preface the following detail by saying that my experience with Zowe (and knowledge of the Open Mainframe Project as a whole) until 30 days ago was zero, I had not even heard of it other than in the name "zowe vscode". I have been on a "journey of discovery" that equates to a month of blindly stumbling around until I find tidbits of information here and there.

In the IBM z/OS Unix Files REST API service API, version org.zowe.unix.files v2.0.0:

I could not find docs on what strings are allowed for the type body in the create unix file /api/v2/unixfiles/{path} endpoint - that being { "type": "<some-type>", "permissions": "rwxrw-r--" }. There is FILE in the example, but how am I meant to know what other types are there, or what the type for directory is? DIRECTORY does not work.

These endpoints URLs are wrongly showing they should have a wildcard /** at the end of them. If you think about it for a minute, the part before that is the path of the unix file. Let's say you want to get dev/u/user1/hello.txt, the URL would be /api/v2/unixfiles/dev/u/user1/hello.txt/**. This would not allow it to find the file because of the wildcard added onto the end.

If I just omit the /**, it works fine. So not sure why the wildcards is specified there? If it is to show that wildcards is valid, why not mention that in the description below instead of looking like it's required in the endpoint URL? In the API Catalog/Swagger manual testing, the example keeps the wildcard, which would break even if the forward slash encoding was fixed to not break it first. Also, the return code for the endpoint being wrong is a 400 bad request, as with most other endpoints... but that is undocumented (even if it seems obvious).

"Try it out" request body:

curl -X 'GET' \
  'https://host:port/api/v2/unixfiles/dev%2Fu%2Fuserid%2Ftest1.txt/**' \
  -H 'accept: application/json'

In the IBM z/OS Datasets REST API service API, version org.zowe.data.sets v2.0.0:

For the set dataset content /api/v2/datasets/{datasetName}/content endpoint, the docs say it should return a 200 for OK, but I actually always get a 204 for OK (and there is no mention of 204 at all in the docs).

Naming wise, there is /api/v2/datasets/{filter} to get datasets matching the filter that contains attributes in the response, and then /api/v2/datasets/{filter}/list which is the same thing but without the attributes, so why is the one without attributes having /list in its name but the first one not? It makes the first one sound like it's not getting a list, when it actually is. Is there a reason for not having them in the same endpoint and then just having a boolean (even an opt-in one) in the request body?

In the create dataset /api/v2/datasets/ endpoint, if I deliberately create wrong dataset attributes, for example I make block size not a multiple of record length, I get a 500 server error, not something sensible like a 400 with an error message returned from zosmf API (that says block size should be multiple of record length). It is fine (returns 201) when the attributes are correct. I also have notes of getting a 400 bad request that sometimes had no error message returned at all, and sometimes did, but can't replicate now as unsure what exactly I was doing at the time.

Finally, when creating a partitioned dataset member (i.e. with the () notation), the API does not state anywhere what to do. It would make sense to have the support for it in the create dataset endpoint, except in zosmf API, the way it actually works, is the only way to create this member is to call the write to dataset endpoint, and it will automatically create the member in the PDS if it does not exist. Now, the zosmf API has some docs explaining this. However, the API catalog has 0 docs explaining this. How is one meant to know? I did not even realise that the API ML was using the zosmf API in the backend until I did a bunch of digging.

[balhar-jakub]

Thanks for the answer.

Jobs and Datasets while being Java services are both using z/OSMF in the backend. On top of that API Mediation Layer itself uses the z/OSMF for authentication in the default setup.

The issue and reason for very limited documentation around these services is that they were deprecated with V2 of the Zowe and even removed from Zowe with V3.

Is there an issue for you using the z/OSMF APIs directly?

[JWaters02]

The issue and reason for very limited documentation around these services is that they were deprecated with V2 of the Zowe and even removed from Zowe with V3.

Wait, really? Is there some information tidbit somewhere that explains why?

Is there an issue for you using the z/OSMF APIs directly?

No, that is what I have done, as soon as I realised API ML used zosmf in the backend by default, I decided to investigate and realised how more mature it is so instantly switched to it (and subsequently to Zowe Node SDK for my purposes).

I just didn't want to give up on API ML given it could be better long term, however with this information that the useful parts of the REST API have been deprecated, I don't see any point of continuing on with it further.

Thanks for your replies and for clearing this confusion up for me!

[balhar-jakub]

The information was published as part of the preparation for V3 - https://docs.zowe.org/stable/whats-new/release-notes/v3_0_0#breaking-changes-in-api-ml

The primary reason was that the functionality was already superseded by the z/OSMF one and by the fact, that the services were deprecated in V2 and disabled by default and when asking about their usage we didn't find any user.

The state of the projects is visible on zowe.org

As for the confusion, the primary goal for API ML is to mediate APIs and provide information, SSO and MFA.

There are discussions about whether and in what exact way the API Mediation Layer should also provide access to datasets and jobs and here I invite @dkelosky as he knows more on the specific topic.

[JWaters02]

Okay, I think this is beginning to all make sense to me now.

So, as I understand it, API ML can support multiple backend products that expose their APIs in one "central hub" which also means that SSO and MFA is easier to do for any given product in one place.

Previously, what I thought, was that through its main APIs (e.g. jobs, datasets, unix, etc.) it would allow for multiple backends to exist while the API endpoints not changing. For example if Broadcom made their own set of tools, that could be hooked into the existing API endpoints without them changing for the frontend stuff while the backend stuff essentially running on something entirely different. I thought it made sense that those deprecated APIs are buggy because of how technically challenging it would've been to do that. But clearly that is not the case.

Thanks!

[balhar-jakub]

But what you are writing about is actually pretty interesting use case, that may make a lot of sense for the DevOps related Use Case of Zowe.

@dkelosky This may be a valuable input to the work you are doing about supporting different backends for Zowe CLI.

[dkelosky]

Thanks @balhar-jakub ...

@JWaters02 - it's nice to hear that you are using the Zowe Node SDK. This piece backs Zowe CLI and Zowe Explorer.

We don't normalize APIs at the API ML as you've noted. For example, IBM RSE, Zowe ZSS, and IBM z/OSMF all provide REST APIs to access z/OS data sets. All of these APIs can be accessed through API ML, but you cannot generically access z/OS data sets via a common API ML end point such that it routes to one of those 3 possibilities depending on what is available at a site.

We also don't normalize operations in the Node SDK or Zowe CLI. We do however do something like this in Zowe Explorer, the VS Code extension. That is, you can request, z/OS jobs, data sets, and z/OS UNIX files in Zowe Explorer in the exact same way through Zowe Explorer UI, but depending on how it is configured, it will use z/OSMF, RSE, or FTP (and likely SSH soon).