Use as a look up table

[znichollscr]

One of the proposed uses of the variable registry is to use it as a look up table. For example, given a branded variable name, I would like to look up the cell methods and dimensions.

At first this seems innocuous. In practice, it introduces a uniqueness constraint (I think): for a given branded variable name, there must be only one valid value of cell methods and dimensions (i.e. the combination of branded variable name, cell methods and dimensions must be unique). With the current branding algorithm, this is not necessarily true because cell methods can have stuff in brackets which the algorithm doesn't look at, so we may run into issues if we don't do a bit more thinking about this (fortunately, with the current data request, this uniqueness is there by coincidence, as confirmed here znicholls/CMIP-branded-variable-mapper#34 (comment)).

So, if we do want to support such look up table use, then we need:

1. identify the look ups which should be unique i.e. provide unambiguous answers
2. add tests to ensure that this uniqueness constraint is respected as we add new information to the registry

[taylor13]

I (think I ) agree that cell_methods and dimensions should be inferable from the branded variable name, and I think that is now the case. If we found that this wasn't the case for some newly proposed variable, then I think we would need to possibly generate a new brand label and modify the algorithm for generating the branding suffix so it remains true.

[znichollscr]

I think we would need to possibly generate a new brand label and modify the algorithm for generating the branding suffix so it remains true

agree. this potential instability in the algorithm is what makes me nervous that we haven't finished requirements capture and therefore aren't at mature/stable stage

[taylor13]

Well CMIP is a research project and to be nimble and productive it can never meet the standards of an operational project. The data structures have remained quite stable over several CMIP phases, so I think the current formulation of branded variables should serve us well and and tweaks needed in the future will be relatively minor. If some major changes are made down the line to meet research needs, we'll have to rethink how we name variables.

[znichollscr]

Well CMIP is a research project and to be nimble and productive it can never meet the standards of an operational project

I think we disagree on this and the implications of not having standards (I would say they make us slow and make work harder).

If some major changes are made down the line to meet research needs, we'll have to rethink how we name variables

Yes ok makes sense. That would be a post-CMIP7 thing (although I guess we've already had this discussion which is why we're relying on the data request for now, so I think we're already on the same page)

[taylor13]

I wholly support having standards. That's what we've been working on for decades. I was trying to make the point that we have found a way, without inventing new table names and too many new root names, to assign unique "branded variable" names to each variable in the data request (once we remove the "frequency", "region", and "grid" axes included in the data request). If a new data structure beyond the current ones accounted for in the branded variable algorithm are needed for future data requests, they will not be approved by the WIP (for inclusion in the variable register) until we find a way to uniquely represent them in the branded variable name. In that sense the current algorithm is subject to future change (I wouldn't say "unstable"). How would we make it more stable now?

[znichollscr]

How would we make it more stable now?

Well, the clearest way would be to have a very clearly defined set of requirements. That way we could say, "We support these use cases and have the tests to prove it. If you find a use case that doesn't work with our flow, then we consider this out of scope and you will need a different solution".

When I read, "If some major changes are made down the line to meet research needs, we'll have to rethink how we name variables", what I read is, "If it turns out the requirements change, then we'll just have to accept that and change the tool/infrastructure/software too". To me, that's the opposite of stable. To get something stable: the requirements come first, then we implement the thing that meets the requirements, then we keep it stable.

It's not unreasonable to say, "No, this is supporting a research project, which is inherently unstable", but the implication of that is sacrificing stability to support new, currently unknown research applications. That's not objectively wrong, but as an underlying philosophy for a core piece of infrastructure, it seems problematic to me. To me: infrastructure should be stable first, even if that means that some use cases require new or other tools to support them in future (fine, we can develop new tools if we need, but let's keep existing ones stable for as long as we can).

[znichollscr]

(The fact that we found a use case, i.e. tmax and tmin, very recently is what makes me nervous that we haven't fully captured our requirements and are therefore building an inherently unstable thing (I think the frequency of changes to and length of discussion around the tables in the google doc also shows that these requirements are still moving). Building a key part of our infrastructure around a developing idea (ok, yes, I concede, no longer rapidly developing) makes me nervous. I'm not saying it's clearly a bad idea, I'm just saying that these moments of, "Yes, we didn't think of that", make me nervous about what else we haven't thought of and what the future implications might be when we have communicate these realisations to potentially lots of users who have built on top of the variable registry under the assumption that it would be structured and function in a certain way.)

[znichollscr]

That's also the point of this issue: we have an algorithm that takes dimensions, cell methods and the variable root and gives back a branded variable and suffix. What I am asking is, does this algorithm need to be invertible (is this a requirement of the approach)? If yes, how do we do that? I thought about this in znicholls/CMIP-branded-variable-mapper#11 and I can't see how I would write the inverse algorithm. So, if inversion is key, then I don't think there's a trivial solution so we should turn our minds to it ASAP in case the current approach is actually unable to be inerted i.e. we can't fulfil a key requirement.

[taylor13]

I'll respond to this last point, which I think poses the question in a way that we can decide on whether any action might be needed, in the relevant znicholls/CMIP-branded-variable-mapper#11 (comment).

I don't think we need to urgently discuss how "stable" things are. [In my opinion they'll always be unstable to a certain extent, but that's not particularly helpful, I suppose.]

[znichollscr]

Sounds good, thanks

…

On Thu, Oct 2, 2025, 00:03 taylor13 ***@***.***> wrote: *taylor13* left a comment (WCRP-CMIP/Variable-Registry#110) <#110 (comment)> I'll respond to this last point, which I think poses the question in a way that we can decide on whether any action might be needed, in the relevant znicholls/CMIP-branded-variable-mapper#11 (comment) <znicholls/CMIP-branded-variable-mapper#11 (comment)>. I don't think we need to urgently discuss how "stable" things are. [In my opinion they'll always be unstable to a certain extent, but that's not particularly helpful, I suppose.] — Reply to this email directly, view it on GitHub <#110 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3KEXH4TMF4FTLO2S7DX3FD3VRFUJAVCNFSM6AAAAACHOIU2Y2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGNJYGMYTQNBSHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[znichollscr]

Wherever we land in znicholls/CMIP-branded-variable-mapper#11, should be recorded here too. At the moment, it seems like the answer, "Use as a look up table via the branded variable is not supported, you need to use [insert other unique ID here] instead", is a possibility

[taylor13]

I suppose it would be "nice" to be able to infer cell_methods and dimensions from the branded variable name alone, but that's impossible because multiple cell methods (e.g. "where sea_ice (mask=siconca)" and "where sea_ice (mask=siconc)" share the same branding suffix (but have different root names). Because of this, a lookup table is necessary.

If that's the only example (and I can check but not now), then we could probably just remove the non-CF-standard dictated ("mask-siconc*) from cell_methods, as I've advocated for some time now, and the reverse would be possible. I'm pretty sure, however, that there are other problems.

Is there a reason anyone would want to construct the cell_methods and dimensions given the branded variable name alone?

[znichollscr]

If that's the only example (and I can check but not now), then we could probably just remove the non-CF-standard dictated ("mask-siconc*) from cell_methods, as I've advocated for some time now, and the reverse would be possible. I'm pretty sure, however, that there are other problems.

ys exactly. We might be able to reverse the set of definitions we have now but the system wouldn;t have this by design - it would just happen to be true for the set we have now but there would always be the risk of adding something that broke the reversibility in future.

Is there a reason anyone would want to construct the cell_methods and dimensions given the branded variable name alone?

simplicity I think. I think the DR thought, "All we need to store is the branded variable, the rest we can look up as needed". That look up clearly only works if the variable registry can use branded variable as a unique identifier i.e. there is a 1:1 mapping from branded variable to cell measures/dimensions.

I think it's ok if we say, "No, you can't use branded variable as the ID. You have to use [some other unique ID for variables in the variable registry] instead." It's just that, if we're going to say this, we need to make sure that we don't forget to say it as I think the current assumption from others (e.g. DR) might be that the branded variable name will already be this ID.

[znichollscr]

Making it very simple, the question is: can the branded variable be used as a unique ID in the variable register?

I think we both agree that the answer is no, because the same branded variable can be generated from subtly different cell methods/dimensions.

That is fine, but I think we need to write it down and make it clear because I believe the DR TT was thinking the opposite i.e. that they could use the branded variable name as a unique ID for looking up entries in the variable register.