Interested in building a more full-featured, registered version

[myersm0]

Hi, I've recently written a set of registered Julia packages packages for working with CIFTI data. See for example my write-ups about it on the Julia discourse site here and here.

My packages also use surface data, but currently I have to resort to a workaround to get what I need for that. Specifically I need the pointset/coordinates, the triangles, and the medial wall components from surf.gii files. Anyway, it is the missing piece of the puzzle in the current state of my surface-space fMRI framework. I get around it currently by simply using a .jld file containing surface data that I've manually put together. Most prospective users of my packages probably are not going to be willing to do that, though.

To remedy this gap, I would like to use your existing unregistered package as at least a starting point towards developing a more full-featured GIFTI.jl, to be submitted to the Julia General Registry.

Let me know if you have any interest in collaborating on this and/or is you have any thoughts that you'd like to discuss.

Thanks.

Also adding a tag for @PavanChaggar

[Tokazama]

I'd be happy to support this. I should be able to respond to PR's or discussion within 24-48 hours for the next couple weeks before I could be working odd hours.

[myersm0]

Great, I'm very invested in working on this. And I will need to work quickly because I have other things that I have to move on to very soon. So maybe we can wrap this up within a week or so.

First off, a note about where I'm coming from. I think Julia is the perfect language for working with fMRI data, but its handling of NIFTI/CIFTI/GIFTI filetypes is lacking. Notwithstanding your existing NIfTI.jl package which works great, and my own CIFTI.jl which I think is quite nice too albeit pretty bare-bones. So to complete support for this trio of important filetypes, GIFTI.jl should be registered and brought up to a similar (or why not even better?) standard.

I'm kind of looking at this from three perspectives:

1. Coming up with a representation that makes sense in terms of the GIFTI specifications, and bringing those concepts into the world of idiomatic Julia
2. Giving existing users of your unregistered GIFTI.jl what they need, without departing too much from code and structs that you've already established there
3. Having it fit in well with my existing framework of registered fMRI packages, too (note: ideally I should have worked out this GIFTI stuff before developing that framework, but I didn't have time, I was trying to get the code out for a paper)

So far, I have drafted up this sketch of a more fully-featured GIFTI package. It was mostly written by Claude AI. Consider it just a proof of concept. I'm not saying the API is right yet or the code organization is ideal, but it works over a variety of different gifti types that I've tried (surf.gii, func.gii, shape.gii), and the GiftiStructs that it reads in are pretty nice to work with. It doesn't produce the Homogeneous Mesh that you all need, but we could add in a constructor or whatever to handle that, once the core design is out of the way.

One thing stands out to me, is that I/we should perhaps build a shared dependency package called something like CiftiGiftiSpecs.jl (or maybe, more inclusively, NiftiSpecs.jl?). For example, in my CIFTI package I define a CIFTI BrainStructure enum type containing things like CORTEX_LEFT and AMYGDALA_LEFT etc (see here), which I would then need to either duplicate in GIFTI.jl or inelegantly import from that package. See here for a mockup of the various constants/enums/whatever that we need to define for GIFTI. If we were to factor out these specification-related things to a separate package and make that a shared dependency for both CIFTI.jl and GIFTI.jl, this is what it might look like. (The API is certainly not right yet as is, so just consider this a conceptual sketch for the moment.)

Now ideally this shared dependency package might be called NiftiSpecs.jl, to be more inclusive. Since I understand "NIfTI" is not just a filetype but also, confusingly, the name of the "initiative" under which these three distinct filetypes fall ("Neuroimaging Informatics Technology Initiative"). But perhaps it would add too much complexity.

I'd be happy to hear your (plural, in case there are others interested/invested) thoughts on all this.

[myersm0]

Also as for what the final interface should look like to users, I think we should just have GIFTI.load(fname). Later I could probably try to register the *.gii filetypes with FileIO.jl so that users can just do load(filename). And then of course there will be accessors, and constructors for conversion to alternative representations (like GeometryBasics.Mesh and so on), etc.

[Tokazama]

There's definitely a lot of overlap with the constants/enums between cifti, gifti, and nifti. I common package for those would inded be ideal. That would save a lot of code and provide a good place for documentation. I think that's the way to go but the exact implementation is a bit difficult. Looking at what you have so far, it's clear you've put some good thought into it, so I'll go ahead and share some issues I've run into.

1. In order to make enums discoverable (instead of just a bunch of symbols floating around the module) you have to enforce some scoping manually. You also have to manually do a lot of the documentation since things like ?instance_name can't be documented in line (like fields of a struct can) and it doesn't even return documentation for it's parent type. So you have to do something like

function Docs.getdoc(x::MyEnum)
   if x === my_enum_name_1
       "my_enum_name_1::MyEnum - indicates bytes for ...."
   elseif x === my_enum_name_2
      ....
   end
end

2. Julia enums aren't always the fastest for looking stuff up programmatically (it's a statically sized lookup of names a few names but uses a hash mapped dictionary), so things like NiftiDataType don't always prove as useful at runtime where you really just want to do something like nifti_code_to_datatype(read(io, UInt16)), instead of holding on to the raw bytes. Therefore, before committing to a bunch of new stuff that you'll need to document, consider benchmarking a simple function that ties it together.

3. Things like anatomy may need something more abstract than enums that bind documentation and concepts to tagged bytes. It's definitely useful to have some sort of system in place, but you get different atlas systems that divy things up a little differently depending on population, modality, species, etc. You have atlases that parcellate the cerebellum into a bunch of different parts but then you have things like orientation and laterality that should be universal. I haven't tried solving this one as much. It would be great if there was something that made it easy to build on a common interface to create a brain atlas package for each brain atlas/parcellation, plus it was able to offer users a consistent and useful way to interact with that template specific package.

4. Things like intent codes are a bit odd. Sometimes SPM or FSL will use them for something specific, but most of the time they don't do anything, except take up space. Then you have things like statistical distributions that you could theoretically map to something like Distributions.jl but then what do you do about all the other statistical packages out their that use a different type system for stats distributions? Maybe a good place for package extensions?

Sorry, that was rambly. Hopefully, it provided some food for thought that was helpful but a lot was kind of opinion. Whatever comes out of this it only really matters if the end design is useful, performant, and comprehensable by others. So if you have an idea you feel strongly about I'd be happy to take a look at it.

[myersm0]

All good food for thought, thank you. A few quick responses and then I'll get around to trying some things out. Some thoughts on a shared dependency package, tentatively to be named NiftiSpecs.jl:

Re: your first point, about discoverability of enums, all true, but I would imagine most people probably don't care about most of these things, unless they're developing or maintaining a nifti-related IO framework like we are. Ideally I think it would be sufficient if we could get away with just saying: here's our design philosophy for translating the specification into Julia, go to the actual specification pdf document if you need more info. Most people probably just want to get their data matrix and then do some work with it. To the extent that there are other things you need to know from your header data, I think it can be the job of downstream packages to deal with that. For example, in CIFTI.jl, users probably need to know what grayordinate indices correspond to which brain structures, so dealing with that must be clear, and it will be the job of CIFTI.jl to do that.

Re: your point 3. Yeah conceptually there is some low-hanging fruit that's not exactly a part of the NIfTI specs but which is kind of implied. Like laterality. E.g. the specification has CIFTI_STRUCT_CORTEX_LEFT and CIFTI_STRUCT_AMYGDALA_LEFT but it doesn't have an explicit hierarchy of laterality and of cortex, subcortex. We could easily impose a very simple such scheme, like define laterality(CIFTI_STRUCT_CORTEX_LEFT) = LeftLateral and is_subcortical(CIFTI_STRUCT_CORTEX_LEFT) = false etc. I'm not sure that's the job of this package, though -- I mean the NiftiSpecs.jl shared dep package. Maybe, I'm not opposed to it upfront. Maybe it makes sense to implement at least the starting point of such a scheme, that people can flesh out for particular use cases as they see fit.

Re: point number 4, yeah, this kind of ties into what I said about point number 1, that a lot of this stuff is just there in the files and in the specifications, and few if any people are going to care. For example I've never seen a cifti/gifti file that uses statistical intent codes. But if it's in your data, we should be able to represent it according to the specs.

Basically I see this hypothetical NiftiSpecs.jl as kind of a dictionary -- in a conceptual sense I mean -- that knows how to interpret codes and such things from the nifti specs, and turn them into actual Julia data types and so on. I imagine people would not generally be interested in using this NiftiSpecs pacakge themselves directly, it would just serve to simplify and unify operations for CIFTI.jl and GIFTI.jl (and perhaps even NIfTI.jl should anyone care to go back and do a refactor), and provide a kind of documentation that will ease downstream maintenance of those packages.

I'll put some more prototype stuff together later, with an eye towards performance and ease of documentation etc.

[myersm0]

I think I've more or less worked out the design and interface of our new GIFTI.jl, here. I invite any feedback. I think people might not agree what I've done with singular vs. plural accessors like pointset() vs. pointsets() and triangle() vs triangles(), so I'm perfectly willing to drop that -- but I'm not sure yet what might work better in its place.

For the moment I'm letting the idea of a shared dependency NiftiSpecs.jl go to the back burner as a possible future improvement. The new code works as is and it's probably fast enough. It's essentially the same speed as the old unregistered GIFTI.jl, but it also does more by bringing in a fuller representation of the GIFTI data arrays and metadata, and also handles more filetypes like shape.gii. (Though a shape.gii will not be able to be turned into a Mesh, because it lacks coordinates and triangles.)

To get an equivalent result as you would get in the old GIFTI.jl:

filename = "my_file.surf.gii"
g = GIFTI.load(filename)
my_mesh = GeometryBasics.Mesh(g)

I will begin adding tests and doing some cosmetic updates to the code later this afternoon. I may see also about some performance engineering to see if we can not only match the old package's speed but beat it. (Even though I would expect that people generally are not needing very high performance for reading GIFTI files, since they tend to be pretty small files of just some coordinate sets and such, rather than big functional or connectivity data. But it could be wrong.)

[myersm0]

I'm also going to:

• register the *gii file extension with FileIO.jl
• come up with an Artifacts.toml that has some test gifti files packaged in it, to develop tests and demos
• docstrings once people confirm that they like the API as demonstrated in the readme

[myersm0]

I think we've now got a nice implementation with testing and demos that could be ready for a version 0.1.

Except perhaps for this part, from our Mesh constructor, for backwards compatibility with parse_gifti_mesh() in the unregistered GIFTI.jl. :
vert_array = data(pointset_arrays[1]) / 100

I'm not sure how standard that division by 100 is. Maybe it would be better to leave it to the user to do that if they want? For example they could:

g = GIFTI.load(fname)
data(pointset(g)) ./= 100  # updates the raw array in RAM but doesn't affect the gifti on disk
GeometryBasics.Mesh(g)

Once we get to the point of being ready for an initial submission to the Julia registry -- and like I said, I think we're about there already -- we will have to figure out how that's going to work practically speaking, e.g. do you want me to transfer it to JuliaNeuroscience org but remain the maintainer? A PR to the original repo probably doesn't make sense.

[myersm0]

I've gotten the ball rolling on registration. If anyone has comments on the implementation or on whether registration should begin from my github account or from here in JuliaNeuroscience (it can be moved later, I think), there are three days, so plenty of time to block or revise the process if necessary. But I was aiming to be able to have it registered by this weekend so that I could update my surface visualization package to utilize it, and then and publish a full Documenter.jl system on the whole thing, demo notebooks, etc.

The door is still open. Thanks, @Tokazama , for the feedback.

[Tokazama]

I'll take a closer look at the code after work. The JuliaNeuroscience group is good because you can share permissions more naturally and improve discoverability. That makes it ileasier for others to step in and review trivial PRs if you're unavailable

[myersm0]

OK sure. We should perhaps also consider moving my existing CIFTI.jl to JuliaNeuroscience, too, if it can get your blessing. Then we'll have all three packages within the same org.

[Tokazama]

OK sure. We should perhaps also consider moving my existing CIFTI.jl to JuliaNeuroscience, too, if it can get your blessing. Then we'll have all three packages within the same org.

I'm cool with whatever helps science move forward for everyone. I sent an invite to the organization

[myersm0]

I've initiated a transfer of CIFTI.jl. I couldn't do so on GIFTI.jl, yet, because it collides with this existing repo's name.

[myersm0]

By the way, I've done some performance measurements versus Python's nibabel. It seems we're currently about 20% slower than nibabel in parsing ASCII-encoded gifti data, but slightly faster than nibabel in base64 and gzipped-base64 encodings. I've got some ideas on a performance-refactor that I have going in another branch.

Edit: come to think of it, that's actually really good, considering that we have an additional burden of transposing/permuting the data as a result of row major vs column major, while Python does not.

[Tokazama]

In cases where we really need to permute the array for things to work correctly we might be able to get away with PermutedDimsArray to perform a lazy permutation.