[FOUNDATIONAL] Facilitate small atomic enhancements

[yarikoptic]

This is a brief issue to possibly start discussion and so we do not forget.
Attn @tsalo @satra

ATM, to accomplish the job, many BEPs need to introduce changes to

• BIDS common principles (e.g. "Indexable metadata" of BEP001)
• add new "entities" (_keys) (Add "part" entity for magnitude and phase data #367).

Quite often those changes are of interest for multiple BEPs, (e.g. _part for BEP001 (multiple contrasts) and BEP004 (SWI)), or even could be argued to be applicable to existing "raw" BIDS itself (e.g. bids dataset provenance, #369).

But being introduced in dedicated specialized BEPs they

• lack proper review by wider BIDS community, and thus have more potential to introduce conceptual inconsistencies within BIDS
• make BEPs development/review/acceptance stagnate
• make more difficult for support tools (validator, pybids) to review/adopt them

I would propose or us to

• improve contributing guide, in particular for BEPs, to RECOMMEND (if not REQUIRE) introducing any such changes as separate PRs against BIDS, with references to existing (or proposed) BEPs
• work between/with existing BEPs to identify such atomic changes and propose them as separate PRs
• for PRs add labels for each BEP (BEP001, ...) so we could see which PRs would be of use for which PR

[yarikoptic]

@bids-standard/bep_leads @bids-standard/derivatives - I would value your input on this proposal

[robertoostenveld]

I would welcome this. The BEP guideline seems not to have been updated for quite some time, is not that easy to find, and might not be that well known to current BEP contributors.

A change in style of working that I realize we went through while extending MRI->MEG->EEG+iEEG, and time-wise somewhere between MEG and EEG+iEEG switching from google docs to github for managing the specification, is that the extension process now expects diffs/PRs, whereas it initially it was more like writing a (partially copied) specification that would be merged manually. Quite some older BEPs are still in the old style (full spec), whereas some of the newer BEPs seem to skip google docs (and thereby risk loosing part of the audience/stakeholders) and only do diffs/PRs.

Wrt the github process: each issue/PR is treated by github (the website) on its own. That causes confusion between the dependencies of issues/PRs (e.g. there is still a mix-up between common-derivatives and imaging-derivatives). Bugzilla and Jira have explicit mechanisms for this. Adding "BEPxxx" to the name and colored tags is a step in the right direction, but still relatively poor (although possibly the best that we can do within github limits).

We might also consider github projects to structure the relation between, and work on issues/PRs. E.g. one project per BEP.

And/or we might consider to more explicitly use branches under the main bids-specification repo, e.g. by having BEP-draft branches (which could also be rendered on RTD). PRs that are specific to a BEP could initially be merged in the BEP specific branch.

[emdupre]

work between/with existing BEPs to identify such atomic changes and propose them as separate PRs

I'm a strong +1 on this.

An immediate example is one @jasmainak and I discovered a while back -- the concerns in BEP21 on re-annotating electrophysiological data strongly relate to concerns with re-annotating naturalistic stimuli for features of interest (such as used in @adelavega 's NeuroScout ) and I think corresponding to BEP02.

I completely agree with @yarikoptic that there's a strong danger of individual BEP leaders not realizing these connections (simply because it would require keeping all of the other BEPs in mind !) and therefore adopting different approaches to the same problem. Personally, I think this would be an ideal role for the steering committee, though I wonder what @robertoostenveld thinks.

[emdupre]

Pinging @bids-standard/steering for opinions on this point:

agree with @yarikoptic that there's a strong danger of individual BEP leaders not realizing these connections (simply because it would require keeping all of the other BEPs in mind !) and therefore adopting different approaches to the same problem. Personally, I think this would be an ideal role for the steering committee

and any thoughts on how to actualize it 😸

[yarikoptic]

and any thoughts on how to actualize it

I would have continued along "I would propose or us to" lines in the original description. May be make them a bit more specific with rules such as

• any new _key-value pair (entity) should be submitted as a separate PR open for discussion (we could add a specific template for such PRs)

And then act on those few cases I have already identified (e.g., _part) and submit them as PRs, referencing corresponding BEPs.

[emdupre]

Sorry @yarikoptic, I meant to ask specifically about how BIDS Steering could help with identifying cross-BEP points of convergence going forward. But yes, I think your suggestions make sense !

[HenkMutsaerts]

Thanks, actually I could need some help with the PR structure! BW, Henk

…

On 14 Jan 2020, at 16:15, Yaroslav Halchenko ***@***.***> wrote: and any thoughts on how to actualize it I would have continued along "I would propose or us to" lines in the original description. May be make them a bit more specific with rules such as any new _key-value pair (entity) should be submitted as a separate PR open for discussion (we could add a specific template for such PRs) And then act on those few cases I have already identified (e.g., _part) and submit them as PRs, referencing corresponding BEPs. — You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub <#371?email_source=notifications&email_token=AGT42LTTIOLVBXQJ23H7FP3Q5XJJRA5CNFSM4JM677LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEI47BYI#issuecomment-574222561>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGT42LUNVF7XV3RNLUGJ6ALQ5XJJRANCNFSM4JM677LA>.

[robertoostenveld]

I think this would be an ideal role for the steering committee, though I wonder what @robertoostenveld thinks.

I don't think the steering group has a better overview per see than other community members, at least I do not have that overview and don't know BEP details (except for the ones I am involved in on a personal title). The "content" is provided by the community members with domain specific knowledge, and the combined wisdom/insight/expertise of the community is needed in the consensus process. The steering group can at best only guide the process.

I do think that we have to improve the signal-to-noise ratio in the communication. That would make it easier for all stakeholders to see where a specific BEP links to other work. My previous comment wrt the github process (projects, branches) might actually not help for that, it would just add another layer and hence more complexity that stands in the way of recognizing what is going on.

I think we (=the community) should update the BEP guideline and that the steering group might help in BEP leads following it. E.g., following up on the community forum meeting yesterday, I asked - on behalf of the steering group - all BEP leads to update their main document to make clear where their BEP specific discussion happens, and some have already done so. Such a process-level review (in this case: ensure that communication is on an open channel that others can join) is something the steering group might be able to take up.

I realize that by looking at the different BEPs, we can learn a lot from each other and improve good practices. The BEP process is not only on merging PRs in the spec (where I agree that overlap should be identified as potential conflicts between modalities, but also as potential added-value between modalities). The BEP process is also about making (and discussing) examples, and ensuring the validator remains consistent with the spec. The suggestions at the start of this issue by @yarikoptic are very good, and we should also highlight other existing practices that we value, e.g., summarizing proposed changes to the spec explicitly, as in BEP001, or explicitly considering the overlap in entities, as @sappelhoff introduced while working on MEG, EEG and iEEG extensions.

[poldrack]

It might be useful to come up with a set of common "user stories" and map out the current path that the user would have to take to find the relevant information. then we could consider refactoring the current environment to optimize those use cases.

…

On Wed, Jan 15, 2020 at 1:12 AM Robert Oostenveld ***@***.***> wrote: I think this would be an ideal role for the steering committee, though I wonder what @robertoostenveld <https://github.com/robertoostenveld> thinks. I don't think the steering group has a better overview per see than other community members, at least I do not have that overview and don't know BEP details (except for the ones I am involved in on a personal title). The "content" is provided by the community members with domain specific knowledge, and the combined wisdom/insight/expertise of the community is needed in the consensus process. The steering group can at best only guide the process. I do think that we have to improve the signal-to-noise ratio in the communication. That would make it easier for all stakeholders to see where a specific BEP links to other work. My previous comment <#371 (comment)> wrt the github process (projects, branches) might actually not help for that, it would just add another layer and hence more complexity that stands in the way of recognizing what is going on. I think we (=the community) should update the BEP guideline <https://docs.google.com/document/d/1pWmEEY-1-WuwBPNy5tDAxVJYQ9Een4hZJM06tQZg8X4/edit?usp%3Dsharing&sa=D&ust=1537468908724000> and that the steering group might help in BEP leads following it. E.g., following up on the community forum meeting yesterday, I asked - on behalf of the steering group - all BEP leads to update their main document to make clear where their BEP specific discussion happens, and some have already done so. Such a process-level review (in this case: ensure that communication is on an open channel that others can join) is something the steering group might be able to take up. I realize that by looking at the different BEPs, we can learn a lot from each other and improve good practices. The BEP process is not only on merging PRs in the spec (where I agree that overlap should be identified as potential conflicts between modalities, but also as potential added-value between modalities). The BEP process is also about making (and discussing) examples, and ensuring the validator remains consistent with the spec. The suggestions at the start of this issue by @yarikoptic <https://github.com/yarikoptic> are very good, and we should also highlight other existing practices that we value, e.g., summarizing proposed changes <https://github.com/bids-standard/bep001/blob/master/ProposedChangesToSpecification.md> to the spec explicitly, as in BEP001, or explicitly considering the overlap in entities <https://bids-specification.readthedocs.io/en/stable/99-appendices/04-entity-table.html>, as @sappelhoff <https://github.com/sappelhoff> introduced while working on MEG, EEG and iEEG extensions. — You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub <#371?email_source=notifications&email_token=AAGUVEDCQN4DU64W5HSMNKDQ53HPDA5CNFSM4JM677LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEI7S2VQ#issuecomment-574565718>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGUVEEBGVDNIO5MS6VFS5TQ53HPDANCNFSM4JM677LA> .

-- Russell A. Poldrack Albert Ray Lang Professor of Psychology Professor (by courtesy) of Computer Science Bldg. 420, Jordan Hall Stanford University Stanford, CA 94305 poldrack@stanford.edu http://www.poldracklab.org/

[dorahermes]

In iEEG we felt that is was the responsibility of the BEP leads to be up to date on the most closely related extensions and ensure compatibility with these and with the main spec. Some examples:

• We had to communicate very closely with the EEG extension and be up to date on the derivatives proposal to make sure that we were aligned and did not add potential derivatives in the iEEG extension. This required many rounds of updates on the spec, examples and consistency in the validator. We were very fortunate to have a close communication with the EEG and derivatives leads and they were involved in the iEEG extension to make this happen and @sappelhoff closely monitored both extensions.
• We used the google doc to tag other leads when we needed input/consensus for the iEEG spec and decide what to include/exclude before we moved to Github.
• We reviewed the EEG and derivatives spec for compatibility several times.
• We used github issues when it came to the merging stage.
• We used the mailing list for issues that could potentially have larger impact or to request input from the community at large (e.g. on topics of units, spaces etc).
• The amazing BEP steering person at that time guided us through this process and helped put us in touch with the relevant people.

Let me know if you'd like a user story on this.