Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.93%. Comparing base (37b11ec) to head (a9d3f28).
⚠️ Report is 991 commits behind head on master.

@@           Coverage Diff           @@
##           master    #1791   +/-   ##
=======================================
  Coverage   87.93%   87.93%           
=======================================
  Files          16       16           
  Lines        1351     1351           
=======================================
  Hits         1188     1188           
  Misses        163      163           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

conflict since since then JSON schema definition was added thus making it not overridable, ref: describing the difference between two types of columns:

• Provide clear(er) and visible to users distinction between columns which could be "overridden" in JSON sidecars and which could not #2048

so @effigies, to get back on this one and to allow to overload units for the age (needed or desired for animal and babies research and #1839 ) -- I should replace current definition: for age with YAML style one, correct?

The JSON-sidecar-style column definitions are overridable by JSON sidecars, so age is overridable.

What we have here is something that's not describable by the JSON sidecar definitions. There's no notion of "anyOf" in column definitions in BIDS.

so what do you think we should do then @effigies? could/should we switch back to "BIDS YAML" definition?

I suppose our options are:

1. Revert to JSON schema in YAML and
   1. Lose the ability to override the age column OR
   2. Allow any column to be overridden, losing the ability to enforce columns whose types shouldn't change OR
   3. Come up with some way to indicate that a column may be overridden at the cost of expressiveness.
2. Figure out a way to at least unify the BIDS column description and JSON schema, mindful to cover this case.
3. Hack this particular case and see how far it gets us.

For option (3), we could allow "Units" and "Levels" to turn into (pardon the mix of YAML and Python):

anyOf:
  - type: number
  - type: string
    enum: {Levels.keys()}

Then we would update this description to be:

   definition: {
     "LongName": "Subject age",
     "Description": "Subject age in postnatal years",
     "Units": "year",
+    "Levels": {
+      "89+": "Subject at least 89 years old"
+    }
   }

cc @rwblair for thoughts.

eh, I wish we had discussed this issue and overall dichotomy of two types of column definitions (#2048) at the recent BIDS maintainers meeting, since it remains very confusing, non-intuitive (to me at least), and not documented AFAIK.

$>  for j in ds*/participants.json; do jq '.age.Units' $j; done | sort | uniq -c
jq: error (at ds002873/participants.json:577): Cannot index array with string "age"
      1 "full year"
      2 "Measurement units. [<prefix symbol>]<unit symbol> format following the SI standard is RECOMMENDED"
    344 null
      1 "week olds"
      1 "weeks"
     17 "year"
      1 "(years)"
    303 "years"
     29 "Years"
      3 "years old"
      1 "years (rounded down)"
      3 "Years, with one quantile precision"
for j in ds*/participants.json; do; jq '.age.Units' $j; done  28.85s user 2.61s system 65% cpu 47.874 total
sort  0.00s user 0.02s system 0% cpu 47.875 total
uniq -c  0.00s user 0.00s system 0% cpu 47.874 total

and likely validator does not even warn about those unknown "Units" values, as e.g. in https://openneuro.org/datasets/ds003505/versions/1.1.2 which has "Units": "weeks" and likely tools just assume that it is years. And if I see it right, with current

  definition: {
    "LongName": "Subject age",
    "Description": "Subject age in postnatal years",
    "Units": "year",
  }

we do not even validate that it is a numeric value and thus ok with anything, including 89+ or 99+ etc ?

In case of going with

1. Revert to JSON schema in YAML and
   i. Lose the ability to override the age column OR
   ii. Allow any column to be overridden, losing the ability to enforce columns whose types shouldn't change OR

don't we still leave people ability to override Units or those additional "descriptive" fields per existing listing in specification ? When would people need more "complete" override of this column?

Maybe it would be good to have a call? We've gone back and forth several times, and I'm not really sure what else to write.

Worst comes to worst, let's meet and discuss late July I am back to East coast, may be in some convenient BBQ atmosphere.