BIDS Schema issues

# BIDS Schema issues ## Revised list * [x] 0:INTERNAL_ERROR -> Implementation detail * [x] 1:NOT_INCLUDED -> errors.yaml * [ ] 22:TSV_EMPTY_ROWS -> Instructions * [x] 23:TSV_EMPTY_CELL -> type check * [ ] 24:TSV_IMPROPER_NA -> todo * [ ] 38:INCONSISTENT_SUBJECTS -> Implementor choice * [ ] 39:INCONSISTENT_PARAMETERS -> Implementor choice * [ ] 52:STIMULUS_FILE_MISSING -> todo * [ ] 77:UNUSED_STIMULUS -> Instructions * [ ] 103:MULTIPLE_COMMAS_IN_AUTHOR_FIELD -> Implementor choice * [ ] 114:INCOMPLETE_DATASET -> Implementor choice * [ ] 129:SCANS_FILENAME_NOT_MATCH_DATASET -> todo * [x] 213:README_FILE_SMALL * [x] 221:PIXEL_SIZE_INCONSISTENT * [ ] 224:OPTIONAL_FIELD_INCONSISTENT * [x] 226:UNSUPPORTED_BIG_TIFF * [x] 227:INCONSISTENT_TIFF_EXTENSION ## Original list ### Doable, need implementation ([spec#1402](https://github.com/bids-standard/bids-specification/issues/1402)) * [x] TOO_FEW_AUTHORS (`length(json.Authors) > 1`, select dataset_description.json) * [x] SLICETIMING_VALUES_GREATOR_THAN_REPETITION_TIME * [x] EFFECTIVEECHOSPACING_TOO_LARGE * [x] ECHOTIME1_2_DIFFERENCE_UNREASONABLE (phasediff only; implementing for phase1/phase2 would be a bonus) * [x] SLICETIMING_ELEMENTS * [x] REPETITIONTIME_PREPARATION_NOT_CONSISTENT * [x] VOLUME_TIMING_AND_REPETITION_TIME_MUTUALLY_EXCLUSIVE (needs issue annotation, possibly reimplement in checks) * [x] NIFTI_DIMENSION * [x] NIFTI_UNIT * [x] NIFTI_PIXDIM * [x] BOLD_NOT_4D * [x] SFORM_AND_QFORM_IN_IMAGE_HEADER_ARE_ZERO * [ ] ~~NIFTI_PIXDIM4~~ (duplicate) * [x] MAGNITUDE_FILE_WITH_TOO_MANY_DIMENSIONS * [x] T1W_FILE_WITH_TOO_MANY_DIMENSIONS * [x] REPETITION_TIME_GREATER_THAN * [x] ECHO_TIME_GREATER_THAN * [ ] ~~ECHO_TIME_DIFFERENCE_GREATER_THAN~~ (not BIDS) * [x] TOTAL_READOUT_TIME_GREATER_THAN * [x] REPETITION_TIME_MISMATCH * [x] EFFECTIVEECHOSPACING_LARGER_THAN_TOTALREADOUTTIME * [x] POST_LABELING_DELAY_GREATER * [x] BOLUS_CUT_OFF_DELAY_TIME_GREATER * [x] LABELING_DURATION_GREATER * [ ] ~~ECHO_TIME_ELEMENTS~~ (duplicate) * [ ] "Participants age 89 or higher" (yes, this really is the code; change it to something more following the convention) * [x] MISSING_TSV_COLUMN_IEEG_CHANNELS - Update `rules.tabular_data.ieeg.iEEGChannels.initial_columns` * [x] MISSING_TSV_COLUMN_IEEG_ELECTRODES - Update `rules.tabular_data.ieeg.iEEGElectroes.initial_columns` * [x] STIMULUS_FILE_MISSING - `checks: [exists(columns.stim_file, "stimuli") == length(columns.stim_file)]` ### Doable, possibly need new validator path * [x] Dataset-level checks (do not apply to files, so run once) ([spec#1402](https://github.com/bids-standard/bids-specification/issues/1402)) * [x] SAMPLES_TSV_MISSING * [x] SUBJECT_FOLDERS * [x] PARTICIPANT_ID_MISMATCH * [x] PHENOTYPE_SUBJECTS_MISSING * [ ] Enforce enums in column contents (can also do with match() on arrays, at cost of duplication of information in objects and rules) ([validator#1607](https://github.com/bids-standard/bids-validator/issues/1607)) * [ ] PARTICIPANT_ID_PATTERN * [ ] SAMPLE_ID_PATTERN * [ ] CHANNELS_COLUMN_TYPE * [ ] ASLCONTEXT_TSV_INCONSISTENT ### Doable, need to update expression language * [x] Need `sorted()` function ([spec#1080](https://github.com/bids-standard/bids-specification/issues/1080)) * [x] VOLUME_TIMING_NOT_MONOTONICALLY_INCREASING * [x] BOLUS_CUT_OFF_DELAY_TIME_NOT_MONOTONICALLY_INCREASING * [ ] Update `count()` to count values in nested lists * [ ] TSV_IMPROPER_NA, also need context access to full array * [ ] Update `match()` or similar for arrays * [x] ACQTIME_FMT (can we do `match(columns.acq_time, schema.objects.formats.datetime`) * [x] CHANNELS_COLUMN_STATUS * [x] ASLCONTEXT_TSV_INCONSISTENT (enum) * [x] CHANNELS_COLUMN_TYPE (enum) * [x] CHANNELS_COLUMN_TYPE_UPPER_CASE * [x] PARTICIPANT_ID_PATTERN (enum) * [x] SAMPLE_ID_PATTERN (enum) * [x] TSV_EMPTY_CELL * [ ] EFFECTIVEECHOSPACING_TOO_LARGE * Need `index(["i", "j", "k"], sidecar.PhaseEncodingDirection[0])` or other way to map values * Can we use `[]` for single character takes from strings or do we need to use `substr()``? ### Doable, need to update context * [ ] README_FILE_SMALL - need file size (stat or some equivalent) or possibly contents as byte array (more expensive, but more potential) * [ ] PIXEL_SIZE_INCONSISTENT - need `ome` metadata context with `PhysicalSize{X,Y,Z}` fields * [ ] TSV_EQUAL_ROWS - Simple check would be `length(rows[0]) == length(columns.headers)` * [ ] Need TIFF context: * [ ] UNSUPPORTED_BIG_TIFF (this one might be an implementation detail) * [ ] INCONSISTENT_TIFF_EXTENSION * `tiff.version` (42 for TIFF, 43 for BIGTIFF) * `tiff.ome` for OME header (unless we want generic `ome` for OME-ZARR or OME-TIFF) * [ ] A `stage` context variable could be set to `dataset`, `subject`, `file`, `post` to replace `path == '/dataset_description.json'` hack ### Done but reconsider * [x] VOLUME_TIMING_AND_REPETITION_TIME_MUTUALLY_EXCLUSIVE implemented with "prohibited" requirement level. Maybe should just be checks if we haven't already implemented? * Note that I did not find any additional need for a `prohibited` requirement level. I vote for replacing with checks. ### May require implementor instructions * [ ] UNUSED_STIMULUS (`There are files in the /stimuli directory that are not utilized in any _events.tsv file.`) - This probably needs to be solved on a per-validator basis. Would that be `errors.yaml`? Or maybe not a schema problem? ### Maybe not schema problems? (Validator courtesy, should not be generalized to all implementations) * [ ] MULTIPLE_COMMAS_IN_AUTHOR_FIELD - I think this was mostly misuse of the OpenNeuro author field UI element; delegate to ON interface? * [ ] INCONSISTENT_SUBJECTS * [ ] INCOMPLETE_DATASET (`This dataset contains remote files. If you would like to validate with remote files, use the --remoteFiles option.`) ### Questions * [ ] NOT_INCLUDED - is this superseded or implemented by filename checks? (for @rwblair) * errors.yaml * [ ] How to do dataset level checks? * For now, select on dataset_description.json * What if it's missing? Check this? * `context.stage`? * [x] QUICK_VALIDATION_FAILED - any reason to keep this? * Drop * [ ] INCONSISTENT_PARAMETERS * Recommender tool * [ ] INTERNAL ERROR - Should this be in errors.yaml or just dropped and let each validator error in its own way? * Drop for now; validator is doing it * [x] Not currently implemented, should they be? (via Stefan, probably not) * [x] CHANNELS_COLUMN_SFREQ * [x] CHANNELS_COLUMN_LOWCUT * [x] CHANNELS_COLUMN_HIGHCUT * [x] CHANNELS_COLUMN_NOTCH * [ ] Not currently implemented, should they be? (open PR [validator#1355](https://github.com/bids-standard/bids-validator/pull/1355)) * [ ] INVALID_TSV_UNITS * [ ] NO_VALID_JSON seems like it's covered by required metadata. Maybe there was a special reason for this?