---
# System prepended metadata

title: 'Import / fail-fast & template mismatch, missing meta'

---

# Import / fail-fast & template mismatch, missing meta

IMPORT:FAILURE BEHAVIOR
- --fail-fast None
    - \d+(\.\d+)?%?(/\d+(\.\d+)?%?)
    - LEFT: EXPLICIT OF TOTAL / RIGHT: ??? CONSECUTIVE < LEFT ???
    - 1
    - 0.1
    - 10%
    - 5/100
- --fail-threshold / --fail-max
    - num and/or percent?
    - ~~distinguish fail-fast from final fail status~~
    - ~~does anyone even want to say <5% fail == success?~~
- --fail-fast-min???
    - the minimum start (eg. if 1st task fails it would be a 100% fail-rate...)
    - should this be opt-in or default? (eg. netsplit, x tasks failed, just give up already)
    - distinguish user-config from netsplit (consecutive failure) case?
        - how to distinguish netsplit from "shitty data clustering (directory)"
- xfer.import
    fail_fast: int | float  # validator nom-nom str
    (int == n_items, float == %_total)

IMPORT:DEPTH/TEMPLATE MISMATCH/MISSING META
- template doesn't capture min required (pattern/default/override)
  - absolute minimum required: level.label
  - default  minimum required: level.label, level-1.label, ...
  - fail early, before creating import (note: fs-like, binfile only)
- template misses depth-
  - distinguish: include/exclude (skip ok) vs template mismatch (skip ok???)
  - skip <<< current behavior
  - item fail (=> op fail)
  - ~~upload to level-1 if we have everything lvl conflict!~~
  - alt: fail by default (and support/recommend depth filter)
      - failed to extract minimum required metadata
          - from FIELDS via RULE_N [PATTERNS]
      - refine the filters[depth] and/or the patterns OR use --skip-???
          - invalid / incomplete / partial / missing meta[data]
          - template / pattern / mapping / rule mismatch
      - --missing-meta=fail|skip[|fill]
          - --ignore-incomplete-meta

      - unknown (aka reaper/scheduled/unattended use-case):
        - defaults missing cont labels to unknown (proj=unsorted)
        - upsert like that (TODO check support in xfer/core models/handlers)
        - adding import-op + source-path becomes even more important!
  - ^ covers depth+ too via filenames not allowing /: `[^/]+`
  - opt: consider unique migration merging [""/None/unknown/Unknown] containers
- template misses depth+
  - skip <<< current behavior
  - fail
  - ~~flatten tail slashes to filename - would re-upload acq data at sess lvl?~~
  - TODO `file.name .+ -> [^/]+`
- how to encourage / default depth?
- how to support "unknown/unsorted"?
