# Renku-Python Releases

## Migration Types

### Major Metadata Migrations

E.g. v9 to v10.

Not backwards or forwards compatible, require a `renku migrate`, usually for major changes and happen rarely.

Determined by `Project.version` in metadata.

### Minor Migrations

Small changes like adding a new property to a domain model with a default value. Backwards compatible.

Happen automatically when a newer version writes to an older version project.

Enforced by `Project.minimum_renku_version` in metadata. Check can be skipped with `RENKU_SKIP_MIN_VERSION_CHECK=1` env var (For dev purposes only).

## Current status

flowchart TD

    ghrelease([Github Release])
    tag[Tag 2.4.0]
    ghaction([Github Action])
    pypipre[Pypi 2.4.0rc2 release]
    chartrel[Chart Release 2.4.0]
    coresvcrel["Core Service 2.4.0 release \n(from git tag)\ncreates new projects\nwith 2.4.0 pip version"]
    autopr["Main Renku Auto-PR\n(acc tests fail since\n2.4.0 not on pypi)"]
    pypirel[Pypi 2.4.0 full release]
    renkurel[Renku Release]
    
    master --> ghrelease
    ghrelease --> tag
    tag --> ghaction    
    ghaction --> pypipre
    ghaction --> chartrel
    ghaction --> coresvcrel
    chartrel & coresvcrel --> autopr
    autopr --> renkurel
    
    pypipre-. manually,later ..-> pypirel

# Solutions

## (1) Back to the way things were

No prereleases, just full release off of RP git.

- People might make their repos unaccessible through the UI, potentially for quite a while in some cases
- Less likely to happen now that we don't nag users anymore
- Doesn't give actors like Omnibenchmark much headsup to catch up with changes

## (2) Current way but force tests

Add an env variable/chart value to manually pass in `2.4.0rc2` for acceptance tests & core-svc project creation until `2.4.0` proper is released. Should be released before first rollout to a deployment.

- A bit hacky, but would work with the way things are now
- Allows us to full release as close to (first)rollout as possible
- Could be useful for other things, deployment can switch contained cli version independent of core-svc version
- Tests could still fail if core-service creates project with an unsupported previous version (Only when we do a major metadata migration)

## (3) Use a private pypi repo
Essentially same as (2), but we do a full release immediately to a private pypi repo which is used on dev and in acceptance tests, and we publish to pypi proper on first rollout.

- More effort to set up the repo
- A bit cleaner in terms of version consistency
- Might actually be nice to have a private pip repo in Renku?

## (4) Pre-Release all artifacts

Make RC versions of chart, core-svc and wheel.

- tests pass since everything is consistent
- Auto-PR would be pointless since we should merge a pre-release
- When making the full release later, we're back at square one with the same issues we had so far 🚫

# Easing User Pain

## (5) Make the CLI forward compatible

Older versions would carry along new, unknown properties, without knowing about them.

- Not really an option for major migrations
- potential for unforeseen bugs
- would only work for versions after when it is implemented, unles we backport

## (6) Two-Way Migrations

Allow downgrading a project (without losing work), allowing users to recover from accidentally upgrading.

- Unclear how to deal with users that used new functionality already
- More effort
- Potential for extra bugs/bad metadata due to downgrade

## (7) Finer-grained versions

E.g. a change to dataset metadata doesn't prevent workflows from working on an older version. Probably not workable.

- A lot of extra effort
- Doesn't help in all cases
- Confusing to users

# Long-Term

- remove/rip out dependency of kg on cli
- Probably can't decouple core-svc and cli version, as they both act on the same metadata -> some issues will persist, at least in some cases (major migrations)
- In these cases, maybe we can have better user feedback and options to resolve it, at least