# Native Projects packages

flowchart TD
    secrets --> proj_svc["project service"]
    secrets --> user_svc["user service"]
    proj_svc --> git_oauth["OAuth for git providers"]
    git_oauth --> proj_cli["CLI for projects"]
    proj_svc --> cdn["CDN/data hosting"]
    cdn --> proj_env["Environments on projects"]
    git_oauth --> proj_env
    proj_env --> dev_cont_sess["Devcontainer sessions"]
    user_svc --> native_crc["CRC native users/groups"]
    proj_svc --> proj_datasets["Project datasets"]
    proj_svc --> proj_search["Project Search"]
    proj_env --> env_search["Environment search"]
    proj_svc --> prov_svc["Provenance service"]
    prov_svc --> cli_prov["CLI provenance tracking"]
    proj_cli --> cli_prov
    proj_env --> better_cli["More CLI support"]
    proj_datasets --> better_cli
    proj_env --> hist_data["Historical data tracking"]
    proj_datasets --> hist_data
    cli_prov --> hist_data
    proj_env --> env_reuse["Environment publishing/reuse"]
    git_oauth --> repo_templ["Repository templates"]
    repo_templ --> templ_search["Search/publish repo templates"]
    env_search --> env_reuse["Environment reuse/import"]
    cdn --> env_reuse
    dev_cont_sess --> ssh["Improved SSH"]
    proj_datasets --> dataset_pub["Publish datasets"]
    dev_cont_sess --> third_part_compute["Third party compute"]
    user_svc --> perm_management["Permission management"]
    

## Secret Storage

already discussed

## Project service

Simple CRUD Service for Projects

Nice to have: store non-sensitive preferences

## User/Group service

native users and groups (based on keycloak).

Nice to have: store non-sensitive preferences

## OAuth Apps for project service
Connect to github and gitlab (maybe two pitches?), in project service. New "Repository" entity inside project.

## CLI for project service
simple CLI that interacts with project service and can cache data locally

- login
- Read for projects, no create/update/delete initially
- can checkout repos

## Data hosting/CDN
Hosting for internal use data. E.g. project images
Can also be used for CSS etc.

## Environment definition at project level
Based on devcontainer in project. CRUD service for defining this. Very simple support only initially.
Essentially a name, project_id, pointer to a repo, path to a devcontainer file, (commit reference?)

## Run devcontainers as sessions

maybe in amalthea, maybe separate component

## Rewire CRC to use native users/groups

As per title. Instead of only keycloak users.

## Dataset links on project entity

Similar to storage service. essentially (project_id, dataset_url, repo_id) tuples. Probably not copied inside a repo but living alongside repos.

Import/Edit|Update/Remove verbs

## Search

Aggregate datat for unified search.
ONLY search, no CRUD here.
Go for most efficient solution.

Search for projects only.

Search for users/groups nice to have.

## Integrate storage service with native projects

Switch from gitlab project ids to native project ids.

Nice to have: associate with users/groups as well

## CLI for more local support

Mount storage, run environments, load datasets, syncing metadata commands, more edit operations
May be multiple shapeups

## Provenance service
store project provenance. probably a Graph db.
- ingestion
- visualization
- search?

immutable

## CLI for provenance tracking

`renku run` with less features but more detailed tracking. Should work with select workflow engines (snakemake?)

## APIs for various services
Like a python client library, also other languages if possible, for interacting with all the other stuff
Lower priority, possibly multiple different pitches. Ideally dogfooded.

## historical data for entities
keep history tables for relevant entities. Like what environment and dataset was used at which point in time.
By time, not commit.
Add web api for this.

Probably UI support?

## CLI support for historical data
CLI support to see historical data.
CLI support to recreate a point in time.

## Environment import/reuse
importing environments from other projects

## Project templates to repository templates
existing project templates are now repository templates
Consider moving to something more standard like cookiecutter
track template use on project/repository entity

## Template search/reuse
allow templates to be created from repositories and published, with search

## SSH solution
allow sshing into sessions. Better handling of keys

## Dataset publishing
support for publishing datasets to ZenodoVerse based on a project

## Third party compute for sessions
Allow running sessions on third party compute (e.g. add credentials and run on AWS)

## Permission management
Manage permissions on a group/project level
With an admin UI for admins