Julep: Applications

This Julep discusses support for "apps" in the Julia package manager. Text in blue could potentially be left out from a first implementation. In this Julep, an app is defined as a Julia package with a @main function, intended to be run by the user as appname args to app. It should feel opaque to the user that the app is written in Julia. It's assumed that Julia is installed and serves as the "driver" to start up the app. This type of app thus differs from:

PackageCompiler apps, which bundle the Julia runtime, libraries, and artifacts to be completely standalone.
Statically compiled Julia binaries (juliac/StaticCompiler) that run without a runtime. It should however be possible to incorporate static apps into the system presented here as later work.
JLL applications. These are executables that are provided by JLLs. We also want to seamlessly incorporate these into the system so that one can add e.g. ImageMagick from a jll and use it as a normal executable.

High-Level Notes

The entry point of an app can be one of the following:
- Package.main
- Package.SubModule.main
- app/app.jl or app/App/app.jl with module app function @main ... end end in it.
  - This mimics how the code for an extension can exist in ext/Ext.jl or ext/Ext/Ext.jl.
  - Maybe apps/Project.toml + apps/Manifest.toml.
    - For adding extra dependencies to app that are only installed once the package is installed as an app. This might require regisrty support (AppDeps.toml)
Each package has a separate environment for its app(s) (.julia/environments/apps/PackageName).
The "shim" that starts Julia and runs the entry point for the app is installed in .julia/bin, which needs to be in the user's PATH.

`Project.toml` Additions

For packages exposing an app to users, a new [app] section lists the apps of the package. For example, a package (Rot13.jl) providing the app rot13:

# Project.toml
name = "Rot13"
uuid = "..."
version = "..."

[deps]
...


[compat]
...
ArgParse = "1"


[apps]
rot13 = {}
# This means rot13 -> Rot13.@main(ARGS)

Possible other ways of defining an app with a different entry point than `Rot13.main` could under `[app]` be:

[apps]

[apps.invrot13]
entry = "Rot13.InvRot13" # submodule of package

[apps.rot26]
entry = "rot13.jl" # file in Rot13/apps/
# or entry = "rot13/rot13.jl"

When the entry point is a file, it must be precompiled in the same way as an extension. Both submodules and files should have a `@main` function.

Pkg Additions

A new submodule App(s) is added to Pkg as well as new REPL commands pkg> app command. The section only describes the new "REPL" commands but the "API"-versions are straightforward to derive from it:

`add`

pkg> app add Rot13

Rot13, a registered package, is downloaded. The project file is examined for an app section.
An environment is created in .julia/environments/apps/Rot13:
- If bundle_manifest is false resolve with Rot13 as the only dependency.
- If bundle_manifest is true, copy the manifest and do an instantiate.

Update the .julia/environments/apps/AppManifest.toml (which is a manifest file containing metadata for all the apps). It should be possible to generate all the shims in .julia/bin from the AppManifest.toml file.

In this case, the AppManifest.toml is similar to a normal manifest but instead of each package having a list of deps they have a list of apps. It would look something like:

julia_version = "1.10.0"
manifest_format = "2.0"
project_hash = "622b0771..."

[[deps.Rot13]]
uuid = "..."
git-tree-sha1 = "8eb7b4d..."

[[deps.Rot13.apps]]
name = "rot13"
julia = "dev"


[[deps.lld_jll.apps]]
name = "lld"
#

The Julia path could either be an absolute path to the julia executable or alternatively a juliaup julia version specificer.

For each app, a "shim" bash/batch file is created in .julia/bin that launches the app. An example shim:

# julia_app_version = "1.0" (version of the shim generator)

# Check if $julia_executable_path exists; if not, throw a warning
JULIA_LOAD_PATH=$(homedir)/.julia/environments/apps/Rot13
exec $julia_executable_path \\
  --startup-file=no \\
  -m $(pkgname) \\
  "$@"

The app is here run in an unstacked environment and the -m flag to start the entry point for a package is used (https://github.com/JuliaLang/julia/pull/52103).

Either automatically put .julia/bin in the users .bashrc etc or check that Sys.which returns the installed app, if not, warn the user about their PATH.

rm
- pkg> app rm Rot13: Remove all shims for the Rot13 package (findable from the AppManifest.toml file) and the .julia/environments/apps/Rot13 folder
- pkg> app rm rot13: Remove the rot13 shim. If all apps for Rot13 are removed, remove the environment folder.
status
- pkg> app status: For each package with app, for each app, show some info about the app e.g:
```
Rot13
=====
rot13: {julia = "/path/to/executable/julia", ...}
...
```
- pkg> app status Rot13: Show info for all apps of Rot13 and show all the dependencies. Almost equivalent to pkg> activate "~/.julia/environments/apps/Rot13"; status
- pkg> app status rot13 same as above but only show info for rot13 app.
update
- pkg app update Rot13, activate "~/.julia/environments/apps/Rot13" and run an update on it.
- pkg app update Run the above on all packages in AppManifest.toml
- pkg app update Rot13 julia set the julia variable (the julia executable) to the current julia path.
dev
- pkg> app dev Rot13, download Rot13 in the same way as pkg> dev Rot13 does, set the path entry for Rot13 in AppManifest.toml and point the JULIA_LOAD_PATH to the package folder in the shim.
precompile
- Need to precompile with command line flags "valid"
- pkg> app precompile Rot13

Registry changes

There are strictly no needed registry changes for the proposal here to function. For things like auto-complete (e.g. app add Fo<TAB>) it could be good to have a bit in the registry to know if a package has any apps in it so that only those are auto-completed once you are tab-completing inside an app command. An error could also quicker be given if one does app add on a package with no apps if that information is present in the registry.

Code loading changes.

With the exception of allowing apps/app.jl in a package, there should be no need for any changes to loading.jl, everything is built on top of the existing code-loading system.

If apps/app.jl is allowed, there needs to be some code (which will look very similar to the package extension code) that maps an app to a UUID etc so it can be precompiled.

Environment variables

Environment variables controlling app behavior start with JULIA_APP_ (similar to e.g. JULIA_PKG_).
This environment variable takes precedence over the app-specific one.

JULIA_APP_ARGS: Extra arguments that are passed to the Julia executable that runs the app, for example, JULIA_APP_ARGS="--optimize=3 --threads=2"
JULIA_APP_JULIA: Path to a julia executable to run the app with.
- It would be good if JULIA_APP_JULIA="julia +1.9" worked for nice compat with juliaup.

Open questions

Exact interaction with DEPOT_PATH
Should Pkg itself install an app (thereby obsoleting jlpkg)
OS compatability on apps? Currently, no OS compat support is available in Pkg / Registry.
Should a package be able to be a library (currently all Julia packages are libraries) and at the same time be able to provide apps?
- It would be wasteful to require registering two entities to provide the Rot13 library and the rot13 app. On the other hand, there is no need for an arg parser dependency in Rot13-library but the app probably wants on so if those are forced to share a module the library would probably need to load the arg parser.
Avoid the bash/batch shim, hook into juliaup?
- This would avoid the duplication of info from AppManifest.toml and the shim. Something like juliaup could just read the manifest file directly.
- rot13 ARGS -> juliaup launchapp rot13 ARGS -> launch julia -m Rot13 ARGS
Libraries kicking out app precompile files, annoying? Should there exist .julia/compiled/apps/...? Or _app.ji which is not kicked out by loading.
pkg> app add lld_jll?
Tab completion for apps bash-completion

TODOs

Store julia version in AppManifest
Call julia +v if juliaup is installed instead of absolute path
Precompile, show precompile in status window.
REPL completions (complete installed apps / packages)
- Add to all commands
Make an ExampleApp (similar to Example.jl)
Env prefix remove when in app mode
Show warning if app not in PATH

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`	在筆記中貼入程式碼
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.

Julep: Applications

High-Level Notes

`Project.toml` Additions

Pkg Additions

`add`

`rm`

`status`

`update`

`dev`

`precompile`

Registry changes

Code loading changes.

Environment variables

Open questions

TODOs

Julep: Applications

High-Level Notes

Project.toml Additions

Pkg Additions

add

rm

status

update

dev

precompile

Registry changes

Code loading changes.

Environment variables

Open questions

TODOs

`Project.toml` Additions

`add`

`rm`

`status`

`update`

`dev`

`precompile`