Multi-Platform App Dev in Rust: Systems Challenges

# Multi-Platform App Dev in Rust: Systems Challenges ## Building Apps with their Dependencies (incl. UI toolkits) * Need to handle complex, multi-step builds beyond just Rust/Cargo * Cargo can't and won't support arbitrary post-build actions * Security issues, lack of maintainer bandwidth, out of scope for cargo itself * Pre-compilation build.rs helps, but doesn't suffice * Good for building native dependencies (Robius does this for Android platform support crates) * Good for generating code, data, etc that should make it into the final binary * Not sufficient for * Doesn't support packaging, signing, or any other post-compilation steps * Need build systems that have better interop with Rust, Cargo, and rustc * Configuring a Cargo/rustc build is not easy with existing build tools * Build tools typically introduce a *third* configuration space -- too many for a dev to manage * Cargo's configuration space: `features`, cargo flags, profiles, etc * Rustc's configuration space: `cfg`, rustflags, target spec, etc * Need to support *easily* building non-Rust (native) code * Cannot avoid native code on some platforms (primarily Android) * e.g., must compile custom Java classes needed for passing to/from Android platform APIs * Usually avoidable or automatic when building on a Linux host, thanks to `pkg-config` * Frequently needed on macOS/Windows where package managers are nonstandardized * Must be invokable seamlessly as part of a Rust build process (ideally only when necessary) * Must be triggerable from third-party crates' build process * That is, any crate in the dependency tree must be able to build its own native code * Not buildable only from a top-level crate or first-party app * Must know about, discover, and support platform-specific toolchains * `cargo-makepad` handles a lot of this for Makepad apps * But uses its own custom versions of each tool for a minimal install * Ideally, UI tool shouldn't require custom build infra * But many do, mostly out of necessity because nothing else existed when they first started work * Future goal: UI tools can be built with a common Robius/OSIRIS top-level build tool * *(✅ partially solved)* Standard configuration of non-Rust builds * Robius's [`android-build`](https://github.com/project-robius/android-build) crate handles this for Android * Discovery of installed Android SDK/NDK on Windows, Linux, and macOS hosts * Clear docs about env vars and how the dev can customize the build * Similar to what `cc-rs` offers for C/C++ builds * Need standardized way for each crate in dependency chain to specify any required permissions/metadata * Right now the top-level app must know about and declare *all* permissions needed by the entire dependency chain. * This is obviously not feasible long-term * Potential solution: use cargo metadata * Define standard format for key-value metadata that a build script or top-level build tool can parse * Also need tool for merging multiple app manifests into a single one ## Bundling, Packaging, Distributing * Most app stores and package managers require apps to be: * Signed, by the app dev with a pre-provided key * Bundled, using standardized or special packaging tools * Published, using the app store tooling * Most of this is done in proprietary ways with special tools * These tools do not typically offer APIs for external programmatic access * There's not a lot we can actually do here... 🙁 * Must thoroughly document procedure to correctly direct developer actions * Mobile platforms, particular Apple, are the most difficult * Complex workflow with many proprietary, closed tools (Xcode) * Currently, OSIRIS *does* support bundling and distributing via TestFlight * But dev must first create a provisioning profile for their app using Xcode * Our tool can collect package metadata from an app's Cargo.toml * e.g., app pkg name, icons, version info, *maybe * permissions * Then, guide developers through the rest of the procedures via docs * Google Play store requires submitting intermediary build artifacts now * Can't just submit full APK any more. * Thus, local development build must differ from a final publishing build * Build infra must explicitly support various build modes * Dev/debug, Dev/release, Publish/release * Bundling/packaging tools have strict requirements * e.g., no arbitrary executables beyond what your app's manifest specifies * **Important**: signing/bundling is ***required for certain platform APIs to work*** * Example: Apple does not display system notifications at all if the app code is not packaged into a signed bundle. ## Integration with UI toolkits * Each UI toolkit is vastly different * We mostly care about what platform resources the UI toolkit "owns" and controls * How do they internally represent such states? * How can external crates interface with or access those states? * Platform support crates must be easy to integrate with UI system internals * Example: [`robius-android-env`](https://github.com/project-robius/robius-android-env/) exists to simplify how third-party feature crates can access UI-owned JNI env and current activity context set up by the UI toolkit ## Exposing & Abstracting Platform features / OS services * Obvious challenge: there are *many* categories of features and *many* platforms * Many hundreds of unique combinations * Platforms tend to be *very* different * Very little continuity across mobile platforms * Some minor similarities on desktop (especially Unix-like) * Majority of mobile-specific features have not ever been attempted by the Rust community * But some desktop features have been tackled ### Goals / guiding philosophy * Native-first (or close to it as possible); abstractions later * Don't reimplement something in Rust just "because" * e.g., context menus, IMEs, file/image pickers * Keep implementations simple, with no fluff * We don’t want to impose unavoidable overhead just to provide Rust access to a platform API. * Developers should never be forced to drop down to platform-specific code * But, these crates should allow easy access to underlying platform if desired * Minimize dependencies * Keeps compile times low * Live update provided by UI toolkits isn't very useful if compilation takes 30 minutes * Reduces size of code and binaries * For ex, early on in our journey we were using `tokio`’s one-shot channels for callbacks in some of the platform crates * But then we realized just how heavyweight certain deps like that can be (same for things like serde), so we only now use such deps if they’re unavoidable. * In the case where we actually do need them, we can offer a minimal pared-down version of that crate that can be used by the framework by default * Then, if an application does need the full crate, it can replace it with a Cargo-level patch statement ### Platform features & OS APIs of interest > > **Legend**: > > * *Orange:* In-progress > > * *Green:* Completed > > * *Clear:* In backlog (not started) * System notifications * Partially handled, only for desktop platforms * Biometrics & authentication * ✅ Fully supported on all platforms via [`robius-authentication`](https://github.com/project-robius/robius-authentication) * File/image picker dialogs * Context menus (OS-native) * Menu bar (desktop only) * Tauri's [`muda` crate](https://github.com/tauri-apps/muda) should suffice * Rich clipboard * Plaintext clipboard already supported in Makepad * Drag & Drop * Simple cases supported in Makepad * Default URI/Intent handling * Will be impl'd by Robius by mid-2024 * Content sharing (open URI) in foreign app * ✅ Fully supported on all platforms via [`robius-open`](https://github.com/project-robius/robius-open) * Native font access * Will be handled by Robius by end of 2024 * Input Method Editors (IME) * Very complex. Needs collab between Robius, Makepad, and Linebender * Native gesture recognition * Connectivity management * WiFi, Bluetooth, NFC, nearby devices * GPS/Location access * Work-in-progress by Robius * Multimedia capture * Device discovery & configuration * Camera – snapshots, video, settings * Partially supported by Makepad * Audio – input, MIDI, playback * Partially supported by Makepad * Sensors * Native alarms, timers * Keychain (secret storage) * Power management/status * App lifecycle state transitions * Robius added some of this to Makepad (missing full state transition map for Apple platforms) * See the [first few variants of the `Event` enum here](https://github.com/makepad/makepad/blob/8180b32d8fd301c8970d91b7f5344dd70fadb581/platform/src/event/event.rs#L29) * And [this relevant commit](https://github.com/makepad/makepad/commit/c6edd5522f27fc6fd93f0025146d68fc59f8e93d) defining the new Foreground/Background events and how they map to Android's platform-native Activity lifecycle callbacks * Robius intends to extract this into a separate crates to support external observers of lifecycle transitions * and many more... ### Platforms of interest All of the following are fully supported at the moment, except for those listed under `Other`. * Mobile * Android * iOS * Secondary targets: Apple's watchOS & tvOS * Desktop * macOS * Linux (Debian-based distros) * Windows * Web (w/ WASM) * Other (unsupported) * OpenHarmony OS * Microcontrollers, select peripherals (?) ## Managing concurrency: multi-threading, async/sync * UI & other select workloads must run on “main” thread * Certain platform APIs only work if called from the same thread or thread group as what draws the UI * A lot of existing SDKs require async, some don't allow it * e.g., Matrix Rust SDK only exposes `async fn`s for many key features * Other system APIs should *not* be invoked from an async context * If a system API will block the OS-native thread, calling it from an async context is **bad** because it will cause all other async tasks running on that same native OS thread to be blocked until the * Never want to block the main UI thread --> never invoke async fns on main thread * Must weave complex web of inter-context communication (e.g., using channels, condvars) * Rust enums do make this constant message passing fairly easy and safe, but it is still very difficult to debug if something doesn't work * Opinion: app devs shouldn't have to deal with 3 different types of execution contexts * Can be hard to figure out what should run where, and what can/cannot run where * Goal: create an abstraction crate that provides macros/helpers to auto-offload certain workloads to background sync threads or background async tasks * Future: discover and expose heterogeneous CPU topology * e.g., ARM's `bigLITTLE` (now `dynamIQ`) * Allow app logic, UI toolkit, and platform support crates to define which class of CPU a given workload should run on * Set affinity for background low-priority workloads to prefer LITTLE/low-power CPUs *