Kids
Younes: 5th Grade
Hamza: 5th Grade - 11 yr old in Nov
Belal: 3th Grade - 9 yr old in Oct
Belal: 2nd Grade - 8 yr old in April
Yaseen: 4th Grade -
Mohamed: 8th Grade - 13
Modo: 12 years old
Omar: 5th Grade
Moaaz : 8 years old
Linux Foundation AI&Data Flyte.org
The Workflow Automation Platform for Complex, Mission-Critical Data and ML Processes at Scale
Haytham Abuelfutuh
haytham@union.ai
Why
Flyte is used in production at Lyft, Spotify, Freenome and many others.
Flyte is Battle-Tested and Truly Open-Source
Flyte system consists of multiple components. For the purposes of this document, let's categorize them into server-side and client-side components:
Admin: A server-side control plane component accessible from console, cli and other backends.
Catalog: A server-side control plane component accessible from console, cli and other backends.
Console: A client-side single page react app.
Cli: A client-side (Command Line Interface) component that interacts with Admin and Catalog.
Propeller: A server-side data plane component that interacts with both admin and catalog services.
Required Flows
User authN using OpenIdConnect
Added CoPilot configuration to sandbox manifest
Updated and streamlined documentation navigation and themes. Better content organization coming soon!
Performance
Improved execution performance including:
Reduced cache lookups
Improved GetWorkflowExecution performance
Capped max number of nodes in each propeller round
Katrina Rogan changed 4 years agoView mode Like Bookmark
The number of shards in the cluster is completely dynamic...
numShards == numReplicas
The pod horizontal autoscaler can scale up and down based on the the CPU/Mem (or number of WFs in the system).
Algorithm:
Admin assigns a unique Int [0, maxBins] per execution randomly.
Propeller starts up, creates Leases that match the number of replicas
It attempts to acquire all the leases.
As soon as it acquires one, it drops the rest and starts processing.