Try   HackMD

Abstract

Arrow UDF is a User-Defined Function Framework for Apache Arrow.

Proposal

Arrow UDF allows user to easily create and run user-defined functions (UDF) in Rust, Python, Java or JavaScript based on Apache Arrow. The functions can be executed natively, or in WebAssembly, or in a remote server via Arrow Flight.

Arrow UDF was originally designed to be used by the RisingWave project but is now being used by Databend and several database startups.

We believe that the Arrow UDF project will provide diversity value to the entire Arrow community.

Background

Arrow UDF is being developed by an open-source community from day one and is owned by RisingWaveLabs. The project has been launched in December 2023.

Initial Goals

By transferring ownership of the project to the Apache Arrow, Arrow UDF expects to ensure its neutrality and further encourage and facilitate the adoption of Arrow UDF by the community.

Current Status

Contributors: 5

Users:

  • RisingWave: A Distributed SQL Database for Stream Processing.
  • Databend: An open-source cloud data warehouse that serves as a cost-effective alternative to Snowflake.

Documentation

The document of Arrow UDF is hosted at https://docs.rs/arrow-udf/latest/arrow_udf/.

Initial Source

The project currently holds a GitHub repository and multiple packages:

Rust:

Python:

Those packge will retain its name, while the repository will be moved to apache org.

Required Resources

Mailing Lists

We can reuse the existing mailing lists that arrow have.

Git Repositories

Option 1

Maintian all arrow udf implemantion in the same repo.

From

To

Option 2

Add Arrow UDF implementation to the corresponding language repository.

For example:

From

To

Issue Tracking

The project would like to continue using GitHub Issues.

Other Resources

The project has already chosen GitHub actions as continuous integration tools.

Initial Committers

Last changed by 
 
Xuanwo
5
434

Read more

Untitled

Justin has identified the following two types of issues: Refer to OpenDAL appropriately in accordance with Apache branding guidelines. Ensure all OpenDAL domains are under control. The OpenDAL PPMC has addressed all the issues. Below is a summary. For "Refer to OpenDAL appropriately in accordance with Apache branding guidelines": There are three kinds of websites:

Jan 15, 2024
数据迁移服务功能场景化描述

Create / Start Task 在界面上创建并开始任务。 流程如下 点击 创建任务 填写源端配置 填写源端类型 填写源端配置（根据类型不同，配置项也不同）

Mar 19, 2021
Read more from Xuanwo
Published on HackMD