* This doc has notes from previous OpenDAL discussions and agendas for upcoming syncs. * Content here will be concluded to the maillist after each sync. * Please add agenda items that you’d like to discuss. > Useful Links > * Google Meet Links: https://meet.google.com/xvv-vcny-jxn > * OpenDAL Community: https://opendal.apache.org/community/ --- # Upcoming Sync --- # Previous Sync Notes ## 2024-05-31 ### Releases v0.47 is about to come. ## 2024-05-10 ### Releases v0.46 has been released. ## 2024-04-19 ### Highlights - OpenDAL oio Refactor - Upcoming v0.46.0 release ### Releases No release yet. ### Discussion No discussion yet. ## 2024-03-29 ### Highlights - New oio API - ofs progress - GSoC Event - OSPP Event ### Releases No release yet. ### Discussion ## 2024-03-08 ### Highlights - Happy new year! ### Releases #### v0.45.1 The first release for all packages. ### Discussion - All in async trait from ```rust trait opendal::raw::oio::Read { fn poll_read(&mut self, cx: Context<'_>,bs: &mut [u8]) -> Poll<Result<usize>>; } ``` into ```rust trait opendal::raw::oio::Read { async read(&mut self, bs: &mut [u8]) -> Result<usize>; } trait opendal::raw::oio::ReadDyn { fn read_dyn(&mut self, bs: &mut [u8]) -> Pin<Box<dyn Future<Output=Result<usize>>>> } ``` ## 2024-01-26 ### Highlights - New Committer, Wenkang Xu [@WenyXu](https://github.com/wenyxu). - [Apache OpenDAL Graduated](https://opendal.apache.org/blog/apache-opendal-graduated) - New PMC Member, Liuqing Yue [@dqhl76](https://github.com/dqhl76) - New Committer, Yang Shuai [@hoslo](https://github.com/hoslo) ### Releases **v0.44.2** - **New Cloud Service Support:** Added support for pCloud, Yandex Disk, Koofr, and Apple iCloud Drive, enhancing the range of cloud storage options available. - **Concurrent Writing Enhancements:** Implemented various features for concurrent writing and associated behavior tests, significantly improving concurrent data processing capabilities. - **HDFS and WebHDFS Improvements:** Introduced atomic write features and multi-write via CONCAT for HDFS, increasing reliability and efficiency in HDFS operations. - **GDrive and S3 Updates:** Implemented path caching for GDrive and introduced a trash mechanism for file deletion. Improved error handling in S3 when credentials are empty, enhancing user experience and error management. - **Bug Fixes and Refactoring:** Addressed various bugs and refactored code across several services, including fixing issues in RangeWriter and BlockWrite, and reorganizing test layouts. - **Documentation and CI Updates:** Improved documentation wording, updated copyright and trademark information, and made adjustments in CI tests and dependency updates, ensuring better clarity and smoother development workflow. ### Discussion - [Looking for GSoC 2024 Project Ideas](https://github.com/apache/opendal/discussions/4076) - ofs (Need an RFC) - SDPK blobstore ## 2024-01-05 ### Highlights - New Committer: Congyi - Graduation Started! - New sub-project: ofs - New Services: pcloud (Thanks @hoslo) - Happy New Year! - New RFC: Concurrent Write ### Releases #### v0.44.0 - New Services: seafile (Thanks @hoslo) - New Services: upyun (Thanks @hoslo) - New RFC: Buffered Reader - OpenDAL is available on WASM32 - s3 - azfile - gdrive - gcs #### v0.44.1 - ASF branding changes ### Discussion - New RFC: Deleter API ## 2023-12-15 ### Highlights - New PPMC Member Xiangdong (@G-XD) - Huggingface File System Support - ConcurrentFutures to remove the depends on tokio  - [Almost all graduation preparations have been completed.](https://github.com/apache/incubator-opendal/issues/3283) - Dependencies Analysis (thanks @Xuanwo) - Maturity Model (thanks @suyanhanx) - Versioned Documents with Nightlies ### Releases - V0.43 - Upgrade to v0.43 - Public API - List Recursive - List Prefix -> changed the behavior of stat a dir path - Raw API - Lister Align - New Features - Enhancements in language bindings - retry layer, capability support and read/write stream for nodejs (thanks @suyanhanx) - Extended support for new services and storage solutions - alluxio and b2 support (thanks @hoslo) - Changes - Refactoring in testing and workflow planning - Refactor the testing workflow for 7 services. - Documentation - Add basic usage for java binding ### Discussion - [\[DISCUSS\] How about changing OpenDAL Community Sync time?](https://lists.apache.org/thread/nfggny089hswrldjxqngkf4zppn6ph9k) - [\[DISCUSS\] Graduate OpenDAL as TLP?](https://lists.apache.org/thread/kq00ynqtbbwsh2n7485s5vypzjropck6) - [\[RFC\] Buffered Reader](https://github.com/apache/incubator-opendal/pull/3734) ## 2023-11-24 ### Highlights * OpenStack Swift support (Thanks @morristai) * Alluxio rest API support (Thanks @hoslo) * B2 support (Thanks @hoslo) * Add File support for python binding (Thanks @xuanwo) * Service Config * New User! * questdb: An open source time-series database for fast ingest and SQL queries * RFC: Concurrent Stat in List (Thanks @morristai) * Change List API into poll based and return one entry instead (Thanks @xuanwo) * Benchmark vs s3 & vs fs * Capability and layer support for nodejs binding (Thanks @suyan) ### Releases * v0.42 * New Features * Enhancements in language bindings * including new behavior testing, async operations support, and renaming capabilities. * Examples include feat(binding/python): Support AsyncOperator tests and feat(binding/java): add rename support. * Extended support for new services and storage solutions * feat(service/libsql): support libsql * feat(service/azfile): add azure file service support. * Introduction of new monitoring and metrics features for Prometheus, like * feat(prometheus): add bytes metrics as counter * feat(layer/prometheus): Support custom metric bucket for Histogram. * Changes * Refactoring in testing and workflow planning * Introduce OpenDAL Workflow Planner * refactor(services/gdrive): Extract folder search logic. * Enhancements in code cleanliness and consistency, like * refactor(bindings/C): Implement error with error message * refactor(core): use list_with in Operator::list. * Fixes: * Bug fixes ensuring stability and reliability, like * fix: Fix read result not full * fix(services/s3): Accept List responses without ETag. * Documentation: * Addition of new documentation and updates for better understanding and usage of features, such as * docs: Add docs in website for sqlite/mysql/postgresql services and docs: update release doc. ### Discussion * List Prefix Progress (from @xuanwo) * For example, we can speed up the list(“aaa”) for this dir. * aaa1 * aaa2 * aaa3 * bbb1 * bbb2 * OpenDAL benchmark vs AWS SDK s3 * Which language that AWS SDK s3 written in? * Rust ## 2023-11-03 ### Highlights * New Committers @manjusaka! * Milvus’s OpenDAL C binding integration has been merged (Thanks @jiaoew and @yah01) * https://github.com/milvus-io/milvus/pull/25642 * New Test Framework * Core’s New Test Framework * Java’s New Test Framework (Thanks @tison and @G-XD) * Python’s New Test Framework (Thanks @manjusaka) * Nodejs’s New Test Framework (Thanks @eryue0220) * Core * Lazy Reader support (Thanks @Xuanwo) * C binding * add reader support (Thanks @jiaoew) * add error message support (Thanks @jixinyou) * API refactor (remove the blokcing prefix) (Thanks @jixinyou) ### Releases * No release so far. * We have a release plan in next week ### Discussion * OpenDAL Incubator Report - November 2023 * https://lists.apache.org/thread/6c4fd4bxh7bpokf8d8h3r7l6dhk0lcmv * Prepare for graduation * OpenDAL RFC Policy * https://lists.apache.org/thread/60580mcwtzjxo8bcbd56j0kql0k3cm8t * OpenDAL Release Policy * https://lists.apache.org/thread/fyd2fz39w8ygc5rqkw1olhytw016scp5 * When will we make a formal C binding release? (from @jiaoew) * We need to address some issues first: * Add test framework for C binding * https://github.com/apache/incubator-opendal/issues/3472 * Add API Docs and Guide for C binding ## 2023-10-13 ### Highlights * New Committer G-XD! * New Services! * mysql * sqlite * d1 * libsql * New Test Framework! * java * python * New Workflow Planner! (@xuanwo) * New Users! * pants * dalfs * manas * terraphim-ai ### Releases * V0.41 * Rust Core * Add PrometheusClientLayer (thanks @flaneur2020) * Support set offset in ghac write and other polish (thanks @Huon Wilson) * Add mysql support (thanks @Zheaoli) * Add postgresql connection pool support (thanks @Zheaoli) * Add sqlite support (thanks @Zheaoli) * Java Binding * New Test Framework (thanks @G-XD) * oay * Extract dav-server-fs-opendal (thanks @Young-Flash) ### Discussion * 2023-09-22 ### Highlights * OwO #1 * OpenDAL v0.40 has been released ### Releases * v0.40 * Rust Core * API Cleanup * Merge append into write * Add lister and list to replace list * Merge range_read into read * oio::Write::copy_from removed * New Features * Buffered Write * Java Binding * Release setup * Blocking layer enabled * Cpp Binding * Release setup * Oay * Webdav supported ### Discussion * Are we ready to graduate from the incubator? * https://incubator.apache.org/guides/graduation.html * Move root from services config to operator? * https://github.com/apache/incubator-opendal/issues/3151 * Expose services options so that users can working on struct? * https://github.com/apache/incubator-opendal/issues/3152 ## 2023-09-01 ### Highlights * The first testing sync happened! * Gdrive test has been setup (Thanks, suyan!) * Oay’s webdav test has been passed (Thanks, young-flash!) * Java binding release workflow (Thanks, tison!) * Cpp binding is started (Thanks, silver!) * New Users * FileCodeBox (python binding) * Milvus (c binding) ### Releases ### Discussion * OpenDAL new release workflow * https://opendal.apache.org/docs/contributing/release * OpenDAL v0.40 Release Plan * Some crazy ideas * OpenDAL FS * OpenDAL REST API * OpenDAL Cache