This was extracted (@ 2025-04-16 22:10) from a list of minutes
which have been approved by the Board.
Please Note
The Board typically approves the minutes of the previous meeting at the
beginning of every Board meeting; therefore, the list below does not
normally contain details from the minutes of the most recent Board meeting.
WARNING: these pages may omit some original contents of the minutes.
Meeting times vary, the exact schedule is available to ASF Members and Officers, search for "calendar" in the Foundation's private index page (svn:foundation/private-index.html).
## Description: The mission of Apache Celeborn is the creation and maintenance of software related to an intermediate data service for big data computing engines to boost performance, stability, and flexibility ## Project Status: Current project status: Ongoing Issues for the board: None ## Membership Data: Apache Celeborn was founded 2024-03-20 (a year ago) There are currently 24 committers and 16 PMC members in this project. The Committer-to-PMC ratio is 3:2. Community changes, past quarter: - Fei Wang was added to the PMC on 2024-12-31 - Sanskar Modi was added as committer on 2024-12-20 - Weijie Guo was added as committer on 2024-11-28 ## Project Activity: Software development activity: - Release 0.5.4 is in the process of discussion. - CIP-6 Support Flink hybrid shuffle is finished. - CIP-7 Celeborn CLI is finished. - CIP-8 Partition writer refactor is almost finished. - CIP-10 Introduce Celeborn Chaos Testing Framework is under development. - CIP-11 Support worker tags is finished. - CIP-12 Support HARD_SPLIT in PushMergedData is finished. - CIP-13 Support automatic scaling is under development. - CIP-14 Support CPP SDK is under development. - Skew partition optimization is almost finished. Meetups and Conferences: - 1 talk was given in Apache Spark & Paimon Meetup 2024. - 2 talks were given in FFA 2024. Recent releases: - 0.5.3 was released on January 6th, 2025. - 0.4.3 was released on December 10th, 2024. - 0.5.2 was released on November 26th, 2024. ## Community Health: Overall community health is good. In the past quarter, the dev mail list mail number had a 6% decrease in the past quarter to 152, the issues mail list had a 31% decrease to 1160, the PMC considered it as normal because the absolute number is relatively high. We have been performing extensive outreach for our users, and encouraging them to contribute back to the project. Also, we are active in making a voice in various conferences to attract more users.
No report was submitted.
## Description: The mission of Apache Celeborn is the creation and maintenance of software related to an intermediate data service for big data computing engines to boost performance, stability, and flexibility ## Project Status: Current project status: Ongoing Issues for the board: None ## Membership Data: Apache Celeborn was founded 2024-03-20 (8 months ago) There are currently 22 committers and 15 PMC members in this project. The Committer-to-PMC ratio is roughly 3:2. Community changes, past quarter: - Mridul Muralidharan was added to the PMC on 2024-09-20 - No new committers. Last addition was Fei Wang on 2024-07-23. ## Project Activity: Software development activity: - Release 0.5.2 is in the process of voting. - CIP-6 Support Flink hybrid shuffle is almost finished. - CIP-7 Celeborn CLI is under development. - CIP-11 Support worker tags is under development. - CIP-12 Support HARD_SPLIT in PushMergedData has passed a vote. - CIP-14 Support CPP SDK has passed a vote. - Support for Apache Tez is almost finished. Meetups and Conferences: - 1 related talk was given in Apache CoC NA 2024. Recent releases: - 0.5.1 was released on July 29th, 2024. - 0.4.2 was released on July 26th, 2024. - 0.5.0 was released on June 24th, 2024. ## Community Health: Overall community health is good. In the past quarter, the dev mail list mail number had a 50% decrease in the past quarter to 129, the PMC considered it as normal because the absolute number is relatively low. The issues mail list had a 14% increase to 1638. We have been performing extensive outreach for our users, and encouraging them to contribute back to the project. Also, we are active in making a voice in various conferences to attract more users.
## Description: The mission of Apache Celeborn is the creation and maintenance of software related to an intermediate data service for big data computing engines to boost performance, stability, and flexibility ## Project Status: Current project status: Ongoing Issues for the board: None ## Membership Data: Apache Celeborn was founded 2024-03-20 (5 months ago) There are currently 22 committers and 14 PMC members in this project. The Committer-to-PMC ratio is roughly 3:2. Community changes, past quarter: - Nicholas Jiang was added to the PMC on 2024-07-23 - Fei Wang was added as committer on 2024-07-23 ## Project Activity: Software development activity: - We released 0.5.1 on July 29th. - We released 0.4.2 on July 26th. - We released 0.5.0 on June 24th. - Support for Apache Flink 1.20 is merged. - Support for Apache Tez is under development. - Several CIPs have been discussed and voted, including Support Flink hybrid shuffle, Celeborn CLI, Chaos Testing Framework, etc. Meetups and Conferences: - 4 related talks were given in Apache CoC Asia 2024. Recent releases: - 0.5.1 was released on July 29th, 2024. - 0.4.2 was released on July 26th, 2024. - 0.5.0 was released on June 24th, 2024. ## Community Health: Overall community health is good. In the past quarter, dev mail list had a 73% increase in past quarter. We have been performing extensive outreach for our users, and encouraging them to contribute back to the project. Also, we are active in making a voice in various conferences to attract more users.
## Description: The mission of Apache Celeborn is the creation and maintenance of software related to an intermediate data service for big data computing engines to boost performance, stability, and flexibility ## Project Status: Current project status: New Issues for the board: None ## Membership Data: Apache Celeborn was founded 2024-03-20 (3 months ago) There are currently 21 committers and 13 PMC members in this project. The Committer-to-PMC ratio is roughly 3:2. Community changes, past quarter: - No new PMC members (project graduated recently). - Chandni Singh was added as committer on 2024-03-21 - Mridul Muralidharan was added as committer on 2024-04-29 ## Project Activity: Software development activity: - We released 0.4.1 on May 28th. - We are in the process of releasing 0.5.0. - Memory file storage is merged to main. - An optimized AQE support is under review. Meetups and Conferences: - A talk was given in Apache Local Conference Hangzhou on May 28th. Recent releases: - 0.4.1 was released on May 22th, 2024. - 0.4.0-incubating was released on February 6th, 2024. ## Community Health: Overall community health is good. In the past quarter, user mail list had a 450% increase in traffic, dev mail list had a 6% decrease in traffic, the PMCs treat it as normal fluctuation. We have been performing extensive outreach for our users, and encouraging them to contribute back to the project. Also, we are active in making a voice in various conferences to attract more users.
## Description: The mission of Apache Celeborn is the creation and maintenance of software related to an intermediate data service for big data computing engines to boost performance, stability, and flexibility. ## Project Status: Current project status: New Issues for the board: None ## Membership Data: Apache Celeborn was founded 2024-03-20 (2 months ago) There are currently 21 committers and 13 PMC members in this project. The Committer-to-PMC ratio is roughly 3:2. Community changes, past quarter: - No new PMC members (project graduated recently). - Chandni Singh was added as committer on 2024-03-21 - Mridul Muralidharan was added as committer on 2024-04-29 ## Project Activity: Software development activity: - We are preparing to release 0.4.1 in May. - We are preparing to release 0.5.0 in May. - Security support (authentication and SSL) has been merged. - Memory storage is close to being merged. Meetups and Conferences: - An online meetup was held on April 16th with some developers. - An online meetup was held on April 25th with some users. Recent releases: - 0.4.0-incubating was released on 2024-02-06. - 0.3.2-incubating was released on 2024-01-08. ## Community Health: Overall community health is good. In the past quarter, dev/issues/users mail lists had 4%/2%/1100% increase in traffic respectively. In addition to encourage more discussions on mailing lists, we will also synchronize important information/conclusions to the mailing lists if they happen elsewhere. We have been performing extensive outreach for our users, and encouraging them to contribute back to the project. Also, we are active in making a voice in various conferences to attract more users.
The Apache Celeborn is an intermediate data service for Big Data compute engines (i.e. Apache Spark, Apache Flink, Apache Hadoop, etc.) to boost performance, stability, and flexibility. Intermediate data typically include shuffle and spilled data. The project state is New. It graduated on 2024-03-21. The project has completed the necessary graduation tasks[1], including update incubator status records, source repo changes, update website, pom.xml, and other resources to point to the new repository location. There are no Board-level issues at this time. Two releases were published in the past quarter: 2024-2-6 Release 0.4.0-incubating, 2024-1-8 Release 0.3.2-incubating The project stays active. In the last quarter, one new PPMC and two new committers were added, the number of contributors reaches 83. The key features the project is working on include dashboard, memory storage, enhanced AQE support, and authentication. Also there are planned key features, including support Tez, enhance Flink integration with Reduce Partition, support Object storage, adaptivity in splitting partitions, etc. The community plans to make at least two new releases in the current quarter. The project has written a draft to press@apache.org to make a major announcement for its graduation. The press has reviewed and commented with a question about which two quotes should be included in the announcement, the PMC has discussed and replied. The most recent date on which a committer was added is 2024-03-21, and the most recent date on which PMC member was added is 2024-02-24. There are no known branding or naming issues, either in the project or externally. There are no known legal issues or questions. [1] https://github.com/apache/celeborn/issues/2415
Establish the Apache Celeborn Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software, for distribution at no charge to the public, related to an intermediate data service for big data computing engines to boost performance, stability, and flexibility. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the "Apache Celeborn Project", be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache Celeborn Project be and hereby is responsible for the creation and maintenance of software related to an intermediate data service for big data computing engines to boost performance, stability, and flexibility; and be it further RESOLVED, that the office of "Vice President, Apache Celeborn" be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache Celeborn Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache Celeborn Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache Celeborn Project: * Becket Qin <jqin@apache.org> * Cheng Pan <chengpan@apache.org> * Duo Zhang <zhangduo@apache.org> * Ethan Feng <ethanfeng@apache.org> * Fu Chen <fchen@apache.org> * Jiashu Xiong <rexxiong@apache.org> * Kerwin Zhang <kerwinzhang@apache.org> * Keyong Zhou <zhouky@apache.org> * Lidong Dai <lidongdai@apache.org> * Willem Ning Jiang <ningjiang@apache.org> * Wu Wei <powerwu@apache.org> * Yi Zhu <angerszhuuuu@apache.org> * Yu Li <liyu@apache.org> NOW, THEREFORE, BE IT FURTHER RESOLVED, that Keyong Zhou be appointed to the office of Vice President, Apache Celeborn, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the Apache Celeborn Project be and hereby is tasked with the migration and rationalization of the Apache Incubator Celeborn podling; and be it further RESOLVED, that all responsibilities pertaining to the Apache Incubator Celeborn podling encumbered upon the Apache Incubator PMC are hereafter discharged. Special Order 7C, Establish the Apache Celeborn Project, was approved by Unanimous Vote of the directors present.
Celeborn is an intermediate data service for big data computing engines to
boost performance, stability, and flexibility.
Celeborn has been incubating since 2022-10-18.
### Three most important unfinished issues to address before graduating:
1. Grow the community
2. Release more versions
### Are there any issues that the IPMC or ASF Board need to be aware of?
None
### How has the community developed since the last report?
1. One new PPMC joined the community.
2. One new committer is proposed and under voting.
### How has the project developed since the last report?
1. One new release was published.
2. Important features like supporting Flink, graceful upgrade are GA.
### How would you assess the podling's maturity?
Please feel free to add your own commentary.
- [ ] Initial setup
- [ ] Working towards first release
- [X] Community building
- [ ] Nearing graduation
- [ ] Other:
### Date of last release:
2023-07-27, 0.3.0-incubating was released.
### When were the last committers or PPMC members elected?
2023-07-01, Jiashu Xiong was elected as our new PPMC member.
### Have your mentors been helpful and responsive?
Our mentors have been very helpful and responsive.
### Is the PPMC managing the podling's brand / trademarks?
The VP, Brand has approved the project name.
### Signed-off-by:
- [ ] (celeborn) Becket Qin
Comments: I'm glad to see the new version has been released.
- [X] (celeborn) Lidong Dai
Comments:
- [ ] (celeborn) Willem Ning Jiang
Comments:
- [ ] (celeborn) Duo Zhang
Comments:
- [X] (celeborn) Yu Li
Comments: It's great to see the latest 0.3.0 release supports more
computing engine (Apache Flink) thus extends its ecology. New
contributors joining also indicates a good growth of the community.
### IPMC/Shepherd notes:
Celeborn is an intermediate data service for big data computing engines to
boost performance, stability, and flexibility.
Celeborn has been incubating since 2022-10-18.
### Three most important unfinished issues to address before graduating:
1. Grow the community
### Are there any issues that the IPMC or ASF Board need to be aware of?
- N/A
### How has the community developed since the last report?
1. Three new committers joined the community.
### How has the project developed since the last report?
1. Two releases are published.
2. Supporting Flink is reaching GA.
### How would you assess the podling's maturity?
Please feel free to add your own commentary.
- [ ] Initial setup
- [ ] Working towards first release
- [X] Community building
- [ ] Nearing graduation
- [ ] Other:
### Date of last release:
- 2023-03-29, 0.2.1-incubating was released.
### When were the last committers or PPMC members elected?
- 2023-03-17, binjieyang was added as a new committer.
### Have your mentors been helpful and responsive?
- Our mentors have been very helpful and responsive.
### Is the PPMC managing the podling's brand / trademarks?
- The VP, Brand has approved the project name.
### Signed-off-by:
- [ ] (celeborn) Becket Qin
Comments:
- [X] (celeborn) Lidong Dai
Comments:
- [X] (celeborn) Willem Ning Jiang
Comments:
- [X] (celeborn) Duo Zhang
Comments:
- [X] (celeborn) Yu Li
Comments: Congratulations on successfully publishing the first apache
release, and glad to see new committers joining the community
### IPMC/Shepherd notes:
jmclean: Please list three things under import issues.
Celeborn is an intermediate data service for big data computing engines to
boost performance, stability, and flexibility.
Celeborn has been incubating since 2022-10-18.
### Three most important unfinished issues to address before graduating:
1. Publish our first release.
2. Grow the community.
### Are there any issues that the IPMC or ASF Board need to be aware of?
* N/A
### How has the community developed since the last report?
1. 2 new contributors have joined the community.
### How has the project developed since the last report?
1. Created 3 release candidates.
### How would you assess the podling's maturity?
Please feel free to add your own commentary.
- [X] Initial setup
- [X] Working towards first release
- [ ] Community building
- [ ] Nearing graduation
- [ ] Other:
### Date of last release:
* N/A
### When were the last committers or PPMC members elected?
* 2022-11-21, nafiyaix was added as new committer.
### Have your mentors been helpful and responsive?
* Our mentors have been very helpful and responsive.
### Is the PPMC managing the podling's brand / trademarks?
* The VP, Brand has approved the project name.
### Signed-off-by:
- [ ] (celeborn) Becket Qin
Comments:
- [ ] (celeborn) Lidong Dai
Comments:
- [X] (celeborn) Willem Ning Jiang
Comments: Podling is working on the first Apache release
- [X] (celeborn) Duo Zhang
Comments:
- [X] (celeborn) Yu Li
Comments: The first Apache release is in the RC phase and now handling
license/dependency issues, and expected to come out soon.
### IPMC/Shepherd notes:
Celeborn is an intermediate data service for big data computing engines to
boost performance, stability, and flexibility.
Celeborn has been incubating since 2022-10-18.
### Three most important unfinished issues to address before graduating:
1. Publish our first release
2. Grow the community
3. Improve our website
### Are there any issues that the IPMC or ASF Board need to be aware of?
* N/A
### How has the community developed since the last report?
1. Committed our LOGO
2. Resolved the suitable name search for "Apache Celeborn"
3. Improved our website
### How has the project developed since the last report?
1. Prepare and start voting for our first apache release
2. Support rolling upgrade
3. Support load-aware slot allocation
### How would you assess the podling's maturity?
Please feel free to add your own commentary.
- [ ] Initial setup
- [X] Working towards first release
- [ ] Community building
- [ ] Nearing graduation
- [ ] Other:
### Date of last release:
XXXX-XX-XX
### When were the last committers or PPMC members elected?
* 2022-11-21, nafiyaix was added as new committer.
### Have your mentors been helpful and responsive?
* Our memtors have been very helpful and responsive.
### Is the PPMC managing the podling's brand / trademarks?
* The VP, Brand has approved the project name.
### Signed-off-by:
- [ ] (celeborn) Becket Qin
Comments:
- [X] (celeborn) Lidong Dai
Comments:
- [X] (celeborn) Willem Ning Jiang
Comments: Podling is Working on the first release of ASF.
- [X] (celeborn) Duo Zhang
Comments:
- [X] (celeborn) Yu Li
Comments: From the dev mailing list discussion I could see the
community is working actively towards the first apache release, and I
believe the team could work it out in the next report cycle.
### IPMC/Shepherd notes:
Celeborn is an intermediate data service for big data computing engines to boost performance, stability, and flexibility. Celeborn has been incubating since 2022-10-18. ### Three most important unfinished issues to address before graduating: 1. Publish our first release 2. Apply for the Trademark 3. Grow the community ### Are there any issues that the IPMC or ASF Board need to be aware of? * N/A ### How has the community developed since the last report? 1. One new committer was elected 2. 4 new contributors have joinied the community 3. Hosted 4 community meetups ### How has the project developed since the last report? 1. The main repo and website repo have migrated to Apache organization 2. We have designed the LOGO, and are preparing to commit it 3. Migrated issue tracking to JIRA ### How would you assess the podling's maturity? Please feel free to add your own commentary. - [ ] Initial setup - [X] Working towards first release - [ ] Community building - [ ] Nearing graduation - [ ] Other: ### Date of last release: * N/A ### When were the last committers or PPMC members elected? * 2022-11-21, nafiyaix was added as new committer. ### Have your mentors been helpful and responsive? * Our mentors have been very helpful and responsive. ### Is the PPMC managing the podling's brand / trademarks? * We have designed our LOGO and are preparing to commit it. ### Signed-off-by: - [ ] (celeborn) Becket Qin Comments: - [ ] (celeborn) Lidong Dai Comments: - [ ] (celeborn) Willem Ning Jiang Comments: - [ ] (celeborn) Duo Zhang Comments: - [X] (celeborn) Yu Li Comments: It's good to see new committer joining, and the community is actively collaborating towards its first Apache release ### IPMC/Shepherd notes:
Celeborn is an intermediate data service for big data computing engines to
boost performance, stability, and flexibility.
Celeborn has been incubating since 2022-10-18.
### Three most important unfinished issues to address before graduating:
1. Finish the process of Initial Code Import
2. Publish our first release
3. Grow the community
### Are there any issues that the IPMC or ASF Board need to be aware of?
* N/A
### How has the community developed since the last report?
* This is our first report.
### How has the project developed since the last report?
* This is our first report.
### How would you assess the podling's maturity?
Please feel free to add your own commentary.
- [X] Initial setup
- [ ] Working towards first release
- [ ] Community building
- [ ] Nearing graduation
- [ ] Other:
### Date of last release:
* N/A
### When were the last committers or PPMC members elected?
* 2022-10-18, when the project was accepted into the incubator.
### Have your mentors been helpful and responsive?
* Our mentors have been very helpful and responsive.
### Is the PPMC managing the podling's brand / trademarks?
* We are in process of designing our LOGO.
### Signed-off-by:
- [ ] (celeborn) Becket Qin
Comments:
- [ ] (celeborn) Lidong Dai
Comments:
- [X] (celeborn) Willem Ning Jiang
Comments:
- [X] (celeborn) Duo Zhang
Comments:
- [X] (celeborn) Yu Li
Comments: Podling bootstrap almost done, repository migration in
progress. PPMC is very cooperative and responsive.