This was extracted (@ 2023-03-23 00:10) from a list of minutes
which have been approved by the Board.
Please Note
The Board typically approves the minutes of the previous meeting at the
beginning of every Board meeting; therefore, the list below does not
normally contain details from the minutes of the most recent Board meeting.
WARNING: these pages may omit some original contents of the minutes.
Meeting times vary, the exact schedule is available to ASF Members and Officers, search for "calendar" in the Foundation's private index page (svn:foundation/private-index.html).
A report was expected, but not received
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: There are no issues requiring board attention. ## Membership Data: Apache Airflow was founded 2018-12-19 (4 years ago) There are currently 51 committers and 29 PMC members in this project. The Committer-to-PMC ratio is roughly 7:4. Community changes, past quarter: - No new PMC members. Last addition was Daniel Standish on 2022-09-22. - Niko Oliveira was added as committer on 2022-12-19 - Andrey Anshin was added as committer on 2022-11-28 ## Project Activity: We are on a pretty steady release cadence. Next release (2.5.1) is being voted upon as we are submitting this report. - Provider packages 2023-01-02 was released on 2023-01-05. - Provider packages apache-hive, microsoft-azure 2022-12-14 was released on 2022-12-19. - Apache Airflow 2.5.0 was released on 2022-12-02. ## Community Health: We are a very lively community with lots of activity. Work is underway for another Airflow Summit that for the first time will be in-person. - dev@airflow.apache.org had a 8% increase in traffic in the past quarter (390 emails compared to 358) - users@airflow.apache.org had a 31% decrease in traffic in the past quarter (53 emails compared to 76) - 1499 commits in the past quarter (8% increase) 223 code contributors in the past quarter (3% increase) - 1075 PRs opened on GitHub, past quarter (-6% change) - 1073 PRs closed on GitHub, past quarter (-5% change) - 422 issues opened on GitHub, past quarter (-7% change) - 418 issues closed on GitHub, past quarter (-6% change)
No report was submitted.
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: There are no issues requiring board attention ## Membership Data: Apache Airflow was founded 2018-12-19 (4 years ago) There are currently 48 committers and 27 PMC members in this project. The Committer-to-PMC ratio is roughly 8:5. Community changes, past quarter: - No new PMC members. Last addition was Jedidiah Cunningham on 2022-01-04. - No new committers. Last addition was Ping Zhang on 2022-05-31. We are in the process of inviting a couple of new committers and PMC members ## Project Activity: Apache Airflow 2.4.0 was released 2022-09-19 and was a big release for us as it adds some exciting new capabilities (that we are calling "Data-aware scheduling"). A write up of this new release with more detail is available at https://airflow.apache.org/blog/airflow-2.4.0/ Main package Releases: - Apache Airflow 2.4.0 was released on 2022-09-19. - Apache Airflow 2.3.3 was released on 2022-07-09. - Apache Airflow 2.3.4 was released on 2022-08-23. Sub/smaller package releases. - Provider packages 2022-08-15: 2022-09-08 were released on 2022-09-08. - Provider packages 2022-08-15 was released on 2022-08-18. - Provider packages 2022-08-10 was released on 2022-08-14. - Apache Airflow Python Client v2.3.0 was released on 2022-08-03. - Provider packages 2022-07-17 was released on 2022-07-20. - Provider packages 2022-07-13 was released on 2022-07-16. - Provider packages 2022-06-15 was released on 2022-06-22. Each of these "provider packages" are a sub-component of Airflow that we have split out into their own packages to a) allow easier upgrading for our users without affecting "Core" scheduling, and b) to have quicker release cycles of those components (they are more isolated and require less testing than the core scheduler/workflow engine.) ## Community Health: dev@airflow.apache.org had a 8% increase in traffic in the past quarter (390 emails compared to 358) users@airflow.apache.org had a 31% decrease in traffic in the past quarter (53 emails compared to 76) 1479 commits in the past quarter (-31% change) 234 code contributors in the past quarter (12% increase) 1093 PRs opened on GitHub, past quarter (-18% change) 1141 PRs closed on GitHub, past quarter (-14% change) 425 issues opened on GitHub, past quarter (-17% change) 465 issues closed on GitHub, past quarter (-22% change)
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: There are no issues requiring board attention. ## Membership Data: Apache Airflow was founded 2018-12-19 (3 years ago) There are currently 48 committers and 27 PMC members in this project. The Committer-to-PMC ratio is roughly 8:5. Community changes, past quarter: - No new PMC members. Last addition was Jedidiah Cunningham on 2022-01-04. - Ping Zhang was added as committer on 2022-05-31 ## Project Activity: We released the long-awaited 2.3.0 release with some exciting features. This was the biggest release so far after introducing Airflow 2 and it it involved quick turnaround of 2.3.1 and 2.3.2 (already released) with fixing teething problems encountered - but we got a very good involvement of our users who not only reported problems but also were willing to take part in testing of those issues and were very helpful in test phase. Seems like the community expectations and engagement is pretty well aligned with our approach with new releases. No big dramas even with such a big release. The list of features implemented: * Dynamic task mapping - AIP-42 (huge improvement in the way how users can build Dags) * New GridView replacing Task View that modernizes the “first” experience the users have with Airflow DAG monitoring * Tooling to purge historical data for Airflow Database * LocalKubernetesExecutor that allows for simpler deployments for users who wish to optimize their experiences when it comes to resource usage * DagProcessorManager as standalone process (AIP-43) which is the first of many steps towards improved security and full multi-tenant setup * JSON serialization for connections that simplifies credentials management for our users * Offline generation of DB migration scripts and “downgrade” feature - that make migration process reversible and allows power users to optimize their migration experience * ARM architecture support - for users who wish to run Airflow on ARM for both development and deployment (experimental) including docker image support After 2.3.* effort we focus on further, incremental steps of improving Airflow features - through Airflow Improvement Proposals (AIP) as part of the upcoming Airflow 2.4.* effort: * Improving security and multi-tenancy of Airflow AIP-44 Airflow Internal API * Implementing DAG versioning AIP-36 DAG Versioning * Adding data dependency management to Airflow AIP-48 Data Dependency Management and Data Driven Scheduling * Improving System Testing for Airflow for external integrations AIP-47 New design of Airflow System Tests Also smaller improvements and optimisations: * AIP-45 Remove double dag parsing in airflow run * AIP-46 Runtime isolation for airflow tasks and dag parsing ### Releases: * Apache Airflow 2.3.2 was released on 2022-06-04. * Apache Airflow 2.3.1 was released on 2022-05-25. * Apache Airflow Helm Chart 1.6.0 was released on 2022-05-20. * Provider packages 2022-05-12 was released on 2022-05-16. * 2.3.0 was released on 2022-04-30. * Provider packages 2022-04-30 was released on 2022-04-30. * Provider packages 2022-04-13 was released on 2022-04-22. * Provider packages 2022-04-07 was released on 2022-04-11. * Apache Airflow 2.2.5 was released on 2022-04-04. * Provider packages 2022-03-23 was released on 2022-03-26. * Provider packages 2022-03-15 was released on 2022-03-19. * Apache Airflow Helm Chart 1.5.0 was released on 2022-03-11. * Provider packages 2022-03-07 was released on 2022-03-10. # Events The Airflow Summit has been successfully completed. We got more than 7500 online users and more than 400 local participants in local events, via something that we consider as a first-of-a-kind distributed (not hybrid) event. We held local events in 13 location and we had 5 informal watch parties: * Local events: London, Tokyo, Lagos, New York, Bengaluru, SF Bay Area, Paris, Warsaw, Melbourne, Sydney, Sao-Paulo, Tel-Aviv, Seattle. * Watch parties: Atlanta, Chenai, Lisbon, Pune, Skopje. The event was completely free to attend. Talks were transmitted online and made immediately available to all registered users. We had 53 talks during the week and 5 interactive workshops with ~ 1900 USD of income (we will eventually donate all the workshop proceedings to the Apache Software Foundation - all workshops were done by volunteers) ## Community Health: Airflow continues to be the top ASF-contributed project (in terms of number of contributors) - we surpassed 2000 contributors overall (2066 on 4th of June 2022). We continue to attract new contributors - with seemingly accelerating speed. We put on hold some of the new discussions and features because of the - already packed - 2.3 release and Airflow Summit, but we are resuming the effort to complete the discussions on new features and lead some of them to completion in 2.4.0 release.
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: ## Membership Data: Apache Airflow was founded 2018-12-19 (3 years ago) There are currently 47 committers and 27 PMC members in this project. The Committer-to-PMC ratio is roughly 3:2. Community changes, past quarter: - Jedidiah Cunningham was added to the PMC on 2022-01-04 - Josh Fell was added as committer on 2022-02-18 - Malthe Borch was added as committer on 2022-02-18 ## Project Activity: ### In-progress work We are gearing up for a new MINOR release that should address AIP-42 Dynamic Task Mapping important feature mentioned in the previous report. The earlier investment and stabilization of the Airflow codebase enabled us to start actively working on bringing more improvements to Airflow in future versions. What is important, those improvements are coming from multiple stakeholders and coming from needs and often contributed code that has been or is being implemented by the stakeholders. * Improving security and multi-tenancy of Airflow * AIP-43 DAG Processor separation * AIP-44 Airflow Internal API * Implementing DAG versioning * AIP-36 DAG Versioning * Adding data dependency management to Airflow * AIP-48 Data Dependency Management and Data Driven Scheduling * Improving System Testing for Airflow for external integrations * AIP-47 New design of Airflow System Tests * Also smaller improvements and optimisations: * AIP-45 Remove double dag parsing in airflow run * AIP-46 Runtime isolation for airflow tasks and dag parsing ### Releases 2.2.4 was released on 2022-02-22. Provider packages 2022-02-14 was released on 2022-02-18. Provider packages 2022-02-09 was released on 2022-02-13. Provider packages 2022-01-08 was released on 2022-01-12. Apache Airflow Helm Chart 1.4.0 was released on 2022-01-10. Provider packages 2021-12-31 was released on 2022-01-06. 2.2.3 was released on 2021-12-21. Provider packages 2021-12-07 was released on 2021-12-15. ### Events The Airflow Summit Steering Committee continues preparation to the Airflow Summit which is supposed to be both online and local offsite event for the Airflow community where local communities will have a chance to meet in-person and take part in global event at the same time ## Community Health: * Airflow continues to be the top ASF-contributed project (in terms of number of contributors) - we are getting close to 2000 contributors overall (1966 on 6th of March 2022). Well ahead 2nd Apache Spark (1791). * In the first week of March 2022 Airflow exceeded 10.000 Forks and 25.000 Stars in Github * We continue to have a steady flow of contributions from both seasoned contributors and new contributors * We effectively utilize Github Discussions to separate Bugs/Features from discussions. We have around 200 unique new / 600 unique regular monthly users of Discussions * Some of the topics focus around special interest groups (multi-tenancy) where we semi-regularly meet and discuss the progress (video chats which we record, publish and summarize for all community to follow) - this is especially when we have multiple stakeholders interested in the areas
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: ## Membership Data: Apache Airflow was founded 2018-12-19 (3 years ago) There are currently 45 committers and 26 PMC members in this project. The Committer-to-PMC ratio is roughly 3:2. Community changes, past quarter: - No new PMC members. Last addition was Ephraim Anierobi on 2021-08-31. - No new committers. Last addition was Brent Bovenzi on 2021-08-27. ## Project Activity: Recent releases: Provider packages 2021-11-30 was released on 2021-12-06. 2.2.2 was released on 2021-11-15. Provider packages (Amazon) 2021-11-04 was released on 2021-11-11. Airflow Helm Chart Check 1.3.0 was released on 2021-11-08. Provider packages 2021-10-30 was released on 2021-11-04. 2.2.1 was released on 2021-10-29. 2.2.0 was released on 2021-10-11. Provider packages 2021-10-08 was released on 2021-10-11. Provider packages 2021-09-30 was released on 2021-10-05. Apache Airflow Helm Chart 1.2.0 was released on 2021-09-28. 2.1.4 was released on 2021-09-18. Provider Packages 2021-09-04 was released on 2021-09-08. ## Community Health: - There's quite a lot of interesting discussions going on, those include introducing multi-tenancy or redesigning Airflow’s executors as well as possible drop in support for MySQL. All those discussions show that Apache Airflow is actively used and developed and there are still things that can be improved. - To reward some of Airflow users and contributors we are running an end-of-year campaign to send swags to the most active and creative participants in the campaign. - Community voted a crucial Airflow Improvement Proposals that will have huge impact on future of Airflow: [AIP-42 Dynamic Task Mapping][1] ### Events: The Airflow Summit Steering Committee has started planning 2022 summits for the Airflow community. [1]: https://s.apache.org/od4lx
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines. ## Membership Data: There are currently 45 committers and 26 PMC members in this project. The Committer-to-PMC ratio is roughly 2:1. Community changes, past quarter: - Elad Kalif was added to the PMC on 2021-08-30 - Ephraim Anierobi was added to the PMC on 2021-08-31 (Just to mention it here: Ephraim is an alumni of the Outreachy program that Apache participated in. This is a cool story!) - Aneesh Joseph was added as committer on 2021-07-06 - Brent Bovenzi was added as committer on 2021-08-27 - Jed Cunningham was added as committer on 2021-06-25 - Tzu-ping Chung was added as committer on 2021-06-25 ## Project Activity: ### Releases: - Provider Packages 2021-09-04 was released on 2021-09-08. - Provider Packages 2021-08-30 was released on 2021-09-03. - Provider Packages 2021-08-24 (cncf.kubernetes) was released on 2021-08-27. - 2.1.3 was released on 2021-08-23. - Provider packages 2021-07-27 was released on 2021-08-02. - Apache Airflow Helm Chart 1.1.0 was released on 2021-07-26. - 2.1.2 was released on 2021-07-14. - 2.1.1 was released on 2021-07-02. - Provider packages 2021-06-26 was released on 2021-07-02. - upgrade-check-1.4.0 was released on 2021-06-26. - Provider packages 2021-06-18 was released on 2021-06-23. As of June 17, 2021 the 1.10.X version of Apache Airflow reached end of life. ### CVEs published - CVE-2021-38540: Apache Airflow: Variable Import endpoint missed authentication check < https://lists.apache.org/thread.html/rb34c3dd1a815456355217eef34060789f771b6f77c3a3dec77de2064%40%3Cusers.airflow.apache.org%3E > - CVE-2021-35936: Apache Airflow: No Authentication on Logging Server < https://lists.apache.org/thread.html/r53d6bd7b0a66f92ddaf1313282f10fec802e71246606dd30c16536df%40%3Cusers.airflow.apache.org%3E > ### Events: Airflow Summit 2021 <https://airflowsummit.org> took place July 8-16, 2021 which was attended by more than 10,000 participants from all over the world. That was up from 6000 attendees in Airflow 2020 and brought much higher quality content. We engaged meetup groups from 8 different cities around the globe, and hosted in 2 different time blocks to accommodate attendees from Europe, Americas and Asia. We are looking forward to the 2022 edition which - hopefully - will bring back the physical component, keeping the “free to attend” option for online conference. ## Community Health: The community seems to be very active despite the decrease in traffic on dev@airflow (probably due to the summer season). The number of commits and code contributions increased by 12% and 24% correspondingly. While the current big stakeholders - mainly Astronomer - grow stronger when it comes to a number of committers and PMC members, we also added committers and PMC members coming from different backgrounds and companies. There is also an increased activity from other stakeholders - Google and Amazon.
## Description:
The mission of Apache Airflow is the creation and maintenance of software
related to workflow automation and scheduling that can be used to author and
manage data pipelines.
## Membership Data:
There are currently 41 committers and 24 PMC members in this project. The
Committer-to-PMC ratio is roughly 2:1.
Community changes, past quarter:
- No new PMC members. Last addition was Xiaodong Deng on 2020-12-25.
- Xinbin Huang was added as committer on 2021-04-01
- Daniel Standish was added as committer on 2021-03-01
- Yu Qian was added as committer on 2021-04-06
## Project Activity:
### Releases:
- Apache Airflow Python Client v2.1.0 was released on 2021-05-31.
- Apache Airflow Helm Chart 1.0.0 was released on 2021-05-19.
- Provider packages 2021-05-01 was released on 2021-05-06.
- Apache Airflow Python Client v2.0.0 was released on 2021-04-23.
- 2.0.2 was released on 2021-04-19.
- Provider packages 2021-04-7 was released on 2021-04-13.
- Backport Provider Packages 2021.4.10 was released on 2021-04-10.
- Backport Provider Packages 2021.3.17 was released on 2021-03-18.
- 1.10.15 was released on 2021-03-17.
- Apache Airflow Elasticsearch Provider 1.0.3 was released on 2021-03-16.
- upgrade-check-1.3.0 was released on 2021-03-14.
- Backport Provider Packages 2021.3.13 was released on 2021-03-13.
- Provider packages 2021-03-08 was released on 2021-03-11.
- Backport Provider Packages 2021.3.3 was released on 2021-03-07.
- Provider packages 2021-02-27 was released on 2021-03-07.
With June 17, 2021 the 1.10.X version of Apache Airflow will reach end of
life. We are also proud that the first ever official version of the Airflow
Helm chart was released. We believe that this is a big step in further
adoption of Airflow.
### CVEs published
- CVE-2021-28359: Apache Airflow Reflected XSS via Origin Query Argument in
URL
<https://lists.apache.org/thread.html/ra8ce70088ba291f358e077cafdb14d174b7a1ce9a9d86d1b332d6367%40%3Cusers.airflow.apache.org%3E>
### Events:
- We are getting ready for Airflow Summit 2021
<https://airflowsummit.org> which is going to take place July 8-16,
2021. There’s going to be a lot of interesting talks from diverse speakers,
first-time contributor workshops, dedicated first-time contributor and
first-time users days with talks.The summit is done with cooperation
with 8 meetups: Bangalore, Bay Area, London, Melbourne, New York City,
Telaviv, Tokyo, Warsaw
- There was an Airflow Tel-Aviv meetup 10th of March,
https://www.meetup.com/pl-PL/tel-aviv-apache-airflow-meetup/events/276448622/
- There was an Airflow virtual meetup on the 29th of April that was hosted in
2 languages: English and Spanish. This is a larger effort to bring Airflow
content to Spanish speaking communities.
https://www.crowdcast.io/e/j3pqcxkf/register
## Community Health:
- The community is very active (dev@airflow had a 34% increase in mails). We
believe it’s strongly connected with the long awaited Airflow 2.0 release
and also with huge support from companies like Astronomer who widley promote
Airflow. While we observed a decrease in the number of commits and opened
PRs (-24% and -22%) we think it’s nothing to worry about as it can be
related with cooldown after preparing and polishing Airflow 2.0.
- Community voted on three crucial Airflow Improvement Proposals that will
have huge impact on future of Airflow:
- AIP-38: Modern Web Application
<https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-38+Modern+Web+Application>
- AIP-39: Richer scheduler_interval
<https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-39+Richer+scheduler_interval>
- AIP-40: Deferrable ("Async") Operators
<https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177050929>
- As we are firm believers in inclusivity we’ve completed renaming of the HEAD
branch of airflow repository to “main”. We are going to continue with
keeping our eye on non-inclusive language in our projects
- The conflict between two members of the PMC has been solved and they both
cooperate including submitting joint talk at ApacheCon
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines. ## Membership Data: There are currently 39 committers and 24 PMC members in this project. The Committer-to-PMC ratio is roughly 7:4. Community changes, past quarter: - Xiaodong Deng was added to the PMC on 2020-12-25 - Daniel Standish was added as committer on 2021-03-01 - Elad Kalif was added as committer on 2021-02-26 - Ephraim Anierobi was added as committer on 2021-02-17 - James Timmins was added as committer on 2021-02-26 - Vikram Koka was added as committer on 2021-01-05 ## Project Activity: ### Releases: - 2.0.1 was released on 2021-02-08. - 2.0.0 was released on 2020-12-17. - 1.10.14 was released on 2020-12-10 - upgrade-check-1.2.0 was released on 2021-02-12. - A batch of provider packages (a "sub"/plugin package of apache-airflow) were released on 2021-02-08 and again on 2021-03-08. - Backport Provider Packages (the same code as the normal provider packages, but compatible with 1.10.x series) were released on 2021-02-10 and again on 2021-03-08 ### CVEs published Airflow published four CVEs this past quarter: - CVE-2020-17513: Apache Airflow Server-Side Request Forgery (SSRF) in Charts & Query View - CVE-2021-26697: Apache Airflow: Lineage API endpoint for Experimental API missed authentication check - CVE-2021-26559: CWE-284 Improper Access Control on Configurations Endpoint for the Stable API - CVE-2020-17526: Apache Airflow Incorrect Session Validation in Airflow Webserver with default config - CVE-2020-17511: Apache Airflow Airflow admin password gets logged in plain text - CVE-2020-17515: Apache Airflow Reflected XSS via Origin Parameter ### Events: - “Airflow Virtual Meetup” was organized on 2021-01-28 - “Bangalore Airflow Virtual Meetup” was organized on 2021-02-12 - Airflow Virtual meet-up will be held on 2021-04-08 - We are working on organizing a virtual Airflow Summit 2021 conference which will be held on May 20-28 2021. The call for papers is already open and will close on April 12. ## Community Health: Despite the decrease in activity we are pretty healthy. - We created a community-managed Airflow publication on medium.com and dev.to to gather Airflow-related blog posts under one umbrella to promote the project. - We have two recurring community calls organized every fortnight: - Airflow Dev Call - Airflow Issue Triage process - Apache Airflow project started using Github Discussions and we observed a constant flow of users' questions. - We are also releasing a “Airflow Newsletter” to dev@airflow once a month. - We conducted a community survey among Airflow users and the results are here: http://airflow.apache.org/blog/airflow-survey-2020/ - In Q4 2020, there was a very large effort to make major upgrades in preparation for 2.0. This effort, while necessary, was also an unsustainable effort (lots of nights and weekends from committers/PMC members). Since 2.0 is now released, it is both reasonable and healthy for there to be a drop in the commit and PR pace (48% decrease in commits vs the previous quarter). However overal commits are still continuing at a healthy pace - 1535 commits form 199 contributors in in Q1 - We have seen a slight decrease in issues being opened (543, down 16%) possibly as a result of enabling GitHub Discussions, meaning users have an alternative means of asking for help rather than opening GitHub issues or posting on the lists.
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: ## Membership Data: Apache Airflow was founded 2018-12-19 (2 years ago) There are currently 35 committers and 24 PMC members in this project. The Committer-to-PMC ratio is roughly 3:2. Community changes, past quarter: - No new PMC members. Last addition was Daniel Imberman on 2020-07-07. - Ryan Hamilton was added as committer on 2020-10-29 ## Project Activity: Apache Airflow 1.10.13 was released on Nov 25. Although it contains a bug related to task_concurrency and depends_on_past: Issue Link. The fix for it has been merged and 1.10.14rc1 would be cut this week and if the VOTE passes, 1.10.14 will be released early next week apache-airflow-upgrade-check 1.0.0 was released on Nov 25 too and is available for download via PIP: pip install apache-airflow-upgrade-check We have received credits from AWS to speed up our CI! [PAST EVENTS] Nov 17 — https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/274289245/ [To Be Confirmed]: We’ve just closed all the Airflow Summit proceedings and we are going to donate $8,863.95 profit from Airflow Summit to the Apache Software Foundation. Many thanks to Software Guru for ]managing the conference efforts] - or something similar! 1.10.13 was released on 2020-11-25. upgrade-check-1.0.0 was released on 2020-11-25. Backport Provider Packages 2020.11.23 was released on 2020-11-24. Backport Provider Packages 2020.11.13 was released on 2020-11-13. Backport Provider Packages 2020.10.29 was released on 2020-11-02. Backport Provider Packages 2020.10.5 was released on 2020-10-06. ## Community Health: We are pretty healthy :-). The numbers speak for themselves we think: dev@airflow.apache.org had a 23% increase in traffic in the past quarter (669 emails compared to 540) 2825 commits in the past quarter (53% increase) 201 code contributors in the past quarter (-13% decrease) 1422 PRs opened on GitHub, past quarter (21% increase) 1410 PRs closed on GitHub, past quarter (21% increase) 633 issues opened on GitHub, past quarter (23% increase) 491 issues closed on GitHub, past quarter (63% increase)
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: We would like guidance from the Board about what constitutes a release when it comes to Docker images and Helm Charts. For context: https://lists.apache.org/x/thread.html/rcb608739206d788785081073a0deb417ffa9981634975fc5525dc769@%3Cdev.community.apache.org%3E The ASF release policy says: "Every ASF release must contain a source package, which must be sufficient for a user to build and test the release provided they have access to the appropriate platform and tool" Our specific question: - Does the reproducibility of a Helm chart include the images it depends on? - Or do we count the Helm chart as "stand-alone" (it is just text files/text templates and references which docker files to use. We (Airflow PMC) weren't able to reach a conclusion ourselves https://github.com/apache/airflow/issues/10753 ## Membership Data: Apache Airflow was founded 2018-12-19 (2 years ago) There are currently 34 committers and 24 PMC members in this project. The Committer-to-PMC ratio is roughly 3:2. Community changes, past quarter: - Daniel Imberman was added to the PMC on 2020-07-07 - Kamil Breguła was added to the PMC on 2020-07-07 - Tomasz Urbaszek was added to the PMC on 2020-07-07 - QP Hou was added as committer on 2020-07-13 - Leah Cole was added as committer on 2020-07-20 - Ry Walker was added as committer on 2020-07-16 ## Project Activity: - 1.10.12 was released on 2020-08-25. - 1.10.11 was released on 2020-07-10. - "Backport Provider Packages" was released on 2020-06-24. (This is a early release of some components that will be available in Airflow 2.0 but available on the 1.10 series.) - Weekly dev calls for "Releasing 2.0" are happening (minutes in Confluence and on list), strong focus on releasing 2.0 this year. ## Community Health: - AirflowSummit 2020 was a huge success - with over 6,000 people registering and 3,000 watching the (virtual) sessions live.
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: There are no issues requiring board attention. ## Membership Data: Apache Airflow was founded 2018-12-19 (a year ago) There are currently 31 committers and 21 PMC members in this project. The Committer-to-PMC ratio is roughly 4:3. Community changes, past quarter: - No new PMC members. Last addition was Kengo Seki on 2019-12-01. - Jiajie Zhong was added as committer on 2020-04-03 ## Project Activity: Recent releases: 1.10.10 was released on 2020-04-09. 1.10.8 was released on 2020-02-07. 1.10.9 was released on 2020-02-07. The Airflow Summit is now scheduled for early July (6-17) and will of course be virtual. https://airflowsummit.org/ ## Community Health: The community is very healthy. We are in the process of moving from JIRA to Github issues. 868 PRS have been closed and 881 PRs have been filed. Its always hard to keep up with reviewing although this has been made a lot easier with some of the development tools that have been contributed. Mailinglists are healthy and the split between user and dev seems to work well. dev@airflow.apache.org had a 11% decrease in traffic in the past quarter (746 emails compared to 831) users@airflow.apache.org had a 64% decrease in traffic in the past quarter (79 emails compared to 214) 53 issues opened in JIRA, past quarter (-93% decrease) 129 issues closed in JIRA, past quarter (-80% decrease) 1211 commits in the past quarter (-27% decrease) 202 code contributors in the past quarter (1% increase) 881 PRs opened on GitHub, past quarter (-5% decrease) 868 PRs closed on GitHub, past quarter (-10% decrease) 602 issues opened on GitHub, past quarter (3662% increase) 344 issues closed on GitHub, past quarter (6780% increase)
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: Due to Corona virus concerns the first ever Airflow Summit was delayed until further notice. ## Membership Data: Apache Airflow was founded 2018-12-19 (a year ago) There are currently 30 committers and 21 PMC members in this project. The Committer-to-PMC ratio is roughly 5:4. Community changes, past quarter: - No new PMC members. Last addition was Kengo Seki on 2019-12-01. - Tomasz Urbaszek was added as committer on 2019-12-20 ## Project Activity: 1.10.9 was released on 2020-02-07 (quick fix release). 1.10.8 was released on 2020-02-07. 1.10.7 was released on 2019-12-24. ## Community Health: We have a very healthy community with lots of activity. Apache Airflow jobs rank 4 for data engineering jobs. Airflow's Slack channels are active. dev@airflow.apache.org had a 12% increase in traffic in the past quarter (823 emails compared to 733) users@airflow.apache.org had a 115% increase in traffic in the past quarter (198 emails compared to 92) 801 issues opened in JIRA, past quarter (6% increase) 654 issues closed in JIRA, past quarter (35% increase) 1607 commits in the past quarter (36% increase) 199 code contributors in the past quarter (11% increase) 910 PRs opened on GitHub, past quarter (7% increase) 944 PRs closed on GitHub, past quarter (12% increase)
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: ## Membership Data: Apache Airflow was founded 2018-12-19 (a year ago) There are currently 29 committers and 21 PMC members in this project. The Committer-to-PMC ratio is roughly 4:3. Community changes, past quarter: - Aizhamal Nurmamat kyzy was added to the PMC on 2019-11-21 - Kevin Yang was added to the PMC on 2019-11-21 - Jarek Potiuk was added to the PMC on 2019-10-18 - Kengo Seki was added to the PMC on 2019-12-01 - Kengo Seki was added as committer on 2019-12-01 ## Project Activity: Recent releases: 1.10.6 was released on 2019-10-28. 1.10.5 was released on 2019-09-04. 1.10.4 was released on 2019-08-06. ## Community Health: We have quite a few active discussions on Airflow Improvement Proposals (AIP) which are more and more being picked up by the community to steer large changes. Mailinglist and PRs remain very active as well. The growth of committers and PMC members is therefore very welcome. dev@airflow.apache.org had a 8% decrease in traffic in the past quarter (735 emails compared to 791) users@airflow.apache.org had a 360% increase in traffic in the past quarter (92 emails compared to 20) 751 issues opened in JIRA, past quarter (7% increase) 474 issues closed in JIRA, past quarter (no change) 1117 commits in the past quarter (no change) 176 code contributors in the past quarter (-12% decrease) 841 PRs opened on GitHub, past quarter (21% increase) 826 PRs closed on GitHub, past quarter (13% increase)
## Description: The mission of Apache Airflow is the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines ## Issues: * There are no issues requiring board attention ## Membership Data: Apache Airflow was founded 2018-12-19 (9 months ago) There are currently 28 committers and 17 PMC members in this project. The Committer-to-PMC ratio is roughly 7:5. Community changes, past quarter: - No new PMC members. Last addition was Alex Van Boxel on 2018-12-19. - Felix Uellendall was added as committer on 2019-09-05 - Chao-Han Tsai was added as committer on 2019-08-17 ## Project Activity: * Two new releases have been made since the last report * A new logo has been designed and voted upon ## Community Health: * We started the user mailing list to reduce the amount of message on dev. It is not very active yet * The slack community is very active across many channels * We have grown to 922 contributors, with 198 active in the last quarter * A ridiculous 1081 commits have been made in the past quarter
## Description:
- Apache Airflow (or simply Airflow) is a platform to programmatically
author, schedule, and monitor workflows. When workflows are defined as
code, they become more maintainable, versionable, testable, and
collaborative. Use Airflow to author workflows as directed acyclic graphs
(DAGs) of tasks. The Airflow scheduler executes your tasks on an array of
workers while following the specified dependencies. Rich command line
utilities make performing complex surgeries on DAGs a snap. The rich user
interface makes it easy to visualize pipelines running in production, monitor
progress, and troubleshoot issues when needed.
## Issues:
- there are no issues requiring board attention at this time
## Activity:
- Increasing community engagement from several companies (e.g. Astronomer,
Google)
- Airflow has been accepted as part of Google's Summer of Docs. This is
effort coordinated by Aizhamal on Airflow's side.
- We received and addressed three vulnerability reports (two were the same
issue manifesting in different ways plus one duplicate.) in the 1.10.3
release: CVE-2019-0216, CVE-2019-0229. We also received a duplicate report
after the release.
## Health report:
- We have grown from 743 to 807 contributors since our last report. Many only
submit one small fix and don't commit further.
- We have invited a 7 new committers (and confirmed) since our last report.
- 158 commits have made to the 1.10 release branch since our last release[1]
- 339 commits to master branch (which will become Airflow 2.0.0) since our
last report
- We have started planning a roadmap for Airflow 2.0
## PMC changes:
- Currently 17 PMC members.
- No new PMC members added in the last 3 months
- Last PMC addition was Alex Van Boxel on Wed Dec 19 2018
## Committer base changes:
- Currently 26 committers.
- New commmitters:
- Aizhamal Nurmamat kyzy was added as a committer on Thu May 30 2019
- Bas Harenslak was added as a committer on Mon Apr 22 2019
- Daniel Imberman was added as a committer on Tue Mar 19 2019
- Joshua Carp was added as a committer on Tue Apr 30 2019
- Kamil Breguła was added as a committer on Fri May 03 2019
- Kevin Yang was added as a committer on Tue Apr 30 2019
- Jarek Potiuk was added as a committer on Thu Apr 18 2019
## Releases:
- 1.10.3 was released on Wed Apr 10 2019
## Mailing list activity:
- users mailinglist was just opened up
- users@airflow.apache.org:
- 36 subscribers (up 36 in the last 3 months):
- 4 emails sent to list (0 in previous quarter)
- dev@airflow.apache.org:
- 598 subscribers (up 19 in the last 3 months):
- 884 emails sent to list (616 in previous quarter)
## JIRA activity:
- 644 JIRA tickets created in the last 3 months
- 507 JIRA tickets closed/resolved in the last 3 months
## Description:
- Apache Airflow (or simply Airflow) is a platform to programmatically
author, schedule, and monitor workflows. When workflows are defined as
code, they become more maintainable, versionable, testable, and
collaborative. Use Airflow to author workflows as directed acyclic graphs
(DAGs) of tasks. The Airflow scheduler executes your tasks on an array of
workers while following the specified dependencies. Rich command line
utilities make performing complex surgeries on DAGs a snap. The rich user
interface makes it easy to visualize pipelines running in production, monitor
progress, and troubleshoot issues when needed.
## Issues:
- there are no issues requiring board attention at this time
## Activity:
- New committer added: Xiaodong Deng
- Healthy discussions on the mailinglist around DAG serialization
## Health report:
- We have grown from 712 to 743 contributors since our last report. We do
monitor for potential committers, but we also see a lot of "hit and run"
contributors that particularly add to the 'periphery' of our code.
- Since latest release 1058 commits have been made
## PMC changes:
- Currently 17 PMC members.
- No new PMC members added in the last 3 months
## Committer base changes:
- Currently 19 committers.
- Xiaodong Deng was added as a committer on Sat Mar 09 2019
## Releases:
- 1.10.2 was released on Tue Jan 22 2019
## Mailing list activity:
- dev@airflow.apache.org:
- 578 subscribers (up 0 in the last 3 months):
- 665 emails sent to list (757 in previous quarter)
## JIRA activity:
- 578 JIRA tickets created in the last 3 months
- 420 JIRA tickets closed/resolved in the last 3 months
## Description:
- Apache Airflow (or simply Airflow) is a platform to programmatically
author, schedule, and monitor workflows. When workflows are defined as
code, they become more maintainable, versionable, testable, and
collaborative. Use Airflow to author workflows as directed acyclic graphs
(DAGs) of tasks. The Airflow scheduler executes your tasks on an array of
workers while following the specified dependencies. Rich command line
utilities make performing complex surgeries on DAGs a snap. The rich user
interface makes it easy to visualize pipelines running in production,
monitor progress, and troubleshoot issues when needed.
## Issues:
## Activity:
- Most of our assets have been moved to TL. We might have some remnants here
and there, which we try to track down
- We released a new version (1.10.2)
- We are in the process (or have done so) in promoting one of the
contributors to committer.
- We are, measured by GitHub stars, the most popular project for data
orchestration comparing Apache Oozie, Spotify's Luigi, Pinterest's Pinball
and LinkedIn's Azkaban
- Note: Apologies for not responding to the question by posting comments, it
was unclear how this worked
## Health report:
- We have grown from 674 to 712 contributors since our last report. We do
monitor for potential committers, but we also see a lot of "hit and run"
contributors that particularly add to the 'periphery' of our code.
- Since the 1.10.2 release 914 commits have been made
## PMC changes:
- Currently 17 PMC members.
- No new PMC members added in the last 3 months
## Committer base changes:
- Currently 18 committers.
- In process of adding one.
## Releases:
- 1.10.2 was released on Tue Jan 22 2019
## Mailing list activity:
- dev@airflow.apache.org:
- 576 subscribers (up 5 in the last 3 months):
- 660 emails sent to list (883 in previous quarter)
## JIRA activity:
- 545 JIRA tickets created in the last 3 months
- 383 JIRA tickets closed/resolved in the last 3 months
## Description:
- Apache Airflow (or simply Airflow) is a platform to programmatically
author, schedule, and monitor workflows. When workflows are defined as
code, they become more maintainable, versionable, testable, and
collaborative. Use Airflow to author workflows as directed acyclic graphs
(DAGs) of tasks. The Airflow scheduler executes your tasks on an array of
workers while following the specified dependencies. Rich command line
utilities make performing complex surgeries on DAGs a snap. The rich user
interface makes it easy to visualize pipelines running in production,
monitor progress, and troubleshoot issues when needed.
## Issues:
## Activity:
- Apache Airflow just graduated from the incubator. We are in the process of
moving our assets to top level.
## Health report:
- We have grown from 580 contributors to 674 since October 2018
- Since the November release 729 commits have been made
## PMC changes:
- Currently 17 PMC members.
- No new PMC members added in the last 3 months
## Committer base changes:
- Currently 18 committers.
- No changes (the PMC was established in the last 3 months)
## Releases:
- Latest release, v1.10.1, is from November 2018. This was during incubation.
## Mailing list activity:
- dev@airflow.apache.org:
- 578 subscribers (up 24 in the last 3 months)
## JIRA activity:
- 476 JIRA tickets created in the last 3 months
- 325 JIRA tickets closed/resolved in the last 3 months
WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software, for distribution at no charge to the public, related to workflow automation and scheduling that can be used to author and manage data pipelines. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the "Apache Airflow Project", be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache Airflow Project be and hereby is responsible for the creation and maintenance of software related to workflow automation and scheduling that can be used to author and manage data pipelines; and be it further RESOLVED, that the office of "Vice President, Apache Airflow" be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache Airflow Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache Airflow Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache Airflow Project: * Alex Guziel <saguziel@apache.org> * Alex Van Boxel <alexvanboxel@apache.org> * Arthur Wiedmer <arthur@apache.org> * Ash Berlin-Taylor <ash@apache.org> * Bolke de Bruin <bolke@apache.org> * Chris Riccomini <criccomini@apache.org> * Dan Davydov <davydov@apache.org> * Fokko Driesprong <fokko@apache.org> * Hitesh Shah <hitesh@apache.org> * Jakob Homan <jghoman@apache.org> * Jeremiah Lowin <jlowin@apache.org> * Joy Gao <joygao@apache.org> * Kaxil Naik <kaxilnaik@apache.org> * Maxime Beauchemin <maximebeauchemin@apache.org> * Siddharth Anand <sanand@apache.org> * Sumit Maheshwari <msumit@apache.org> * Tao Feng <tfeng@apache.org> NOW, THEREFORE, BE IT FURTHER RESOLVED, that Bolke de Bruin be appointed to the office of Vice President, Apache Airflow, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the Apache Airflow Project be and hereby is tasked with the migration and rationalization of the Apache Incubator Airflow podling; and be it further RESOLVED, that all responsibilities pertaining to the Apache Incubator Airflow podling encumbered upon the Apache Incubator Project are hereafter discharged. Special Order 7F, Establish the Apache Airflow Project, was approved by Unanimous Vote of the directors present.
Airflow is a workflow automation and scheduling system that can be used to
author and manage data pipelines.
Airflow has been incubating since 2016-03-31.
Three most important issues to address in the move towards graduation:
1. Once we make a release with the licensing fix we will move forward with
graduation.
2.
3.
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
No!
How has the community developed since the last report?
1. Since our last podling report 1 month ago (i.e. between September 4 &
October 1, inclusive), we grew our contributors from 548 to 580 (32 new
contributors)
2. Since our last podling report 1 month ago (i.e. between September 4 &
October 1, inclusive), we resolved 151 pull requests (currently at 3114
closed PRs)
3. Since being accepted into the incubator, the number of companies
officially using Apache Airflow has risen from 193 to 206, 13 new from
the last podling report 1 month ago.
How has the project developed since the last report?
See above : 151 PRs resolved, 32 new contributors, & 13 new companies
officially using it.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[ ] Initial setup
[ ] Working towards first release
[ ] Community building
[x] Nearing graduation
[ ] Other:
Date of last release:
2018-08-27
When were the last committers or PPMC members elected?
Tao Feng on Aug 3
Have your mentors been helpful and responsive or are things falling
through the cracks? In the latter case, please list any open issues
that need to be addressed.
One of our mentors is pretty active and helpful, one of our mentors has
never been involved, and one is in the middle.
Signed-off-by:
[ ](airflow) Chris Nauroth
Comments:
[x](airflow) Hitesh Shah
Comments:
[x](airflow) Jakob Homan
Comments:
IPMC/Shepherd notes:
Airflow is a workflow automation and scheduling system that can be used to
author and manage data pipelines.
Airflow has been incubating since 2016-03-31.
Three most important issues to address in the move towards graduation:
1.Once we make a release with the licensing fix we will move forward
with graduation.
2.
3.
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of? None
How has the community developed since the last report?
1. Since our last podling report 1 month ago (i.e. between August 5 &
September 4, inclusive), we grew our contributors from 512 to 548 (36 new
contributors)
2. Since our last podling report 1 month ago (i.e. between August 5 &
September 4, inclusive), we resolved 129 pull requests (currently at 2963
closed PRs)
3. Since being accepted into the incubator, the number of companies
officially
using Apache Airflow has risen from 30 to 193, 10 new from the last
podling report 1 month ago.
How has the project developed since the last report?
See above : 129 PRs resolved, 36 new contributors, & 10 new companies
officially using it. We also released 1.10.0, our 4th Apache Release while
in the incubator as seen on https://pypi.org/project/apache-airflow/ #history
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[ ] Initial setup
[ ] Working towards first release
[ ] Community building
[x] Nearing graduation
[ ] Other:
Date of last release:
2018-08-27
When were the last committers or PPMC members elected?
Kaxil Naik on May 7 & Tao Feng on Aug 3
Signed-off-by:
[ ](airflow) Chris Nauroth
Comments:
[ ](airflow) Hitesh Shah
Comments:
[x](airflow) Jakob Homan
Comments:
IPMC/Shepherd notes:
Airflow is a workflow automation and scheduling system that can be used to
author and manage data pipelines.
Airflow has been incubating since 2016-03-31.
Three most important issues to address in the move towards graduation:
1.Once we make a release with the licensing fix we will move forward with
graduation.
2.
3.
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of? None
How has the community developed since the last report?
1. Since our last podling report 5 months ago (i.e. between March
28 & August 5, inclusive), we grew our contributors from 427 to 512
2. Since our last podling report 5 months ago (i.e. between March
28 & August 5, inclusive), we resolved 554 pull requests (currently at
2834 closed
PRs)
3. Since being accepted into the incubator, the number of companies
officially
using Apache Airflow has risen from 30 to 183, 34 new from the last
podling
report 5 months ago.
How has the project developed since the last report?
See above : 554 PRs resolved, 85 new contributors, & 34 new companies
officially using it. We have also added 2 new committers : Kaxil Naik on
May 7 & Tao Feng on Aug 3.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[ ] Initial setup
[ ] Working towards first release
[ ] Community building
[x] Nearing graduation
[ ] Other:
Date of last release:
2017-12-15
When were the last committers or PPMC members elected?
2 new committers : Kaxil Naik on May 7 & Tao Feng on Aug 3
Signed-off-by:
[ ](airflow) Chris Nauroth
Comments:
[ ](airflow) Hitesh Shah
Comments:
[ ](airflow) Jakob Homan
Comments:
IPMC/Shepherd notes:
Airflow is a workflow automation and scheduling system that can be used to
author and manage data pipelines.
Airflow has been incubating since 2016-03-31.
Three most important issues to address in the move towards graduation:
1. Once we make a release with the licensing fix we will move forward with
graduation.
2.
3.
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
of?
None
How has the community developed since the last report?
1. Since our last podling report 1.5 months ago (i.e. between Feb 8 & March
28, inclusive), we grew our contributors from 375 to 427
2. Since our last podling report 1.5 months ago (i.e. between Feb 8 & March
28, inclusive), we resolved 152 pull requests (currently at 2280 closed
PRs)
3. Since being accepted into the incubator, the number of companies officially
using Apache Airflow has risen from 30 to 149, 17 new from the last podling
report 1.5 months ago.
How has the project developed since the last report?
See above : 152 PRs resolved, 52 new contributors, & 17 new companies
officially using it.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[ ] Initial setup
[ ] Working towards first release
[ ] Community building
[x] Nearing graduation
[ ] Other:
Date of last release:
2018-01-02
When were the last committers or PPMC members elected?
2017-11-30 joygao a.k.a Joy Gao (committer/PMC)
Signed-off-by:
[ ](airflow) Chris Nauroth
Comments:
[x](airflow) Hitesh Shah
Comments:
[x](airflow) Jakob Homan
Comments: Ready for graduation
IPMC/Shepherd notes:
I checked into the licensing issue and it is resolved, but it seems that a
release will be needed before graduation. Dave Fisher
Airflow is a workflow automation and scheduling system that can be used to
author and manage data pipelines.
Airflow has been incubating since 2016-03-31.
Three most important issues to address in the move towards graduation:
1. We have had 4 releases and are working toward our 5th. We are getting better at releases.
2.
3.
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
None
How has the community developed since the last report?
1. We had our forth official release 1.9.0 on Jan 2, 2018.
2. Since our last podling report 4 months ago (i.e. between Sept 25 & Feb 8,
inclusive), we grew our contributors from 315 to 385
3. Since our last podling report 4 months ago (i.e. between Sept 25 & Feb 8,
inclusive), we resolved 331 pull requests (currently at 2128 closed PRs)
4. Since being accepted into the incubator, the number of companies officially
using Apache Airflow has risen from 30 to 132, 18 new from the last podling
report.
How has the project developed since the last report?
See above : 331 PR resolved, 70 new contributors, & 18 new companies
officially using it.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[ ] Initial setup
[ ] Working towards first release
[ ] Community building
[x] Nearing graduation
[ ] Other:
Date of last release:
2018-01-02
When were the last committers or PPMC members elected?
2017-11-30 joygao a.k.a Joy Gao (committer/PMC)
Signed-off-by:
[ ](airflow) Chris Nauroth
Comments:
[x](airflow) Hitesh Shah
Comments:
[x](airflow) Jakob Homan
Comments: Ready for graduation
Airflow is a workflow automation and scheduling system that can be used to
author and manage data pipelines.
Airflow has been incubating since 2016-03-31.
Three most important issues to address in the move towards graduation:
1. We are working on our 4th apache release 1.9.0 and are getting better at releases
2.
3.
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
None
How has the community developed since the last report?
1. We had our third official release 1.8.2 on Aug 7th 2017.
2. Since our last podling report 3 months ago (i.e. between July
5 & Sept 25, inclusive), we grew our contributors from 280 to 315
4. Since our last podling report 3 months ago (i.e. between July
5 & Sept 25, inclusive), we resolved 167 pull requests (currently at 1797 closed
PRs)
5. Since being accepted into the incubator, the number of companies
officially using Apache Airflow has risen from 30 to 114, 16 new from the last podling report.
How has the project developed since the last report?
See above : 167 PR resolved, 35 new contributors, & 16 new companies officially using it.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[ ] Initial setup
[ ] Working towards first release
[ ] Community building
[x] Nearing graduation
[ ] Other:
Date of last release:
2017-08-07
When were the last committers or PPMC members elected?
2017-01-01 fokko a.k.a Fokko Driespong (committer/PMC)
Signed-off-by:
[ ](airflow) Chris Nauroth
Comments:
[x](airflow) Hitesh Shah
Comments:
[ ](airflow) Jakob Homan
Comments:
IPMC/Shepherd notes:
A quick look at this month's email and it looks like Airflow is ready to graduate.
Dave Fisher
Airflow is a workflow automation and scheduling system that can be used to
author and manage data pipelines.
Airflow has been incubating since 2016-03-31.
Three most important issues to address in the move towards graduation:
1. We are working on our third apache release 1.8.2 to get more experience
with the process
2.
3.
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
None
How has the community developed since the last report?
1. We had our second official release. 1.8.1 on May 9th 2017.
2. Since our last podling report 3 months ago (i.e. between Apr 1 and July
5, inclusive), we grew our contributors from 256 to 280
4. Since our last podling report 3 months ago (i.e. between Apr 1 and July
5, inclusive), we resolved 151 pull requests (currently at 1630 closed
PRs)
5. A new meet-up group formed in Tokyo, Japan - they held their first
meet-up on May 11.
6. Since being accepted into the incubator, the number of companies
officially using Apache Airflow has risen from 30 to 98, 15 new from
last podling report.
How has the project developed since the last report?
See above : 151 PR resolved, 24 new contributors, & 15 new companies
officially using it.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[ ] Initial setup
[ ] Working towards first release
[ ] Community building
[x] Nearing graduation
[ ] Other:
Date of last release:
2017-05-09
When were the last committers or PPMC members elected?
As mentioned on
https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements#Announcements-Mar14,2017
Alex Guziel joined the Apache Airflow PPMC/Committer group.
Signed-off-by:
[ ](airflow) Chris Nauroth
Comments:
[X](airflow) Hitesh Shah
Comments:
[ ](airflow) Jakob Homan
Comments:
Airflow is a workflow automation and scheduling system that can be used to
author and manage data pipelines.
Airflow has been incubating since 2016-03-31.
Three most important issues to address in the move towards graduation:
1. We are working on our second apache release 1.8.1 to get more experience
with the process
2.
3.
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
None
How has the community developed since the last report?
1. We had our first official release. 1.8.0 on March 19th 2017.
2. We elected 1 new PPMC Member/Committer: Alex Guziel (a.k.a saguziel)
3. Since our last podling report 3 months ago (i.e. between Jan 1 and Mar
31, inclusive), we grew our contributors from 224 to 256
4. Since our last podling report 3 months ago (i.e. between Jan 1 and Mar
31, inclusive), we resolved 216 pull requests (currently at 1479 closed
PRs)
5. Three meet-ups, one in San Francisco, CA hosted by Clover Health, one in
New York, NY hosted by Blue Apron and one in San Jose, CA hosted by PayPal
were held by the community.
6. Since being accepted into the incubator, the number of companies
officially using Apache Airflow has risen from 30 to 83.
How has the project developed since the last report?
1. As noted above, 216 pull requests were merged since our last report
(i.e. between Jan 1 and Mar 31, inclusive).
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[ ] Initial setup
[ ] Working towards first release
[ ] Community building
[X] Nearing graduation
[ ] Other:
The Airflow community continues to grow, and we have successfully created
our first Apache release. We want to continue on this momentum and create
another release to solidify our process and tools around it, but we feel
we are nearing graduation. We are open to feedback and guidance to make
sure we can do so.
Date of last release:
2017-03-19
When were the last committers or PPMC members elected?
As mentioned on
https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements#Announcements-Mar14,2017
Alex Guziel joined the Apache Airflow PPMC/Committer group.
Signed-off-by:
[ ](airflow) Chris Nauroth
Comments:
[x](airflow) Hitesh Shah
Comments:
[x](airflow) Jakob Homan
Comments:
Airflow is a workflow automation and scheduling system that can be used to
author and manage data pipelines.
Airflow has been incubating since 2016-03-31.
Three most important issues to address in the move towards graduation:
1. Getting an Apache release out
2.
3.
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
None
How has the community developed since the last report?
1. We elected 1 new PPMC Member/Committer : Alex Van Boxel a.k.a.
alexvanboxel
2. Since our last podling report 3 months ago (i.e. between Oct 4 and Dec
31, inclusive), we grew our contributors from 191 to 224
3. Since our last podling report 3 months ago (i.e. between Oct 4 and Dec
31, inclusive), we resolved 202 pull requests (currently at 1263 closed
PRs)
4. One meet-up, hosted at WePay, was held by the community
5. Since being accepted into the incubator, the number of companies
officially using Apache Airflow has risen from 30 to 74
How has the project developed since the last report?
See above
Date of last release:
None. First ASF release currently being discussed.
When were the last committers or PPMC members elected?
As mentioned on
https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements#Announcements-Nov28,2016,
Alex Van Boxel joined the Apache Airflow PPMC/Committer group.
Signed-off-by:
[X](airflow) Chris Nauroth
[x](airflow) Hitesh Shah
[ ](airflow) Jakob Homan
Airflow is a workflow automation and scheduling system that can be used to author and manage data pipelines. Airflow has been incubating since 2016-03-31. Three most important issues to address in the move towards graduation: 1. Getting an Apache release out 2. 3. Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? We discovered a private Gitter channel for committers that was created during the days when the Git project was owned by Airbnb. When the Github project’s ownership moved to the ASF, so did that of all the Gitter channels. We killed the channel - all committers left the channel and can no longer even see/discover it. How has the community developed since the last report? 1. We elected 2 new PPMC Members/Committers (Li Xuanji a.k.a. Zodiac & Sumit Maheshwari a.k.a. msumit) 2. Since our last podling report 3 months ago (i.e. between Jul 7 and Oct 4, inclusive ), we grew our contributors from 160 to 191 3. Since our last podling report 3 months ago (i.e. between Jul 7 and Oct 4, inclusive ) , we merged >142 pull requests 4. Two meet-ups (One for HUG Warsaw, the other an Airflow meet-up @Stripe HQ in SF) and several public talks (e.g. DataDaySeattle, PyData SF) were held by the community 5. Since being accepted into the incubator, the number of companies using Apache Airflow has risen from 30 to 58 How has the project developed since the last report? 1. Podling name search approved : https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-110 2. Very active discussion list, public Gitter channels and JIRA (more than 500 issues) Date of last release: None. First ASF release currently being discussed. When were the last committers or PMC members elected? As mentioned on https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements, 1. On Aug 16, Sumit Maheshwari (a.k.a. msumit) was elected to the PPMC and Committer group 2. On Aug 19, Li Xuanji (a.k.a. zodiac) was elected to the PPMC and Committer group Signed-off-by: [x](airflow) Chris Nauroth [x](airflow) Hitesh Shah [x](airflow) Jakob Homan
Airflow is a workflow automation and scheduling system that can be used to author and manage data pipelines. Airflow has been incubating since 2016-03-31. Three most important issues to address in the move towards graduation: 1. Getting an Apache release out. 2. Podling name search has to happen. 3. Verify distribution rights Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? We've done a lot of work on distribution rights (3), but we need to wrap it up. We've been pushing out our release due to stability concerns and the need for a testing framework How has the community developed since the last report? - Since our last podling report 1 month ago, we grew our contributors from 148 to 160. - In June 2016 we had 56 commits. - We organized a meet up at WePay. We also had one before at Airbnb. - We had several committers meetings. - 1 new committer. 2 offers were given but 1 was declined. How has the project developed since the last report? - Migrated all files to contain Apache 2.0 license headers. - Replaced charting library that was non-Apache 2.0 compliant (a large effort). - Release Airflow incubator website, and added appropriate branding and disclaimers. Date of last release: None (had a non-Apache release on 2016-06-13)* * Release was discussed and approved here: https://mail-archives.apache.org/mod_mbox/incubator-general/201605.mbox/%3CEF1D08AF-80DD-4614-BC74-BBCED9251BE1@gmail.com%3E When were the last committers or PMC members elected? 2016-05-20 Signed-off-by: [X](airflow) Chris Nauroth [x](airflow) Hitesh Shah [x](airflow) Jakob Homan
Airflow is a workflow automation and scheduling system that can be used to
author and manage data pipelines.
Airflow has been incubating since 2016-03-31.
Three most important issues to address in the move towards graduation:
1. Releases
2. Grow up user and contributor communities
3. Improve and extend documentation and samples on the website
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
None
How has the community developed since the last report?
* Since our last podling report 1 month ago, we grew
our contributors from 137 to 148
* Since our last podling report 1 month ago, we
accepted/merged 51 PRs
* We voted on the following matters according to Apache guidelines:
* We voted to make all current and future committers part of the PPMC
* We voted in a new committer and PPMC member : Steven Yvinec-Kruyk and
he accepted
* We voted in a commit policy of "RTC with a +1 vote from a committer
other than the author (assuming no vetos)"
How has the project developed since the last report?
* All resources have been created and all migrations are complete - code
donation included
* The website is in progress
Date of last release:
N/A - we are still working out releases as we are new to Apache.
When were the last committers or PMC members elected?
On May 20, 1 committer/PPMC member(Steven Yvinec-Kruyk) was elected and he
accepted.
Signed-off-by:
[X](airflow) Chris Nauroth
[x](airflow) Hitesh Shah
[x](airflow) Jakob Homan
Airflow is a workflow automation and scheduling system that can be used to author and manage data pipelines. Airflow has been incubating since 2016-03-31. Three most important issues to address in the move towards graduation: 1. Continued releases 2. Grow up user and contributor communities 3. Improve and extend documentation and samples on the website Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? None How has the community developed since the last report? * Since we applied to the incubator in March 17 (proposal email), we grew our contributors from 100 to 137 * Since we applied to the incubator in March 17 (proposal email), we accepted/merged 119 PRs * Dev mailing lists activity increased How has the project developed since the last report? * All resources have been created (website, Jira, git & github mirror, ...) but not all migrations are complete * The code donation (and Pull Request migration) is in progress * The website is in progress * The previous wiki has been migrated to confluence, Jira is now used instead of the old Github issues, and the old google group is being EOLd on May 2 as users migrate to the dev mailing list Date of last release: N/A - we are still working out releases as we are new to Apache. When were the last committers or PMC members elected? N/A - we are still new to Apache and have not gotten to the point of electing new committers/PMC members. Signed-off-by: [ ](airflow) Chris Nauroth [X](airflow) Hitesh Shah [x](airflow) Jakob Homan