Community-led development "The Apache Way"
This was extracted (@ 2021-04-07 00:10) from a list of minutes
which have been approved by the Board.
Please Note
The Board typically approves the minutes of the previous meeting at the
beginning of every Board meeting; therefore, the list below does not
normally contain details from the minutes of the most recent Board meeting.
Meeting times vary, the exact schedule is available to ASF Members and Officers, search for "calendar" in the Foundation's private index page (svn:foundation/private-index.html).
Apache Kafka is a distributed event streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We released the 2.7.0, which includes the following key features. - Configurable TCP connection timeout and improve the initial metadata fetch - Enforce broker-wide and per-listener connection creation rate - Throttle Create Topic, Create Partition and Delete Topic Operations - Add TRACE-level end-to-end latency metrics to Streams - Add Broker-side SCRAM Config API - Support PEM format for SSL certificates and private key - Add RocksDB Memory Consumption to RocksDB Metrics - Add Sliding-Window support for Aggregations We’ve continued steady progress toward the task of replacing ZooKeeper in Kafka. We expect to have some limited functionalities in the upcoming 2.8.0 release that demonstrate much higher scalability for the metadata. Tiered Storage work continues and unlocks infinite scaling and faster rebalance times via KIP-405. The KIP proposal is getting closer for voting. Community =========== Dev mailing list had a 39% increase in traffic in the past quarter (3246 emails compared to 2333). User mailing list had a 33% decrease in traffic in the past quarter (378 emails compared to 562). We didn't add any new committer or PMC member in the last quarter. We last added 3 new committers Chia-Ping Tsai and David Jacot on Oct. 15, 2020, and Sophie Blee-Goldman on Oct. 19, 2020. We last added a new PMC member John Roesler on Aug. 9, 2020. With the help from ASF branding committee, Kafka PMC passed a relaxed naming policy for non-java clients and connectors. Quite a few non-java clients and connectors have been using Kafka in its name. Those products form a critical eco-system for Apache Kafka since Apache Kafka doesn't provide them directly. We feel that requiring all of them to change their product name is detrimental to the Kafka community. The relaxed policy can be found at https://kafka.apache.org/trademark Releases =========== 2.7.0 was released on Dec. 21, 2020. 2.6.1 was released on Jan. 7, 2021.
Apache Kafka is a distributed event streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We are voting the 2.7.0 release, which includes the following key features. - connection rate limit - partition mutation quota - sliding window aggregation in KStreams - feature versioning support - Support PEM format for SSL certificates and private key We’ve continued steady progress toward the task of replacing ZooKeeper in Kafka. Now there is a separate "raft" module containing the core consensus protocol. Seven other KIPs are in active development to provide support for more partitions per cluster, simpler operation, and tighter security. Tiered Storage work continues and unlocks infinite scaling and faster rebalance times via KIP-405. Community =========== Dev mailing list had a 22% decrease in traffic in the past quarter (2448 emails compared to 3124). User mailing list had a 23% decrease in traffic in the past quarter (572 emails compared to 741). We added 3 new committers, Chia-Ping Tsai and David Jacot on Oct. 15, 2020, and Sophie Blee-Goldman on Oct. 19, 2020. We last added a new PMC member John Roesler on Aug. 9, 2020. The virtual Kafka Summit Austin (August 24-25, 2020) had more than 35,000 people registered and 19,000 people attended. Releases =========== 2.6.0 was released on Aug. 3, 2020. 2.5.1 was released on Agu. 8, 2020.
Apache Kafka is a distributed event streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We released 2.6.0, which includes the following new features: - TLSv1.3 has been enabled by default for Java 11 or newer. - Significant performance improvements, especially when the broker has large numbers of partitions - Smooth scaling out of Kafka Streams applications - Kafka Streams support for emit on change - New metrics for better operational insight - Kafka Connect can automatically create topics for source connectors - Improved error reporting options for sink connectors in Kafka Connect - New Filter and conditional SMTs in Kafka Connect - The default value for the `client.dns.lookup` configuration is now `use_all_dns_ips` - Upgrade Zookeeper to 3.5.8 This release also includes other features, 74 improvements, 175 bug fixes, plus other changes. We are also in the process of releasing 2.5.1, which fixes 72 issues since the 2.5.0 release. We are actively working on replacing Zookeeper with an internal Raft based consensus service. This will simplify the operation of Kafka and allow Kafka to handle many more partitions. We are also actively working on tiered storage support which decouples the storage from the compute in Kafka. Community =========== Dev mailing list had a 20% increase in traffic in the past quarter (3248 emails compared to 2685). We added 2 new PMC members, Mickael Maison on Jul. 30, 2020 and John Roesler Aug. 9, 2020. We added 2 new committers, Boyang Chen on Jun. 22, 2020 and Xi Hu on Jun. 23, 2020. The now virtual Kafka Summit Austin (August 24-25, 2020) has more than 20,000 registrations so far. We also have a new layout for the Apache Kafka website. Releases =========== 2.6.0 was released on Aug. 3, 2020.
Apache Kafka is a distributed event streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We released 2.5.0, which includes the following new features: - TLS 1.3 support (1.2 is now the default) - Co-groups for Kafka Streams - Incremental rebalance for Kafka Consumer - New metrics for better operational insight - Upgrade Zookeeper to 3.5.7 - Deprecate support for Scala 2.11 We also released 2.4.1, which fixes more than 39 issues. We are actively working on replacing Zookeeper with an internal Raft based consensus service. This will simplify the operation of Kafka and allow Kafka to handle many more partitions. Community =========== Lots of activities in the mailing list. We had 2,855 emails in the dev mailing list, 34% more than the last 3 months. We had 8962 emails in the in the user mailing list, 21% more than the last 3 months. We had 5.358 JIRA activities, 92% more than the last 3 months. We didn't add any new PMC member. We last added 3 new PMC members, Colin McCabe, Manikumar Reddy and Vahid Hashemian on Jan 13, 2020. We added one new committer Konstantine Karantasis on Feb. 26, 2020. Due to COVID-19, Kafka Summit London (Apr 27-28, 2020) was cancelled and Kafka Summit Austin (August 24-25, 2020) has been changed to a virtual one. Releases =========== 2.5.0 was released on Apr. 15, 2020. 2.4.1 was released on Mar. 12, 2020.
Apache Kafka is a distributed event streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We released 2.4.0, which includes the following new features: - Allow consumers to fetch from closest replica - Support for incremental cooperative rebalancing to the consumer rebalance protocol - MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter replication engine - New Java authorizer Interface - Support for non-key joining in KTable - Administrative API for replica reassignment - Securing Internal connect REST endpoints - API to delete consumer offsets and expose it via the AdminClient. We also released 2.2.2, which fixes more than 30 issues. Community =========== Lots of activities in the mailing list. We had 2,203 emails in the dev mailing list, 15% less than the last 3 months, likely due to the holiday season. We had 814 emails in the in the user mailing list, 18% more than the last 3 months. We had 2,963 JIRA activities, 7% less than the last 3 months. We added 3 new PMC members, Colin McCabe, Manikumar Reddy and Vahid Hashemian in January, 2020. We didn't add any new committers in the last 3 months. We last added a new committer on Nov. 11, 2019. We plan to have 2 Kafka Summits in 2020, London on Apr. 27/28 and Austin on Aug. 24/25. Releases =========== 2.4.0 was released on Dec. 16, 2019. 2.2.2 was released on Dec. 1, 2019.
Apache Kafka is a distributed event streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We are in the process of preparing 2.4.0 release which includes new features such as allowing consumers to fetch from closest replica, incremental consumer rebalance, sticky partitioner for better producer throughput, administrative API for replica assignment, and MirrorMaker 2. We released 2.3.1, which fixes more than 40 critical issues. We accepted KIP-500, a significant effort to remove ZooKeeper dependency in Kafka for simplified deployment and more scalability. Community =========== Lots of activities in the mailing list. We had 2,758 emails in the dev mailing list, 13% more than the last 3 months. We had 771 emails in the in the user mailing list, 15% more than the last 3 months. We had 3,350 JIRA activity, 7% more than the last 3 months. We added two new committers Mickael Maison and John Roesler in November. We didn't add any new PMC member in the last 3 months. We last added a new PMC member on Apr. 18, 2019. Kafka Summit San Francisco occurred on Sep. 30 and Oct. 1, 2019, with more than 2,100 participants. Releases =========== 2.3.1 was released on Oct. 24, 2019.
Apache Kafka is a distributed event streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We released 2.3.0 which includes new features such as incremental cooperative rebalancing in Connect, external configuration stores, in-memory session store and window store in KStreams, improved metrics and AdminClient. We also released 2.2.1 which fixes more than 40 critical issues. We are working on some of the new Kafka Improvement Proposals (KIP). In particular, KIP-500 proposes to remove the dependency on Zookeeper to improve the scalability and the dependency in Kafka. Community =========== Lots of activities in the mailing list. We have 2824 subscribers in the user mailing list, slightly more than the last 3 months. We have 1341 subscribers in the dev mailing list, slightly more than the last 3 months. We have 432 JIRA tickets created and 220 JIRA tickets closed in the last 3 months. We didn't add any new committer/PMC member in the last 3 months. We last added a new committer on Feb. 14, 2019 and we last added a new PMC member on Apr. 18, 2019. Kafka Summit London completed on May 14, 2019, with more than 1000 participants. Releases =========== 2.3.0 was released on Jun. 24, 2019. 2.2.1 was released on Jun. 2, 2019.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development We released 2.2.0 which includes new features such as separate control and data planes, re-authentication in SASL, better fencing with broker epoch, more functionalities in KStreams, etc. We also released 2.1.1, which fixes more than 30 issues. We are actively working on new features such as incremental client rebalancing, MirrorMaker 2.0, improvements in KStreams and AdminClient, etc. Community Lots of activities in the mailing list. We have 2806 subscribers in the user mailing list, about the same as in the last 3 months. We have 624 emails in the user mailing list in the last 3 months, about the same as in the previous cycle. We have 1305 subscribers in the dev mailing list, about the same as in the last 3 months. We have 2152 emails in the dev mailing list in the last 3 months, more than the 1947 in the previous cycle. We added one new committer Randall Hauch on Feb. 14, 2019. We added two new PMC members Sriharsh Chintalapan and Matthias J. Sax on Apr. 18, 2019. We also elected a third new PMC member. Unfortunately, that person declined the invitation due to shifted focus. Kafka Summit New York completed on Apr. 2, 2019, with more than 700 participants. Confluent published two public guidelines on respecting ASF policies. https://www.confluent.io/apache-guidelines https://www.confluent.io/apache-engineering-guidelines Releases =========== 2.2.0 was released on Mar. 26, 2019. 2.1.1 was released on Feb. 19, 2019.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We are preparing the next major release 2.2.0, which plans to include new features such as separate control and data planes, re-authentication in SASL, better fencing with broker epoch, reducing consumer group rebalance, more functionalities in KStreams, etc. We are also preparing a bug fix release 2.1.1, which fixes more than 30 issues. Community =========== Lots of activities in the mailing list. We have 2809 subscribers in the user mailing list, up 37 in the last 3 months. We have 657 emails in the user mailing list in the last 3 months, a bit less than the 811 in the previous cycle (likely due to year end holidays). We have 1310 subscribers in the dev mailing list, up 38 in the last 3 months. We have 2056 emails in the dev mailing list in the last 3 months, a bit less than the 2689 in the previous cycle. We elected two new committers, Vahid Hashemian on Jan 15, 2019 and Bill Bejeck on Feb. 8, 2019. We didn't add any new PMC members. We last elected one new PMC member Dong Lin on Aug. 4, 2018. An EU sponsored Bug Bounty program for Apache Kafka with a focus on security is in progress. Releases =========== 2.1.0 was released on Nov. 21, 2018. 2.0.1 was released on Nov. 9, 2018.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We are voting on the next major release 2.1.0. This release includes 28 new KIPs and features such as Java 11 support, Zstandard support, more intuitive user timeouts in the producer, better fencing for replication, various Streams API improvements and the admin client improvements. We released 2.0.1, which fixes 49 issues. Community =========== Lots of activities in the mailing list. We have 2767 subscribers in the user mailing list, up 28 in the last 3 months. We have 864 emails in the user mailing list in the last 3 months, a bit less than the 953 in the previous cycle. We have 1259 subscribers in the dev mailing list, up 45 in the last 3 months. We have 2972 emails in the dev mailing list in the last 3 months, a bit more than the 2878 in the previous cycle. We elected two new committers, Colin McCabe on Sep 29, 2018 and Manikumar Reddy on Oct 11, 2018. We elected one new PMC member Dong Lin on Aug. 4, 2018. Kafka Summit in San Francisco completed with more than 1200 attendees. We published the very first Kafka blog in Apache (https://blogs.apache.org/kafka/entry/apache-kafka-supports-more-partitions). Releases =========== 2.0.1 was released on Nov. 9, 2018. 1.1.1 was released on Jul. 20, 2018.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We shipped 4 releases this quarter. The major release was 2.0.0, which includes 40 KIPs and more than 240 fixes. This release includes major features such as prefixed ACL, OAuth authentication, improved replication protocol to avoid log divergence, better error control in Connect, better secret configuration in Connect, and header support in Streams. This is also the release where we dropped Java 7 support. We are working on new features such as improving the consumer offset semantics, making the log cleaner more robust, allowing the broker to prioritize controller requests, etc. Community =========== Lots of activities in the mailing list. We have 2735 subscribers in the user mailing list, up 69 in the last 3 months. We have 990 emails in the user mailing list in the last 3 months, a bit less than the 1027 in the previous cycle. We have 1212 subscribers in the dev mailing list, up 68 in the last 3 months. We have 3074 emails in the dev mailing list in the last 3 months, much higher than the 2367 in the previous cycle (likely due the releases). The vote for a new PMC member Dong Lin pased on Aug. 4, 2018. We are waiting for the board notice peroid before sending an invitation. We didn't elect any new committer this cycle. Some concern on committer diversity have been raised in the private mailing list. We are reviewing our committer guidelines now and will resume the committer nomination soon. We last elected a new committer on Mar. 28, 2018. Releases =========== 2.0.0 was released on Jul. 30, 2018. 1.1.1 was released on Jul. 20, 2018. 0.11.0.3 was released on Jul. 4, 2018. 0.10.2.2 was released on Jul. 4, 2018.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We released 1.0.1, which fixes 49 critical issues. We also released 1.1.0, which includes major features such as JBOD, delegation tokens, controller improvements, dynamic broker configs, improved consumption performance with a large number of partitions, and Kafka Streams API improvements. We are actively fixing critical issues that have been reported since the release. Community =========== Lots of activities in the mailing list. We have 2665 subscribers in the user mailing list, up 36 in the last 3 months. We have 1044 emails in the user mailing list in the last 3 months, a bit less than the 1152 in the previous cycle. We have 1143 subscribers in the dev mailing list, up 39 in the last 3 months. We have 2521 emails in the dev mailing list in the last 3 months, a bit less than the 2869 in the previous cycle. We elected one new committer, Dong Lin on Mar. 28, 2018. We didn't elect any new PMC member in this cycle. We last elected a new PMC member on Jan. 17, 2018. Kafka Summit in London completed with more than 500 attendees. Releases =========== 1.0.1 was released on Mar. 6, 2018. 1.1.0 was released on Mar. 29, 2018.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We are voting the second RC of release 1.0.1, which fixes 49 critical issues. We will also be releasing 1.1.0 in Feb, which includes major features such as JBOD, delegation tokens, controller improvements, dynamic broker configs, improved consumption performance with a large number of partitions, and Kafka Streams API improvements. Community =========== Lots of activities in the mailing list. We have 2625 subscribers in the user mailing list, up 64 in the last 3 months. We have 1180 emails in the user mailing list in the last 3 months, a bit less than the 1185 in the previous cycle. We have 1100 subscribers in the dev mailing list, up 43 in the last 3 months. We have 2819 emails in the dev mailing list in the last 3 months, down from the 4172 in the previous cycle, likely due to holidays. We elected one new committer, Matthias Sax on Jan. 12, 2018. We elected one new PMC member, Rajini Sivaramon on Jan. 17, 2018. The agenda for the next Kafka Summit in London in Apr. 2018 has been published. Releases =========== 0.11.0.1 was released on Sep. 13, 2017. 1.0.0 was released on Nov. 1, 2017.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We released 1.0.0! This is a big milestone of Apache Kafka after 7+ years of active development, which has accumulated key features such as intra cluster replication, compacted topics, security, connect, kstream, exactly-once semantics, among others. 108 people contributed to the 1.0.0 release. More than 340 people have contributed to Apache Kafka over time. We are actively working on new features such as JBOD, delegation tokens, additional improvements of the controller, various improvements in kstream and connect, etc. Community =========== Lots of activities in the mailing list. We have 2551 subscribers in the user mailing list, up 78 in the last 3 months. We have 1268 emails in the user mailing list in the last 3 months, a bit less than the 1631 in the previous cycle. We have 1059 subscribers in the dev mailing list, up 55 in the last 3 months. We have 4409 emails in the dev mailing list in the last 3 months, a bit less from the 6349 in the previous cycle. We elected one new committer, Onur Karaman on Oct. 31, 2017. We elected one new PMC member, Becket Qin on Aug. 23, 2017. The next Kafka Summit in London in Apr. 2018 is now open for submission. Releases =========== 0.11.0.1 was released on Sep. 13, 2017. 1.0.0 was released on Nov. 1, 2017.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We released 0.11.0.0 on Jun. 28, 2017, which includes 32 KIPs, over 400 bug fixes and improvements, and more than 700 pull requests. The main features in this release include exactly-once semantic, new admin apis, request quotas, etc. We are actively working on new features such as JBOD, delegation tokens and improvements of the controller. The next release will be 1.0.0, a big milestone in Kafka. Community =========== Lots of activities in the mailing list. We have 2474 subscribers in the user mailing list, up 44 in the last 3 months. We have 1705 emails in the user mailing list in the last 3 months, about the same as 1736 in the previous cycle. We have 1003 subscribers in the dev mailing list, up 24 in the last 3 months. We have 6526 emails in the dev mailing list in the last 3 months, up from 6248 in the previous cycle. We elected one new committer, Damian Guy on Jun. 9, 2017. We elected two new PMC members, Ismael Juma on Jul. 5, 2017 and Jason Gustafson on Jul. 11, 2017. We plan to have a 1-day Kafka Summit in SF on Aug 28. Releases =========== 0.11.0.0 was released on Jun. 28, 2017.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We released 0.10.2.0 on Feb. 21, 2017, which includes new features such as session window and global table in streams, client compatibilty, multiple listeners in the same security protocol and SASL/SCRAM. We released 0.10.2.1 on Apr. 26, 2017, which includes 29 critical bug fixes. We are actively working on new features such as exactly-once semantic, admin apis, request quotas, and controller improvement for the next release. Community =========== Lots of activities in the mailing list. We have 2427 subscribers in the user mailing list, up 89 in the last 3 months. We have 1824 emails in the user mailing list in the last 3 months, about the same as 1833 in the previous cycle. We have 977 subscribers in the dev mailing list, up 54 in the last 3 months. We have 6779 emails in the dev mailing list in the last 3 months, up from 6066 in the previous cycle. We elected one new committer, Rajini Sivaram on Apr. 22, 2017. We last elected a new PMC member Gwen Shapira on Aug. 17, 2016. We held a 1-day Kafka Summit in NY on May 8. It's a success with 24 technical sessions and more than 500 attendees. Releases =========== 0.10.2.1 was released on Apr. 26, 2017.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We released 0.10.1.1, which fixes 30 critical issues. We are in the process of releasing 0.10.2.0, which includes new features such as session window and global table in streams, client compatibilty, multiple listeners in the same security protocol and SASL/SCRAM. This is our second time-based release and we expect it to be released on time as well. We are actively working on new features such as exactly-once semantic, admin apis, and better JBOD support. Community =========== Lots of activities in the mailing list. We have 2324 subscribers in the user mailing list, up 79 in the last 3 months. We have 1954 emails in the user mailing list in the last 3 months, up from 1630 in the previous cycle. We have 912 subscribers in the dev mailing list, up 44 in the last 3 months. We have 6537 emails in the dev mailing list in the last 3 months, up from 5156 in the previous cycle. We elected one new committer, Grant Henke on Jan. 10, 2017. We last elected a new PMC member Gwen Shapira on Aug. 17, 2016. We had several Kafka meetups in various cities in this quarter. One feedback from the community is that it's a bit hard to find an answer by searching the Apache mailing list. We are wondering if Apache has considered some sort of integration with things like Stack Overflow that does a better job at Q/A. Releases =========== 0.10.1.1 was released on Dec. 19, 2016.
Apache Kafka is a distributed streaming platform for efficiently storing and processing a large number of records in real time. Development =========== We switched to a time-based release model and plan to do a release every 4 months. We released 0.10.1.0 which includes new features such as interactive queries, time-based seek, replication quotas, and quota by user. We plan to do the next release in Feb, 2017. Community =========== Lots of activities in the mailing list. We have 2243 subscribers in the user mailing list, up 48 in the last 3 months. We have 1718 emails in the user mailing list in the last 3 months, slightly down from 1912 in the previous cycle. We have 871 subscribers in the dev mailing list, up 18 in the last 3 months. We have 5355 emails in the dev mailing list in the last 3 months, up from 5318 in the previous cycle. We elected two new committers, Jason Gustafson and Jiangjie Qin, on Sep. 6 and Oct, 31, 2016, respectively. We elected a new PMC member Gwen Shapira on Aug.17, 2016. We had several Kafka meetups in various cities in this quarter. Releases =========== 0.10.1.0 was released on Oct. 20, 2016.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We released 0.10.0.0 which includes new features such as Kafka stream, rack-awareness support, and native support of timestamp in the message. We also did a bug fix release 0.10.0.1 which fixed 53 issues in 0.10.0.0. We are actively working on new features such as admin apis, queryable state in Kafka stream, replication throttling, etc. Community =========== Lots of activities in the mailing list. We have 2187 subscribers in the user mailing list, up 129 in the last 3 months. We have 2043 emails in the user mailing list in the last 3 months, up from 1839 in the previous cycle. We have 852 subscribers in the dev mailing list, up 72 in the last 3 months. We have 5631 emails in the dev mailing list in the last 3 months, down from 6291 in the previous cycle, mostly because 0.10.0.0 has been released. We last elected a new committer Ismael Juma on Apr. 26, 2016. We are voting on a new PMC member right now. We had several Kafka meetups in various cities in this quarter. Releases =========== 0.10.0.0 and 0.10.0.1 were released on May. 24 and Aug. 10, 2016, respectively.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We released 0.9.0.1 that fixes more than 70 critical issues in 0.9.0.0. We are actively working on the 0.10.0.0 release which includes new features such as Kafka stream, rack-awareness support, and native support of timestamp in the message. A release candidate of 0.10.0.0 is being voted on. Community =========== Lots of activities in the mailing list. We have 2050 subscribers in the user mailing list, up 192 in the last 3 months. We have 1952 emails in the user mailing list in the last 3 months, up from 1377 in the previous cycle. We have 777 subscribers in the dev mailing list, up 80 in the last 3 months. We have 6845 emails in the dev mailing list in the last 3 months, up from 5378 in the previous cycle. We last elected a new committer Ismael Juma on Apr. 26, 2016. We had the very first Kafka Summit on Apr. 26, 2016. It'a one-day conference that attracted more than 500 people. Releases =========== 0.9.0.1 was released on Feb. 19, 2016.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We released 0.9.0.0, which include the following major features: (1) a new java-based consumer; (2) authorization (through ssl and sasl) and authentication; (3) quotas; (4) Kafka connect: a framework for copying data into and out of Kafka. We are actively working on critical bug fixes from 0.9.0.0 and new features such as KStream. Community =========== Lots of activities in the mailing list. We have 1828 subscribers in the user mailing list, up 148 in the last 3 months. We have 1513 emails in the user mailing list in the last 3 months, up from 1197 in the previous cycle. We have 687 subscribers in the dev mailing list, up 51 in the last 3 months. We have 5779 emails in the dev mailing list in the last 3 months, up from 5344 in the previous cycle. We last elected a new committer Ewen Cheslack-Postava on Dec. 8, 2015. Releases =========== 0.9.0.0 was released on Nov. 24, 2015.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We just rolled out RC 1 for the 0.9.0.0 release, which include the following major features: (1) a new java-based consumer; (2) authorization (through ssl and sasl) and authentication; (3) quotas; (4) Kafka connect: a framework for copying data into and out of Kafka. Community =========== Lots of activities in the mailing list. We had several Kafka improvement proposals post the 0.9.0 release. We last elected a new committer Sriharsh Chintalapani on Sep. 21, 2015. We had Kafka meetups in Austin, Los Angeles, and London this quarter. Releases =========== 0.8.2.2 was released on Sep. 13, 2015.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We are actively developing multiple features in the next release. These include (1) a new java-based consumer; (2) security (authorization and authentication); (3) quotas; (4) a framework for copying data into and out of Kafka; (5) a stream processing api integrated with Kafka. Community =========== Lots of activities in the mailing list. kafka-user has 435, 278, 86 emails in Feb, Mar and Apr, respectively (down from 453 in Apr). kafka-dev has 1532, 800, 402 emails in May, Jun and Jul, respectively (down from 2001 in Apr). We last elected a new committer Gwen Shapira on Jul. 6, 2015. We also added Guozhang Wang to pmc on Jun. 10, 2015. We had two Kafka meetups in the bay area on May 19 and Jun. 16. Releases =========== 0.8.2.1 was released on Mar. 11, 2015.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We released Kafka 0.8.2.1, which fixes 4 critical issues in 0.8.2.0. The highlights in the 0.8.2.x releases include (1) a java-based new producer, which provides better performance and a cleaner api; (2) a more scalable Kafka-based offset storage management; (3) automated leader balancing and controlled shutdown; (4) delete topic support; (5) more than 200 bug fixes and improvements. We are actively developing a new java-based consumer and the security support in trunk. Community =========== Lots of activities in the mailing list. kafka-user has 768, 776, 453 emails in Feb, Mar and Apr, respectively (up from 665 in Jan). kafka-dev has 1742 1854, 2001 emails in Feb, Mar and Apr, respectively (up from 1632 in Jan). We last elected a new committer Guozhang Wang on Nov. 19, 2014. We had two Kafka talks and a meetup in ApacheCon 2015. We had a Kafka meetup in Mountain View, California on Mar. 24, 2015. Releases =========== 0.8.2.1 was released on Mar. 11, 2015.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We released Kafka 0.8.2.0. The highlights in this release include (1) a java-based new producer, which provides better performance and a cleaner api; (2) a more scalable Kafka-based offset storage management; (3) automated leader balancing and controlled shutdown; (4) delete topic support; (5) more than 200 bug fixes and improvements. We are actively developing a new java-based consumer and the security support in trunk. Community =========== Lots of activities in the mailing list. kafka-user has 550, 464, 665 emails in Nov, Dec and Jan, respectively (up from 563 in Oct). kafka-dev has 1214, 881, 1632 emails in Nov, Dec and Jan (about the same as 1633 in Jul). We last elected a new committer Guozhang Wang on Nov. 19, 2014. We had a Kafka meetup in Mountain View, California on Jan. 27, 2015. Releases =========== 0.8.2.0 was released on Feb. 3, 2015.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We released Kafka 0.8.2 beta. The highlights in this release include (1) a java-based new producer, which provides better performance and a cleaner api; (2) a more scalable Kafka-based offset storage management; (3) automated leader balancing and controlled shutdown; (4) delete topic support; (5) more than 100 bug fixes and improvements. We expect to release 0.8.2 final by end of Nov. Community =========== Lots of activities in the mailing list. kafka-user has 431, 383, 563 emails in Aug, Sep and Oct, respectively (up from 429 in Jan). kafka-dev has 1129, 1257, 1633 emails in Aug, Sep and Oct (about the same as 1357 in Jul). A new committer is being voted now. The voting thread was started on Nov 11, 2014. Releases =========== 0.8.2 beta was released on Oct 27, 2014.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We continue to stablize two main features for the 0.8.2 release. The first one is Kafka-based offset management, which allows consumer offsets to be stored in a more scalable way. The second one is the new producer, which provides better performance and a cleaner api. We expect to release 0.8.2 in a month or two. We also finalized the api and the design for the new consumer and started the development. This feature is intended for the 0.9.0 release. Community =========== Lots of activities in the mailing list. kafka-user has 422, 560, 429 emails in May, June and July, respectively (up from 388 in Apr). kafka-dev has 789, 610, 1357 emails in May, June and July (sligtly higher than 1272 in Apr). We had a Kafka meetup in June. We last elected two committers in Oct. 2013 and are actively grooming new potential committers. Releases =========== 0.8.1 was released on Mar 21, 2014 and 0.8.1.1 was released on Apr 29, 2014.
pache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We released Kafka 0.8.1 and 0.8.1.1, which include a few improved admin tools and new features like log compaction. We are currently stablizing the new producer, which provides better performance and a cleaner api. We are finalizing the api and the design for the new consumer. Community =========== Lots of activities in the mailing list. kafka-user has 480, 450, 388 emails in Feb, Mar and Apr, respectively (slightly less than 538 in Jan). kafka-dev has 1120, 1047, 1272 emails in Feb, Mar and Apr (slightly less than 540 in Jan). There were three Kafka related talks in ApacheCon 2014. We last elected two committers in Oct. 2013. Releases =========== 0.8.1 was released on Mar 21, 2014 and 0.8.1.1 was released on Apr 29, 2014. Infra =========== The recent Apache mail server issue impacted many projects. I am sure there will be some post-mortem on how to prevent this in the future. Could the post-mortem results be shared with all PMCs?
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We released Kafka 0.8.0 and have since improved existing admin tools. An 0.8.1 release branch has been created and we intend to release 0.8.1 in a few weeks. We are actively working on a new version of the producer client, which provides better performance and a cleaner api. After that, we expect to rewrite the consumer client for the same reasons. Community =========== Lots of activities in the mailing list. kafka-user has 538, 461, 436 emails in Jan, Dec and Nov, respectively (slightly less than 582 in Oct). kafka-dev has 540, 509, 540 emails in Jan, Dec and Nov (slightly less than 581 in Oct). At least one Kafka talk is submitted to ApacheCon 2014. We last elected two committers in Oct. 2013. Releases =========== 0.8.0 was released on Dec 3, 2013.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We have fixed critical bugs and improved the tools since the Kafka 0.8 beta1 (supports intra-cluster replication). We are in the process of releasing 0.8.0 final. Next, we will be focusing on stablizing trunk and releasing 0.8.1 (including various server side performance improvements). After that, we will start improving the client library. An initial design is in the wiki. Community =========== Lots of activities in the mailing list. kafka-user has 665, 278, 582 emails in Jul, Jun and May, respectively (up from 425 in Jul). kafka-dev has 581, 509, 595 emails in Oct, Sep and Aug (up from 290 in Apr). Two new Kafka committers were elected: David Arthur and Sriram Subramanian. Releases =========== 0.8.0 final release candidate 3 is being voted now.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We released Kafka 0.8 beta1 (supports intra-cluster replication). We have updated the website with documentation and our new logo. We are fixing critical bugs in the 0.8 branch and expect to release the 0.8 final version soon. Community =========== Lots of activities in the mailing list. kafka-user has 425, 442, 421 emails in Jul, Jun and May, respectively (up from 390 in Apr). kafka-dev has 290, 290, 562 emails in Jul, Jun and May (down from 567 in Apr since 0.8 dev is winding down). There are patches being contributed by non-committers. No committers added yet since graduation. Last committer was added in May, 2012. Releases =========== 0.8.0 beta1: released on Jun. 27, 2013 0.8.0 final release is expected in three weeks
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We are in the final phase of testing Kafka 0.8 (supports intra-cluster replication). We have updated the related docs and expect to release 0.8 beta in a couple weeks. Community =========== Lots of activities in the mailing list. kafka-user has 390, 357, 174 emails in Apr, Mar and Feb, respectively (up from 243 in Jan). kafka-dev has 567, 565, 457 emails in Apr, Mar and Feb (down from 804 in Jan since 0.8 dev is winding down). There are patches being contributed by non-committers. Releases =========== 0.7.2: released Oct. 11, 2012 0.8.0 beta: expected to release in two weeks
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We found a few more blocker issues while testing Kafka 0.8 (supports intra-cluster replication). The 0.8 release is likely delayed to Mar or Apr. Community =========== Lots of activities in the mailing list. kafka-user has 243 emails in Jan. (up from 241 in Dec). kafka-dev has 804 emails in Dec (up from 498 in Dec). We accepted a few more patches from non-committers. There will be a Kafka presentation in ApachCon 2013 in Feb. Infrastructure =========== This is the third month since Kafka became a top-level project. All infra work is now done.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== We are actively testing Kafka 0.8 (supports intra-cluster replication) and fixing blocking issues that show up. We expect to release 0.8 towards the end of Jan. Community =========== Lots of activities in the mailing list. kafka-user has 241 emails in Dec (down from 335 in Nov, mostly due to the holidays). kafka-dev has 498 emails in Dec (up from 448 in Nov). More patches are being contributed by non-committers. Infrastructure =========== This is the second month since Kafka became a top-level project. Apache infra has helped us migrate the website and move from svn to git.
Apache Kafka is a distributed pub/sub system for efficiently collecting and delivering a large number of messages to both offline and online systems. Development =========== Kafka 0.8 (supports intra-cluster replication) is in the final debugging stage. We gave out a trivial version on Nov 26. We expect to release 0.8 in Jan. 2013. Community =========== Lots of activities in the mailing list. kafka-user has 335 emails in Nov (up from 236 in Oct). kafka-dev has 448 emails in Nov (down from 758 in Oct, mostly because of fewer active jiras). There will be a Kafka presentation (introducing 0.8 features) in ApacheCon in Feb., 2013. Infrastructure =========== This is the first month since Kafka became a top-level project. Apache infra has helped us migrate the svn repository and the mailing list, and is in the process of migrating our website.
WHEREAS, the Board of Directors deems it to be in the best
interests of the Foundation and consistent with the
Foundation's purpose to establish a Project Management
Committee charged with the creation and maintenance of
open-source software, for distribution at no charge to the
public, related to a distributed publish/subscribe stream
processing framework.
NOW, THEREFORE, BE IT RESOLVED, that a Project Management
Committee (PMC), to be known as the "Apache Kafka Project",
be and hereby is established pursuant to Bylaws of the
Foundation; and be it further
RESOLVED, that the Apache Kafka Project be and hereby is
responsible for the creation and maintenance of software
related to a distributed publish/subscribe stream processing
framework; and be it further
RESOLVED, that the office of "Vice President, Apache Kafka" be
and hereby is created, the person holding such office to
serve at the direction of the Board of Directors as the chair
of the Apache Kafka Project, and to have primary responsibility
for management of the projects within the scope of
responsibility of the Apache Kafka Project; and be it further
RESOLVED, that the persons listed immediately below be and
hereby are appointed to serve as the initial members of the
Apache Kafka Project:
* Jun Rao <junrao@apache.org>
* Jay Kreps <jkreps@apache.org>
* Neha Narkhede <nehanarkhede@apache.org>
* Jakob Homan <jghoman@apache.org>
* Joel Koshy <jjkoshy@apache.org>
* Prashanth Menon <prashanthmenon@apache.org>
* Joe Stein <joestein@apache.org>
NOW, THEREFORE, BE IT FURTHER RESOLVED, that Jun Rao
be appointed to the office of Vice President, Apache Kafka, to
serve in accordance with and subject to the direction of the
Board of Directors and the Bylaws of the Foundation until
death, resignation, retirement, removal or disqualification,
or until a successor is appointed; and be it further
RESOLVED, that the initial Apache Kafka PMC be and hereby is
tasked with the creation of a set of bylaws intended to
encourage open development and increased participation in the
Apache Kafka Project; and be it further
RESOLVED, that the Apache Kafka Project be and hereby
is tasked with the migration and rationalization of the Apache
Incubator Kafka podling; and be it further
RESOLVED, that all responsibilities pertaining to the Apache
Incubator Kafka podling encumbered upon the Apache Incubator
Project are hereafter discharged.
Special Order 7C, Establish the Apache Kafka Project, was
approved by Unanimous Vote of the directors present.
(introduced to Apache incubator on Jul 4, 2011)
Kafka provides an extremely high throughput distributed publish/subscribe
messaging system. Additionally, it supports relatively long term persistence
of messages to support a wide variety of consumers, partitioning of the
message stream across servers and consumers, and functionality for loading
data into Apache Hadoop for offline, batch processing.
Three most important issues to address in the move towards graduation:
None.
Previously, there was some concern about the diversity of the project
that we believe has been addressed:
- The project now has active committers from four companies
- and continues to receive patches from other contributors from various
institutions.
We are considering whether to pursue graduation immediately or focus
exclusively on the development of 0.8.
Any issues that the Incubator PMC or ASF Board might wish/need to be aware of:
None.
How has the community developed since the last report:
- The mailing list continues to be very active with bug reports, patch
submissions, feature requests and use cases.
[(Apr,May,Jun): kafka-users (151, 261, 251); kafka-dev (142, 195, 380)].
- Added two new committers.
- Conducted the first Kafka user group meeting with over 50 attendees
(archived video link available here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations).
This was a very successful event that gave users and contributors an
opportunity to meet and discuss Kafka usage at various companies, ongoing
development efforts, feature requests, etc.
- Received and reviewed several major patches (including contributions from
non-committers).
How has the project developed since the last report:
- Intra-cluster replication (KAFKA-50)
- Considerable progress has been made on the design and implementation of
this much-anticipated feature.
- Regular status updates are being sent out on the kafka-dev mailing list.
- Another significant project that is under discussion and development is
the consumer redesign (KAFKA-364) and consumer coordinator implementation
(KAFKA-264).
- Released Kafka 0.7.1 in June. This is the second Kafka release since its
introduction to Apache incubator
Signed off by mentor: Alan Cabrera
Shepherd: Matt Franklin
(introduced to Apache incubator on Jul 4, 2011) Kafka provides an extremely high throughput distributed publish/subscribe messaging system. Additionally, it supports relatively long term persistence of messages to support a wide variety of consumers, partitioning of the message stream across servers and consumers, and functionality for loading data into Apache Hadoop for offline, batch processing. A list of the most important issues to address in the move towards graduation - Invite diverse new active committers Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? - None How has the community developed since the last report? - Added a new committer. Continued contribution to source code in fixes and features and continued activity on the mailing lists. How has the project developed since the last report? - Work was begun on 0.8 release which includes our replication feature and also requires a breaking change for existing consumers/producer to the wiring protocol. Signed off by mentor: cdouglas, adc
(introduced to Apache incubator on Jul 4, 2011) Kafka provides an extremely high throughput distributed publish/subscribe messaging system. Additionally, it supports relatively long term persistence of messages to support a wide variety of consumers, partitioning of the message stream across servers and consumers, and functionality for loading data into Apache Hadoop for offline, batch processing. A list of the most important issues to address in the move towards graduation 1. Invite diverse new active committers Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? We incurred significant overhead during our first release; see http://apache.markmail.org/message/rsxjgrrufc6khlqy?q=list:org%2Eapache%2Eincubator%2Egeneral+from:%22junrao%22 for reference. Firstly, there is no good example to follow for a new podling. Secondly, the licensing rules are complex and the documentation is not enough for a new podling to understand. This gets more complicated since different IPMC members have different views and apply those rules slightly differently. This delays releases and causes overhead to the team. We hope that some of these things can be addressed to make the process easier to other projects. How has the community developed since the last report? Activities in the mailing list increased 4x since the last report. Several patches from non-committers have been submitted, reviewed and committed. How has the project developed since the last report? We have successfully completed the first release of Kafka after joining the incubator. This releases adds the end-to-end compression feature and mirroring support in Kafka. In addition to this, it also fixes more than 40 issues.
Introduced to Apache incubator on Jul 4, 2011 Kafka provides an extremely high throughput distributed publish/subscribe messaging system. Additionally, it supports relatively long term persistence of messages to support a wide variety of consumers, partitioning of the message stream across servers and consumers, and functionality for loading data into Apache Hadoop for offline, batch processing. A list of the most important issues to address in the move towards graduation 1. Successful podling release. 2. Invite diverse new active committers Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? Not at this time. How has the community developed since the last report? Mailing list traffic for July-August-September[12th] (user: 40, 65, 19; dev: 116, 371, 63) both show healthy growth trends. Qualitatively the -dev discussion has trended away from topics like "how should we configure Jira?" to "What's the best way to deal with multiple language bindings?". Several patches were submitted by first time contributors. How has the project developed since the last report? The general theme over the past month has been polish: fixing bugs, better unit tests, getting log levels right, build system cleanup etc.
Introduced to Apache incubator on Jul 4, 2011 Kafka provides an extremely high throughput distributed publish/subscribe messaging system. Additionally, it supports relatively long term persistence of messages to support a wide variety of consumers, partitioning of the message stream across servers and consumers, and functionality for loading data into Apache Hadoop for offline, batch processing. A list of the most important issues to address in the move towards graduation 1. Successful podling release. 2. Invite diverse new active committers Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? Not at this time. How has the community developed since the last report? Mailing list traffic for July-August-September[12th] (user: 40, 65, 19; dev: 116, 371, 63) both show healthy growth trends. Qualitatively the -dev discussion has trended away from topics like "how should we configure Jira?" to "What's the best way to deal with multiple language bindings?". Several patches were submitted by first time contributors. How has the project developed since the last report? The general theme over the past month has been polish: fixing bugs, better unit tests, getting log levels right, build system cleanup etc.
(introduced to Apache incubator on Jul 4, 2011) Kafka provides an extremely high throughput distributed publish/subscribe messaging system. Additionally, it supports relatively long term persistence of messages to support a wide variety of consumers, partitioning of the message stream across servers and consumers, and functionality for loading data into Apache Hadoop for offline, batch processing. A list of the three most important issues to address in the move towards graduation 1. Package, document and release at least one Apache Incubator release. 2. Develop a strong community with organizational diversity. Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? No, not at this time. How has the community developed since the last report? Just get started. The mailing list is pretty active, including people other than the initial committers. How has the project developed since the last report? * Moved everything (svn, website, mailing list, jira, wiki) in Kafka to Apache. CCLA from LinkedIn filed with Apache. * The end-to-end compression feature is almost complete. * Received various patches from 3 different companies, including dynamic topic discovering for cross-cluster mirroing, mx4j support, .net client, etc.