Apache Logo
The Apache Way Contribute ASF Sponsors

This was extracted (@ 2017-08-04 00:10) from a list of minutes which have been approved by the Board.
Please Note The Board typically approves the minutes of the previous meeting at the beginning of every Board meeting; therefore, the list below does not normally contain details from the minutes of the most recent Board meeting.

2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010 | 2009 | 2008 | 2007 | 2006 | 2005 | 2004 | 2003 | 2002 | 2001 | 2000 | 1999 | Pre-organization meetings

Beam

21 Jun 2017 [Davor Bonaci / Chris]

## Description:

Apache Beam is a unified programming model for both batch and streaming data
processing, enabling efficient execution across diverse distributed execution
engines and providing extensibility points for connecting to different
technologies and user communities.

## Issues:

There are no issues that require the Board's attention at this time.

## Activity:

We have achieved a major milestone for the project -- the completion of the
first stable release, version 2.0.0. It signifies a statement from the
community that it intends to maintain API stability with all releases for the
foreseeable future, and making Beam suitable for enterprise deployment.
Additionally, version 2.0.0 improves user experience across the project,
focusing on seamless portability across execution environments, including
engines, operating systems, on-premise clusters, cloud providers, and data
storage systems.

Beam continues to interconnect additional execution engines and data
storage/messaging systems, and serves as a glue in the ecosystem. On the
execution side, the work continues on the Apache Gearpump (incubating) runner,
and a new effort on the JStorm runner has started. On the IO connector side,
connectors for Apache Cassandra and Apache Hive’s HCatalog have been
contributed, and additional connectors for Redis, Apache DistributedLog
(incubating), Apache Solr, Apache Parquet, RabbitMQ, and Advanced Message
Queuing Protocol (AMQP) are in progress. Finally, we have started a major
effort to create a SQL extension, based on Apache Calcite.

We have published a press release and a blog post regarding the first stable
release:
- https://blogs.apache.org/foundation/entry/the-apache-software-foundation-
 announces12
- https://beam.apache.org/blog/2017/05/17/beam-first-stable-release.html
We have also refreshed the design of our website.

Beam was covered at seven major industry conferences over the past quarter,
including the "Apache: Big Data" conference in Miami, FL, where we have had 4
talks, a birds-of-a-feather session and a social event. Additionally, we
organized the first meetup in the Bay Area, hosted by Hortonworks and Future
of Data.

Going forward, the main focus continues to be on the user growth, with
outreach continuing across conferences and meetups. On the technical side, the
next major milestone is the completion of the portability framework across all
components of the project, which would, among other benefits, extend Python
SDK to all Beam runners.

## Health report:

The community continues to grow steadily, as follows:
- The number of contributors continues to increase. We are now at 179 unique
 code contributors, with 76 individuals contributing to the latest release
 alone (which spanned less than 2 months).
- Releases continue at a regular pace of 1-2 months per release.
- The activity on the user@ mailing list more than doubled.

## PMC changes:

Currently 14 PMC members. No new PMC members have been added since graduation
six months ago.

We are watching for potential new PMC members.

## Committer base changes:

Currently 24 committers. Four new committers have been added since the last
report:
- Aviem Zur was added as a committer on Fri Mar 17 2017.
- Chamikara Jayalath was added as a committer on Fri Mar 17 2017.
- Ismaël Mejía was added as a committer on Fri Mar 17 2017.
- Eugene Kirpichov was added as a committer on Fri Mar 17 2017.

## Releases:

Since the last report, Apache Beam has published two releases:
- 0.6.0 was released on Mon Mar 13 2017.
- 2.0.0 was released on Mon May 15 2017.

Version 2.0.0 is the first release that comes with API stability guarantees.
Going forward, we expect to publish a release every 1-2 months.

## Mailing list activity:

Mailing list activity continues to increase across all metrics, with the
number of user@ emails more than doubling compared to the previous quarter.

- dev@beam.apache.org
 - 424 subscribers (up 63 in the last 3 months).
 - 1162 emails sent to list (1094 in previous quarter).

- user@beam.apache.org
 - 384 subscribers (up 73 in the last 3 months).
 - 547 emails sent to list (250 in previous quarter).

## JIRA activity:

JIRA activity continues to increase across all metrics, with the number of
resolved issues nearly doubling.

- 725 JIRA tickets created in the last 3 months (542 in the previous quarter).
- 650 JIRA tickets closed/resolved in the last 3 months (347 in the previous
 quarter).

15 Mar 2017 [Davor Bonaci / Rich]

## Description:

Apache Beam is a unified programming model for both batch and streaming data
processing, enabling efficient execution across diverse distributed execution
engines and providing extensibility points for connecting to different
technologies and user communities.

## Issues:

There are no issues that require the Board's attention at this time.

## Activity:

Apache Beam was established as a top-level project at December’s Board
meeting. This is the third in the series of three consecutive monthly reports
for new projects.

Since last month's report, we have started work on the next release, version
0.6.0. This will be the first release with the new Python SDK, a highly
anticipated component that opens up a new user community. Pipelines built with
Python SDK currently run on a limited number of runners, but work is ongoing
to extend runner support.

Beam continues to interconnect additional execution engines and data
storage/messaging systems. Since the last report, IO connector for Apache
HBase has been contributed, and additional connectors for Redis, Apache
Cassandra, Apache DistributedLog, Apache Parquet, Apache Solr, RabbitMQ, and
Advanced Message Queuing Protocol (AMQP) are in progress. The work has resumed
on the Apache Gearpump (incubating) runner.

Going forward, the main focus continues to be on the community growth,
particularly users. Beam will be covered at 6 major conferences over the next
2 months, including 2 talks and a tutorial at the upcoming Apache: Big Data
North America 2017 conference.

On the technical side, the next major milestone is the availability of the
first stable release, which will include backward-compatibility guarantees.
This stabilization effort has started recently.

## Health report:

The community continues to grow steadily, as follows:

- The number of contributors continues to increase.
- Releases continue at a regular pace of 1-1.5 months per release.
- Mailing list activity continues to increase significantly.

## PMC changes:

Currently 14 PMC members. No new PMC members have been added since graduation
three months ago.

## Committer base changes:

Currently 20 committers. Three new committers have been added since
graduation:

- Ahmet Altay was added as a committer on Tue Jan 31 2017.
- Pei He was added as a committer on Tue Jan 31 2017.
- Stas Levin was added as a committer on Tue Jan 31 2017.

## Releases:

In the two months following graduation, Apache Beam has published two
releases:

- 0.4.0 was released on Sun Jan 01 2017.
- 0.5.0 was released on Mon Feb 06 2017.

In addition, the 0.6.0 release is in progress.

## Mailing list activity:

Mailing list activity continues to increase across all metrics.

- dev@beam.apache.org
- 351 subscribers (up 60 in the last 3 months)
- 1161 emails sent to list (866 in previous quarter)

- user@beam.apache.org
- 298 subscribers (up 58 in the last 3 months)
- 282 emails sent to list (241 in previous quarter)

## JIRA activity:

- 542 JIRA tickets created in the last 3 months
- 347 JIRA tickets closed/resolved in the last 3 months

27 Feb 2017 [Davor Bonaci / Isabel]

## Description:

Apache Beam is a unified programming model for both batch and streaming data
processing, enabling efficient execution across diverse distributed execution
engines and providing extensibility points for connecting to different
technologies and user communities.

## Issues:

There are no issues that require the Board's attention at this time.

## Activity:

Apache Beam was established as a top-level project at December’s Board
meeting. This is the second in the series of three consecutive monthly reports
for new projects.

Since last month's report, we have:
- published the second post-graduation release, version 0.5.0,
- added 3 new committers from two different organizations,
- promoted the Python SDK to the master branch with support for two runners.

Over the last month, Apache Beam graduation has been covered in more than a
dozen technical publications and received endorsements from multiple
organizations.

Beam continues to interconnect additional execution engines and data
storage/messaging systems. Since the last report, IO connectors for
Elasticsearch and MQ Telemetry Transport have been released, and additional
connectors for Redis, Apache Cassandra, Apache DistributedLog, Apache Parquet,
RabbitMQ, and Advanced Message Queuing Protocol (AMQP) are in progress.

Going forward, the main focus continues to be on the community growth. On the
technical side, the next major milestone is the availability of the first
stable release, which will include backward-compatibility guarantees.

## Health report:

The community continues to grow steadily, as follows:
- The number of contributors continues to increase.
- Releases continue at a regular pace of 1-1.5 months per release.
- Mailing list activity continues to increase significantly.

## PMC changes:

Currently 14 PMC members. No new PMC members have been added since graduation
two months ago.

## Committer base changes:

Currently 20 committers. Three new committers have been added in the last
month:
- Ahmet Altay was added as a committer on Tue Jan 31 2017.
- Pei He was added as a committer on Tue Jan 31 2017.
- Stas Levin was added as a committer on Tue Jan 31 2017.

## Releases:

In the two months following graduation, Apache Beam has published two
releases:
- 0.4.0 was released on Sun Jan 01 2017.
- 0.5.0 was released on Mon Feb 06 2017.

## Mailing list activity:

Mailing list activity continues to increase across all metrics.

- dev@beam.apache.org
- 332 subscribers (up 56 in the last 3 months)
- 1032 emails sent to list (762 in previous quarter)

- user@beam.apache.org
- 276 subscribers (up 50 in the last 3 months)
- 301 emails sent to list (203 in previous quarter)

## JIRA activity:

- 481 JIRA tickets created in the last 3 months
- 322 JIRA tickets closed/resolved in the last 3 months

## Appendix:

More details about graduation media coverage are available in the “media
recap” blog post:
https://beam.apache.org/blog/2017/02/01/graduation-media-recap.html

18 Jan 2017 [Davor Bonaci / Shane]

## Description:

Apache Beam is a unified programming model for both batch and streaming data
processing, enabling efficient execution across diverse distributed execution
engines and providing extensibility points for connecting to different
technologies and user communities.

## Issues:

There are no issues that require the Board's attention at this time.

## Activity:

Apache Beam was established as a top-level project at last month's Board
meeting. This is the first in the series of three consecutive monthly reports
for new projects.

Since becoming a top-level project, we have:
* completed administrative and infrastructure-related tasks to transition from
 a podling to a TLP,
* published the press release and a follow-up blog,
* published the first non-incubating release, version 0.4.0.

In addition, since the last report, we have participated in major conferences
and meetups, including:
* presented at Apache: Big Data Europe 2016 and ApacheCon's Podling Shark
 Tank, as well as and the Birds of Feather session,
* presented at QCon San Francisco 2016,
* presented at Strata + Hadoop World Singapore 2016, along with a hands-on
 Beam tutorial,
* co-organized a meetup with an Apache Apex user group, and presented at
 another meetup.

Beam continues to interconnect additional execution engines and data
storage/messaging systems. Since the last report, a runner for Apache Apex was
merged from a feature branch and released, and IO connectors for Elasticsearch
and MQ Telemetry Transport have been contributed.

Going forward, the main focus continues to be on community growth. On the
technical side, the next major milestone is the availability of the first
stable release, which will include backward-compatibility guarantees.

## Health report:

The community continues to grow steadily, as follows:
* The number of contributors continues to increase, with an expectation of
 additional committers in the near future.
* Releases continue at a regular pace of 1-1.5 months per release.
* Mailing list activity continues to increase, with some metrics doubling
 quarter-over-quarter (see below).

## PMC changes:

Currently 14 PMC members. No new PMC members have been added since graduation
a month ago.

## Committer base changes:

Currently 17 committers. No new committers have been added since graduation a
month ago.

## Releases:

The first post-graduation release, version 0.4.0, was published on January 1,
2017.

## Mailing list activity:

Mailing list activity continues to increase, with some metrics doubling
quarter-over-quarter.

- dev@beam.apache.org:
  - 310 subscribers (up 49 in the last 3 months)
  - 1079 emails sent to list (519 in previous quarter)

- user@beam.apache.org:
  - 261 subscribers (up 54 in the last 3 months)
  - 231 emails sent to list (246 in previous quarter)

## JIRA activity:

- 512 JIRA tickets created in the last 3 months
- 338 JIRA tickets closed/resolved in the last 3 months

21 Dec 2016

Establish the Apache Beam Project

 WHEREAS, the Board of Directors deems it to be in the best
 interests of the Foundation and consistent with the
 Foundation's purpose to establish a Project Management
 Committee charged with the creation and maintenance of
 open-source software, for distribution at no charge to
 the public, related to a unified programming model for both
 batch and streaming data processing, enabling efficient
 execution across diverse distributed execution engines
 and providing extensibility points for connecting to different
 technologies and user communities.

 NOW, THEREFORE, BE IT RESOLVED, that a Project Management
 Committee (PMC), to be known as the "Apache Beam Project",
 be and hereby is established pursuant to Bylaws of the
 Foundation; and be it further

 RESOLVED, that the Apache Beam Project be and hereby is
 responsible for the creation and maintenance of software
 related to a unified programming model for both batch and
 streaming data processing, enabling efficient execution across
 diverse distributed execution engines and providing extensibility
 points for connecting to different technologies and user
 communities; and be it further

 RESOLVED, that the office of "Vice President, Apache Beam" be
 and hereby is created, the person holding such office to
 serve at the direction of the Board of Directors as the chair
 of the Apache Beam Project, and to have primary responsibility
 for management of the projects within the scope of
 responsibility of the Apache Beam Project; and be it further

 RESOLVED, that the persons listed immediately below be and
 hereby are appointed to serve as the initial members of the
 Apache Beam Project:

   * Tyler Akidau <takidau@apache.org>
   * Davor Bonaci <davor@apache.org>
   * Robert Bradshaw <robertwb@apache.org>
   * Ben Chambers <bchambers@apache.org>
   * Luke Cwik <lcwik@apache.org>
   * Stephan Ewen <sewen@apache.org>
   * Dan Halperin <dhalperi@apache.org>
   * Kenneth Knowles <kenn@apache.org>
   * Aljoscha Krettek <aljoscha@apache.org>
   * Maximilian Michels <mxm@apache.org>
   * Jean-Baptiste Onofré <jbonofre@apache.org>
   * Frances Perry <frances@apache.org>
   * Amit Sela <amitsela@apache.org>
   * Josh Wills <jwills@apache.org>

 NOW, THEREFORE, BE IT FURTHER RESOLVED, that Davor Bonaci
 be appointed to the office of Vice President, Apache Beam, to
 serve in accordance with and subject to the direction of the
 Board of Directors and the Bylaws of the Foundation until
 death, resignation, retirement, removal or disqualification,
 or until a successor is appointed; and be it further

 RESOLVED, that the initial Apache Beam PMC be and hereby is
 tasked with the creation of a set of bylaws intended to
 encourage open development and increased participation in the
 Apache Beam Project; and be it further

 RESOLVED, that the Apache Beam Project be and hereby
 is tasked with the migration and rationalization of the Apache
 Incubator Beam podling; and be it further

 RESOLVED, that all responsibilities pertaining to the Apache
 Incubator Beam podling encumbered upon the Apache Incubator
 Project are hereafter discharged.

 Special Order 7C, Establish the Apache Beam Project, was
 approved by Unanimous Vote of the directors present.

16 Nov 2016

Apache Beam is an open source, unified model and set of language-specific
SDKs for defining and executing data processing workflows, and also data
ingestion and integration flows, supporting Enterprise Integration Patterns
(EIPs) and Domain Specific Languages (DSLs). Beam pipelines simplify the
mechanics of large-scale batch and streaming data processing and can run on
a number of runtimes such as Apache Flink, Apache Gearpump, Apache Apex,
Apache Spark, and Google Cloud Dataflow. Beam also brings SDKs in different
languages, allowing users to easily implement their data integration
processes.

Beam has been incubating since 2016-02-01.

The most important issue to address in the move towards graduation:

  1. Make it easier for the Beam community to to learn, use, and grow by
     expanding and improving the Beam documentation, code samples, and the
     website

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
of?

  None.

How has the community developed since the last report?

  * 441 closed/merged pull requests
  * High engagement on dev and user mailing lists (742 / 179 messages)
  * Several public talks, articles, and videos including:
    - @Scale San Jose (“No shard left behind: APIs for massive parallel
      efficiency in Apache Beam”)
    - Strata + Hadoop World NYC (“Learn stream processing with Apache Beam”)
    - Paris Spark Meetup (“Introduction to Apache Beam”)
    - Hadoop Summit Melbourne (“Stream/Batch processing portable across
      on-prem (Spark, Flink) and Cloud with Apache Beam”)
    - Hadoop User Group Taipei (“Stream Processing with Beam and Google Cloud
      Dataflow”)
    - Data Science Lab London (“Apache Beam: Stream and Batch Processing;
      Unified and Portable!”)

How has the project developed since the last report?

  Major developments on the project since last report include the following:

  * Second and third incubating release (0.2.0 and 0.3.0) and a release
    guide [1]
  * New DirectRunner support for testing streaming pipelines[2]
  * Continued improvements to the Flink, Spark, and Dataflow runners
  * Added support for new IO connectors, including MongoDB, Kinesis, and JDBC
    with Cassandra, MQTT support pending in pull requests
  * Addition of the Apache Apex runner on a feature branch, and continued
    work on the Apache Gearpump runner and Python SDK feature branches. [3]
  * Continued reorganization and refactoring of the project
  * Continued improvements to documentation and testing

 [1]: http://beam.incubator.apache.org/contribute/release-guide/
 [2]: http://beam.incubator.apache.org/blog/2016/10/20/test-stream.html
 [3]: http://beam.incubator.apache.org/contribute/work-in-progress/#feature-branches

Dates of last releases:

  * 2016/08/07 - 0.2.0-incubating
  * 2016/10/31 - 0.3.0-incubating

When were the last committers or PMC members elected?

  The following committers were elected on 2016/10/20:

  * Thomas Weise
  * Jesse Anderson
  * Thomas Groh

Signed-off-by:
 [X](beam) Jean-Baptiste Onofré
 [ ](beam) Venkatesh Seetharam
 [ ](beam) Ted Dunning

17 Aug 2016

Apache Beam is an open source, unified model and set of language-specific SDKs
for defining and executing data processing workflows, and also data ingestion
and integration flows, supporting Enterprise Integration Patterns
(EIPs) and Domain Specific Languages (DSLs). Beam pipelines simplify the
mechanics of large-scale batch and streaming data processing and can run on a
number of runtimes such as Apache Flink, Apache Gearpump, Apache Spark, and
Google Cloud Dataflow (a cloud service). Beam also brings SDKs in different
languages, allowing users to easily implement their data integration
processes.

Beam has been incubating since 2016-02-01.

Three most important issues to address in the move towards graduation:

 1. Additional and continued Beam releases
 2. Grow the community of Beam users and contributors
 3. Add to and improve upon documentation, code samples, and project
    website

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
of? None.

How has the community developed since the last report?

 * 425 closed/merged pull requests
 * High engagement on dev and user mailing lists (590 / 455 messages)
 * Several public talks, articles, and videos including:
 * Hadoop Summit San Jose ("Apache Beam: A Unified Model for Batch and
   Streaming Data Processing" & "The Next Generation of Data Processing &
   OSS")
 * O’Reilly & The New Stack ("Future-proof and scale-proof your code")
 * QCon NY ("Apache Beam: The Case for Unifying Streaming API's")
 * JBCN Barcelona ("Introduction to Apache Beam")

How has the project developed since the last report?

 Major developments on the project since last report include the following:

 * First incubating release (0.1.0-incubating)
 * Second incubating release (0.2.0-incubating)
 * Addition of Apache Beam Python SDK
 * Addition of the Apache Gearpump runner
 * Added support for writing to Apache Kafka clusters
 * Added support for reading from and writing to Java Message Services,
   including Apache ActiveMQ, GeronimoJMS, and RabbitMQ
 * Ratified new Beam model APIs to improve efficiency and failure handling:
   DoFn setup, teardown, and reuse
 * Optimized key components such as data serialization and shuffle
 * Continued improvements to the Flink, Spark, and Dataflow runners
 * Continued reorganization and refactoring of the project
 * Continued improvements to documentation and testing

Date of last release:

 * 2016/06/15 - 0.1.0-incubating
 * 2016/08/08 - 0.2.0-incubating)

When were the last committers or PMC members elected?

 N/A - no changes since last report.

Signed-off-by:

 [X](beam) Jean-Baptiste Onofre
 [ ](beam) Venkatesh Seetharam
 [X](beam) Bertrand Delacretaz
 [X](beam) Ted Dunning

18 May 2016

Apache Beam is an open source, unified model and set of language-specific SDKs
for defining and executing data processing workflows, and also data ingestion
and integration flows, supporting Enterprise Integration Patterns
(EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the
mechanics of large-scale batch and streaming data processing and can run on a
number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow
(a cloud service). Beam also brings DSL in different languages, allowing users
to easily implement their data integration processes.

Beam has been incubating since 2016-02-01.

Three most important issues to address in the move towards graduation:

 1. Continued releases
 2. Grow up user and contributor communities
 3. Improve and extend documentation and samples on the website

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
of?

 None

How has the community developed since the last report?

 * Both user and dev mailing lists activity increased
 * We sustain a high level of activity on the pull request cycles (submit,
   review, ...)

How has the project developed since the last report?

 * All resources have been created (website, Jira, git & github mirror, ...)
 * The code donation has been completed
 * The website has been published, we are still in the process of donated
   documentation and samples resources
 * We renamed all package to match the Apache convention
 * We started the re-organization and refactoring of the project structure
   (isolating and moving some modules)

Date of last release:

 N/A

When were the last committers or PMC members elected?

 N/A

Signed-off-by:

 [X](beam) Jean-Baptiste Onofre
 [X](beam) Jim Jagielski
 [X](beam) Venkatesh Seetharam
 [ ](beam) Bertrand Delacretaz
 [X](beam) Ted Dunning