Apache Logo
The Apache Way Contribute ASF Sponsors

This was extracted (@ 2018-10-06 21:10) from a list of minutes which have been approved by the Board.
Please Note The Board typically approves the minutes of the previous meeting at the beginning of every Board meeting; therefore, the list below does not normally contain details from the minutes of the most recent Board meeting.

2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010 | 2009 | 2008 | 2007 | 2006 | 2005 | 2004 | 2003 | 2002 | 2001 | 2000 | 1999 | Pre-organization meetings

Spot

19 Sep 2018

Report was filed, but display is awaiting the approval of the Board minutes.

15 Aug 2018

Apache Spot is a platform for network telemetry built on an open data model
and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 1. More activity in the User and Dev mail lists
 2. Revive Community activity (Discussion, commits)
 3. Develop a better release process

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

Lack of community discussion and activity is being addressed.

How has the community developed since the last report?

Started Bi-weekly (2 weeks) project wide community live meetings to try to
revive development on the project.

How has the project developed since the last report?

Community revival is underway. Bi-weekly meetings are taking place in order
to achieve momentum in the project. Goals are to prioritize Pull Requests
and promote the creation of new features in the community.

How would you assess the podling's maturity?
Please feel free to add your own commentary.

 [ ] Initial setup
 [ ] Working towards first release
 [X] Community building
 [ ] Nearing graduation
 [ ] Other:

Date of last release:

 2017-09-08

When were the last committers or PPMC members elected?

 2018-01-18

Signed-off-by:

 [ ](spot) Jarek Jarcec Cecho
    Comments:
 [ ](spot) Brock Noland
    Comments:
 [ ](spot) Andrei Savu
    Comments:
 [X](spot) Uma Maheswara Rao G
    Comments: Recently there was some improvement in discussions and
conducting community meetings etc.

IPMC/Shepherd notes:

21 Mar 2018

Apache Spot is a platform for network telemetry built on an open data model
and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 1. More activity in the User and Dev mail lists
 2. Integrate Open Data Model on top of Spot modules
 3. Develop a better release process

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?



How has the community developed since the last report?

A. more interaction from partners
B. Seeing more interest and usage of Spot leading to more contributions in on the way

How has the project developed since the last report?

A. More development around Open Data Model
B. New data sources being integrated with ODM is maturing the structure
C. Adding Kerberos support to the project
D. Ingest Redesign

How would you assess the podling's maturity?
Please feel free to add your own commentary.

 [ ] Initial setup
 [ ] Working towards first release
 [X] Community building
 [ ] Nearing graduation
 [ ] Other:

Date of last release:

 2017-09-08

When were the last committers or PPMC members elected?

 2018-01-18

Signed-off-by:

 [ ](spot) Jarek Jarcec Cecho
    Comments:
 [ ](spot) Brock Noland
    Comments:
 [ ](spot) Andrei Savu
    Comments:
 [X](spot) Uma Maheswara Rao G
    Comments: Dev list discussions are low and community should give more attention to this point.

21 Feb 2018

Apache Spot is a platform for network telemetry built on an open data model
and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 1. Develop a better release process

 2. Handle additional data types for ingestion and enrichment into ODM shema

 3. Fostering more activity in the user, dev and private mail lists

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

a. Issues providing podling reports during December, January.
This is being addressed moving forward.

How has the community developed since the last report?

a. Seeing more pull requests from new contributors

How has the project developed since the last report?

a. Development on the ODM branch has been moving forward and will continue to push towards adoption into the master branch.
b. Ingest redesign underway

How would you assess the podling's maturity?
Please feel free to add your own commentary.

  [ ] Initial setup
  [ ] Working towards first release
  [x] Community building
  [ ] Nearing graduation
  [ ] Other:

Date of last release:

2017-09-08

When were the last committers or PPMC members elected?

2018-01-18

Signed-off-by:

  [ ](spot) Jarek Jarcec Cecho
     Comments:
  [X](spot) Brock Noland
     Comments:
  [ ](spot) Andrei Savu
     Comments:
  [ ](spot) Uma Maheswara Rao G
     Comments:

IPMC/Shepherd notes:
  Dave Fisher: It's not clear to me where this podling has development discussions.
  It would be good for the mentors to guide Spot to at least reflect discussions
  on the dev@ list from Slack, Github or wherever these are happening.

18 Oct 2017

Apache Spot is a solution stack that provides the capability to ingest IT related telemetry (network flows, domain name service DNS, and proxy server logs) and provide unsupervised machine learning capabilities to identify suspicious activity.  The information is organized and presented using operational analytics so that a security analyst can investigate the most suspicious connections. Apache Spot is built on and using Apache Spark and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

  1. Build diverse community
  2. Continue building a strong and diverse body of PPMC members for long term project continuity
  3. Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
 * Recently Facebook decided to re-license React under MIT license, to enable compatibility with Apache licensed projects and many others. If there are not other indications from Incubator board we’ll call out a vote to keep it for now: https://code.facebook.com/posts/300798627056246/relicensing-react-jest-flow-and-immutable-js/


How has the community developed since the last report?

 * Committers that are specialized on Ingest functions are merging branches for Open Data Model (ODM) development.
 * Same case for views in order to simplify the process of normalization through the UI.
 * Datasets with both benign and malicious traffic have been crafted to allow community to test/evaluate ML executions, and facilitate visualization exercises when doing Threat Hunting .
 * Number of issues and discussion have increased with both feedback/RFE and bug fixes, which probes adoption, of course Community growth also consists on continue expanding the Committer baseline.


How has the project developed since the last report?

 * The Spot project completed the first official Apache release for Spot v1.0, approved and voted by Incubator PMC.
 * Open Data Model, development efforts continue over its branch in both directions specification and code pushes.


How would you assess the podling's maturity?
Please feel free to add your own commentary.

  [ ] Initial setup
  [ ] Working towards first release
  [X] Community building
  [ ] Nearing graduation
  [ ] Other:

 Comments:

 * Completion of the first Apache release was a significant milestone. The focus now needs to be converting the large number of project observers into project committers to foster a more diverse community, creating project awareness through technology blog posts, and Meetups is a significant effort that needs to happen for project evangelization

Date of last release:

  2017-09-08

When were the last committers or PPMC members elected?

 2017-06-02

Signed-off-by:

  [X](spot) Jarek Jarcec Cecho
     Comments:
  [ ](spot) Brock Noland
     Comments:
  [ ](spot) Andrei Savu
     Comments:
  [X](spot) Uma Maheswara Rao G
     Comments:

IPMC/Shepherd notes:
  Project is active and moving forward well. They do use ReactJS and will need to upgrade and release as soon as the relicensed version is available.
  Dave Fisher

20 Sep 2017

 IPMC/Shepherd notes:

 - Drew Farris (shepherd): No mentor activity apparent from the mailing lists,
   ongoing project activity observed.
 - John D. Ament:  Podling needed some reminding about the board report, I suspect
   next report will be in and of high quality.

21 Jun 2017

Apache Spot is a solution stack that provides the capability to ingest IT
related telemetry (network flows, domain name service DNS, and proxy server
logs) and provide unsupervised machine learning capabilities to identify
suspicious activity.  The information is organized and presented using
operational analytics so that a security analyst can investigate the most
suspicious connections. Apache Spot is built on an open data model using
Apache Spark and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 1.  Move infrastructure and development to ASF (code, issues, mailing list,
   ...)
 2.  Build diverse community
 3.  Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
of? Two out of the four mentors are the ones showing the most active
participation and involvement on the project so we need to discuss with IPMC
our options to get the other two replaced.

How has the community developed since the last report? By remaking the
Community section on the Project webpage and driving users on Slack to limit
the usage of the same tool just for quick interactions, the traffic over Mail
Lists has increased, we still need to continue driving usage questions to USER
list. A group of Developers from Brazil is coming with a proposal for UX and
UI, based on the merits of the proposal, and the potential that the same has
we have invited them to become Committers on the projects.
The initial purpose of the Project on Network & Security needs to expand the
scope to IT Analytics to cover other uses cases and/or enrich content with
User and Endpoint data. While a normalization framework has been drafted to
what we call Open Data Model we still need more work and contributions on
technical delivery to make it a reality. Contributions to improve the concept
of Ingestion Pipeline & Data Lake are delivered with more cadence, there is a
recognition from Community to normalize data to open the space for other
sources of Ingestion

How has the project developed since the last report? The team has created the
epic on ASF JIRA towards the first Apache release, we’re working now including
the user stories that will be part of release, and doing checks to what can be
delivered before we do the the formal announcement on DEV for the code freeze.
Integrations for context enrichment are critical, however also the need of
bringing a plugin manager where does can be handled, the Spec was drafted, and
the team now is finalizing the last Pull requests to begin merging
functionality onto project code. Documentation is an area of focus so from May
to July committers we’ll continue improving & detailing more, architecture
definitions, deployment options, and UI usage sections. As the whole delivery
is concluded we’ll vote for a FAQ section which can be a point of reference
for common issues/situations. Being part of ASF is important Community
requests that project enables integration/consumption of other Apache
projects, one of our DEVOPS committers, for that reason having an API
framework now will open the door for more smooth integrations.

Date of last release: N/A

When were the last committers or PMC members elected? Last committer was
elected on 06/02

How does the podling rate their own maturity? By know we’re almost ready to do
our first Apache Release which is good, but we still need to mature Community,
the ramping of other Contributors has been slow, we have a strong base of
Consumer positive aspect as it probed consumption, however more Commits from
other interested parties will accelerate functionality delivery.


Signed-off-by:
 [X](spot) Jarek Jarcec Cecho
    Comments:
 [ ](spot) Brock Noland
    Comments:
 [ ](spot) Andrei Savu
    Comments:
 [X](spot) Uma Maheswara Rao G
    Comments:

15 Mar 2017

Apache Spot is a solution stack that provides the capability to ingest IT
related telemetry (network flows, domain name service information and proxy
server logs) and provide unsupervised machine learning capabilities to
identify suspicious activity.  The information is organized and presented
using operational analytics so that a security analyst can investigate the
most suspicious connections. Apache Spot is built on an open data model using
Apache Spark and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 1. Move infrastructure and development to ASF (code, issues, mailing
    list, ...)
 2. Build diverse community
 3. Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of?

 None

How has the community developed since the last report?

 Based on initial feedback, process for issues was updated on the project
 website, with primary objective to facilitate the workflow and interaction
 with Community, and having diverse ownership of them until resolution.

 Slack channel has become very popular, however in order to adhere to ASF
 official communications channels, those conversations that involve quick
 question will be kept there, any extended discussion on code, pipeline, and
 architecture, are being ported to Dev mail list.

 There has been a concern to get continuous improvement of the UI on Apache
 Spot, committers have been working on process to get UX interviews that
 help to understand and address community and compliance needs.

How has the project developed since the last report?

 The project got a major architecture change in order to remove CSV’s,
 offload storage from the OS file system and instead leverage HDFS (for
 distributed architecture).

 Per community request through JIRA issue, and in order to facilitate the
 installation/adoption of the project for scale deployments, an installation
 script consolidating the ones that were built per pipeline component has
 been pulled to repository, it will reduce the complexity of porting files
 and configuration variables, by having a central point of deployment.

 Open Data Models, which serves as a framework for data sources
 normalization, initially for ingestion, was ported through its
 Specification to the project repository, this will bring more contributions
 to facilitate the design of the NoSQL table, and then determine technology
 selection through discussion and voting through mailing list.

February Metrics:

   +---------------------------------------------+
   |   Metric                          | counts  |
   +---------------------------------------------+
   | Active Contributors (incl mentors)|    14   |
   | Jira New Issues                   |    6    |
   | Resolved Issues                   |    1    |
   | Pull Requests merged              |    3    |
   | Pull Requests proposed            |    3    |
   +---------------------------------------------+

Date of last release:

 N/A

When were the last committers or PMC members elected?

 Last committer was elected on 2/24

How does the podling rate their own maturity?

 There was a strong focus on code delivery over the last quarter, towards
 Apache release, community adoption keeps growing, now by adding a layer of
 process for opening issues and commits, plus interaction over channels
 we're bringing contributions from others, towards Apache release.

Signed-off-by:

 [x](spot) Jarek Jarcec Cecho
    Comments:
 [ ](spot) Brock Noland
    Comments:
 [ ](spot) Andrei Savu
    Comments:
 [x](spot) Uma Maheswara Rao G
    Comments:

27 Feb 2017

Apache Spot is a solution stack that provides the capability to ingest IT
related telemetry (network flows, domain name service information and proxy
server logs) and provide unsupervised machine learning capabilities to
identify suspicious activity.  The information is organized and presented
using operational analytics so that a security analyst can investigate the
most suspicious connections. Apache Spot is built on an open data model
using Apache Spark and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 1.  Move infrastructure and development to ASF (code, issues, mailing
     list, ...)
 2.  Build diverse community
 3.  Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

 IPMC this point has been posted on the last two report, we have not
 received any feedback in terms of how it goes, please comment.

 Please elaborate on the process for code scanning and IP reviews. On the
 last report this was brought to attention but no communication was
 received.

How has the community developed since the last report?

 Community is pushing more aggressively on project interaction which is
 good, by opening issues on ASF JIRA, some of them have been closed with
 their respective PR for bug fixing. Also developers in Brazil are
 proposing a new UI schema by their own, which once is voted will
 incorporate as contribution/new functionality towards our first Apache
 release.

 Adoption rate and popularity of the project has been recognized by
 Industry receiving an award from Infoworld as Technology of the year and
 also finalist on Edison Awards.

How has the project developed since the last report?

 The Spot team  launched a new project webpage version which incorporate
 the long term vision and objectives of Apache Spot, and also includes a
 new documentation package which is similar in structure to other ASF
 projects.

 Two new processes were published to explain on commits and how to open
 issues, providing a structured frame that Community in general can follow
 up in order to facilitate follow-up in combination of technology tools,
 that can facilitate the life cycle continuity.

 During this period 14 pull request were merged over different pipeline
 components , as committers work also on the new design for data storage
 through non-relational database technologies.

 Also facilitating a quick demo of the project, we have updated our
 container demo, and added automation in order to reflect those changes as
 commits to UI are pulled to ASF Git repository.

Date of last release:

 N/A

When were the last committers or PMC members elected?

 Last committer was elected on 1/24

Signed-off-by:

 [ ](spot) Jarek Jarcec Cecho
 [ ](spot) Brock Noland
 [ ](spot) Andrei Savu
 [X](spot) Uma Maheswara Rao G

21 Dec 2016

Apache Spot is a solution stack that provides the capability to ingest
network related telemetry (network flows, domain name service
information and proxy server logs) and provide unsupervised machine
learning capabilities to identify suspicious activity.  The information
is organized and presented using operational analytics so that a
security analyst can investigate the most suspicious connections. Apache
Spot is built on an open data model using Apache Spark and Apache
Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

  1.  Move infrastructure and development to ASF (code, issues, mailing list, ...)
  2.  Build diverse community
  3.  Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

  Please elaborate on the process for code scanning and IP reviews.

How has the community developed since the last report?

  Community interaction has ramped velocity, as new adopters of the project
  have been posting questions through Twitter and Slack. We have added
  approximately 20% to our Slack channel and added several contributors
  through the ICLA.

How has the project developed since the last report?

  We have moved our repo to ASF as of December 8, 2016.

  We have removed our LDA-C dependencies by switching to Spark LDA,
  eliminating licensing issues on this particular branch.

Date of last release:

  N/A

When were the last committers or PMC members elected?

  N/A

Signed-off-by:

 [ ](spot) Jarek Jarcec Cecho
 [ ](spot) Brock Noland
 [X](spot) Andrei Savu
 [X](spot) Uma Maheswara Rao G

16 Nov 2016

Apache Spot is a solution stack that provides the capability to ingest
network related telemetry (network flows, domain name service information and
proxy server logs) and provide unsupervised machine learning capabilities to
identify suspicious activity.  The information is organized and presented
using operational analytics so that a security analyst can investigate the
most suspicious connections. Apache Spot is built on an open data model using
Apache Spark and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 - Move infrastructure and development to ASF (code, issues, mailing list, …)
 - Build diverse community
 - Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of?

 We currently have a hard dependency via our LDA implementation that
 requires a GPL license.  A rewrite is in progress to move the code from
 LDA-C to Spark LDA.

How has the community developed since the last report?

 This is first report after accepting Apache Spot to incubator and we still
 have not transitioned everything to ASF. We do however see increased
 interest in the project, primarily on our Slack channel.

How has the project developed since the last report?

 This is first report.

Date of last release:

 N/A

When were the last committers or PMC members elected?

 N/A

Signed-off-by:

 [x](spot) Jarek Jarcec Cecho
 [ ](spot) Brock Noland
 [ ](spot) Andrei Savu
 [X](spot) Uma Maheswara Rao G