Apache Logo
The Apache Way Contribute ASF Sponsors

This was extracted (@ 2018-04-18 20:10) from a list of minutes which have been approved by the Board.
Please Note The Board typically approves the minutes of the previous meeting at the beginning of every Board meeting; therefore, the list below does not normally contain details from the minutes of the most recent Board meeting.

2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010 | 2009 | 2008 | 2007 | 2006 | 2005 | 2004 | 2003 | 2002 | 2001 | 2000 | 1999 | Pre-organization meetings

Spot

21 Mar 2018

Apache Spot is a platform for network telemetry built on an open data model
and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 1. More activity in the User and Dev mail lists
 2. Integrate Open Data Model on top of Spot modules
 3. Develop a better release process

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?



How has the community developed since the last report?

A. more interaction from partners
B. Seeing more interest and usage of Spot leading to more contributions in on the way

How has the project developed since the last report?

A. More development around Open Data Model
B. New data sources being integrated with ODM is maturing the structure
C. Adding Kerberos support to the project
D. Ingest Redesign

How would you assess the podling's maturity?
Please feel free to add your own commentary.

 [ ] Initial setup
 [ ] Working towards first release
 [X] Community building
 [ ] Nearing graduation
 [ ] Other:

Date of last release:

 2017-09-08

When were the last committers or PPMC members elected?

 2018-01-18

Signed-off-by:

 [ ](spot) Jarek Jarcec Cecho
    Comments:
 [ ](spot) Brock Noland
    Comments:
 [ ](spot) Andrei Savu
    Comments:
 [X](spot) Uma Maheswara Rao G
    Comments: Dev list discussions are low and community should give more attention to this point.

21 Feb 2018

Apache Spot is a platform for network telemetry built on an open data model
and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 1. Develop a better release process

 2. Handle additional data types for ingestion and enrichment into ODM shema

 3. Fostering more activity in the user, dev and private mail lists

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

a. Issues providing podling reports during December, January.
This is being addressed moving forward.

How has the community developed since the last report?

a. Seeing more pull requests from new contributors

How has the project developed since the last report?

a. Development on the ODM branch has been moving forward and will continue to push towards adoption into the master branch.
b. Ingest redesign underway

How would you assess the podling's maturity?
Please feel free to add your own commentary.

  [ ] Initial setup
  [ ] Working towards first release
  [x] Community building
  [ ] Nearing graduation
  [ ] Other:

Date of last release:

2017-09-08

When were the last committers or PPMC members elected?

2018-01-18

Signed-off-by:

  [ ](spot) Jarek Jarcec Cecho
     Comments:
  [X](spot) Brock Noland
     Comments:
  [ ](spot) Andrei Savu
     Comments:
  [ ](spot) Uma Maheswara Rao G
     Comments:

IPMC/Shepherd notes:
  Dave Fisher: It's not clear to me where this podling has development discussions.
  It would be good for the mentors to guide Spot to at least reflect discussions
  on the dev@ list from Slack, Github or wherever these are happening.

18 Oct 2017

Apache Spot is a solution stack that provides the capability to ingest IT related telemetry (network flows, domain name service DNS, and proxy server logs) and provide unsupervised machine learning capabilities to identify suspicious activity.  The information is organized and presented using operational analytics so that a security analyst can investigate the most suspicious connections. Apache Spot is built on and using Apache Spark and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

  1. Build diverse community
  2. Continue building a strong and diverse body of PPMC members for long term project continuity
  3. Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
 * Recently Facebook decided to re-license React under MIT license, to enable compatibility with Apache licensed projects and many others. If there are not other indications from Incubator board we’ll call out a vote to keep it for now: https://code.facebook.com/posts/300798627056246/relicensing-react-jest-flow-and-immutable-js/


How has the community developed since the last report?

 * Committers that are specialized on Ingest functions are merging branches for Open Data Model (ODM) development.
 * Same case for views in order to simplify the process of normalization through the UI.
 * Datasets with both benign and malicious traffic have been crafted to allow community to test/evaluate ML executions, and facilitate visualization exercises when doing Threat Hunting .
 * Number of issues and discussion have increased with both feedback/RFE and bug fixes, which probes adoption, of course Community growth also consists on continue expanding the Committer baseline.


How has the project developed since the last report?

 * The Spot project completed the first official Apache release for Spot v1.0, approved and voted by Incubator PMC.
 * Open Data Model, development efforts continue over its branch in both directions specification and code pushes.


How would you assess the podling's maturity?
Please feel free to add your own commentary.

  [ ] Initial setup
  [ ] Working towards first release
  [X] Community building
  [ ] Nearing graduation
  [ ] Other:

 Comments:

 * Completion of the first Apache release was a significant milestone. The focus now needs to be converting the large number of project observers into project committers to foster a more diverse community, creating project awareness through technology blog posts, and Meetups is a significant effort that needs to happen for project evangelization

Date of last release:

  2017-09-08

When were the last committers or PPMC members elected?

 2017-06-02

Signed-off-by:

  [X](spot) Jarek Jarcec Cecho
     Comments:
  [ ](spot) Brock Noland
     Comments:
  [ ](spot) Andrei Savu
     Comments:
  [X](spot) Uma Maheswara Rao G
     Comments:

IPMC/Shepherd notes:
  Project is active and moving forward well. They do use ReactJS and will need to upgrade and release as soon as the relicensed version is available.
  Dave Fisher

20 Sep 2017

 IPMC/Shepherd notes:

 - Drew Farris (shepherd): No mentor activity apparent from the mailing lists,
   ongoing project activity observed.
 - John D. Ament:  Podling needed some reminding about the board report, I suspect
   next report will be in and of high quality.

15 Mar 2017

Apache Spot is a solution stack that provides the capability to ingest IT
related telemetry (network flows, domain name service information and proxy
server logs) and provide unsupervised machine learning capabilities to
identify suspicious activity.  The information is organized and presented
using operational analytics so that a security analyst can investigate the
most suspicious connections. Apache Spot is built on an open data model using
Apache Spark and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 1. Move infrastructure and development to ASF (code, issues, mailing
    list, ...)
 2. Build diverse community
 3. Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of?

 None

How has the community developed since the last report?

 Based on initial feedback, process for issues was updated on the project
 website, with primary objective to facilitate the workflow and interaction
 with Community, and having diverse ownership of them until resolution.

 Slack channel has become very popular, however in order to adhere to ASF
 official communications channels, those conversations that involve quick
 question will be kept there, any extended discussion on code, pipeline, and
 architecture, are being ported to Dev mail list.

 There has been a concern to get continuous improvement of the UI on Apache
 Spot, committers have been working on process to get UX interviews that
 help to understand and address community and compliance needs.

How has the project developed since the last report?

 The project got a major architecture change in order to remove CSV’s,
 offload storage from the OS file system and instead leverage HDFS (for
 distributed architecture).

 Per community request through JIRA issue, and in order to facilitate the
 installation/adoption of the project for scale deployments, an installation
 script consolidating the ones that were built per pipeline component has
 been pulled to repository, it will reduce the complexity of porting files
 and configuration variables, by having a central point of deployment.

 Open Data Models, which serves as a framework for data sources
 normalization, initially for ingestion, was ported through its
 Specification to the project repository, this will bring more contributions
 to facilitate the design of the NoSQL table, and then determine technology
 selection through discussion and voting through mailing list.

February Metrics:

   +---------------------------------------------+
   |   Metric                          | counts  |
   +---------------------------------------------+
   | Active Contributors (incl mentors)|    14   |
   | Jira New Issues                   |    6    |
   | Resolved Issues                   |    1    |
   | Pull Requests merged              |    3    |
   | Pull Requests proposed            |    3    |
   +---------------------------------------------+

Date of last release:

 N/A

When were the last committers or PMC members elected?

 Last committer was elected on 2/24

How does the podling rate their own maturity?

 There was a strong focus on code delivery over the last quarter, towards
 Apache release, community adoption keeps growing, now by adding a layer of
 process for opening issues and commits, plus interaction over channels
 we're bringing contributions from others, towards Apache release.

Signed-off-by:

 [x](spot) Jarek Jarcec Cecho
    Comments:
 [ ](spot) Brock Noland
    Comments:
 [ ](spot) Andrei Savu
    Comments:
 [x](spot) Uma Maheswara Rao G
    Comments:

27 Feb 2017

Apache Spot is a solution stack that provides the capability to ingest IT
related telemetry (network flows, domain name service information and proxy
server logs) and provide unsupervised machine learning capabilities to
identify suspicious activity.  The information is organized and presented
using operational analytics so that a security analyst can investigate the
most suspicious connections. Apache Spot is built on an open data model
using Apache Spark and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 1.  Move infrastructure and development to ASF (code, issues, mailing
     list, ...)
 2.  Build diverse community
 3.  Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

 IPMC this point has been posted on the last two report, we have not
 received any feedback in terms of how it goes, please comment.

 Please elaborate on the process for code scanning and IP reviews. On the
 last report this was brought to attention but no communication was
 received.

How has the community developed since the last report?

 Community is pushing more aggressively on project interaction which is
 good, by opening issues on ASF JIRA, some of them have been closed with
 their respective PR for bug fixing. Also developers in Brazil are
 proposing a new UI schema by their own, which once is voted will
 incorporate as contribution/new functionality towards our first Apache
 release.

 Adoption rate and popularity of the project has been recognized by
 Industry receiving an award from Infoworld as Technology of the year and
 also finalist on Edison Awards.

How has the project developed since the last report?

 The Spot team  launched a new project webpage version which incorporate
 the long term vision and objectives of Apache Spot, and also includes a
 new documentation package which is similar in structure to other ASF
 projects.

 Two new processes were published to explain on commits and how to open
 issues, providing a structured frame that Community in general can follow
 up in order to facilitate follow-up in combination of technology tools,
 that can facilitate the life cycle continuity.

 During this period 14 pull request were merged over different pipeline
 components , as committers work also on the new design for data storage
 through non-relational database technologies.

 Also facilitating a quick demo of the project, we have updated our
 container demo, and added automation in order to reflect those changes as
 commits to UI are pulled to ASF Git repository.

Date of last release:

 N/A

When were the last committers or PMC members elected?

 Last committer was elected on 1/24

Signed-off-by:

 [ ](spot) Jarek Jarcec Cecho
 [ ](spot) Brock Noland
 [ ](spot) Andrei Savu
 [X](spot) Uma Maheswara Rao G

21 Dec 2016

Apache Spot is a solution stack that provides the capability to ingest
network related telemetry (network flows, domain name service
information and proxy server logs) and provide unsupervised machine
learning capabilities to identify suspicious activity.  The information
is organized and presented using operational analytics so that a
security analyst can investigate the most suspicious connections. Apache
Spot is built on an open data model using Apache Spark and Apache
Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

  1.  Move infrastructure and development to ASF (code, issues, mailing list, ...)
  2.  Build diverse community
  3.  Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

  Please elaborate on the process for code scanning and IP reviews.

How has the community developed since the last report?

  Community interaction has ramped velocity, as new adopters of the project
  have been posting questions through Twitter and Slack. We have added
  approximately 20% to our Slack channel and added several contributors
  through the ICLA.

How has the project developed since the last report?

  We have moved our repo to ASF as of December 8, 2016.

  We have removed our LDA-C dependencies by switching to Spark LDA,
  eliminating licensing issues on this particular branch.

Date of last release:

  N/A

When were the last committers or PMC members elected?

  N/A

Signed-off-by:

 [ ](spot) Jarek Jarcec Cecho
 [ ](spot) Brock Noland
 [X](spot) Andrei Savu
 [X](spot) Uma Maheswara Rao G

16 Nov 2016

Apache Spot is a solution stack that provides the capability to ingest
network related telemetry (network flows, domain name service information and
proxy server logs) and provide unsupervised machine learning capabilities to
identify suspicious activity.  The information is organized and presented
using operational analytics so that a security analyst can investigate the
most suspicious connections. Apache Spot is built on an open data model using
Apache Spark and Apache Hadoop.

Spot has been incubating since 2016-09-23.

Three most important issues to address in the move towards graduation:

 - Move infrastructure and development to ASF (code, issues, mailing list, …)
 - Build diverse community
 - Demonstrate ability to create releases

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of?

 We currently have a hard dependency via our LDA implementation that
 requires a GPL license.  A rewrite is in progress to move the code from
 LDA-C to Spark LDA.

How has the community developed since the last report?

 This is first report after accepting Apache Spot to incubator and we still
 have not transitioned everything to ASF. We do however see increased
 interest in the project, primarily on our Slack channel.

How has the project developed since the last report?

 This is first report.

Date of last release:

 N/A

When were the last committers or PMC members elected?

 N/A

Signed-off-by:

 [x](spot) Jarek Jarcec Cecho
 [ ](spot) Brock Noland
 [ ](spot) Andrei Savu
 [X](spot) Uma Maheswara Rao G