Community-led development "The Apache Way"
This was extracted (@ 2020-12-16 21:10) from a list of minutes
which have been approved by the Board.
Please Note
The Board typically approves the minutes of the previous meeting at the
beginning of every Board meeting; therefore, the list below does not
normally contain details from the minutes of the most recent Board meeting.
Meeting times vary, the exact schedule is available to ASF Members and Officers, search for "calendar" in the Foundation's private index page (svn:foundation/private-index.html).
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
### Three most important unfinished issues to address before graduating:
1. Make official SDAP (Incubating) Release
2. Improve committer participation
3. Improve/create user guide documentation
### Are there any issues that the IPMC or ASF Board need to be aware of?
None.
### How has the community developed since the last report?
Thomas Huang delivered a (virtual) keynote address at the 2020 ApacheCon
@Home on 9/29 and discussed our creation of the Apache Science Data
Analytics Platform (SDAP) as a professional open source Analytics Center
Framework (ACF) and its growing list of NASA and international
applications.
The COVERAGE (CEOS Ocean Variables Enabling Research and Applications for
GEO, https://coverage.ceos.org/technology/#software) development team at
JPL completed an initial deployment of SDAP on European cloud platform,
WEkEO (https://wekeo.eu/). Paired with a previously deployed SDAP on
Amazon Web Services (AWS), an objective of COVERAGE is to enable joint
analytics leveraging geographically distributed SDAP installations that
host distinct yet compatible American and European satellite-based and in
situ ocean-related datasets.
Additional deployments of SDAP that are currently being maintained:
- NASA Estimating the Circulation and Climate of the Ocean (ECCO) (
https://ecco-group.org)
- NASA Gravity Recovery & Climate Experiment (GRACE) Follow-On (
https://grace.jpl.nasa.gov)
- NASA Sea Level Change Portal (https://sealevel.nasa.gov)
### How has the project developed since the last report?
Three new features:
- SDAP-292 Add Cassandra authentication support to delete_by_query.py
script
- SDAP-278 Cassandra authentication for webapp
- SDAP-258 Update Deseason Algorithm to Use Precomputed Climatologies
Two improvements:
- SDAP-284 Collection Manager to support recursive directory structures
- SDAP-285 Solr not indexing lat/lon
Eight bug fixes:
- SDAP-282 Support configuring dimension names and projection during
ingestion
- SDAP-291 netcdf parsing error with GMPE dataset in Granule Ingester
- SDAP-280 Collection Manager should not block when checking ingestion
history
- SDAP-279 Collection Manager does not pick up fs events with NAS mount
- SDAP-267 config operator, manage git url or branch update in the helm
chart
- SDAP-261 bug fixes for new ingestion system (granule ingestion &
collection manager)
- SDAP-277 Better error handling in Granule Ingester
- SDAP-281 bigdata ingestion feedback (reading time)
### How would you assess the podling's maturity?
Please feel free to add your own commentary.
- [ ] Initial setup
- [X] Working towards first release
- [X] Community building
- [ ] Nearing graduation
- [ ] Other:
### Date of last release:
XXXX-XX-XX
### When were the last committers or PPMC members elected?
Two new committers were elected in April 2020.
### Have your mentors been helpful and responsive?
No issues, mentors are responsive.
### Is the PPMC managing the podling's brand / trademarks?
The PPMC is not managing the podling's brand / trademarks
### Signed-off-by:
- [ ] (sdap) Jörn Rottmann
Comments:
- [X] (sdap) Trevor Grant
Comments:
### IPMC/Shepherd notes:
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
### Three most important unfinished issues to address before graduating:
1. Make official SDAP (Incubating) Release
2. Improve committer participation
3. Improve/create user guide documentation
### Are there any issues that the IPMC or ASF Board need to be aware of?
Subscription of PPMC members to the @private mailing list should be
resolved. All PPMC members should now be subscribed to @private. If this
is not accurate, please advise.
### How has the community developed since the last report?
New committers have been added.
### How has the project developed since the last report?
1. Integrated SDAP analytics with Esri ArcGIS front end.
2. Added new temporal variance (2D map) analytic algorithm.
3. Further improvements in Kubernetes-based deployment procedures.
### How would you assess the podling's maturity?
Please feel free to add your own commentary.
- [ ] Initial setup
- [X] Working towards first release
- [X] Community building
- [ ] Nearing graduation
- [ ] Other:
### Date of last release:
XXXX-XX-XX
### When were the last committers or PPMC members elected?
Two new committers were elected in April 2020.
### Have your mentors been helpful and responsive?
Mentors have been helpful.
### Is the PPMC managing the podling's brand / trademarks?
PPMC is not managing the podling's brand / trademarks.
### Signed-off-by:
- [ ] (sdap) Jörn Rottmann
Comments:
- [x] (sdap) Trevor Grant
Comments:
### IPMC/Shepherd notes:
Justin Mclean: Can I ask why the PPMC is not managing the podlings
brand and trademarks? Also there still 1/3 of of your PPMC members
not signed up to your private list. Also please include a date of
your last release in your next report.
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
### Three most important unfinished issues to address before graduating:
1. Make official SDAP (Incubating) Release
2. Improve committer participation
3. Improve/create user guide documentation
### Are there any issues that the IPMC or ASF Board need to be aware of?
No.
### How has the community developed since the last report?
We are continuing to attract new committers.
A NASA-sponsored project to build a cloud workflow for sea ice and ocean
circulation data, in development at JPL, is adapting SDAP for its analytics
platform. Reference: T. Huang, M. DeBellis, I. Fenty, P. Heimbach, J. C.
Jacob, O. Wang, E. Yam, "Analytics Center Framework for Estimating the
Circulation and Climate of the Ocean," 39th IEEE Geoscience and Remote
Sensing Symposium, Jul-Aug 2019, doi: 10.1109/IGARSS.2019.8897904.
### How has the project developed since the last report?
Development of a helm chart for use when deploying to kubernetes has been
opened as a PR.
Further development work has continued and existing installations are being
maintained.
### How would you assess the podling's maturity?
Please feel free to add your own commentary.
- [x] Initial setup
- [x] Working towards first release
- [x] Community building
- [ ] Nearing graduation
- [ ] Other:
### Date of last release:
XXXX-XX-XX
### When were the last committers or PPMC members elected?
Maya Debellis was elected as a committer on 2019-02-08
### Have your mentors been helpful and responsive?
Mentors are helpful and responsive.
### Is the PPMC managing the podling's brand / trademarks?
PPMC is not managing the podling's brand or trademarks.
### Signed-off-by:
- [ ] (sdap) Jörn Rottmann
Comments:
- [X] (sdap) Trevor Grant
Comments: Need to start working towards release. Let me know if I can
help- I may have some cycles in upcoming months.
### IPMC/Shepherd notes:
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
### Three most important unfinished issues to address before graduating:
1. Make official SDAP (Incubating) Release
2. Improve/create user guide documentation
3. Improve committer participation
### Are there any issues that the IPMC or ASF Board need to be aware of?
No.
### How has the community developed since the last report?
Interest in the project has increased after presentation at ApacheCon.
New `#sdap` channel created on the-asf.slack.com
### How has the project developed since the last report?
Frank Greguska attended ApacheCon NA and presented an overview of SDAP.
Discussion about what is necessary to produce a release were beneficial
and the path forward is clear. Goal is to get a source release before the
end of the year.
### How would you assess the podling's maturity?
There are several deployments of SDAP actively being used and interest is
high. However, active participation from project members is low. First
source release should be coming soon.
Please feel free to add your own commentary.
- [x] Initial setup
- [x] Working towards first release
- [x] Community building
- [ ] Nearing graduation
- [ ] Other:
### Date of last release:
XXXX-XX-XX
### When were the last committers or PPMC members elected?
Maya Debellis was elected as a committer on 2019-02-08
### Have your mentors been helpful and responsive?
Yes, Trevor has been helpful and responsive. Have not heard from Jörn
recently.
### Signed-off-by:
- [ ] (sdap) Jörn Rottmann
Comments:
- [x] (sdap) Trevor Grant
Comments:
### IPMC/Shepherd notes:
SDAP is an integrated data analytic framework for Big Science problems.
SDAP has been incubating since 2017-10-22.
### Three most important issues to address in the move towards graduation:
1. Make official SDAP (Incubating) Release
2. Improve/create user guide documentation
3. Improve committer participation
### Are there any issues that the IPMC or ASF Board need to be aware of?
No.
### How has the community developed since the last report?
We are continuing to attract new committers.
### How has the project developed since the last report?
SDAP is still working towards its first release. Work towards a
repeatable build pipeline has been started.
New functionality has been added to the Distributed Oceanographic Matchup
Service (DOMS) and Extensible Data Gateway Environment (EDGE) components
that allow for filtering in situ measurements by a standardized quality
flag.
### How would you assess the podling's maturity?
There are several deployments of SDAP actively being used and interest is
high. However, active participation from project members is low. A push
is needed to cross the t's and dot the i's in regards to meeting the
requirements Apache has for releasing software so that we can claim a 1.0
version of SDAP.
Please feel free to add your own commentary.
- [X] Initial setup
- [X] Working towards first release
- [X] Community building
- [ ] Nearing graduation
### Date of last release:
XXXX-XX-XX
### When were the last committers or PPMC members elected?
Maya Debellis was elected as a committer on 2019-02-08
### Have your mentors been helpful and responsive?
Yes, mentors have been helpful and responsive.
### Signed-off-by:
- [ ](sdap) Jörn Kottmann
Comments:
- [ ](sdap) Trevor Grant
Comments:
- [ ](sdap) Lewis John McGibbney
Comments:
### IPMC/Shepherd notes:
Justin Mclean: Please use the format provided rather than an old format.
SDAP is an integrated data analytic framework for Big Science problems. SDAP has been incubating since 2017-10-22. Three most important issues to address in the move towards graduation: 1. Make official SDAP (Incubating) Release 2. Improve/create user guide documentation 3. Improve committer participation Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? No. How has the community developed since the last report? Committership has been accepted by a community member. SDAP was presented at a Data Science Workshop @ JPL How has the project developed since the last report? SDAP is still working towards its first release. Mentors have provided some suggestions that will be taken into consideration for the Java parts of the project, but the python release process also needs to be better understood. A new tool has been published that helps with reading the output produced from the Matchup algorithm. How would you assess the podling's maturity? Please feel free to add your own commentary. [X] Initial setup [X] Working towards first release [X] Community building [ ] Nearing graduation Date of last release: XXXX-XX-XX When were the last committers or PPMC members elected? Maya Debellis was elected as a committer on 2019-02-08 Have your mentors been helpful and responsive or are things falling through the cracks? In the latter case, please list any open issues that need to be addressed. Yes, mentors have been helpful and responsive. Signed-off-by: [ ](sdap) Jörn Rottmann Comments: [X](sdap) Trevor Grant Comments: IPMC/Shepherd notes:
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
Three most important issues to address in the move towards graduation:
1. Make official SDAP (Incubating) Release
2. Improve/create user guide documentation
3. Grow Community
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
A member of the SDAP podling stepped up to the release manager role which
is great. It shows the the podling is maturing.
Unfortunately, the release candidate which was produced was not in
compliance with the ASF release policy, thankfully Justin highlighted
this and we are in the process of rolling the RC back. Lewis (SDAP
champion) and the release manager are working to resolve this and get the
first incubating release candidate finished.
How has the community developed since the last report?
Two VOTE's have been made over on private@sdap regarding extending
PPMC and Committership to two community contributors. This is very
positive.
How has the project developed since the last report?
SDAP is on the verge of it's first release. With some persistence
the podling will have a release candidate for community VOTE soon.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[X] Initial setup
[X] Working towards first release
[X] Community building
[ ] Nearing graduation
[X] Other: One the release management process is documented we will be
pushing for graduation.
Date of last release:
XXXX-XX-XX
When were the last committers or PPMC members elected?
Elizabeth Lam was elected on 2019-02-06
Maya DeBellis was elected on 2019-02-06
Have your mentors been helpful and responsive or are things falling
through the cracks? In the latter case, please list any open issues
that need to be addressed.
Yes mentors have been responsive providing answers.
Signed-off-by:
[X](sdap) Jörn Kottmann
Comments:
[X](sdap) Suneel Marthi
Comments:
[X](sdap) Lewis John McGibbney
Comments:
IPMC/Shepherd notes:
Dave Fisher: Most of the PPMC listed in Whimsy is not subscribed to
private@.
The number of developers on the dev list is small. A little too
soon to be thinking about graduation if you ask me.
Justin Mclean: The IPMC usually likes to see 2 or 3 releases happen
before the project consider graduation.
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
Three most important issues to address in the move towards graduation:
1. Make formal SDAP (Incubating) Release
2. Improve/create user guide documentation
3. Grow Community
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
None
How has the community developed since the last report?
Added 2 new developers that are becoming familiar with the project and
will soon become committers.
SDAP has been heavily advertised at/in many Earth Science technology
forums.
SDAP was featured at the CUAHSI Biennial Colloquium with an aim towards
growing the community.
How has the project developed since the last report?
Development is still underway to push towards the first incubating
release of SDAP and each of its subcomponents.
How would you assess the podling's maturity?
Podling's still getting established.
[] Initial setup
[X] Working towards first release
[ ] Community building
[ ] Nearing graduation
[ ] Other:
Date of last release:
XXXX-XX-XX
When were the last committers or PPMC members elected?
Kevin M Gill was elected on 2018-03-05
Have your mentors been helpful and responsive or are things falling
through the cracks? In the latter case, please list any open issues
that need to be addressed.
As mentors, we have not been very actively involved - but there's nothing ye
t that required the mentor's attention (the project is in good hands under L
ewis :-)).
Mentors are available nevertheless to help with releases and guiding the pod
ling.
Signed-off-by:
[X](sdap) Jörn Kottmann
Comments:
[X](sdap) Suneel Marthi
Comments:
IPMC/Shepherd notes:
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
Three most important issues to address in the move towards graduation:
1. Grow community
2. Make formal SDAP (Incubating) releases
3. Evangelize SDAP as an integrated data analytic center for Big Science
problems
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
No.
How has the community developed since the last report?
For the last quarter, again SDAP has been heavily advertised
at/in tons of Earth Science technology forums.
Members of SDAP team demonstrated the software at the ESIP
Summer 2018 meeting in Tucson, AZ with the aim of growing
community.
How has the project developed since the last report?
The project is around a month or so away from making it's first
Incubating release. Development is ongoing and positive.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[X] Initial setup
[X] Working towards first release
[X] Community building
[ ] Nearing graduation
[ ] Other:
Date of last release:
XXXX-XX-XX
When were the last committers or PPMC members elected?
Kevin M Gill was elected on 2018-03-05
Signed-off-by:
[ ](sdap) Jörn Rottmann
Comments:
[ ](sdap) Raphael Bircher
Comments:
[ ](sdap) Suneel Marthi
Comments:
[X](sdap) Lewis John McGibbney
Comments:
IPMC/Shepherd notes:
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
Three most important issues to address in the move towards graduation:
1. Grow community
2. Make formal SDAP (Incubating) releases
3. Evangelize SDAP as an integrated data analytic center for Big Science problems
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
No.
How has the community developed since the last report?
SDAP has been heavily advertised at/in tons of Earth Science
technology forums. The SDAP community has plans to continue
in this thread with loads of activity booked over the summer
including for example, a workshop at the forthcoming ESIP
Summer 2018 meeting in Tucson, AZ
http://esipfed.org/meetings/upcoming-meetings/esip-summer-meeting-2018
How has the project developed since the last report?
Development is being sustained with initial discussion of an initial
Incubating release some time soon. The dev@ community has produced
documentation regarding release artifacts. The aim would be to have
a release candidate ready within the next month or so.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[X] Initial setup
[X] Working towards first release
[X] Community building
[ ] Nearing graduation
[ ] Other:
Date of last release:
XXXX-XX-XX
When were the last committers or PPMC members elected?
Kevin M Gill was elected on 2018-03-05
Signed-off-by:
[ ](sdap) Jörn Rottmann
Comments:
[ ](sdap) Raphael Bircher
Comments:
[ ](sdap) Suneel Marthi
Comments:
[X](sdap) Lewis John McGibbney
Comments:
IPMC/Shepherd notes:
johndament: The report was written by a mentor. It seems only this mentor is active on list.
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
Three most important issues to address in the move towards graduation:
1. Grow community
2. Make formal SDAP (Incubating) releases
3. Evangelize SDAP as an integrated data analytic center for Big Science problems
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
No
How has the community developed since the last report?
The community is demonstrating the ability to VOTE decisively.
We acquired quorum to move canonical source code development
to Gitbox.
How has the project developed since the last report?
All infrastructure and codebase(s) have successfully been
transitioned over to the ASF. Three new repositories have been
created and initial source code imports have taken place.
Development is working through JIRA nicely.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[X] Initial setup
[X] Working towards first release
[X] Community building
[ ] Nearing graduation
[ ] Other:
Date of last release:
XXXX-XX-XX
When were the last committers or PPMC members elected?
No new committers or PPMC have been added other than the
original SDAP committers.
Signed-off-by:
[ ](sdap) Jörn Rottmann
Comments:
[ ](sdap) Raphael Bircher
Comments:
[ ](sdap) Suneel Marthi
Comments:
[X](sdap) Lewis John McGibbney
Comments: SDAP is doing very well. Branding,
trademarks, source code compliance are the
primary issues blocking a release right now.
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
Three most important issues to address in the move towards graduation:
1. Grow community 2. Make formal SDAP (Incubating) releases 3. Evangelize
SDAP as an integrated data analytic center for Big Science problems
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
of?
No
How has the community developed since the last report?
Over the Christmas holidays not work has been done. SDAP is however being
presented at the ESIP Winter 2018 meeting taking place in Bethesda, Maryland,
U.S.A. this is a community building opportunity.
How has the project developed since the last report?
All infrastructure and codebase(s) have successfully been transitioned over to
the ASF. We are still working on branding, trademarks and source code
compliance.
How would you assess the podling's maturity? Please feel free to add your own
commentary.
[X] Initial setup
[X] Working towards first release
[X] Community building
[ ] Nearing graduation
[ ] Other:
Date of last release:
XXXX-XX-XX
When were the last committers or PPMC members elected?
No new committers or PPMC have been added other than the original SDAP
committers.
Signed-off-by:
[ ](sdap) Jörn Rottmann
Comments:
[ ](sdap) Raphael Bircher
Comments:
[ ](sdap) Suneel Marthi
Comments:
[X](sdap) Lewis John McGibbney
Comments: SDAP is doing pretty well.
Branding, trademarks and source code compliance are the primary issues
right now. Once these are addressed SDAP can focus on the first Incubating
release.
SDAP is an integrated data analytic center for Big Science problems.
SDAP has been incubating since 2017-10-22.
Three most important issues to address in the move towards graduation:
1. Grow community
2. Make formal SDAP (Incubating) releases
3. Evangelize SDAP as an integrated data analytic center for Big Science problems
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
No
How has the community developed since the last report?
We have not formally reported before. We hope the community
to grow once we make our first Incubating release.
How has the project developed since the last report?
All infrastructure and codebase(s) have successfully been
transitioned over to the ASF. We are working to comply
with branding, trademarks and source code. This work is ongoing.
How would you assess the podling's maturity?
Please feel free to add your own commentary.
[X] Initial setup
[X] Working towards first release
[ ] Community building
[ ] Nearing graduation
[ ] Other:
Date of last release:
XXXX-XX-XX
When were the last committers or PPMC members elected?
No new committers or PPMC have been added other than the
original SDAP committers.
Signed-off-by:
[ ](sdap) Jörn Rottmann
Comments:
[ ](sdap) Raphael Bircher
Comments:
[ ](sdap) Suneel Marthi
Comments:
[X](sdap) Lewis John McGibbney
Comments: SDAP is doing pretty well. Branding,
trademarks and source code compliance are the
primary issues right now. Once these are addressed
SDAP can focus on the first Incubating release.