# 2016-08-25: General discussion Time: 14:00 UTC Hangout link: [](https://hangouts.google.com/call/v5olhwzpfzgzpoq5i3wthjpqpie)[https://hangouts.google.com/call/v5olhwzpfzgzpoq5i3wthjpqpie](https://hangouts.google.com/call/v5olhwzpfzgzpoq5i3wthjpqpie) **Attendees** Jonathan Helmus, Filipe, [Michael Sarahan](https://conda-forge.hackpad.com/ep/profile/yHQTJXZ4gyS), [John Kirkham](https://conda-forge.hackpad.com/ep/profile/wv6uvIZX6h0), Jake VanderPlas, Eric Dill, Ray Donnelly , [Phil Elson](https://conda-forge.hackpad.com/ep/profile/AviM60TiesB) **Standing items** * How many repos? 1035 * How many contributors? 212 (with a few bots) * New core devs? **Notes** * Invite Peter M. Landwehr (pmlandwehr) to be involved with review of staged-recipes. Should we give these type of people a title, Filipe will reach out to. * Governing Open Source Projects at Scale: Lessons from Wikipedia's Growing Pains | Staurt Geiger [](https://www.youtube.com/watch?v=ZSQJYEVcMWM&index=89&list=PLYx7XA2nY5Gf37zYZMw6OqGFRPjB1jCy6)[https://www.youtube.com/watch?v=ZSQJYEVcMWM&index=89&list=PLYx7XA2nY5Gf37zYZMw6OqGFRPjB1jCy6](https://www.youtube.com/watch?v=ZSQJYEVcMWM&index=89&list=PLYx7XA2nY5Gf37zYZMw6OqGFRPjB1jCy6) * Enhancement proposal document, Jonathan has notes will write these up later today. * Governance document - help is welcomed. Also "whos who" or "about" page. [](https://conda-forge.github.io/#about)[https://conda-forge.github.io/](https://conda-forge.github.io/#about)[#about](https://conda-forge.hackpad.com/ep/search/?q=%23about&via=2YkV96cvxPG) * This page could be expanded, should mentioned these meeting. * Removing items from agenda * Prioritize items on agenda which we should/must talk about. * Cross link items to GitHub issues/discussions * Status page: [](https://conda-forge.github.io/status/)[https://conda-forge.github.io/status/](https://conda-forge.github.io/status/) * Linked to "status" repo: [](https://github.com/conda-forge/status)[https://github.com/conda-forge/status](https://github.com/conda-forge/status) * conda-forge code of conduct - Filipe still workin on * Many groups working on new build systems: Filipe, Phil, Continuum * Continuum's plan would allow others to add build workers, perhaps conda-forge could use these in addition to the CI services, especially for long builds * Organize new meeting to discuss this topic * Open sourcing Anaconda Build, should we push to get this released? * Would be helpful to have our own build system rather than being dependent on CI systems. * Travis CI can increase time if we reduce concurrency * Can we switch between longer time and concurrency? How much work is this? * Probably not going to take offer at the moment * Better to find trusted hardware somewhere * Vagrant for OS X builds, can we look into this * Security * If user changes name and someone takes old name can be a security issue: [](https://groups.google.com/forum/#)[https://groups.google.com/forum/#](https://groups.google.com/forum/#%21topic/rustlang-security-announcements/BK_3gbXhSn4)[!topic/rustlang-security-announcements/BK_3gbXhSn4](https://groups.google.com/forum/#%21topic/rustlang-security-announcements/BK_3gbXhSn4) * Can be solved by using unique user ID rather than GitHub username * Want tokens for Anaconda.org which allow writing to a single package (Phil will push Continuum on this) rather than globally. * Metadata unification * Should conda-forge include additional metadata which would make it easier for Continuum to re-use recipes * Should this be required or optional? * Required would likely reduce number of contributors * Will require time/work to add these to all current packages * Add to linter and conda skeleton * Make linter have "warnings" not hard fails * Many of these seem redundant, can we re-use existing metadata? * License file should likely be required * Legal vs. suggested **Agenda** * Marking agenda items as done. * Share status page. :) Also figure out how to direct notifications effectively. * Enhancement proposal document update. * conda-forge code of conduct doc: [](https://docs.google.com/document/d/10dxX0Zse0Rx1HqsxC73Wfsghmy5m8PP8cHuBIOhWKpc/edit)[https://docs.google.com/document/d/10dxX0Zse0Rx1HqsxC73Wfsghmy5m8PP8cHuBIOhWKpc/edit](https://docs.google.com/document/d/10dxX0Zse0Rx1HqsxC73Wfsghmy5m8PP8cHuBIOhWKpc/edit) * Mention Travis-CI offer for more CI time. * We could look at increasing your build time to 180 mins, but we may need to decrease your default concurrency from 5 jobs to 3 as you will be using multiple VMs for a long period at a time. * Mention/Discuss Travis Oliphant's [comment](https://github.com/Anaconda-Platform/support/issues/55#issuecomment-242206322) regarding open sourcing Anaconda Build CI. * Security * Feedstocks philosophy: Explicit vs implicit / reproducible vs redundant * Metadata unification with Continuum - are we OK with adding some fields to about section to match Anaconda standard? * Including license file * Many recipes don't include the license file. * Almost every license has some terms about making the license available. * Should we just start requiring this field. * Note some developers are not including the license file either. * In some cases it has been a struggle to get them to. * CUDA/cuDNN update * Improving infrastructure * Better workflows with staged-recipes * Fast finish AppVeyor on merge ( [conda forge/staged recipes#1142](https://github.com/conda-forge/staged-recipes/pull/1142) ) * Drop Travis CI matrix ( [conda forge/staged recipes#1234](https://github.com/conda-forge/staged-recipes/pull/1234) ) * Use CircleCI for feedstock generation ( [conda forge/staged recipes#916](https://github.com/conda-forge/staged-recipes/issues/916) ) * Keeping recipes out of PRs ( [conda forge/staged recipes#942](https://github.com/conda-forge/staged-recipes/issues/942) ) * Bank work in partial conversion ( [conda forge/staged recipes#915](https://github.com/conda-forge/staged-recipes/issues/915) ) * Notifications (how do we stay on top of them) * MSYS2 * Available on defaults - was in conda 4.1.7, but that was pulled. Coming in 4.1.8. * Discussing Ray Donnelly's work on MSYS2 packages and how we want to use and integrate these into conda-forge. * Some use cases to consider OpenBLAS, FFTW, build tools, others? * Binary data * Do we include it in recipes? * What kinds do we allow if any (e.g. icons)? * How do we verify the licensing? * How do we verify that they are safe? * Dev releases: Where do they happen? * Do we do them at conda-forge? * Maybe add a label. * Do we let others do them with a feedstock on their own repo? * How do we enforce whatever we decide? * Channel mirroring * Can this point be a little bit explained? I thought about this as well and would like to contribute to this point. * Eric Dill has put together a script for copying a package from one channel to another here: [conda forge/conda forge.github.io#134](https://github.com/conda-forge/conda-forge.github.io/pull/134) * I have a really, really crude script that copies all of the packages in one channel to another that I just put at: [](https://gist.github.com/mwcraig/8473cf840f6d29236d6d8af699404555)[https://gist.github.com/mwcraig/8473cf840f6d29236d6d8af699404555](https://gist.github.com/mwcraig/8473cf840f6d29236d6d8af699404555) * conda-build-all can copy from one channel to another: `conda build-all --inspect-channels conda-forge --upload-channels astropy some_packge_recipe` will copy the `some_package` from the channel conda-forge to astropy if it can, or build it if it doesn't exist on conda-forge. Discussion about what the desired behavior should be has started at: [SciTools/conda build all#46](https://github.com/SciTools/conda-build-all/issues/46) * Feedstock history * Is it sacred? * Do we rebase/force push? * If so, under what conditions? * How do we avoid multiple people doing this simultaneously? * I don't think you can. * IMHO, if it's just one author in staged recipes, sure. If feedstock, no force push - only to PRs to feedstock. If people don't mind merge PRs, it sure is a lot simpler to not rebase. I have messed up rebasing a few times recently... =( * Continuum metadata request: can we add these to linter? * example metadata: [](https://github.com/ContinuumIO/anaconda-recipes/blob/master/anaconda-build/meta.yaml#L36-L44)[https://github.com/ContinuumIO/anaconda-recipes/blob/master/anaconda-build/meta.yaml#L36-L44](https://github.com/ContinuumIO/anaconda-recipes/blob/master/anaconda-build/meta.yaml#L36-L44) * Also, distinguish summary (limit of 77 or 80 chars) from description (unlimited) * Anaconda verify: would be nice to meet in the middle, rather than diverge. conda-build may integrate anaconda-verify, would be nice if conda-forge added metadata here. * Drop numpy 1.10 and reduce our build matrix. (Numba now works with numpy 1.11.) * Signing packages * Should be easy to do. ( [](http://conda.pydata.org/docs/signed-packages.html)[http://conda.pydata.org/docs/signed-packages.html](http://conda.pydata.org/docs/signed-packages.html) ) * There has been some interest previously. * HTTPError: 503 Server Error: Service Unavailable: Back-end server is at capacity for url... * Seems we are regularly running into this issue under normal usage conditions. * Had discussed previously caching packages on AppVeyor and trying to reuse those to start. * Maybe we need to consider caching on all CIs. * Building our own Miniconda-like self-extracting scripts with packages via [`constructor`](https://github.com/conda/constructor). * There have been improvements on Continuum's side that should help this. In short, repodata (the package index for a given channel) was being generated for each anaconda.org query. This was unnecessarily high cost, and some caching schemes have been implemented. * Handling removal of unpinned/improperly pinned packages. * Has been done manually thus far. * This doesn't scale well though. * Should we (semi) automate removal? * Should we hot-fix broken packages? ( [conda forge/conda forge.github.io#170](https://github.com/conda-forge/conda-forge.github.io/pull/170) ) * Should we label them as broken * Not currently buildable packages * In particular open source code that is out of scope for CIs. * Examples include Qt4, Qt5, possibly PyQt4, possibly PyQt5, gcc, VTK, etc. * How do we indicate they are built manually? * Are we ok with uploading non-built binaries? * When do we determine something is ok to be built manually? * What procedures should people follow for building manually? * Use a standard build docker image, VM, or vagrant file * Sign package? * Implement reproducible builds where feasible (linux) * [](https://reproducible-builds.org/)[https://reproducible-builds.org/](https://reproducible-builds.org/) * What changes do we need to make in conda-smithy elsewhere? * What other build infrastructure could we utilize? * Would be nice to provide some volunteer builder abstraction, so that we could have an elastic worker farm that would be somewhat resilient. * Standardizing build images is probably (relatively) easy - how to orchestrate, though? * [Staged Releases](https://conda-forge.hackpad.com/DZNKZdgiMbF) * Windows BLAS Solutions * Still don't have a BLAS for Windows yet need something. * Don't build a BLAS * NumPy has a small subset of BLAS functionality. * Not sure what to do with SciPy (unable to find Windows wheels for them either). * Build OpenBLAS with C support only. * Will be pretty slow. * Should work on all Pythons. * Build OpenBLAS with MinGW compilers. * Works with Python 2.7 and 3.4. * Won't work with Python 3.5? * Reuse something like R's BLAS. * Is there a package for something like this? * Will it have the same issues with Python 3.5? * ATLAS?