# 2016-08-12: General discussion

Time: 14:00 UTC

Hangout link: [](https://hangouts.google.com/call/v5olhwzpfzgzpoq5i3wthjpqpie)[https://hangouts.google.com/call/v5olhwzpfzgzpoq5i3wthjpqpie](https://hangouts.google.com/call/v5olhwzpfzgzpoq5i3wthjpqpie)

2016-07-22: General discussion

**Attendees**

Eric Dill

[Phil Elson](https://conda-forge.hackpad.com/ep/profile/AviM60TiesB)

[Michael Sarahan](https://conda-forge.hackpad.com/ep/profile/yHQTJXZ4gyS)

[John Kirkham](https://conda-forge.hackpad.com/ep/profile/wv6uvIZX6h0)

**Standing items**

*   How many repos?
*   How many contributors?
*   New core devs?

**Agenda**

*   Prerelease versions

        *   Python prerelease done at [conda forge/python feedstock#45](https://github.com/conda-forge/python-feedstock/pull/45) - is this an example to follow?
    *   Do we have documentation on how to do this?
    *   Waiting PR: [conda forge/scikit image feedstock#2](https://github.com/conda-forge/scikit-image-feedstock/pull/2) 
    *   Conda itself: [conda/conda#3262](https://github.com/conda/conda/pull/3262)#issuecomment-239410077
    *   proposal for naming pre-release channels:

                *   embed the package name in the anaconda label so that you can specify exactly which pre-release things to install
        *   The conda install command to specify from a label other than `main` is:** **

                        *   **`conda** install -c conda-forge/label/rc <package>`

                *   So  if you embed, for example, "matplotlib-" in the label name, then you  can specifically install *just* the matplotlib pre-release with:

                        *   `conda install -c conda-forge/label/matplotlib-rc matplotlib`

*   Status page

        *   Have dependencies.
    *   Some code for the webservice

*   Feedstocks philosophy:  Explicit vs implicit / reproducible vs redundant
*   OSX - getting back to a usable, coherent, stack

        *   libc++ (clang) vs libstdc++ (gcc/g++)
    *   Minimum OSX required for clang (10.8, I think?)

    *   Actually clang is usable beginning in 10.7. So, this would be viable given your compatibility constraints.
    *   Also, all the refs I have seen suggest that this will still have C++11 support.

    *   Compatibility  with defaults (built on 10.7, uses gcc) - where will people break?  I  think only if mixing packages - how do we assure that we have all the  ones we need?

*   Improving infrastructure

        *   Finish out GitHub API issues ( [conda forge/conda forge.github.io#172](https://github.com/conda-forge/conda-forge.github.io/issues/172) )
    *   Better workflows with staged-recipes

                *   Fast finish AppVeyor on merge ( [conda forge/staged recipes#1142](https://github.com/conda-forge/staged-recipes/pull/1142) )
        *   Drop Travis CI matrix ( [conda forge/staged recipes#1234](https://github.com/conda-forge/staged-recipes/pull/1234) )
        *   Use CircleCI for feedstock generation ( [conda forge/staged recipes#916](https://github.com/conda-forge/staged-recipes/issues/916) )
        *   Keeping recipes out of PRs ( [conda forge/staged recipes#942](https://github.com/conda-forge/staged-recipes/issues/942) )
        *   Bank work in partial conversion ( [conda forge/staged recipes#915](https://github.com/conda-forge/staged-recipes/issues/915) )

*   Low level packaging
*   Basic community practices when PR-ing to staged-recipes.

*   No need to re-discuss this. I am still writing the docs and, if ready, I will send the link tomorrow (or after SciPy ;-)

*   NetCDF (<s>also curl/ca-certificates and Perl packages</s>) - Done?

        *   curl and ca-certificates are done and available. 
    *   Perl is no longer relevant as part of this process

*   Notifications (how do we stay on top of them)
*   Standardizing installs

        *   Mention [`toolchain`](https://github.com/conda-forge/toolchain-feedstock) .

                *   Discuss rollout to feedstocks.

        *   Get feedback on [`python-toolchain`](https://github.com/conda-forge/staged-recipes/pull/642) 

*   MSYS2

        *   Available on defaults - was in conda 4.1.7, but that was pulled.  Coming in 4.1.8.
    *   Discussing Ray Donnelly's work on MSYS2 packages and how we want to use and integrate these into conda-forge.
    *   Some use cases to consider OpenBLAS, FFTW, build tools, others?

*   Binary data

        *   Do we include it in recipes?
    *   What kinds do we allow if any (e.g. icons)?
    *   How do we verify the licensing?
    *   How do we verify that they are safe?

*   Dev releases: Where do they happen?

        *   Do we do them at conda-forge?

                *   Maybe add a label.

        *   Do we let others do them with a feedstock on their own repo?
    *   How do we enforce whatever we decide?

*   Conda-forge installer

        *   We have Python 3.5 and 3.4 now. Would be nice to complete Python 2.7.
    *   Have all dependencies. Though `conda-build` has some kinks to be worked out.
    *   Many open questions about the installer including its name
    *   Where do we host the installers? Git tags?
    *   This can work right now if you pin to conda-build 1.21.7
    *   But,  is realistically blocked due to a setuptools entrypoints issue on  windows that is fixed with conda 4.2, but 4.2 is not released yet.   conda 4.2 is slated to be released by the end of August

*   Channel mirroring

        *   Can this point be a little bit explained? I thought about this as well and would like to contribute to this point.

                *   Eric Dill has put together a script for copying a package from one channel to another here: [conda forge/conda forge.github.io#134](https://github.com/conda-forge/conda-forge.github.io/pull/134)
        *   I have a really, really crude script that copies all of the packages in one channel to another that I just put at: [](https://gist.github.com/mwcraig/8473cf840f6d29236d6d8af699404555)[https://gist.github.com/mwcraig/8473cf840f6d29236d6d8af699404555](https://gist.github.com/mwcraig/8473cf840f6d29236d6d8af699404555)
        *   conda-build-all can copy from one channel to another: `conda build-all --inspect-channels conda-forge --upload-channels astropy some_packge_recipe` will copy the `some_package`  from the channel conda-forge to  astropy if it can, or build it if it  doesn't exist on conda-forge. Discussion  about what the desired  behavior should be has started at: [SciTools/conda build all#46](https://github.com/SciTools/conda-build-all/issues/46)

*   Feedstock history

        *   Is it sacred?
    *   Do we rebase/force push?

                *   If so, under what conditions?
        *   How do we avoid multiple people doing this simultaneously?

                        *   I don't think you can.

                *   IMHO,  if it's just one author in staged recipes, sure.  If feedstock, no  force push - only to PRs to feedstock.  If people don't mind merge PRs,  it sure is a lot simpler to not rebase.  I have messed up rebasing a few  times recently... =(

*   Docker hosting solution

        *   Docker Hub builds were broken for a week and a half.
    *   Have switched to quay.io currently.
    *   Mirroring quay.io image on Docker Hub.
    *   Thoughts about quay.io? Thoughts about hosting in general?

*   Continuum metadata request: can we add these to linter?

        *   example metadata: [](https://github.com/ContinuumIO/anaconda-recipes/blob/master/anaconda-build/meta.yaml#L36-L44)[https://github.com/ContinuumIO/anaconda-recipes/blob/master/anaconda-build/meta.yaml#L36-L44](https://github.com/ContinuumIO/anaconda-recipes/blob/master/anaconda-build/meta.yaml#L36-L44)
    *   Also, distinguish summary (limit of 77 or 80 chars) from description (unlimited)
    *   Anaconda  verify: would be nice to meet in the middle, rather than diverge.   conda-build may integrate anaconda-verify, would be nice if conda-forge  added metadata here.

*   Google    hangouts has a max capacity of 10. Is it worth considering other    methods of communication so everyone who wants to participate can?

*   Maybe this ( [](http://www.freeconferencecalling.com/)[http://www.freeconferencecalling.com/](http://www.freeconferencecalling.com/) ) is an option.
*   Bluejeans 
*   Continuum has webex.  Past experience is that some Linux platforms had trouble connecting

*   Drop numpy 1.10 and reduce our build matrix. (Numba now works with numpy 1.11.)

*   This comment from the PR for graphviz is the best summary I've seen: [conda forge/staged recipes#568](https://github.com/conda-forge/staged-recipes/pull/568)#issuecomment-225315370
*   Thanks for pointing this out. The described solution looks reasonable and is preferable to prefixing package names. Great!

*   What is the benefit?
*   Will   we distinguish between libs and standalone tools, similar to Debian? I   would strongly suggest to do this, because it is (1) established and  (2)  more accessible for the user (if he wants to use a library, he  knows  the language. If he wants to use a standalone, he doesn't care).[ ( ](https://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-package_names)[](https://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-package_names))[https://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-package_names)](https://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-package_names)[ ](https://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-package_names)
*   Will   there be an orchestrated move? If not, how do we deal with   inconsistencies and potential conflicts (installing both python-h5py  and  h5py).

        *   we will probably go with meta-packages for conflicting packages

*   Signing packages

*   Should be easy to do. ( [](http://conda.pydata.org/docs/signed-packages.html)[http://conda.pydata.org/docs/signed-packages.html](http://conda.pydata.org/docs/signed-packages.html) )
*   There has been some interest previously.

*   HTTPError: 503 Server Error: Service Unavailable: Back-end server is at capacity for url...

*   Seems we are regularly running into this issue under normal usage conditions.
*   Had discussed previously caching packages on AppVeyor and trying to reuse those to start.
*   Maybe we need to consider caching on all CIs.
*   Building our own Miniconda-like self-extracting scripts with packages via [`constructor`](https://github.com/conda/constructor).
*   There  have been improvements on Continuum's side that should help this.  In  short, repodata (the package index for a given channel) was being  generated for each anaconda.org query.  This was unnecessarily high  cost, and some caching schemes have been implemented.

*   Handling removal of unpinned/improperly pinned packages.

*   Has been done manually thus far.
*   This doesn't scale well though.
*   Should we (semi) automate removal?
*   Should we hot-fix broken packages? ( [conda forge/conda forge.github.io#170](https://github.com/conda-forge/conda-forge.github.io/pull/170) )

*   Not currently buildable packages

*   In particular open source code that is out of scope for CIs.
*   Examples include Qt4, Qt5, possibly PyQt4, possibly PyQt5, gcc, VTK, etc. 

*   How do we indicate they are built manually?
*   Are we ok with uploading non-built binaries?
*   When do we determine something is ok to be built manually?
*   What procedures should people follow for building manually?

        *   Use a standard build docker image, VM, or vagrant file
    *   Sign package?
    *   Implement reproducible builds where feasible (linux)

                *   [](https://reproducible-builds.org/)[https://reproducible-builds.org/](https://reproducible-builds.org/)

*   What changes do we need to make in conda-smithy elsewhere?
*   What other build infrastructure could we utilize?

        *   Would  be nice to provide some volunteer builder abstraction, so that we could  have an elastic worker farm that would be somewhat resilient.
    *   Standardizing build images is probably (relatively) easy - how to orchestrate, though?

*   Conda RPMs: [](https://github.com/pelson/conda-rpms)[https://github.com/pelson/conda-rpms](https://github.com/pelson/conda-rpms)