# 2020-10-21 conda-forge core meeting [Zoom link](https://flatiron.zoom.us/j/93242638216?pwd=bjRCWmVJRW1oTGJhN09VUmxtTTJOUT09) [What time is the meeting in my time zone](https://arewemeetingyet.com/UTC/2020-08-26/17:00/w/Conda-forge%20dev%20meeting#eyJ1cmwiOiJodHRwczovL2hhY2ttZC5pby9wUk15dFVKV1FmU3NJM2xvMGlqQzJRP2VkaXQifQ==) [last weeks meeting](https://hackmd.io/1mI2fg1wS2O9Vho_if6Z6Q) ## Attendees * Eric D * Matt B * Filipe * Cheng Lee * Isuru * Marius * Crystal * Marcelo * Paul I. * Marcel B. * Connor M. * Bill Batey * Stephanie * John K. * Lori B. * Wolf * CJ ## Agenda ### Standing items * [x] intros for new folks on the call * Bill - new to Anaconda. On the distribution team with Cheng and Connor. * [x] (CJ) budget * current approvals? * Whenever updated numbers land, please screenshare and show the budget. * Link is in Keybase (numfocus_spreadsheets.txt) * [x] (MRB) open votes * Chris Burr for Core * Won't use polys again because something totally went sideways. Maybe we'll try again? #### From previous meeting(s) * [x] (MRB) github releases and conda-forge-sparta * code and data in the regro github org right now * if you want access, LMK * I am also happy to move the repos to the conda-forge org but they may slow rerendering since they use a fair bit of github actions * how it works * all packages that are validated and copied to anaconda.org/conda-forge are also sent to a releases repo via a workflow dispatch event in github actions * the github actions CI job (regro/releases) * downloads the package * computes its repodata information * uploads the package and its repodata as a github release * pushes the repodata shard to another repo with all of the shards * another job on a continuous loop (regro/repodata aka the repoworker) * downloads the shards * figures out which ones are new * applies repodata patching to the shards (will also repatch everything if patches have changed) * computes the repodata and channel data * computes an index of links that point to where one can download the package * uploads all of this data as a github release * kicks the heroku server to download the new links * the herkou server (https://conda-forge-sparta.herokuapp.com/conda-forge-sparta/) * gets a request for a package or repodata file at the standard URL for an anaconda server * looks up where that package is in the links index * issues a 300 to redirect the request to the actual location * a final set of cron jobs runs once an hour to sync our repodata shards with anaconda.org (regro/repodata-shards) * this job catches any missed releases due to bad CI jobs or manual uploads * will update the shards repo and upload to github * ensures eventual consistency on ~1-2 hour time scales if things break * overall time delay from the initial workflow dispatch to a package being available is 5-10 minutes * can go longer if repodata patches are being redone for everything * how could this not depend on anaconda **if we needed it** * replace cf-staging and anaconda.org w/ quetz * limit traffic to those instances to keep the costs down * send the vast majority of download requests to our (possibly beefed up) heroku server and thus github * I really don't want to maintain this for production, but we could if we had to. * For now, we have a system to help index and archive our data in another spot. * [ ] (MRB / ED / SC) Roadmap / Funding? Discussion in gitter: * (SC) What would CF do as a project if it received funding? Maybe having some kind of whitepapers on the conda-forge website would help. Would conda-forge do something very different if some corporation gave 10k, 100k, or 1M to the project via NumFOCUS? * 10k: does it buy enough hosting to do something interesting? * 100k - what does it bring. * A much larger amount: conda-forge may even be able to have funded people directly working on the project. * We can almost certainly find a lot useful stuff that would not hurt getting funding for without necessarily having to get consensus on a roadmap. (something i think would not be controversial, like oh we could use some funding to do some nice web development for better visualization of migrations, or stuff) there are probably a lor of things of that nature. * If we make a list, Santa may just show up... * see the bloomberg note above... * wishlist and ideas: https://hackmd.io/0zGSUS71SbOdBsdLtDmGjg * (ED) Can we add initials for whomever submitted the idea? would be useful to track that if there are follow-on questions * (MRB) Start chatting on this for 15 minutes in the next core meeting * [x] (MvN) tactical things to do about cython? * CPython breaks its apis reguarly enough to force a recythoning of much of the python ecosystem. * Should we patch our cython so that when the `CONDA_BUILD` env var is set, we always recythonize and ignore mtime for files? * [x] (MvN) noarch: python v2 * As we increasingly drop compatibility with py27/35/36 should we automatically add python constraints to all `noarch: python` artifacts using repodata pacthing? * The idea would be to pick an epoch date (say 2020-10-01) and have the artifacts built after that date get a repodata patch adding a python>=3.6 constraint iff it only had a python constraint. * Additionally we may as well upate the noarch adding bot action that adds noarch to strip all lines containing py27/34/35 selectors from the recipe. * This should allow us to move lots of feedstocks that currently depend on some things only for py27 compat to noarch: python cutting down on the growth rate of artifacts. * to do * ( ) [ ] make a hint * ( ) [ ] make an announcement * ( ) [ ] make the hint a lint * [x] (WV) multisheller github.com/wolfv/multisheller ### Your __new__() agenda items * [x] (FF) Outreachy, round 2 * [X] (FF) "Feature parity" between feedstock and staged-recipes. For example: CFEP-05 does not contemplate a dev release on staged recipes when there is no stable release to start a feedstock. https://github.com/conda-forge/cfep/blob/master/cfep-05.md * No need for a conda-forge.yml, one can add a conda config file * The broader issue still is not solved, we cannot do cos7 * staged recipes needs some work! ### Pushed to next meeting ### Active votes ### Subteam updates #### Bot #### ARM #### POWER #### CUDA #### Docs #### staged-recipes #### website #### security+systems ### CI infrastructure #### Compiler upgrade ### CFEP updates #### Open PRs * [cfep-04](https://github.com/conda-forge/conda-forge-enhancement-proposals/pull/7) X11 and CDT policy * INACTIVE - Merge in with some inactive-esque status? * Needs new champion. Thanks for your work on this pkgw! Has unaddressed comments from pkgw as from Jan 10, 2020 * [cfep-06](https://github.com/conda-forge/conda-forge-enhancement-proposals/pull/9) Staged-recipes review lifecycle * INACTIVE - Merge in with some inactive-esque status? * Lingering comment from @saraedum. @jakirkham, can you reply? Has unadressed comment from @saraedum from Jan 8, 2020 * (MRB) The stalebot has solved the worst of the issues here. I think we could defer this one permanently. * [cfep-10](https://github.com/conda-forge/conda-forge-enhancement-proposals/pull/15) Feedstock statuses, unmaintained * INACTIVE - Merge in with some inactive-esque status? * Needs another review. Has unaddressed updates from pkgw as of Jan 11, 2020 * [cfep-12](https://github.com/conda-forge/cfep/pull/23) Removing packages that violate the terms of the source package * Stalled since May 26, 2020 * Active debate about moving to "broken" vs deleting from conda-forge channel * Active vote, ends on 2020-03-11 * What were the results of the vote? * Did we hear back from NumFOCUS? * [cfep-17](https://github.com/conda-forge/cfep/pull/32) Handling pin backports and dependency rebuilds * Stalled debate about implementation details between Isuru, CJ and Matt * **UPDATE 2020-07-22**: We in principle have agreement to render the extra pinnings needed directly in the feedstock on a temporary basis (i.e., until the migration has ended). * [cfep-20](https://github.com/conda-forge/cfep/pull/39) ## Discussion ## Check in on previous action items Copy previous action items from last meeting agenda. ### This meeting 2020-10-21 * [ ] Python 2.7 migration * ( ) [ ] make a hint * ( ) [ ] make an announcement * ( ) [ ] make the hint a lint ### Last meeting 2020-10-07 * [ ] Make sure to add the NVBug info to the cudatoolkit package that conda-forge makes (if we make one) ### 2 meetings ago 2020-09-30 ### Move to Issue Tracker 2020-09-23 * [x] (MRB) * do libgfortran name change * add target platform to hashes * do gfortran migration with bot * bump pinnings 2020-09-16 * [x] Get a call set up with Jon Mease about the kaleido staged recipes PR * Emailed on 2020-09-16 * [x] (FF) Open up a PR on the python feedstock for python 3.9 and see what fails 2020-09-09 * [ ] (ED) Update governance docs with similar voting model as what got put into conda-tools (+3 with no -1 is a pass) * [ ] (SC) Write jinja template to turn institutional partners yaml into a website https://github.com/conda-forge/conda-forge.github.io/blob/2a2d3caaf7d74eb370ac40c679ba337a73d15c8a/src/inst_partners.yaml * [ ] (SC) Document what needs to be done to create an OVH account and get access 2020-08-26 **Docker hub** * [ ] (JK) Check in on Azure build workers to see if they have the docker hub limitation. * [ ] (JK) work with dockerhub to see if we can get OSS status * [ ] Check in again at some point. We haven't heard back as of 2020-09-23 * [x] (MRB) start pushing images to quay (https://github.com/conda-forge/docker-images/pull/152) **OVH** * [ ] (???) build webpage to credit them (and others) * [ ] If we're adding a logo, will want to make sure that we have permission to use it. * [ ] Shout-out on twitter at some point. "Thanks forOVHCloud for providing a VM", etc. (maybe after we ship qt on windows with it?) * [ ] Figure out how to communicate breaking changes to users. Likely should open up an issue immediately for futher discussion. Ping @kkraus, plus capture notes from further up in these meeting notes * [ ] John K. will update the cuda toolkit feedstock on the git repo to note the NVBug link to the internal NVIDIA issue tracker * [ ] Jonathan will update docs to note that some non-exhaustive list of packages (like cuda-toolkit, MKL, etc.) * [ ] Jonathan will review this [PR](https://github.com/AnacondaRecipes/cudatoolkit-feedstock/pull/7) * [ ] (Kale) schedule conda working group * [ ] cfep-10 next steps: CJ to call a vote for feedback * [ ] cfep-06 next steps: Ask staged recipes team to champion this CFEP and move it forward * [ ] jakirkham & CJ-wright to sync on adding CUDA to the migration bot * [ ] (Eric) Scheduling Anaconda <-> conda-forge sync on anaconda.org requirements gathering * Will try and get this scheduled in the next month. * [ ] (Anthony) Reach out to NumFocus to figure out legal ramifications of not including licenses in files. * [ ] (Eric) check internally for funding levels for hotels & flying folks from the community in? * [ ] (Eric) Figure out finances of conda-forge to support themselves? * [ ] (jjhelmus) Open up CFEP for which python's we're going to support * [ ] (jakirkham) write a blog post on CUDA stuff we discussed today * [ ] (jakirkham) update docs on how to add CUDA support to feedstocks * [ ] (jakirkham) will open an issue on conda-smithy to investigate Drone issues. (ping the aarch team) * https://github.com/conda-forge/conda-forge.github.io/issues/954 * [ ] (ED) Who we are page? Some combination of a FAQ and a who is everyone. FAQ things like: * who's the POC for CF <> Anaconda, CF <> NumFocus, CF <> Azure * who's the POC for the various subteams? * Informal information: roles, day jobs, bios, the whole nine yards, why you're here, etc. * Public or internal? I don't really care either way. Anyone feel strongly one way or the other? * opt-in to public bios * software carpentry has a large number of instructors and has https://carpentries.org/instructors * some concern about "yet another place to keep stuff up to date" * [ ] (ED) document strategies for reproducible environments using conda-forge * [ ] (UK) Static libraries stuff * [ ] Add linting hints to builds to find them * [x] Recommend how to package them -> CFEP-18 * [x] We should write docs saying we don't provide support and this is a bad idea. -> CFEP-18