# 2023-03-03 conda-forge bot meeting ###### tags: `bot-meeting-notes` Attendees: - Jaime - Eric - Matt - Vini Notes: - depfinder: problem is because we're fnmatching on a bunch of stuff that includes the full path - https://github.com/conda/grayskull/issues/441 this one? - - DEBUG depfinder:reports.py:168 ******found ignore match for name: /usr/share/miniconda3/envs/test/conda-bld/praw_1677855233037/work/praw/reddit.py - problem 1: we have an env named "test" so the fnmatch for "*/test/*" - problem 2: bot background: - biggest thing that's happened in bot land in the last 2 years are that the bot will now update recipe with dependencies from grayskull. or depfinder actually. it does both. grayskull is broken cause grayskull has a bug. marcelo hasn't had a chance to fix it. things related to carets in the deps which conda doesn't understand. and thene xtra requirements syntax for pip-land. so that's all broken. depfinder broke which we think we just figured out. - https://github.com/conda/grayskull/issues/441 this one? - second big change - slow process - started before bot meetings stopped - completed it in th past week . refactored a lot of the internals of the bots datamodel sothat everythign can run in parallel. much more responsive now cause its on cron jobs. before that was done - vini you were working on the version stuff - if you made any changes to the json blob - one json blob per feedstock - this comes from the github API - what happened was that if you wanted to run diff parts of the bot in parallel they'd all edit the file simultaneously and the whole thing woudl fall apart. that single PR-json file and extracted the different elements into different files. jobs that collect versions, jobs that update PRs, jobs that update fedstocks and remake metadata and some other jobs, those ar ethe main three, all of those jobs can run in parallel now. what happens now is that when the bot starts up it grabs the current state. does use a consistent data model at a given time. if we were to switch to a DB and we allowed updates on the fly, the bot would potentially see a changing data model as it runs. now you just DL a git repo so it all works. that's cut out a huge amount of latency, even though CF has grown by ~2x. now the bot runs on a cycle that takes ~2 hours which is down from 3 which is down from even 4-5 hours not too far ago. - V: some stuff is lingering around. graph, date. another job can start while the prev job is running. - M: bot always had this deadman switch. current job kicks off the next one, as opposed to the cron job. sometimes that step of kicking off the next one fails so we have a cron job that sees if one is running and starts one if its not. switched to GH concurrency feature in their workflows and use the github cron syntax for kicking things off. now you'll see dup jobs coming in. that cron job kicks off a new one but concurrency of 1 means that it doesn't actually run. bot status badges dont mean anything anymore. when github does the cancellation, it doesn't know that we actually are working fine. - M: With Eric's help we moved things back into one repo instead of splitting into two. cf-autotick-bot was merged with cf-scripts - V: knew we had migrations and admin-requests. noticed on [J infra source of truth](https://cf-infra-docs.netlify.app/docs/reference/infrastructure/) there is a [curator app](https://cf-infra-docs.netlify.app/docs/reference/infrastructure/services/#bot-accounts). is this new? - M: Security discussion. not transcribing. - - M: Bot latency is the biggest issue right now. "Half a day" in practice (30 mins min, plus the CI job itself - 2h-ish). Cutting the time down is the biggest impact on users. Splitting tasks in separate jobs could also work (version updates different from migrations). - V: How are tasks splitted in version updates? - M: Some modulo operation on the feedstock name hash (?) - M: chaindb still used for dependency tracking (?), which requires xonsh. If we remove that, we can remove xonsh completely.