--- tags: freebsd, git, workflow --- # A Committer's FreeBSD git workflow Below is my day-to-day workflow for dealing with ports and patches in git. While this is ports focused, I use the same approach for src but significantly less frequently. Comments welcome, and I would *love* to see similar flows for svn. Perhaps we can do a survey once we have a good number of workflows, and find out what the most used ones, and most used features are, that we need to preserve. https://hackmd.io/s/rkQxTwCa4 Most of the discussion so far about git & svn on has been based on individual features, which will clearly never be at parity no matter what the direction of comparison, nor the confirmation bias of the individuals making their (valid) points. I’m interested in hearing how other people use both svn and git, so we can compare real-world workflows, and move past the fruitless discussions that compare isolated features in theoretical use cases. In the example below the upstream is the “official unofffical” git mirror on github, but the workflow doesn’t require nor mandate github at all. This workflow has been used by a number of people who were not necessarily familiar with FreeBSD nor ports, to successfully maintain a custom set of ports and packages that ebbed and flowed as we reviewed and committed changes into the official FreeBSD ports, instead of our custom fork, and used this tree to manage packages for a number of jails and architectures. This has worked really well in allowing us the best of both worlds - updated ports from the project and community, and custom fixes or changes when we needed something urgently to deal with security vulnerabilities or occasionally to pin a port where we needed an older version for a period of time. We also have more extensive documentation on pulling in specific patches from FreeBSD ports when we are *not* ready to bump all the versions, but need a single fix, but I think everybody here is already well familiar with that. ## Overview Origins are full git mirrors of the FreeBSD code base. As these are all originally forked from the project git repo, the bulk of the commits are shared, and the space is not excessive: 1.6 GiB for /usr/ports/distfiles, and 1.9 GiB for /usr/ports/.git. - an origin called "upstream" which is the project's git master - an origin called "dch" which is my private trash heap - additional origins as required for customers and other forks of interest Branches are pointers to an interesting tip commit, and all the preceding commits below it. A local branch can track a remote branch, and will automatically update itself when the origin is refreshed. - upstream tracks upstream/master aka FreeBSD's trunk - I have a branch for each port or feature I am working on - master is my local branch with all the branches I find useful included - I regularly push my local master up to our poudriere server which uses that to build all our ports off Worktrees allow me to have a separate working directory that shares a single bare ".git" directory, where all the branches, metadata, and commits are cached. This is very efficient. I particular, for any major activity, I don't check a branch out to my disk, but to a tmpfs. Obviously I need to commit regularly to ensure that the tmpfs changes are safely recorded in the on-disk bare repo, but this makes for a very fast and reasonably safe workflow. I can commit every few minutes, and finally rebase and squash the intermediary untidy commits into a nice clean logical flow before publishing upstream, or to phabricator for review. ## Pre-requisites ### Ports - arcanist (for phabricator) - git - cdiff ### zfs datasets ``` # set up zfs $ sudo zfs destroy -vrf zroot/usr/ports $ sudo zfs create zroot/usr/ports $ sudo zfs create zroot/usr/ports/distfiles $ sudo chown -R $USER /usr/ports ``` ### initial clone ``` # clone into that directory, using "upstream" as the origin $ git clone -o upstream git@github.com:freebsd/freebsd-ports /usr/ports $ cd /usr/ports # add my private origin where I stash wip and trash # you can have multiple origins and pull, push, and rebase amongst them $ git remote add dch git@git.sr.ht:~dch/ports ``` ## Daily Grind ### Bump our ports tree ``` # fetch updates and float any local patches up over master # in most cases, if the local patch has been committed upstream, # the rebase will Do The Right Thing, or prompt us to skip that # commit, or resolve the conflict if we require. $ git checkout master $ git fetch upstream && git rebase upstream/master # tag it so we can roll back in future if required $ git tag $ (date -u +%Y%m%d-%H%M) # do the poudriere dance and re-deploy affected systems ``` ### Contribute a fix ``` # oh dear, devel/dodgy has *another* bug. Curse you dodgy devs. $ git checkout -t upstream/master -B devel/dodgy # hack hack hack $ git commit -am 'devel/dogdy: fixed shoddy workmanship' # this incancation of arcanist provides a patch that includes *only* # the last commit. I could alternatively use upstream/master if there # were several, and I wanted to send all of them to phabricator. $ arc diff --base git:HEAD~1 ``` If we have changes to integrate, simply repeat the last 2 steps, and the phabricator diff is updated with our changes. ### Test a fix from another developer ``` # ensure we have a clean base to work from $ git checkout upstream # use arcanist to fetch the diff and apply it in a nice clean branch $ arc patch D12345 # we are now on a branch arcpatch-D12345 that contains the commit # from the patch in phabricator # we hack hack hack again and if we need to update I believe it's # possible to push our changes to somebody else's diff - but I've # not personally tried this. ``` ## Merging all the things Personally I *really* like subversion's monotonic commits, so I always rebase my branch off master, and then pull my commits back into git master, to avoid a merge commit. ## Gripes and Grumbles Right now, I also have a full `svn checkout` of ports and src, and when I need to *commit* a patch, I update svn, then use `arc patch D12345` as above, to pull in the patch, review it with `sysutils/cdiff` and commit it. I haven't gotten git's svn commit flow to work, although `jrm@` provided very helpful instructions, it's not really necessary for me right now. The git->phabricator->svn juggle has on at least 3 occasions missed things, often deleted files seem to get lost, and on 1 occasion I ended up with extra garbage in the diff, which luckily I caught before committing. It's likely this is my bad and not the tools. The slowest operation (in both svn and git) is enumerating from the /usr/ports root, all the changes - for example, cdiff, or `git status`. This takes a few seconds, and for both svn and git, is *much* faster when run in the appropriate subdirectory. A git pull from a laptop on a remote connection is very lightweight, and even over comparatively slow links, is IMHO acceptable time-wise. My baseline is ADSL or 3G in New Zealand, so it's not the worst case, but a reasonable one to accept as a baseline I think. ## Worktrees As I mentioned above, I also use worktrees in a tmpfs for speed. Obviously this is not required, but it is useful. Once done, all the usual branch and commit stuff works as expected, and when you commit, the work is saved on the original /usr/ports disk filesystem. ``` # set up a new worktree tracking an existing branch $ git worktree add --track -B net-p2p/libswift /tmp/swift $ cd /tmp/swift # hack hack hack $ git commit 'net-p2p/libswift: L33T h@x0rz' $ git push $ cd - $ git worktree remove /tmp/swift # or if I forget to remove it and the system was rebooted, then: $ git worktree prune # all tidy again ```