Try   HackMD

Building And Releasing CouchDB For Fun And Profit

Introduction

This is a general-purpose guide designed to onboard people into the CouchDB release cycle. Parts of it can also be adapted to troubleshoot building CouchDB on your local device (something that, as an end user, you shouldn't need to do unless we don't support your architecture/OS).

This guide is currently a work in progress.

Prerequisites

To build CouchDB, you will need:

  • A bit of free space on the disk you're building CouchDB on.
  • Erlang/OTP. This can be any version above or equal to OTP 19, but for the sake of convenience with using couchdb-pkg, it should be OTP 20 (I use 20.3.8.26, the latest in the 20 release line). I'd recommend asdf for managing multiple Erlang installations.
  • ICU, this is usually available as libicu on Linux, on recent macOS versions, it's pre-installed.
  • GNU Make
  • GCC or clang, make sure that your clang installation aliases to gcc and g++ if you're using it.
  • Python, 3.5 or up.
  • Node.js, ideally the latest LTS version (16 at the time of writing).
  • Sphinx (installable via pip install sphinx), but more recent Sphinx versions are broken. I've had success with pip install "sphinx==3.1.2".
  • The Sphinx RTD theme (pip install sphinx_rtd_theme).
  • Spidermonkey. This is Mozilla's JS compiler chain, it's also known as libmozjs or just js. CouchDB supports several versions of this: 1.8.5, 60, 68, and 86. One of these versions should ideally be available in your package manager, if you're on a M1 Mac, you can use one of these Homebrew formulas.

For packaging CouchDB, you also need:

  • A piece of hardware running Ubuntu 20 LTS. Our packaging repo is specifically tuned to build DEBs and RPMs from a Ubuntu machine. It also needs Docker installed, so it can't be Ubuntu running in Docker. I'd recommend a dual-boot with an already existing system.
  • (For ARM builds) Something that runs ARM, like a Raspberry Pi 4.
  • Docker
  • curl, gpg and the rpm tools (on Ubuntu, apt install curl gpg rpm)
  • A bit more disk space, because we're going to spit out a bunch of intermediary files and binaries.
  • Windows requirements tbd

Building CouchDB on Linux or macOS

CouchDB follows the regular Unix procedue of building programs from source:

  1. Get a copy of the source code
  2. Run the configure script
  3. Execute a make task

1. Get a copy of the source code

You can get a fresh copy of the CouchDB source code via a Git checkout:

git checkout https://github.com/apache/couchdb.git

If you're building a specific version, you can checkout to the specific tag:

git checkout tags/3.2.1

If you're releasing, check out to the major version branch (3.x at the time of writing):

git checkout 3.x

2. Run the configure script

As is convention for Unix programs, CouchDB contains a Bash script that generates a Makefile tailored for your system. This script is called configure, and you can run it like this once you're in your CouchDB source code directory:

./configure

This'll check out CouchDB's dependencies, and initialize the tooling used to build CouchDB. Notably, it doesn't strictly check for build-time dependencies such as Spidermonkey - this will error during the build step instead.

By default, the configure script will assume you have Spidermonkey 1.8.5 installed. You can specify a different version by setting a command line flag like this:

./configure --spidermonkey-version 68

3. Execute a make task

CouchDB has a variety of make tasks available. The most important ones are these:

  • make all - builds all components. This is the default when you run make without any arguments
  • make dist - creates a release tarball. This is what we're going to need as a basis to build binaries from.
  • make distclean - gets rid of build and distribution artifacts so you can do a clean build

If you're just looking to build CouchDB for your own usage, run make. If you want to package CouchDB after, run make dist.

Building CouchDB on Windows

TBD :)

Releasing CouchDB

Overview

CouchDB releases are distributed in the following ways:

  • As a source tarball
  • Windows x64 executables (complete with installer)
  • A Mac app in a .zip file
  • Software packages for the following distributions:
    • Ubuntu LTS (16, 18, 20)
    • Debian 9 and 10
    • CentOS/RHEL 7 and 8

The CouchDB project does not distribute any packages to Linux/BSD distributions' package archives. This is a task performed by volunteers not in association with the project. Additionally, these packages end up being several years out of date (you can see an overview here).

Mac releases are currently handled by Jan Lehnardt.

Releasing source tarballs

Before we do anything, we need a release tag. This can be an RC version (e.g. 3.3.0-RC1), or a plain release (e.g. 3.3.0). Once that's been created, we checkout to the tag and run make dist. This will create a source tarball that we can distribute.

TODO: how to generate SHA hashes for the tarball

TODO: how to sign this tarball

Releasing DEBs, RPMs and binaries

This is enabled via couchdb-pkg, a repository that provides support files and scripts in order to automatically build DEBs and RPMs for different systems, using Docker. This will need to be run on a machine that supports building DEB files, as building RPM files on a non-RPM machine is easier. Ideally, this would be Ubuntu 20.04.

Make sure you have the dependencies listed under "Prerequisites" above. You will want to clone couchdb-pkg:

git clone https://github.com/apache/couchdb-pkg.git

From here on, your main interaction will be with the build.sh script. To display a short info text about what build.sh can do:

./build.sh

You will need the tarball you built as the first step of the release somewhere where it's accessible. To build for every platform and arch that we support, run:

./build.sh couch-all path/to/your/tarball.tar.gz

This will take a while. Basically, for every arch/platform, it:

  • Downloads a docker image that's normally used in the CouchDB CI process
  • Mounts a volume to the destination package path
  • Copies over the tarball we provided
  • The docker container then builds a release based off this tarball

In the end, we'll have our generated packages in pkgs/couch.

The RPM packages for CentOS/RHEL need to have correct file permissions, so we run:

sudo chmod a+w pkgs/couch/centos-*/*.rpm

TODO: how to sign rpms

TODO: how to upload packages to artifactory

Releasing Docker images

TODO

Releasing Windows executables

Setting up Windows

To build for Windows, you're going to need some kind of Windows installed somewhere. For ideal use of existing hardware, a laptop or something running Windows is ideal, but running it in a VM (as long as you give it enough system resources) is fine, too.

If you're setting up a fresh copy of Windows, make sure it's 64-bit. Any version of Windows above 7 is 64-bit-only, but we recommend Windows 10 (preferrably a N version, as those come without media bloat).

You want to make sure to install all available Windows updates. This can take quite a while, be sure to keep checking the Windows update screen, as it can download multiple updates in sequence instead of at the same time. Once all of your updates are done, create a snapshot of your VM (if you're running Windows in a VM).

Setting up couchdb-glazier

Windows builds are made easier via couchdb-glazier, a collection of scripts that aims to automate the setup and build process, much like couchdb-pkg does for DEBs and RPMs.

Open an Administrator PowerShell, and run the following:

mkdir C:\Relax\
cd C:\Relax\
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
choco feature enable -n allowGlobalConfirmation
choco install git

At this point, you may need to restart PowerShell for it to recognize git as a command. Once that's done:

Set-ExecutionPolicy Bypass -Scope Process -Force
git clone https://github.com/apache/couchdb-glazier
&.\couchdb-glazier\bin\install_dependencies.ps1

This will take a loooong time. Make sure your disk that Windows is on has around 50GB of free space or more.

Your PowerShell may not immediately recognize some of the commands available to you as a result of running that script. If, during the script execution, it complains about pip not being available, simply execute it after you've restarted PowerShell:

pip install --upgrade sphinx docutils pygments nose hypothesis sphinx_rtd_theme

As a shortcut from now on, in case you reopen PowerShell, you can set the important environment variables using the shell.ps1 script:

Set-ExecutionPolicy Bypass -Scope Process -Force
&.\couchdb-glazier\bin\shell.ps1

Building dependencies and CouchDB on Windows

In addition to what the installer has installed for you, you'll want to have pkg-config installed via choco:

choco install pkg-config

Next up, you'll want to build Erlang. This should Just Work™ for you if you follow the relevant section in the glazier readme.

For building SpiderMonkey, save yourself the trouble of cloning the entire gecko-dev repository. Instead, just download the relevant sources in a tarball:

C:\mozilla-build\start-shell.bat
cd /c/Relax
wget.exe https://ftp.mozilla.org/pub/spidermonkey/prereleases/60/pre3/mozjs-60.1.1pre3.tar.bz2
tar xvf mozjs-60.1.1pre3.tar.bz2
cd mozjs-60.1.1pre3/js/src

The only difference in the build process is that you will need to run the configure script without the --with-system-icu option, as we want to use the bundled ICU. You will also need to set the PKG_CONFIG_PATH and PATH to make the Mozilla build tools aware of your library locations:

# set the PATH to include pkg-config, as well as the pkg-config path itself. do this once you're in the mozilla build tools shell
export PATH="/c/ProgramData/chocolatey/bin:$PATH"
export PKG_CONFIG_PATH="/c/Relax/vcpkg/installed/x64-windows/lib/pkgconfig"
# instead of the configure script used in the readme, use this
../configure --disable-ctypes --disable-ion --disable-jemalloc --enable-optimize --enable-hardening --with-intl-api --build-backends=RecursiveMake --with-visual-studio-version=2017 --disable-debug --enable-gczeal --target=x86_64-pc-mingw32 --host=x86_64-pc-mingw32 --prefix=/c/relax/vcpkg/installed/x64-windows

CouchDB's build tools may complain about a missing mozjs-60.lib file later, so after you've finished building SpiderMonkey, you want to make sure CouchDB's LIBPATH includes said .lib file. You can do this either by setting the LIBPATH yourself (TODO how?) or copying the file to somewhere in the existing LIBPATH, like this:

copy C:\Relax\mozjs-60.1.1pre3\js\src\build_OPT.OBJ\js\src\build\mozjs-60.lib 'C:\Program Files\erl19.3.3.14\lib\erl_interface-3.10.2.2\lib'

The rest of the CouchDB build process should now run exactly as described in the relevant section. Hooray!

Last changed by 
 
mokou
0
672

Read more

Adonis plus Storybook

(AdonisJS + Storybook) - React = Success? I'll detail what I found out about how to make Storybook work with AdonisJS WITHOUT adding an extra UI layer on the frontend. Basic pieces Storybook's server adapter. Without this, nothing here would be possible. This allows us to write anything we want on the server, and then for Storybook to dynamically fetch said anything for use in stories. Storybook's {builder,manager}-webpack5 packages. When using Adonis' Encore-based Webpack tooling, it uses Webpack 5. Storybook ships with Webpack 4. You can guess that those different versions don't like each other, so we can tell Storybook to use Webpack 5 instead. Adonis' components. Adonis encourages writing components that can take props and slots, just like in component systems like React. Putting them together Install Storybook

Mar 29, 2023
Opsy GHA Setup

Docs I used: https://docs.github.com/en/actions/guides/building-and-testing-nodejs https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#job-status-check-functions I restricted the branches to master and feat/gha (my branch) first during PR testing. Env Vars

Nov 24, 2021
RustBridge Berlin meeting 5/2/2020

Photo policy: get badges or lanyards or something where you can tell someone's preference from a picture Presentation Part AUN: add cargo --version to slides (in the installation segment) AUN: add visual studio installation to slides (or link to tutorial) finish the port of the tutorial to ggez (rather review things that should be familiar from other languages to get everyone on a similar level) in presentation focus on what people need now. AUN: add xcode to mac

Nov 24, 2021
Organizing BA runs efficiently with Fernehalwes

This guide covers everything you can currently do after you've gotten people to sign up to your BA run. Prioritizing Attendees The central feature of Ferne right now is the ability to easily sort attendees into parties. However, no attendee is created equal - there are some who you might want to guarantee a spot first, such as party leaders or first timers. First things first, you need to see all of the people you want to give priority to in one spot. You can use the attendee list page's search and filter functions for this. For example, party leaders will usually sign up first, so they will be all in one spot if you sort the "signed up at" column. First-timers can be identified by searching for "Never entered", or sorting the column for that field. Once you have all of the people you'd like, you can click the checkboxes to select them. Shift-clicking a checkbox selects all attendees between the previously checked box and that one. You can also use the "select all" checkbox at the top, it'll select every attendee you can currently see (but not the ones you can't see, e.g. if you searched for something). This could look something like this: You can then click on the blue button to the right to select all of these attendees for inclusion. Being selected for inclusion (I'm open for a less confusing term) basically means these people are guaranteed a spot in the event and have priority when being assigned to a party. The list might break a little bit when you do this, that's something I'm working on, it just resets the order of things, but your selections are still locked.

Nov 24, 2021
Read more from mokou
Published on HackMD