# Steps for Building Packages

`Remember to log in to appropriate lab machines before starting with the steps:`

rhs-vm-16.gdev.lab.eng.rdu2.redhat.com(debian 9 strech)
rhs-vm-17.gdev.lab.eng.rdu2.redhat.com (ubuntu 18.04), (bionic, xenial) eoan, disco-reched, groovy eol (present in the doc for historic reference)

***All the packages can be built on rhs-vm-08 including bionic and xenial from 9 series:***

rhs-vm-06.gdev.lab.eng.rdu2.redhat.com - **for JAMMY and KINETIC**

rhs-vm-08.gdev.lab.eng.rdu2.redhat.com (**new ubuntu machines jammy of release 9, focal, hirsute, impish for io-uring, and other updates- glusterfs 9 onwards**)

**NOTE: we're not building glusterfs-9 on ubuntu xenial or debian stretch.**
(domain name changed from storage-dev.lab.eng.bos.redhat.com to gdev.lab.eng.rdu2.redhat.com)

rhs-vm-18.gdev.lab.eng.rdu2.redhat.com (fedora), and
rhs-vm-19.gdev.lab.eng.rdu2.redhat.com (debian 10 buster)
rhs-vm-12.gdev.lab.eng.rdu2.redhat.com (debian 11 bullseye)
rhs-vm-07.gdev.lab.eng.rdu2.redhat.com (debian 12 bookworm)
rhs-vm-05.gdev.lab.eng.rdu2.redhat.com (trixie/sid)

three new arm64 vms for Debian builds:
rhs-vm-24 (stretch/9),
rhs-vm-25 (buster/10) and
rhs-vm-26 (bullseye/11).

### How to run rc builds using the script?
1. open two tabs for the same build machine
2. run rc-build-debian_gluster.sh with required arguments
eg: bash rc-build-debian_gluster.sh debian buster 10 10.0rc0 1
3. When the yes/no prompt opens on terminal, go two second terminal, create required branch and make required changes and then push it to github
4. Go back to previous tab and type yes and then enter if you feel that appropriate changes are made in the code. If not press no, and debug what else is to be corrected.

#### 1. Create a new branch -
> [$user] git checkout -b branch_name origin/remote_branch_name
#### 2. Make changes in Changelog -
> [$user] nano debian/changelog
#### 3. Run the below command making appropriate changes for distribution and mirror -
> [$user] sudo pbuilder create --distribution wheezy --mirror ftp://ftp.us.debian.org/debian/ --debootstrapopts "--keyring=/usr/share/keyrings/debian-archive-keyring.gpg"

> [$user] sudo pbuilder create --distribution cosmic --mirror http://ubuntu.osuosl.org/ubuntu/ --debootstrapopts "--keyring=/usr/share/keyrings/ubuntu-archive-keyring.gpg"

`--distribution`
for debian = stretch/buster/bullseye
for ubuntu = bionic/cosmic/disco/eoan/xenial

` --mirror `
for debian = ftp://ftp.us.debian.org/debian/
for ubuntu = http://ubuntu.osuosl.org/ubuntu/

` --keyring`
for debian = /usr/share/keyrings/debian-archive-keyring.gpg
for ubuntu = /usr/share/keyrings/ubuntu-archive-keyring.gpg

#### 4. Run appropriate scripts for ubuntu and debian 
> [$glusterpackager] ./ubuntu_script.sh

> [$glusterpackager] ./debian_script.sh

#### 5. Commit the changes - 
> git commit -a -m 'msg here'

#### 6. Push changes - 
> git push origin local_branch:remote_branch
eg: git push origin buster-glusterfs-4.1.10:buster-glusterfs-4.1

### OPENSUSE
**OpenSUSE Leap 15.3 and SUSE SLE-15 SP3 are out. We should phase out Leap 15.2 and SLE-15 SP2 builds.**

look for version and enter subproject
create package
check for SLES, LEAP, TumBLEWEED
fill in the details and commit 

Name:
GlusterFS 6.4
Title:
GlusterFS 6.4-1
Description:
GlusterFS-6.4 GA

select appropriate tar and spec files
save(it will start building)
packages to be built for glusterfs 7 - Leap15.1,  SLES12SP4, SLES15SP1, Tumbleweed


Shwetha Acharya,
Hi Kaleb, which is the spec file that is used in building suse packages? tar file can be downloaded from https://download.gluster.org/pub/gluster/glusterfs/
Is it  glusterfs.spec.in which we find in the source code?
Can you please clarify on this.

Kaleb Keithley,
https://github.com/gluster/glusterfs-suse
gluster/glusterfs-suse
gluster/glusterfs-suse/github.com

In general there's a branch for each of leap15, sles12, sles15, and tumbleweed.  Use the leap15 branch for the Leap15.1 subproject builds, sles12 for SLES12SP4, sles15 for SLES15 and SLES15SP1, and tumbleweed for Tumbleweed.
Consult the matrix at https://docs.gluster.org/en/latest/Install-Guide/Community-Packages/ for which subprojects we are currently building for.  I've sent a PR to add SLES15SP1 but it hasn't been merged yet, so keep building for both SLES15 and SLES15SP1; you should be able to use the same .spec for both.

1) modify version and changelog entry in spec files
2) push the changes
3) Get the spec files and sha files ready in folders for easy access
4) goto https://build.opensuse.org/project/subprojects/home:glusterfs
5) create new package using spec and tar files

### Unpacking a Debian (e.g. for stretch) tarball in /pub/gluster/glusterfs/qa-releases/7.0rc0/Debian/strech/amd64/apt

1) scp the stretch...tar.gz file produced by the build to download.rht.gluster.org:/var/www/scratch   (as there's not enough space in /tmp)
2) Then ssh to the download.rht.gluster.org
3) cd /var/www/html/pub/gluster/glusterfs/qa-releases/7.0rc0/Debian/stretch/amd64/apt
4) sudo tar xpf /var/www/scratch/stretch...tar.gz
5) don't forget to delete the /var/www/scratch/stetch....tar.gz afterwards
6) **update the latest**

### To be done in new machines
1) useradd -m username -s /bin/bash
2) passwd username
3) mkdir -p src/github
4) cd src/github
5) clone glusterfs-debian repo

### Testing centos packages
Packages for CentOS 6 & 7 are now in the testing repositories of the
Storage SIG. Please try them out and share results on the list.

    # yum install centos-release-gluster6
    # yum --enablerepo=centos-gluster6-test install glusterfs-server
    ... do your testing now
    

centos8 test mac - 10.70.41.185
### when no space left error
/var/cache/pbuilder/result/
/var/cache/apt/archives

### building RCs
----------------
from kaleb @Sheetal Pamecha @Shwetha Acharya  BTW, please disable gnfs in the builds.  I.e. change --enable-gnfs in rules to --disable-gnfs.
also create new {stretch,buster,bullseye}-gluster-7 branches from the -gluster-6 branches. You probably realized this, sorry for stating the obvious.
"""typically for RCs we have only built for Fedora, Debian, and maybe CentOS. And the Debian and Ubuntu scripts don't work for RCs

Debian packages will have to be built following the HOWTO, not using the script. Don't build Ubuntu or SUSE. I'll make the .../qa-releases/7.0rc0 directory and upload the tarball.   Sheetal and Shwetha still haven't been given accounts on download.gluster.org !!!!"""

### Automating the packaging
[Github repo for reference](https://github.com/gluster/build-jobs/blob/master/build-gluster-org/scripts/centos-regression.sh)
[Script](https://hackmd.io/@spamecha/H1SIWPhWr)
[hari-doc](https://docs.google.com/document/d/1QfS6v-tHAHnnNSds6zwVDNnWhdKN6s9zIf8USbCLPNE/edit)
[hari-script](https://review.gluster.org/#/c/build-jobs/+/23289/2/build-gluster-org/scripts/generic_package.sh)



### Automation steps
For running - generic_package.sh
bash build.sh debian stretch 7 7.4 1

### h3
How to push to build-jobs repo
Create .gitreview file 
`[gerrit]
host=review.gluster.org
port=29418
project=gluster/git-review.git
defaultbranch=master`

git-review




Here's something I sent to Nigel back in June 2017. It's a teeny bit out of date, but mostly everything still holds true. (Some of this will probably utterly meaningless until I explain it, so don't worry if it doesn't make sense right now.)

There are several VMs dedicated to building packages. I suppose they
could be docker containers, or even jenkins slaves. That's something you could do if you have the interest and time.

They are rhs-vm-{112,113,116,17,18,19}.gdev.lab.eng.bos.redhat.com. They all run on rhs-srv-01.storage-dev.lab.eng.bos.redhat.com. This is entirely self managed. If they get shut down, sign on to rhs-srv-01 and restart them with `virsh start $guest` or use VirtManager from you laptop to manage them with the GUI.

rhs-vm-{12,13,16} are deprecated. Their build environment got hopelessly messed up
so I replaced them with rhs-vm-{112,113,116}. We don't build packages for Debian 7 or 8 any more, so you can ignore -112 and -113.

All machines have root/redhat and glusterpackager users. See the README
in ~glusterpackager on any machine for glusterpackager's passwd. Sign in
as root, create an account for yourself and give yourself sudo privs.

rhs-vm-112: debian 7 (trusty)
rhs-vm-113: debian 8 (jessie)
rhs-vm-116: debian 9 (stretch)
rhs-vm-17: ubuntu 18.04
rhs-vm-18: fedora 27
rhs-vm-19: debian 10 (buster/sid/unstable)

rhs-vm-17 has dput and related packages installed. These are used to
upload a .source_changes to Ubuntu Launchpad. The glusterpackager user
there has the gpg key for signing the .source_changes to Launchpad. This
key needs to not be lost. (I have it in an email as well.) I haven't
felt any strong need to update or replace 16.04 with any of the newer
releases. Unless something goes off the rails I will probably wait for
18.04. There is a ~glusterpackager/ubuntu_script.sh that automates 99%
of the work to build packages on Launchpad.

rhs-vm-18 has two purposes: 1) it's where I create glusterpackager gpg
keys, and 2) it's where I sign Fedora packages and create repos for
download.gluster.org. I used gpg to create a key, export it to the the
debian boxes, and then (re)import it with gpg2. Debian packages are
signed with gpg, Fedora packages with gpg2. For reasons I don't fully
understand, you must reimport the key with gpg2 otherwise rpmsign can't
find the key.

The passphrase for each key is in the ~glusterpackager/README file on
each of Debian boxes and the Fedora box. The keys are also exported to
the mit gpg keyserver. The glusterpackager's password is also the
password for the gluster account on OpenSuSE Build Service.

Each of the Debian boxes has all the packages necessary to build
packages, including debuild, pbuilder, and reprepro.
~glusterpackager/HOWTO has the recipe for building packages. In a
nutshel the process is 1) use debuild to create a .dsc (and
.source_changes) file, 2) use pbuilder to build the .debs, and 3) use
reprepro to create the apt repos.

Since Debian 9 just released and debian 10/buster/sid/unstable were just
released and rhs-vm-19 is set up, you won't have to think about this for
a while, but there's some stuff in /var/www/repos/apt/debian/conf that
has to be created by hand. When I set up the new box for buster I just
copied all the stretch bits and s/stretch/buster/.

There's some magic pbuilder stuff for building nfs-ganesha packages in
/etc/pbuilderrc and ~glusterpackager/ganesha-deps on the debian boxes.
This magic stuff creates a repo with the glusterfs and libntirpc
dependencies.

I recommend ssh directly to debian boxes as glusterpackager. (Versus ssh
and su.)  This is the "easy" way to get a tty so that the pinentry util
used by the various package and repo signing apps can prompt for
password. There might be other ways to get a tty or to feed the password
to pinenty without needing a tty. I haven't spent the time to figure
that out.

Every blue moon or so the pbuilder cache of dependency packages will get
too old to build with. There are instructions at the top of
~glusterpackager/HOWTO for building or updating the cache. (I usually
just rebuild it, as I've had mixed results with updating it.) The
instructions point at ftp.us.debian.org, but I actually use
debian.csail.mit.edu for no particular reason. I suspect it may have
more bandwidth.

You don't necessarily need anything special to build packages for SuSE.
Everything can be done via the web UI. I've heard there are CLI commands
available, presumably in packages for SLES and OpenSUSE Leap.

The packaging bits themselves are in two github repos:
https://github.com/gluster/glusterfs-debian and
https://github.com/gluster/glusterfs-suse. I usually have two or three
windows open. One signed into my account where I update the packaging
bits, and another signed in to glusterpackager where I run the build and
packaging commands. In the best case scenario all you do is add a new
entry to .../debian/changelog and proceed to build. But sometimes you
will need to update dependencies in .../debian/control, or add/remove
patches in .../debian/patches, or fix up stuff in .../debian/rules.
That's when things get interesting. As mentioned above, for Ubuntu
Launchpad builds just use the ~glusterpackager/ubuntu_script.sh to build.



FOR ERROR - "No secret key"
gpg --list-secret-keys
gpg --list-keys