# Debian Packaging Notes
###### tags: `Debian` `GNU` `Linux` `Open Source` `Operating System` `Software Packaging` `Shell Script`

Notes for Debian software packaging operations.

## Resources

- [**Debian Policy Manual - Debian Docs**](https://www.debian.org/doc/debian-policy/index.html): Debian system design (in packages) reference manual.
- [**Debian Reference - Debian Docs**](https://www.debian.org/doc/manuals/debian-reference/index.en.html): Debian user's guide covers many aspects of system administration through shell-command examples for non-developers.
- [**Debian Packaging Guide - Debian Wiki**](https://wiki.debian.org/Packaging): Guides for packaging software for Debian.
- [**Ubuntu Packaging Guide - Ubuntu Docs**](https://canonical-ubuntu-packaging-guide.readthedocs-hosted.com/en/latest/): Guides for Ubuntu development and packaging.
- [**Packaging PPA (Personal Package Archive) - Launchpad**](https://help.launchpad.net/Packaging/PPA): Manage source code packages in personal apt repository on Launchpad, automatically build binaries, distribute software, and updates directly to Ubuntu users.

## Download the source code of the Debian package

ref: [Get the source of a package - Ubuntu Docs](https://canonical-ubuntu-packaging-guide.readthedocs-hosted.com/en/latest/how-to/get-package-source/)

### Wget via HTTP

> :warning: *No SSL encryption, make sure to check cryptographic signature.*

#### Download

wget http://archive.ubuntu.com/path/to/the/package.dsc

#### Extract the contents

dpkg-deb -R package.deb ./project-directory

### Git-ubuntu

The modern way of working with Ubuntu source packages.

#### Install

sudo snap install --classic --edge git-ubuntu

#### Clone the Source Git Repository

git-ubuntu clone <package-name> ./project-directory

#### Generate the Original Source Archive (name_version.orig.tar.xz)

cd ./project-directory
git-ubuntu export-orig

### Pull-pkg

Part of the ubuntu-dev-tools package and downloads a specific version of a source package, or the latest version from a specified release.

#### Install

# Install ubtuntu-dev-tools (pull-pkg included)
sudo apt update && sudo apt install ubuntu-dev-tools

#### Pull the Source Repository

##### General Command

pull-pkg [OPTIONS] PACKAGE-NAME [SERIES|VERSION]

##### Option 1. Download Latest from Launchpad

pull-lp-source <package-name>
pull-lp-source <package-name> mantic              # for Ubuntu-mantic-release
pull-lp-source <package-name> <specific-version>  # specific-version

##### Option 2. Download Latest from Personal Package Archive (PPA) on Launchpad

pull-ppa-source --ppa '<user-name>/<package-name>' '<package-name>'
pull-ppa-source --ppa '<user-name>/<package-name>' '<package-name>' 'mantic'
pull-ppa-source --ppa '<user-name>/<package-name>' '<package-name>' '<specific-version>'

##### Option 3. Download Latest from Debian

pull-debian-source '<package-name>'
pull-debian-source '<package-name>' 'sid'                 # for sid-release
pull-debian-source '<package-name>' '<specific-version>'  # specific-version

### Apt-get source

Via built-in apt package manager (may need *deb-src* for checking actual source packages version).

#### Clone the Source Repository

apt source '<package-name>'

# ! Error: You must put some 'deb-src' URIs in your sources.list
sudo uncomment *deb-src* line in /etc/apt/sources.list

### Dget

Download Debian source and binary packages.

> :warning: *No SSL encryption, make sure to check cryptographic signature.*

#### Install

# Install devscripts (dget included)
sudo apt update && sudo apt install devscripts

#### Clone the Source Repository

dget <URL>  # Ex. https://launchpad.net/.../source.dsc

## Update the Package's Latest Upstreams

Most of the source packages contain a *[watch](https://wiki.debian.org/debian/watch)* file in the *debian* folder. This is a configuration file for the *uscan(1)* utility which allows you to automatically search HTTP or FTP sites or *git(1)* repositories for newly available updates of the upstream project.

> :warning: *If manually downloading upstream file(s) is required, ensure verifying package's cryptographic signatures (such as the signing key may be store as debian/upstream/signing-key.asc).*

cd ./project-directory

uscan --safe  # check for the new version

uscan  # --verbose for detailed log

## Debian Package

### What is a Package?

> A (Debian) package is a collection of files that allow for applications or libraries to be distributed via the package management system. The aim of packaging is to allow the automation of installing, upgrading, configuring, and removing computer programs for Debian in a consistent manner (Learn more: [Debian Policy - Debian Docs](https://www.debian.org/doc/debian-policy/index.html)).

A package consists of **Binary Package(s)** & **Source Package**:
#### Binary Package(s)

Binary Package (.deb)
├── usr
|   ├── bin
|   |   └── bin (executable files under system directories)
|   ├── lib
|   |   └── lib.so (shared libraries under system directories)
|   ├── share (data shared across all users)
|   |   ├── man/ (for manuals)
|   |   ├── locale/ (for supporting multi languages)
|   |   └── doc/ (Ex. README)
|   ├── include
|   |   └── header.h (for development & compilation)
|   └── local (bin/lib/data under local directories)
|       ├── bin/
|       ├── lib/
|       └── share/
├── etc (config for systems)
|   └── system.conf
└── DEBIAN (maintainer scripts and package control files for Debian)
    ├── control
    ├── postinst
    └── prerm

:::warning
**System-Wide `/...` vs. User-Program `/usr/...`**

Why seperate those? Disk storage limitation back in old days!

For compatibility reasons, to maintain a cleaner directory manner, we use `usermerge` to do ++symbolic link++. For example:
- `/bin` -> `usr/bin`
- `/sbin` -> `usr/sbin`
- `/lib` -> `usr/lib`
- `/lib64` -> `usr/lib64`

> 
>
> (Source: [Mayank Ahuja - Twitter](https://x.com/techNmak/status/1881036704251080733))

[usrmerge in Debian - SZ Lin](https://szlin.me/2017/12/28/usrmerge-in-debian/)
:::

#### Source Package

Source Package
├── debian/
|   ├── changelog
|   └── control
├── debian/
├── main.h
├── main.c
└── Makefile

- **`<name>_<version>.dsc`**: Metadata about the package and reference both `orig.tar.xz` & `debian.tar.xz` (& `.changes`).
> Description of how to build the Debian package.

- **`<name>_<version>.orig.tar.xz`**: Compressed tarball (`.tar.xz`).
> Compressed source directories & files (source archive).

:::info
**Update a Debian Package**

Usually, for packaging purposes, you need to unpack `<name>_<version>.orig.tar.xz` in the current folder and then place the `debian/` folder inside the unpacked archive. It is important to keep the archive in place.

THEN you do the packaging stuff that you wish, either building binaries, or building the source archives (which results in a `<name>_<version>.changes` file that would be `dput`-ed to a *PPA*, for example).

[What is a '.orig.tar.xz' file in packaging? - Ask Ubuntu](https://askubuntu.com/questions/603157/what-is-a-orig-tar-xz-file-in-packaging)
:::

- **`<name>_<version>.debian.tar.xz`**: A compressed tarball (`.tar.xz`) that only contains the Debian-specific packaging files (`debian/*`) that are used to build the Debian package from the original source code.
> Debian-platform-specific compressed source directories & files.

- **`<name>_<version>_source.changes`**: Source package changes.
> Soruce code patches.

- **`<name>_<version>_<instruction-set>.changes`**: Binary package changes.
> Binary software patches.

:::warning
**`DEBIAN` vs. `debian` Directory?**

When creating the package, you have a `debian` directory, which contains the data needed for making the package (e.g., *control file*, *rules*, *changelog*, etc.). *dpkg-deb* (which requires `DEBIAN`) is not the usual way to make a package - it just offers a quick way to assemble one. You usually *usedpkg-buildpackage* instead.

[DEBIAN or debian in .deb package? - Ask Ubuntu](https://askubuntu.com/questions/1319936/debian-or-debian-in-deb-package)
:::

### [Optional] Modify the Debian Package

All information of the source package related to Debian is store under `package-directory/debian` folder.

- **debian/changelog:** Modified to include a new entry with the package version and changelog details.
- **debian/control:** Often modified to specify build dependencies or package details.
- **debian/rules:** Contain rules for building the package. This file is usually modified to include custom build steps or to override default behavior.
- **debian/compat:** This defines the *debhelper* compatibility level.
- **debian/*.install:** [Optional] Specify where files should be installed.
- **debian/*.manpages:** [Optional] Specify which man pages should be installed.
- **debian/*.lintian-overrides:** [Optional] Suppress lintian warnings or errors.
- **debian/*.links:** [Optional] Create symbolic links during package installation.

#### Add Package Info

- debian/control

Package: Package-Name
Version: 1.0
Maintainer: Developer-Name
Architecture: all
Description: This package includes some updates.
 Details of the updates are these ...

#### Add Messages when Package's Installed, Upgraded or Removed

- [Package maintainer scripts and installation procedure - Debian Docs](https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html)

- [Option 1] debian/postinst

#!/bin/bash
echo "The message after installation."

- [Option 2] debian/rules

...
override_dh_install:
    dh_install
    echo "The message after installation."

#### Add Shell Scripts to Package's File List

- debian/rules

...
override_dh_install:
        dh_install
        cp testing.sh debian/<package>/usr/bin/

> After building the package and installing it, testing.sh should be part of the Debian package.

### Build the Debian Package
- [Guide for Debian Maintainers - Debian Docs](https://www.debian.org/doc/manuals/debmake-doc/index.en.html)
- [How To Package For Debian - Debian Wiki](https://wiki.debian.org/HowToPackageForDebian)
- [Building the package - Debian Docs](https://www.debian.org/doc/manuals/maint-guide/build.html)
- [Debian 套件打包指南 - Lucas Nussbaum - Debian Docs](https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial.zh_TW.pdf)
- [如何製作「deb檔(Debian Package)」 - samwhelp](https://samwhelp.github.io/book-ubuntu-basic-skill/book/content/package/how-to-build-package.html)
- [用Open Source工具開發軟體: 新軟體開發觀念 - 套件包裝製作 - study-area](http://www.study-area.org/cyril/opentools/opentools/x1447.html)

> :warning: chroot (file-system-based environment isolation) is recommended.

#### What does a building process do?
1. **Clean:** Remove any previous build artifacts.
1. **Patch:** Apply any patches from the `debian/patches`.
1. **Configure:** Run *configure* script before compilation.
1. **Compile:** Ex. *make*, *gcc*
1. **Build Binary Package:** Build the binary Debian packages (.deb) from the compiled source code.
1. **Sign:** Sign the binary packages with developer's GPG key.
1. **Build Source Package:** Include *name_version.dsc*, *name_version.orig.tar.xz*, *name_version.debian.tar.xz*, *name_version.changes*

#### 0. Package Init

dh_make

#### 1. Commit Changes (Patches)

dpkg-source --commit

#### 2. [Optional] Changelog & GPG Sign Key

> Only when the package built & uploaded to the public need this changelog & signature process. Otherwise use *debuild -us -uc*.

#### 2-1. Add Changelog for the new Debian package

DEBEMAIL="[e-mail]" EMAIL="[e-mail]" dch -i

- debian/changelog

hello (2.10-2ubuntu3) jammy; urgency=medium

  * Take Home Test.
  * Show personal messages during the installation.
  * Add a testing.sh as an executable script installed on the system.

 -- My Name <[e-mail]>  Sun, 14 Apr 2024 22:35:50 +0800
 
...

> Will need a GPG key that matches the latest author's info of the changelog to sign on the later building process.

#### 2-2. Setup Personal GPG Key Signature (Ex. My Name \<[e-mail]\>)
See [next chapter](#0-1-Setup-Personal-GPG-Key-Signature-Ex-My-Name-ltexamplemailcomgt).

#### 3. Build (Repackage) Package

dpkg-buildpackage # -us -uc

Automatic building process with *dpkg-buildpackage* commands

debuild # -us -uc

:::info
When working on a package, a faster rebuild can be done with *debuild*. But then, all build-dependencies must be satisfied in the installation where the package is built. All necessary packages can be installed automatically with apt-get *build-dep*.

A complete example for building the foo package looks like this:

apt-get source foo
cd foo-0.0.1
sudo apt-get build-dep foo
debuild -us -uc

:::
:::warning
#### Compatibility Issues & Solutions

# ! dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper-compat (= 9)
sudo apt-get install debhelper-compat=9

# ! dh: warning: Compatibility levels before 10 are deprecated (level 9 in use)
vim debian/control  # Change Build-Depends: debhelper-compat (= 10)

# ! Tools not found
sudo apt-get update && sudo apt-get install <package>

# ! dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/<name>_<version>.diff.<hash>
dpkg-source --commit

# Repackage again
debuild -us -uc

:::

### Upload the Package to Public PPA Repositories

To get software into a PPA, you need to build a source package. That includes the source code for the software you want to distribute, along with the instructions for where the application should live in the file system and of any dependencies it has on other software.

- [Personal Package Archives (PPA) Guide - Launchpad](https://help.launchpad.net/Packaging/PPA)
- [Ubuntu Linux 新增與移除 PPA 個人套件庫指令教學 - gtwang](https://blog.gtwang.org/linux/ubuntu-linux-add-and-remove-ppa-command-tutorial/)
- [上傳 debian package 到 PPA - elleryq](https://blog.elleryq.idv.tw/2010/09/debian-package-ppa.html)

#### 0-1. Setup Personal GPG Key Signature (Ex. My Name \<[e-mail]\>)
- [GPG Manual - GNU](https://www.gnupg.org/gph/en/manual/book1.html)

> We skip signature sign on the *debuild* stages (`-us -uc`), here we redo this step & changelog, then debuild again with changes.

gpg --full-generate-key

gpg --list-keys --keyid-format=long

# or (display same things)
gpg --list-secret-keys --keyid-format=long

gpg --delete-secret-keys <key-id>

gpg --delete-key <key-id>  # public key

gpg --delete-secret-and-public-key <key-id>

# edit subkey pairs
gpg --edit-key <key-id>

# revoke a subkey pair
gpg --gen-revoke <key-id> > public-key.asc

# don't forget to update to the public keyservers !

# export public key in ASCII-armored format
gpg --armor --export <key-id>

# store private key in ASCII-armored format
gpg --export-secret-keys --armor <key-id> > private-key.asc

gpg --import private-key.asc

:::info
**Demythify a GPG Key Bundle**
- **General:**
- **Fingerprint:** The hash value for the key.
> Used to locate the key.
- **Key ID:** A shortened *Fingerprint*.
- **Primary Key Pair:** The full GPG public key to verify / sign & encrypt.
> Usually used for signing (`[S]`) & certifying (`[C]`).
- **SubKey Pair(s):** A seperate key pairs but automatically associated with the primary key pair (like brief & revocable aliases).
> Usually used for encrypting (`[E]`), signing (`[S]`), & authentication (`[A]`) such as for SSH connection.

:::warning
**Computing Signatures**

All signatures are formed by producing a hash over the signature data, and then using the resulting hash in the signature algorithm.

When a signature is made over a key, the hash data starts with the octet `0x99`, followed by a two-octet length of the key, and then body of the key packet. A subkey binding signature (type `0x18`) or primary key binding signature (type `0x19`) then hashes the subkey using the same format as the main key (also using `0x99` as the first octet).

[RFC 4880: OpenPGP Message Format](https://www.rfc-editor.org/rfc/rfc4880#section-5.2.4)
:::
- **`--armor`:** Make *unarmored (binary)* data ASCII-armored (readable).
> Often use `base64` for en/decoding.

- **Public Key Shared Packet:** Contain the primary public key & valid public subkeys.
> For others to verify its your (corresponding key) signing, or decrypt/encrypt messages.

:::warning
**Encryption vs. Digital Signature**

Something like the sealing wax of the envelope vs. the stamp on the official letter.
:::

- **Private Key Secret Packet:** Contain the primary private key & valid private subkeys (& all of their corresponding public keys).
> For the owner to certify the message (by signing an encrypted message hash), or decrypt/encrypt messages.

> asymmetric en/decryption:
> 
>
> (Source: [Pretty Good Privacy (PGP) - Wiki](https://en.wikipedia.org/wiki/Pretty_Good_Privacy))

More:
- [Still confused about GPG keys and subkeys - Stack Exchange](https://superuser.com/questions/1522300/still-confused-about-gpg-keys-and-subkeys)
- [Why does gpg's secret and public key have the same keyid? - Stack Exchange](https://superuser.com/questions/1620268/why-does-gpgs-secret-and-public-key-have-the-same-keyid)
:::

#### 0-2. Upload GPG Key to KeyServer.Ubuntu

gpg --keyserver keyserver.ubuntu.com --send-keys <key>

- [Just sent my gpg key to the keyserver but Launchpad can't import it. What should I do? - Launchpad FAQ](https://answers.launchpad.net/launchpad/+faq/79)
- [cannot import gpg key - Launchpad FAQ](https://answers.launchpad.net/launchpad/+question/28068)

> Check the key is uploaded: http://keyserver.ubuntu.com/pks/lookup?search=0x12345678&op=index
> *(where `0x12345678` is the last 8 hex digit of the GPG footprint)*

#### 0-3. Upload & Confirm OpenPGP Key Registration on Lanuchpad

If adding key on Launckpad personal page successfully, they will send you a confirmation email.

Inside the email is a base64 text encrypted by your private key (passphrase involved).

:::warning
If you cannot read the unencrypted instructions below, it may be because your mail reader does not support automatic decryption of "ASCII armored" encrypted text.
:::

The message looks like this:

- encrypt.txt

-----BEGIN PGP MESSAGE-----

hQGMA3Qo4tK9dAu5AQwArsU2NCeUB4RWziL3aiOY23s72lHrTwGHv40tbCGbNQV/
CZWltAD/F9Sz6TS35BtK/Sw/mPiWL7PoQuMg7ev/qFePNklg9bpJLziP+ZXQ8nTv
WN4cS4WmZdfx2XFKmlT7kqHokL3jMqItMFKq2imWFct1O4OfH7D1BjynS52taiKH
GrpQ2G3BJTRNhqiC2PJ1/zIRJSRIcItioz+hAXyKDKN3oq1/2K+UCbs7+FmTTfkl
...

-----END PGP MESSAGE-----

gpg --decrypt encrypt.txt

#### 1. Update the Changelog
- [Required files under the debian directory: changelog - Debian Docs](https://www.debian.org/doc/manuals/maint-guide/dreq.en.html#changelog)

dch -i

- debian/changelog

hello (2.10-2ubuntu3) jammy; urgency=medium

  * Initial release. (Closes: #nnnn)  <nnnn is the bug number of your ITP>
  * Add files.
  * New features.

 -- My Name <example.com>  Sun, 14 Apr 2024 22:35:50 +0800
 
...

#### 2. Rebuild Source Package with GPG Sign Key

debuild -S -sa  # -S: builds the source package
                # -sa: includes the original source

:::danger
**Using debuild -S to build source package changes!**
- *amd64.changes*: Binary package changes.
- *source.changes:* Source package changes. (We want this uploading to Launchpad.)
:::

#### 3-1. Setup FTP Method of Uploading to Launchpad's PPA

- ~/.dput.cf

[my-ppa]
fqdn = ppa.launchpad.net
method = ftp
incoming = ~ppa:<user>/ubuntu/<ppa>/
login = anonymous
allow_unsigned_uploads = 0

#### 3-2. Dput the Source Code Change onto Launchpad

dput --force ppa:<name>/<ppa> <package>_<version>_source.changes


:::warning
:warning: Although the log says "Successfully uploaded packages", it still can fail in the building process at Launchpad, if any problem occurs or the building is successful, Launchpad will send an email with detailed information.
:::

:::warning
:warning: Even if the email indicates the build process is ok, it still has a chance to fail on building.
:::

#### 4-1. Install Package from PPA

sudo add-apt-repository ppa:<user>/<ppa>

sudo apt update

:::warning
**If OpenPGP key on Launchpad isn't registered, you may unable to do PPA download operations.**

:::

#### 4-2. Install

sudo apt install hello


#### 4-3. Uninstall

sudo apt remove hello

## Useful Tools

### Debootstrap with Chroot - Isolated Environment
[Built-In] *debootstrap* is a tool which will install a Debian base system into a subdirectory of another, already installed system.

[Built-In] *chroot* provides file-system-level isolation, allowing to run processes in a restricted environment where they cannot access files outside their designated directory tree.

- [Debootstrap - Debian Wiki](https://wiki.debian.org/Debootstrap)

#### Install

sudo apt-get install debootstrap

#### Create

sudo debootstrap --arch=amd64 xenial /path/to/chroot http://archive.ubuntu.com/ubuntu/

#### Change Root

sudo chroot /path/to/chroot

### Debhelper (dh) - Helper Programs for `debian/rules`
[Built-In] A collection of programs that can be used in a `debian/rules` file to automate common tasks related to building Debian packages.

- [debhelper - Debian Packages](https://packages.debian.org/en/sid/debhelper)

> Programs are included to install various files into your package, compress files, fix file permissions, integrate your package with the Debian menu system, debconf, doc-base, etc. Most Debian packages use debhelper as part of their build process.

- ***dh_install:*** Installs files into the package directories.
- ***dh_auto_configure:*** Automatically configures the package build system.
- ***dh_auto_build:*** Automatically builds the package.
- ***dh_auto_test:*** Automatically runs tests for the package.
- ***dh_auto_install:*** Automatically installs the package into a temporary directory.
- ***dh_clean:*** Cleans up temporary files and directories.

### Lintian - Debian Package Checker
[Built-In] Lintian tries to check for Debian Policy violations and violations of various sub-policies, best practices, common mistakes, and problems that maintainers like to catch before uploads.

- [Lintian - Debian Wiki](https://wiki.debian.org/Lintian)

> Lintian is invoked at the end of *debuild -us -uc*.

Now running lintian hello_2.10-2ubuntu2_amd64.changes ...
W: hello source: configure-generated-file-in-source config.log
W: hello source: configure-generated-file-in-source config.status
W: hello-dbgsym: elf-error In program headers: Unable to find program interpreter name [usr/lib/debug/.build-id/b9/71970b0a1284236bad7994652e5644c44b54c3.debug]
W: hello: maintainer-script-empty [preinst]
W: hello: maintainer-script-ignores-errors [preinst]
W: hello source: maintainer-script-lacks-debhelper-token [debian/preinst]
Finished running lintian.

### Quilt - Patch Management
A management software for handling multiple patch files in Linux. It provides a way to manage changes to source code in a structured manner, making it easier to update, remove, or reapply patches as needed.

[Modifying the source: quilt - Debian Docs](https://www.debian.org/doc/manuals/maint-guide/modify.en.html)

#### Setup Custom *Dquilt*

- ~/.bashrc

...
alias dquilt="quilt --quiltrc=${HOME}/.quiltrc-dpkg"

# provide shell completion feature as same as quilt
. /usr/share/bash-completion/completions/quilt
complete -F _quilt_completion -o filenames dquilt

- ~/.quiltrc-dpkg

d=. ; while [ ! -d $d/debian -a $(readlink -e $d) != / ]; do d=$d/..; done
if [ -d $d/debian ] && [ -z $QUILT_PATCHES ]; then
    # if in Debian packaging tree with unset $QUILT_PATCHES
    QUILT_PATCHES="debian/patches"
    QUILT_PATCH_OPTS="--reject-format=unified"
    QUILT_DIFF_ARGS="-p ab --no-timestamps --no-index --color=auto"
    QUILT_REFRESH_ARGS="-p ab --no-timestamps --no-index"
    QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
    if ! [ -d $d/debian/patches ]; then mkdir $d/debian/patches; fi
fi

### Help2man - Automatic Generated Manual
A tool for automatically generating simple manual pages from program output.

- [help2man Reference Manual - GNU](https://www.gnu.org/software/help2man/)

### Texinfo
Texinfo uses a single source file to produce output in a number of formats (HTML, PDF, DVI, Info, DocBook, LaTeX, EPUB 3).

- [Texinfo - GNU](https://www.gnu.org/software/texinfo/)