# pulp-2to3-migration and FIPS
## Executive Summary
If you are:
* running Pulp3 on a FIPS-enabled machine, **and**
* you have patched Django and Pulp3 with the correct patches, **and**
* you have "md5" and "sha1" included in your ALLOWED_CHECKSUMS, **then** :
No matter what's in your Pulp2 database, it will migrate successfully.
## Issues:
https://pulp.plan.io/issues/7782
https://pulp.plan.io/issues/8453
## Patches:
Django 3.2 patch:
https://github.com/theforeman/pulpcore-packaging/pull/272
Django 2.2 LTS :
https://github.com/theforeman/pulpcore-packaging/blob/rpm/3.7/packages/python-django/0001-FIPS-Mark-use-of-MD5-not-security-relevant.patch
```
wget https://raw.githubusercontent.com/theforeman/pulpcore-packaging/rpm/3.7/packages/python-django/0001-FIPS-Mark-use-of-MD5-not-security-relevant.patch
patch -d /usr/local/lib/pulp/lib/python3.6/site-packages/ -p1 <0001-FIPS-Mark-use-of-MD5-not-security-relevant.patch
```
Pulp3: https://gist.github.com/bmbouter/31c45dac7de68eeccc35a9f9564c0f28
```
wget https://gist.githubusercontent.com/bmbouter/31c45dac7de68eeccc35a9f9564c0f28/raw/de9bda4aed4dd6ed72ac38ead1d0db7629a0c13c/patch_pulp_md5_usedforsecurity.diff
patch -p1 -d /home/vagrant/devel/pulpcore/<patch_pulp_md5_usedforsecurity.diff
```
Without patching Django, you can't even restart Pulp services:
```
(pulp) [vagrant@pulp2-nightly-pulp3-source-fips-a ~]$ pclean
systemctl stop pulpcore-content pulpcore-worker@1 pulpcore-worker@2 pulpcore-resource-manager pulpcore-api
Traceback (most recent call last):
File "/usr/local/lib/pulp/bin/pulpcore-manager", line 33, in <module>
sys.exit(load_entry_point('pulpcore', 'console_scripts', 'pulpcore-manager')())
File "/home/vagrant/devel/pulpcore/pulpcore/app/manage.py", line 11, in manage
execute_from_command_line(sys.argv)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py", line 357, in execute
django.setup()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py", line 114, in populate
app_config.import_models()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/config.py", line 211, in import_models
self.models_module = import_module(models_module_name)
File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/guardian/models/__init__.py", line 1, in <module>
from .models import (
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/guardian/models/models.py", line 69, in <module>
class UserObjectPermission(UserObjectPermissionAbstract):
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 315, in __new__
new_class._prepare()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 367, in _prepare
index.set_name_with_model(cls)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/indexes.py", line 116, in set_name_with_model
'%s_%s' % (names_digest(*hash_data, length=6), self.suffix),
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 221, in names_digest
h = hashlib.md5()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
```
fips-a is a dev-box, and therefore comes pre-patched for the Django/MD5 issue (the patch touches the same code in a slightly-different way):
https://github.com/pulp/pulp_installer/commit/ae47f49ef9b79cd79851d0caa245559ceabbc7ce#diff-8332bf7262838ddb1ec95ae3a42460666336878f00cf2de7bbc5d5f39f466401R245
## Variants:
### Pulp3
1. **UNP**: without pulp3 md5 patch
1. **PAT**: with pulp3 md5 patch
1. **ALL**: ACC includes md5/sha1
1. **SANE**: ACC excludes md5/sha1
#### Combinations:
1. **UNP-ALL** : no patch, all checksums
1. **UNP-SANE** : no patch, sane checksums
1. **PAT-ALL** : patched, all checksums
1. **PAT-SANE** : patched, sane checksums
### Pulp2
1. **+MD5**: pulp2 data includes md5-only repo
1. **-MD5**: pulp2 data excludes md5-only repo
1. **IM**: pulp2 data sync'd immediate
1. **OD**: pulp2 data sync'd on_demand
#### Combinations
1. **+MD5-IM** : MD5 data, with content
1. **-MD5-IM** : no MD5 data, with content
1. **+MD5-OD** : MD5 data, no content
1. **-MD5-OD** : no MD5 data, no content
### Scripts
#### Pulp2 content
```shell=
#Create MD5 repo, immediate
pulp-admin login -u admin -p admin
pulp-admin rpm repo create --repo-id md5 --feed https://fixtures.pulpproject.org/rpm-with-md5/ --download-policy immediate
pulp-admin rpm repo sync run --repo-id md5
```
```shell=
#Create 'normal' repo, immediate
pulp-admin rpm repo create --repo-id rpm --feed https://fixtures.pulpproject.org/rpm-signed/ --download-policy immediate
pulp-admin rpm repo sync run --repo-id rpm
```
```shell=
#Create MD5 repo, on-demand
pulp-admin login -u admin -p admin
pulp-admin rpm repo create --repo-id md5 --feed https://fixtures.pulpproject.org/rpm-with-md5/ --download-policy on_demand
pulp-admin rpm repo sync run --repo-id md5
```
```shell=
#Create 'normal' repo, on-demand
pulp-admin rpm repo create --repo-id rpm --feed https://fixtures.pulpproject.org/rpm-signed/ --download-policy on_demand
pulp-admin rpm repo sync run --repo-id rpm
```
```shell=
#CLEANUP
pulp-admin rpm repo delete --repo-id rpm
pulp-admin rpm repo delete --repo-id md5
pulp-admin orphan remove --all
```
#### Pulp3 migrate
```shell=
# Create/run migration
export HREF=$(pulp migration plan create --plan '{"plugins": [{"type": "rpm"}]}' | jq -r '.pulp_href')
pulp migration plan run --href $HREF
```
## Test Matrix
| |UNP/ALL|UNP/SANE|PAT/ALL|PAT/SANE|
|:-------:|:-----:|:------:|:-----:|:------:|
| +MD5/IM | PASS | FAIL[0]| PASS |FAIL[0] |
| -MD5/IM | PASS | PASS | PASS | PASS |
| +MD5/OD | PASS | FAIL[1]| PASS | PASS |
| -MD5/OD | PASS | PASS | PASS | PASS |
## Notes
https://pulp.plan.io/issues/8453 is the result of running unpatched-django in a FIPS environment.
[0] Migration fails, EXPECTED:
```
(pulp) [vagrant@pulp2-nightly-pulp3-source-fips-a site-packages]$ pulp migration plan run --href $HREF
Started background task /pulp/api/v3/tasks/f6735a2a-12db-4b6d-a227-e55423759250/
..Error: Task /pulp/api/v3/tasks/f6735a2a-12db-4b6d-a227-e55423759250/ failed: 'Checksum algorithm md5 forbidden for this Pulp instance.'
(pulp) [vagrant@pulp2-nightly-pulp3-source-fips-a site-packages]$
```
[1] Migration fails, EXPECTED:
```
(pulp) [vagrant@pulp2-nightly-pulp3-source-fips-a site-packages]$ pulp migration plan run --href $HREF
Started background task /pulp/api/v3/tasks/40394899-04c6-4dc3-b82a-1412dda2b335/
..Error: Task /pulp/api/v3/tasks/40394899-04c6-4dc3-b82a-1412dda2b335/ failed: 'On-demand content located at the url https://fixtures.pulpproject.org/rpm-with-md5/zebra-0.1-2.noarch.rpm contains forbidden checksum type,thus cannot be synced.You can allow checksum type with 'ALLOWED_CONTENT_CHECKSUMS' setting.'
```
## What about 3.7?
If you want to run Pulp3.7 on a FIPS-enabled box, you will need to apply daviddavis' additional patch, that creates/enables the code for the pulp3-md5-patch above:
https://gist.github.com/daviddavis/9a819fae1b18595169c1ed38d1dc72df
**In order**, you will need to apply:
1. the [django-fips-patch](https://github.com/theforeman/pulpcore-packaging/blob/rpm/3.7/packages/python-django/0001-FIPS-Mark-use-of-MD5-not-security-relevant.patch)
2. [daviddavis' patch](https://gist.github.com/daviddavis/9a819fae1b18595169c1ed38d1dc72df)
3. [Pulp3-MD5-enabling patch](https://gist.github.com/bmbouter/31c45dac7de68eeccc35a9f9564c0f28)
To get daviddavis' patch to apply cleanly under pulpcore-3.7, I used the following:
```
wget https://gist.githubusercontent.com/daviddavis/9a819fae1b18595169c1ed38d1dc72df/raw/eb26da1d34c8be69f25e75f083775c1f89ad28bd/hashlib.patch
patch --fuzz 3 -p1 -d /home/vagrant/devel/pulpcore/ -i patch_pulp_md5_usedforsecurity.diff
```
on a system running pulpcore-3.7, pulp-rpm-3.10, and pulp-2to3-migration-0.9.0.
These three patches enabled me to successfully migrate an MD5-repo from my Pulp2 box into my Pulp3.7 FIPS-enabled system.
###### tags: `FIPS`
Sign in with Wallet
Connect another wallet