# pulp-2to3-migration and FIPS ## Executive Summary If you are: * running Pulp3 on a FIPS-enabled machine, **and** * you have patched Django and Pulp3 with the correct patches, **and** * you have "md5" and "sha1" included in your ALLOWED_CHECKSUMS, **then** : No matter what's in your Pulp2 database, it will migrate successfully. ## Issues: https://pulp.plan.io/issues/7782 https://pulp.plan.io/issues/8453 ## Patches: Django 3.2 patch: https://github.com/theforeman/pulpcore-packaging/pull/272 Django 2.2 LTS : https://github.com/theforeman/pulpcore-packaging/blob/rpm/3.7/packages/python-django/0001-FIPS-Mark-use-of-MD5-not-security-relevant.patch ``` wget https://raw.githubusercontent.com/theforeman/pulpcore-packaging/rpm/3.7/packages/python-django/0001-FIPS-Mark-use-of-MD5-not-security-relevant.patch patch -d /usr/local/lib/pulp/lib/python3.6/site-packages/ -p1 <0001-FIPS-Mark-use-of-MD5-not-security-relevant.patch ``` Pulp3: https://gist.github.com/bmbouter/31c45dac7de68eeccc35a9f9564c0f28 ``` wget https://gist.githubusercontent.com/bmbouter/31c45dac7de68eeccc35a9f9564c0f28/raw/de9bda4aed4dd6ed72ac38ead1d0db7629a0c13c/patch_pulp_md5_usedforsecurity.diff patch -p1 -d /home/vagrant/devel/pulpcore/<patch_pulp_md5_usedforsecurity.diff ``` Without patching Django, you can't even restart Pulp services: ``` (pulp) [vagrant@pulp2-nightly-pulp3-source-fips-a ~]$ pclean systemctl stop pulpcore-content pulpcore-worker@1 pulpcore-worker@2 pulpcore-resource-manager pulpcore-api Traceback (most recent call last): File "/usr/local/lib/pulp/bin/pulpcore-manager", line 33, in <module> sys.exit(load_entry_point('pulpcore', 'console_scripts', 'pulpcore-manager')()) File "/home/vagrant/devel/pulpcore/pulpcore/app/manage.py", line 11, in manage execute_from_command_line(sys.argv) File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line utility.execute() File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py", line 357, in execute django.setup() File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py", line 114, in populate app_config.import_models() File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/config.py", line 211, in import_models self.models_module = import_module(models_module_name) File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 994, in _gcd_import File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 665, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 678, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/usr/local/lib/pulp/lib64/python3.6/site-packages/guardian/models/__init__.py", line 1, in <module> from .models import ( File "/usr/local/lib/pulp/lib64/python3.6/site-packages/guardian/models/models.py", line 69, in <module> class UserObjectPermission(UserObjectPermissionAbstract): File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 315, in __new__ new_class._prepare() File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 367, in _prepare index.set_name_with_model(cls) File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/indexes.py", line 116, in set_name_with_model '%s_%s' % (names_digest(*hash_data, length=6), self.suffix), File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 221, in names_digest h = hashlib.md5() ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS ``` fips-a is a dev-box, and therefore comes pre-patched for the Django/MD5 issue (the patch touches the same code in a slightly-different way): https://github.com/pulp/pulp_installer/commit/ae47f49ef9b79cd79851d0caa245559ceabbc7ce#diff-8332bf7262838ddb1ec95ae3a42460666336878f00cf2de7bbc5d5f39f466401R245 ## Variants: ### Pulp3 1. **UNP**: without pulp3 md5 patch 1. **PAT**: with pulp3 md5 patch 1. **ALL**: ACC includes md5/sha1 1. **SANE**: ACC excludes md5/sha1 #### Combinations: 1. **UNP-ALL** : no patch, all checksums 1. **UNP-SANE** : no patch, sane checksums 1. **PAT-ALL** : patched, all checksums 1. **PAT-SANE** : patched, sane checksums ### Pulp2 1. **+MD5**: pulp2 data includes md5-only repo 1. **-MD5**: pulp2 data excludes md5-only repo 1. **IM**: pulp2 data sync'd immediate 1. **OD**: pulp2 data sync'd on_demand #### Combinations 1. **+MD5-IM** : MD5 data, with content 1. **-MD5-IM** : no MD5 data, with content 1. **+MD5-OD** : MD5 data, no content 1. **-MD5-OD** : no MD5 data, no content ### Scripts #### Pulp2 content ```shell= #Create MD5 repo, immediate pulp-admin login -u admin -p admin pulp-admin rpm repo create --repo-id md5 --feed https://fixtures.pulpproject.org/rpm-with-md5/ --download-policy immediate pulp-admin rpm repo sync run --repo-id md5 ``` ```shell= #Create 'normal' repo, immediate pulp-admin rpm repo create --repo-id rpm --feed https://fixtures.pulpproject.org/rpm-signed/ --download-policy immediate pulp-admin rpm repo sync run --repo-id rpm ``` ```shell= #Create MD5 repo, on-demand pulp-admin login -u admin -p admin pulp-admin rpm repo create --repo-id md5 --feed https://fixtures.pulpproject.org/rpm-with-md5/ --download-policy on_demand pulp-admin rpm repo sync run --repo-id md5 ``` ```shell= #Create 'normal' repo, on-demand pulp-admin rpm repo create --repo-id rpm --feed https://fixtures.pulpproject.org/rpm-signed/ --download-policy on_demand pulp-admin rpm repo sync run --repo-id rpm ``` ```shell= #CLEANUP pulp-admin rpm repo delete --repo-id rpm pulp-admin rpm repo delete --repo-id md5 pulp-admin orphan remove --all ``` #### Pulp3 migrate ```shell= # Create/run migration export HREF=$(pulp migration plan create --plan '{"plugins": [{"type": "rpm"}]}' | jq -r '.pulp_href') pulp migration plan run --href $HREF ``` ## Test Matrix | |UNP/ALL|UNP/SANE|PAT/ALL|PAT/SANE| |:-------:|:-----:|:------:|:-----:|:------:| | +MD5/IM | PASS | FAIL[0]| PASS |FAIL[0] | | -MD5/IM | PASS | PASS | PASS | PASS | | +MD5/OD | PASS | FAIL[1]| PASS | PASS | | -MD5/OD | PASS | PASS | PASS | PASS | ## Notes https://pulp.plan.io/issues/8453 is the result of running unpatched-django in a FIPS environment. [0] Migration fails, EXPECTED: ``` (pulp) [vagrant@pulp2-nightly-pulp3-source-fips-a site-packages]$ pulp migration plan run --href $HREF Started background task /pulp/api/v3/tasks/f6735a2a-12db-4b6d-a227-e55423759250/ ..Error: Task /pulp/api/v3/tasks/f6735a2a-12db-4b6d-a227-e55423759250/ failed: 'Checksum algorithm md5 forbidden for this Pulp instance.' (pulp) [vagrant@pulp2-nightly-pulp3-source-fips-a site-packages]$ ``` [1] Migration fails, EXPECTED: ``` (pulp) [vagrant@pulp2-nightly-pulp3-source-fips-a site-packages]$ pulp migration plan run --href $HREF Started background task /pulp/api/v3/tasks/40394899-04c6-4dc3-b82a-1412dda2b335/ ..Error: Task /pulp/api/v3/tasks/40394899-04c6-4dc3-b82a-1412dda2b335/ failed: 'On-demand content located at the url https://fixtures.pulpproject.org/rpm-with-md5/zebra-0.1-2.noarch.rpm contains forbidden checksum type,thus cannot be synced.You can allow checksum type with 'ALLOWED_CONTENT_CHECKSUMS' setting.' ``` ## What about 3.7? If you want to run Pulp3.7 on a FIPS-enabled box, you will need to apply daviddavis' additional patch, that creates/enables the code for the pulp3-md5-patch above: https://gist.github.com/daviddavis/9a819fae1b18595169c1ed38d1dc72df **In order**, you will need to apply: 1. the [django-fips-patch](https://github.com/theforeman/pulpcore-packaging/blob/rpm/3.7/packages/python-django/0001-FIPS-Mark-use-of-MD5-not-security-relevant.patch) 2. [daviddavis' patch](https://gist.github.com/daviddavis/9a819fae1b18595169c1ed38d1dc72df) 3. [Pulp3-MD5-enabling patch](https://gist.github.com/bmbouter/31c45dac7de68eeccc35a9f9564c0f28) To get daviddavis' patch to apply cleanly under pulpcore-3.7, I used the following: ``` wget https://gist.githubusercontent.com/daviddavis/9a819fae1b18595169c1ed38d1dc72df/raw/eb26da1d34c8be69f25e75f083775c1f89ad28bd/hashlib.patch patch --fuzz 3 -p1 -d /home/vagrant/devel/pulpcore/ -i patch_pulp_md5_usedforsecurity.diff ``` on a system running pulpcore-3.7, pulp-rpm-3.10, and pulp-2to3-migration-0.9.0. These three patches enabled me to successfully migrate an MD5-repo from my Pulp2 box into my Pulp3.7 FIPS-enabled system. ###### tags: `FIPS`
Last changed by  
 
Pulp
Design/Discussion/Meeting minutes for https://pulpproject.org/
757

Read more

CLI Team

https://github.com/pulp/pulp-cli/pulls

4/24/2024
Pulp Services Deployment Meetings

Edge Mgmt Road Map View - https://issues.redhat.com/secure/PortfolioPlanView.jspa?id=1411&sid=1416&vid=8158#plan/backlog

4/24/2024
Deployers Team Meeting

Triage Query:https://github.com/pulp/pulp-oci-images/issues?q=is%3Aissue+is%3Aopen+label%3ATriage-Neededhttps://github.com/pulp/pulp-operator/issues?q=is%3Aissue+is%3Aopen+label%3ATriage-Needed

4/23/2024
Open Floor Agenda

Pulp open floor, every Tuesday at 10:30 ET (either EST or EDT) (That is 16:30 CET) in pulp-meeting on Matrix.See https://pulpproject.org/get_involved/ for more info about this meeting.

4/23/2024
Read more from Pulp
Published on HackMD