HackMD
  • Prime
    Prime  Full-text search on all paid plans
    Search anywhere and reach everything in a Workspace with Prime plan.
    Got it
    • Prime  Full-text search on all paid plans
      Prime  Full-text search on all paid plans
      Search anywhere and reach everything in a Workspace with Prime plan.
      Got it
      • Options
      • Versions and GitHub Sync
      • Transfer ownership
      • Delete this note
      • Template
      • Insert from template
      • Export
      • Dropbox
      • Google Drive
      • Gist
      • Import
      • Dropbox
      • Google Drive
      • Gist
      • Clipboard
      • Download
      • Markdown
      • HTML
      • Raw HTML
      • Sharing Link copied
      • /edit
      • View mode
        • Edit mode
        • View mode
        • Book mode
        • Slide mode
        Edit mode View mode Book mode Slide mode
      • Note Permission
      • Read
        • Owners
        • Signed-in users
        • Everyone
        Owners Signed-in users Everyone
      • Write
        • Owners
        • Signed-in users
        • Everyone
        Owners Signed-in users Everyone
      • More (Comment, Invitee)
      • Publishing
        Everyone on the web can find and read all notes of this public team.
        After the note is published, everyone on the web can find and read this note.
        See all published notes on profile page.
      • Commenting Enable
        Disabled Forbidden Owners Signed-in users Everyone
      • Permission
        • Forbidden
        • Owners
        • Signed-in users
        • Everyone
      • Invitee
      • No invitee
    Menu Sharing Help
    Menu
    Options
    Versions and GitHub Sync Transfer ownership Delete this note
    Export
    Dropbox Google Drive Gist
    Import
    Dropbox Google Drive Gist Clipboard
    Download
    Markdown HTML Raw HTML
    Back
    Sharing
    Sharing Link copied
    /edit
    View mode
    • Edit mode
    • View mode
    • Book mode
    • Slide mode
    Edit mode View mode Book mode Slide mode
    Note Permission
    Read
    Owners
    • Owners
    • Signed-in users
    • Everyone
    Owners Signed-in users Everyone
    Write
    Owners
    • Owners
    • Signed-in users
    • Everyone
    Owners Signed-in users Everyone
    More (Comment, Invitee)
    Publishing
    Everyone on the web can find and read all notes of this public team.
    After the note is published, everyone on the web can find and read this note.
    See all published notes on profile page.
    More (Comment, Invitee)
    Commenting Enable
    Disabled Forbidden Owners Signed-in users Everyone
    Permission
    Owners
    • Forbidden
    • Owners
    • Signed-in users
    • Everyone
    Invitee
    No invitee
       owned this note    owned this note      
    Published Linked with GitHub
    Like BookmarkBookmarked
    Subscribed
    • Any changes
      Be notified of any changes
    • Mention me
      Be notified of mention me
    • Unsubscribe
    Subscribe
    # Fedora Project IAD2 datacenter application/service validation ###### tags: `IAD2 Datacenter` last updated: 2020-05-27 01:17UTC This document serves to provide a checklist/working document of services/applications setup in the new IAD2 datacenter. As services/applications/vm's are tested by their owners or others they can be checked off here and we know they are ready for the switch over from PHX2 datacenter on the week of 2020-06-08. issues here should have a infra ticket opened on them and/or discussed on list or irc. https://pagure.io/fedora-infrastructure/issues?tags=iad2 Please test if your application works, if anything is missing, if you can see anything that would break. Please also note if you are missing any command line history or files in your home dir that you expect/need, or cannot access things you normally do (sudo should work if it works in phx2, etc) ## testing limitations 1. I have dumped/restored databases for all services as of last week (2020-05-24) and restored the last db before bringing up the iad2 version. 2. Some hosts have local data (ex: pkgs02.phx2 has git repos on local disk). We will be rsyncing that data, and at cutover shutting down phx2 service, sync and bring up. 3. With only a few exceptions (where noted) playbooks complete on all iad2 hosts. 4. It will be of course impossible to fully test some apps before they are migrated. Thats fine, just do the best you can testing things so we have less to fix/do on move week. 5. a few things are not completed yet: - [ ] builders for koji - waiting for iscsi storage setup - [ ] resultsdb - waiting to decide where it goes - [ ] odcs - in progress - [ ] mbs - in progress - [ ] openqa hub and workers - waiting for qvmhosts / worker installs - [ ] sign-vault / autosign - waiting for machine installs - [ ] mailman - going to be migrated - [ ] notifs - going to be migrated - [ ] osbs - not yet deplyed (cverna) ## testing access 1. everyone should have the same access they do to machines in iad2 as they do to machines in phx2. Note that you need to put in your .ssh/config: <code> Host \*.iad2.fedoraproject.org 10.3.\*.\* HostName %h ProxyCommand ssh -q bastion-iad01.fedoraproject.org /usr/bin/nc %h 22 </code> 2. proxy01.iad2 is up and running, you may want to use your local /etc/hosts file to use it, (Cf point #5 below). This will allow you to hit iad2 hosts for testing 3. openshift is up and running in iad2. You may need to adjust your playbooks and run them to deploy your application, as of this writing only fas and ipsilon are running. 4. ansible-playbook can/should be run from batcave01.phx2 for now, we will move to batcave01.iad2 soon. 5. In order to test you can use a local /etc/hosts file to point your local brownser to iad2 instead of phx2: ```` 38.145.60.20 fedoraproject.org getfedora.org admin.fedoraproject.org apps.fedoraproject.org mirrors.fedoraproject.org os.fedoraproject.org console.app.os.fedoraproject.org app.os.fedoraproject.org id.fedoraproject.org qa.fedoraproject.org src.fedoraproject.org mdapi.fedoraproject.org greenwave.fedoraproject.org ```` 6. To commit in dist-git in iad, edit your ``/etc/rpkg/fedpkg.conf`` and replace the hostname currently using ``pkgs.fedoraproject.org`` by ``pkgs-iad.fedoraproject.org`` (do remember to remove this after testing to be able to access phx2/normal hosts again) ## a note on vpns Currently iad2 hosts are all using bastion01.iad2 as their vpn hub. This means they can talk to each other, but not any phx2 host. This is to allow for us to test things more easily. a few days before the move we are going to add all the iad2 hosts as 'hostname-iad2' with a 192.168.10.x ip into the vpn.fedoraproject.org dns, then connect them all to bastion01.phx2. This will allow us to switch services over by changing haproxy/apache/dns to point to the hostname-iad2 host instead of hostname. Once everything is moved at the end of the week, we will then repoint everything to bastion01.iad2 and switch names back. ## a note on things that aren't here Please note that due to limited cpu/vm/disk/other capacity, some services will just be down. They will come back after the machines move and are re-setup in iad2, most likely in late june / early july. There is no need to test or note that any of these are down/not here/not working * asknot * Fedocal * Badges * Nuancier * Anitya * elections * koschei * message-tagging-service * review-stats * All staging services (*.stg.fedoraproject.org) * simple-koji-ci * While not technically impacted by the move, we will turn it off so that we lessen the load on the builders Validation by host = Below are a list of applications that we need assistance validating, so if you have knowledge on running any of the below apps, please feel free to test them. If you find they are working properly, please check it off. If you find any of the below apps dont work correctly, please open a ticket and add the tag 'iad2' so we can investigate it. openshift apps: - [ ] bodhi - [ ] compose-tracker - [x] coreos-cincinnati - [x] Check for glaring anomalies in [metrics](https://status.updates.coreos.fedoraproject.org/metrics) - [ ] coreos-koji-tagger - [ ] distgit-bugzilla-sync - [ ] docsbuilding - [x] fas - interface up and running fine - [ ] greenwave - 2 of the 3 deployments worked, the last one is blocked by https://pagure.io/fedora-infrastructure/issue/8977 - [x] ipsilon - [ ] mdapi - Persistent storage issue: https://pagure.io/fedora-infrastructure/issue/8976 - [ ] messaging-bridges - [ ] monitor-gating - [ ] transtats - [x] waiverdb - [ ] websites baremetal machines: - [ ] autosign01 - [ ] sign-vault01 - [ ] aarch64 builders - [ ] ppc64le builders - [ ] access to s390x builders - [ ] backup server virtual machines: - [x] bastion01.iad2.fedoraproject.org - [ ] batcave01.iad2.fedoraproject.org - [ ] blockerbugs01.iad2.fedoraproject.org - [ ] bodhi-backend01.iad2.fedoraproject.org - [ ] bugzilla2fedmsg01.iad2.fedoraproject.org - [ ] busgateway01.iad2.fedoraproject.org - [ ] certgetter01.iad2.fedoraproject.org - [ ] compose-iot01.iad2.fedoraproject.org - [ ] compose-rawhide01.iad2.fedoraproject.org - [ ] compose-x86-01.iad2.fedoraproject.org - [x] datagrepper01.iad2.fedoraproject.org - [x] db01.iad2.fedoraproject.org | needs final sync when moving - [x] db03.iad2.fedoraproject.org | needs final sync when moving - [x] db-datanommer01.iad2.fedoraproject.org | needs final sync when moving - [x] db-fas01.iad2.fedoraproject.org | needs final sync when moving - [x] db-koji01.iad2.fedoraproject.org | needs final sync when moving - [x] dl01.iad2.fedoraproject.org - [x] dl02.iad2.fedoraproject.org - [x] dl03.iad2.fedoraproject.org - [ ] dl04.iad2.fedoraproject.org - [ ] dl05.iad2.fedoraproject.org - [ ] fedimg01.iad2.fedoraproject.org - [ ] github2fedmsg01.iad2.fedoraproject.org - [ ] ipa01.iad2.fedoraproject.org | not yet replicating - [ ] ipa02.iad2.fedoraproject.org | not yet replicating - [x] koji01.iad2.fedoraproject.org - [x] koji02.iad2.fedoraproject.org - [ ] kojipkgs01.iad2.fedoraproject.org - [ ] kojipkgs02.iad2.fedoraproject.org - [ ] log01.iad2.fedoraproject.org | rsyslog not working right - [ ] loopabull01.iad2.fedoraproject.org - [ ] mbs-backend01.iad2.fedoraproject.org - [ ] mbs-frontend01.iad2.fedoraproject.org - [ ] memcached01.iad2.fedoraproject.org - [ ] mm-backend01.iad2.fedoraproject.org - [ ] mm-crawler01.iad2.fedoraproject.org - [ ] mm-frontend01.iad2.fedoraproject.org - [ ] mm-frontend-checkin01.iad2.fedoraproject.org - [ ] noc01.iad2.fedoraproject.org - [x] ns01.iad2.fedoraproject.org - [x] ns02.iad2.fedoraproject.org - [ ] oci-candidate-registry01.iad2.fedoraproject.org - [ ] oci-registry01.iad2.fedoraproject.org - [x] os-control01.iad2.fedoraproject.org - [x] os-master01.iad2.fedoraproject.org - [x] os-master02.iad2.fedoraproject.org - [x] os-master03.iad2.fedoraproject.org - [x] os-node01.iad2.fedoraproject.org - [x] os-node02.iad2.fedoraproject.org - [x] os-node03.iad2.fedoraproject.org - [x] os-node04.iad2.fedoraproject.org - [x] os-node05.iad2.fedoraproject.org - [ ] pdc-backend01.iad2.fedoraproject.org - [ ] pdc-backend02.iad2.fedoraproject.org - [ ] pdc-backend03.iad2.fedoraproject.org - [ ] pdc-web01.iad2.fedoraproject.org - [ ] pkgs01.iad2.fedoraproject.org - [ ] proxy01.iad2.fedoraproject.org - [ ] proxy101.iad2.fedoraproject.org - [ ] proxy10.iad2.fedoraproject.org - [ ] proxy110.iad2.fedoraproject.org - [ ] rabbitmq01.iad2.fedoraproject.org | rabbitmq isn't clustering - [ ] rabbitmq02.iad2.fedoraproject.org | rabbitmq isn't clustering - [ ] rabbitmq03.iad2.fedoraproject.org | rabbitmq isn't clustering - [ ] secondary01.iad2.fedoraproject.org - [ ] sign-bridge01.iad2.fedoraproject.org - [ ] sundries01.iad2.fedoraproject.org - [x] tang01.iad2.fedoraproject.org - [x] tang02.iad2.fedoraproject.org - [ ] value01.iad2.fedoraproject.org - [x] wiki01.iad2.fedoraproject.org | works, but cannot login services: - [ ] packaging pipeline - [x] pkgs.fedoraproject.org clone works over ssh - [x] src.fedoraproject.org clone works over http - w/ SELinux off - [ ] src.fedoraproject.org login works - Strange state: failure - [ ] src.fedoraproject.org web interface works - [x] src.fedoraproject.org https push works - Hooks crash (py3 related) - [ ] src.fedoraproject.org forking works - [x] pkgs.fedoraproject.org ssh push works - [ ] gssapi auth to koji works - [ ] krb5 keytab auth to koji works - [ ] koji build works from src.fedoraproject.org - [ ] bodhi web interface loads - [ ] bodhi web new update works - [ ] waiverdb works - [ ] greenwave works - [ ] ci runs - [ ] bodhi command line new update works - [ ] bodhi updates composes work - [ ] updates sync from compose to ftp works - [ ] rawhide compose works - [x] openshift - passed all its tests fine. - [ ] mirrormanager - [ ] registry - [ ] fedmsg/fedora-messaging - [ ] pdc - [ ] value - [ ] meeting logs can be viewed - [ ] mote works - [ ] koji - [x] web interface - [ ] xmlrpc - [ ] builders checkin - [ ] packages and logs download (kojipkgs) - [ ] amazing service XYZ and its test plan

    Import from clipboard

    Advanced permission required

    Your current role can only read. Ask the system administrator to acquire write and comment permission.

    This team is disabled

    Sorry, this team is disabled. You can't edit this note.

    This note is locked

    Sorry, only owner can edit this note.

    Reach the limit

    Sorry, you've reached the max length this note can be.
    Please reduce the content or divide it to more notes, thank you!

    Import from Gist

    Import from Snippet

    or

    Export to Snippet

    Are you sure?

    Do you really want to delete this note?
    All users will lost their connection.

    Create a note from template

    Create a note from template

    Oops...
    This template is not available.


    Upgrade

    All
    • All
    • Team
    No template found.

    Create custom template


    Upgrade

    Delete template

    Do you really want to delete this template?

    This page need refresh

    You have an incompatible client version.
    Refresh to update.
    New version available!
    See releases notes here
    Refresh to enjoy new features.
    Your user state has changed.
    Refresh to load new user state.

    Sign in

    Forgot password

    or

    By clicking below, you agree to our terms of service.

    Sign in via Facebook Sign in via Twitter Sign in via GitHub Sign in via Dropbox Sign in via Google

    New to HackMD? Sign up

    Help

    • English
    • 中文
    • Français
    • Deutsch
    • 日本語
    • Español
    • Català
    • Ελληνικά
    • Português
    • italiano
    • Türkçe
    • Русский
    • Nederlands
    • hrvatski jezik
    • język polski
    • Українська
    • हिन्दी
    • svenska
    • Esperanto
    • dansk

    Documents

    Tutorials

    Book Mode Tutorial

    Slide Mode Tutorial

    YAML Metadata

    Contacts

    Facebook

    Twitter

    Feedback

    Send us email

    Resources

    Releases

    Pricing

    Blog

    Policy

    Terms

    Privacy

    Cheatsheet

    Syntax Example Reference
    # Header Header 基本排版
    - Unordered List
    • Unordered List
    1. Ordered List
    1. Ordered List
    - [ ] Todo List
    • Todo List
    > Blockquote
    Blockquote
    **Bold font** Bold font
    *Italics font* Italics font
    ~~Strikethrough~~ Strikethrough
    19^th^ 19th
    H~2~O H2O
    ++Inserted text++ Inserted text
    ==Marked text== Marked text
    [link text](https:// "title") Link
    ![image alt](https:// "title") Image
    `Code` Code 在筆記中貼入程式碼
    ```javascript
    var i = 0;
    ```
    var i = 0;
    :smile: :smile: Emoji list
    {%youtube youtube_id %} Externals
    $L^aT_eX$ LaTeX
    :::info
    This is a alert area.
    :::

    This is a alert area.

    Versions

    Versions and GitHub Sync

    Sign in to link this note to GitHub Learn more
    This note is not linked with GitHub Learn more
     
    Add badge Pull Push GitHub Link Settings
    Upgrade now

    Version named by    

    More Less
    • Edit
    • Delete

    Note content is identical to the latest version.
    Compare with
      Choose a version
      No search result
      Version not found

    Feedback

    Submission failed, please try again

    Thanks for your support.

    On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?

    Please give us some advice and help us improve HackMD.

     

    Thanks for your feedback

    Remove version name

    Do you want to remove this version name and description?

    Transfer ownership

    Transfer to
      Warning: is a public team. If you transfer note to this team, everyone on the web can find and read this note.

        Link with GitHub

        Please authorize HackMD on GitHub

        Please sign in to GitHub and install the HackMD app on your GitHub repo. Learn more

         Sign in to GitHub

        HackMD links with GitHub through a GitHub App. You can choose which repo to install our App.

        Push the note to GitHub Push to GitHub Pull a file from GitHub

          Authorize again
         

        Choose which file to push to

        Select repo
        Refresh Authorize more repos
        Select branch
        Select file
        Select branch
        Choose version(s) to push
        • Save a new version and push
        • Choose from existing versions
        Available push count

        Upgrade

        Pull from GitHub

         
        File from GitHub
        File from HackMD

        GitHub Link Settings

        File linked

        Linked by
        File path
        Last synced branch
        Available push count

        Upgrade

        Danger Zone

        Unlink
        You will no longer receive notification when GitHub file changes after unlink.

        Syncing

        Push failed

        Push successfully