###### tags: `data centre move`

# Current Live updated schedule for PHX2 -> IAD2 move

## Acronyms Used in Document
MVFE - Minimal Viable Fedora Environment
PHX2 - Chandler Arizona datacentre
RDU-CC - Morrisville NC datacentre community cage
IAD2 - Ashburn Equinix datacentre
AAA - Authentication, Authorization, Auditing
IPA - Identity, Policy, Audit
FAS - Fedora Account System (older AAA solution)
FreeIPA - Newer account system

#### June 15th Fedora to IAD

**note:** it looks like it will take 2 to 3 days to unrack the systems in PHX2. it will also take 1 week for the hardware to travel across the country.

## Phase 1

### Week 00 (2020-03-02 -> 2020-03-08)
- [x] Hardware shipping needs to be planned out
- [x] Rack layouts for IAD2 and RDU-CC need to be finalized
- [x] Work with RH IT on what they need in network diagrams

### Week 01 (2020-03-09 -> 2020-03-15)
- [x] Get RHEL-8 virthost built
- [x] Deliver network diagrams to RHIT
- [x] Announce at downtime for communishift
- [x] ~~Try to find a place for communishift to temp run? - no~~

### Week 02 (2020-03-16 -> 2020-03-22)
- [x] 2020-03-17 Fedora 32 Beta Date

### Week 03 (2020-03-23 -> 2020-03-29)
- [x] Re-Announce downtime for communishift
- [x] Get updated tasks and timeline set out
- [x] Work out COVID-19 Contingencies

### Week 04 (2020-03-30 -> 2020-04-05)
- [x] Virthost03 skunkworks (kevin)
- [x] RHEL-8 install
- [x] test encryption over bridge
- [x] test TEAM network
- [x] Virthost05 skunkworks (kevin)
- [x] uefi EL8 install instructions
- [x] secure boot.
- [x] Re-Announce downtime for communishift
- [x] Set up noggin instance in AWS to replace communishift (kevin)

### Week 05 (2020-04-06 -> 2020-04-12)
- [x] Create DNS templates for IAD2
- [x] Create template DHCP for mgmt hosts
- [x] Add more items to ship to IAD2
- [x] Get network layout for RDU-CC finalized
- [x] public ip address count and space needed
- [x] mgmt network 172.23.1.??/24
- [x] private network for openshift backnodes 172.23.2.??/24
- [x] Collect all mgmt mac addresses
- [x] Collect all hardware mac addresses for debugging
- [x] Set mgmt interfaces to DHCP before getting shipped
- [x] opengear
- [x] amperes
- [ ] power hardware
- [x] 2020-04-09 Fedora 32 Final Freeze
- [x] Power needs to be ready in RDU-CC cage
- [x] Option A/B meeting for Phase 2
- [x] Option A - We ship items and continue with A
- [ ] Option B - We go with planning out move of items within PHX2 and look at the larger move to IAD2 next year
## Phase 2

### Week 06 (2020-04-13 -> 2020-04-19)
- [x] Put move details into status.fedoraproject.org so people can see what is going on.
- [x] Take down and ship communishift hardware to RDU-CC
- [x] Take down and ship extra hardware to IAD2
- [x] Begin takedown of communishift hardware
- [x] Communishift 13th April - 1st May
- [x] Power off systems in racks
- [x] Work with logistics for pack and move
- [x] Power should be on by 17th - this is power to RDU-CC

### Week 07 (2020-04-20 -> 2020-04-26)
- [x] Hardware should arrive at RDU-CC
- [x] Rerack from 27th 1st may
- [x] Hardware should arrive at IAD2
- [x] work with Shauns team to get systems racked/stacked in 101

### Week 08 (2020-04-27 -> 2020-05-03)
- [x] Work out temporary root password for installs
- [x] 2020-04-28 Fedora 32 release
- ~~[ ] set ip address in mac~~
- ~~[ ] set up admin user~~
- ~~[ ] set ipmi and serial over lan access~~

### Week 09 (2020-05-04 -> 2020-05-10)
- [ ] RDU Bootstrap
- [x] Write howto on Dell mgmt setup (smooge)
- [ ] IAD2 work (see IAD2 bootstrap)
- [ ] Work with IT on any network layout issues left for RDU2 site.
- [ ] communishift proxies with private + external interfaces
- [ ] Move final virthost-cc boxes into new RDU-CC racks
- [ ] Give internal IT new DNS servers ip address
- [ ] Give internal IT new SMTP server ip address

### Week 10 (2020-05-11 -> 2020-05-17)
- [ ] IAD2 work (see IAD2 bootstrap)
- [ ] Do any items in RDU-CC that time allows

### Week 11 (2020-05-18 -> 2020-05-24)
- [ ] IAD2 work (see IAD2 bootstrap)
- [ ]Checkin meeting:
- [ ] Option A: Ship everything left in PHX2 to IAD2 on June 15th
- [ ] Option B: Need more time and ship on June 30

### Week 12 (2020-05-25 -> 2020-05-31)
- [ ] IAD2 work (see IAD2 bootstrap)
- [ ] Rack 103 must be up. Get openqa and other systems in.

## Phase 4

### Week 13 (2020-06-01 -> 2020-06-07)
- [ ] Final checklist and approval of IAD2 MVFE
- [ ] Test email routing through IAD2 proxies
- [ ] Test www proxies
- [ ] Test builds
- [ ] Test openvpn
- [ ] Test rsync
- [ ] Test route to s390x
- [ ] Test route to bugzilla STOMP message bus
- [ ] Bring up openqa in IAD2
- [ ] Change Fedora DNS to shorter times for major change
- [ ] GO/NO-GO meeting:
- [ ] Option A: Ship everything left in PHX2 to IAD2 on June 15th
- [ ] Option B: Look at internal move of PHX2 equipment

### Week 14 (2020-06-08 -> 2020-06-14)
- [ ] Move logical infrastructure to IAD2 MVFE
- [ ] src, koji hubs, koji db, bodhi, downloads all move in one day outage
- [ ] particular applications can move one at a time with their deps
- [ ] Shutdown PHX2 koji and related dbs
- [ ] Do final dump
- [ ] Import data into IAD2 dbs
- [ ] Turn DNS to point to IAD2
- [ ] Move openvpn to IAD2
- [ ] Begin cleaning PHX2 systems as possible
- [ ] Reinstall with base EL8?
- [ ] Power off PHX2 systems as possible

### Week 15 (2020-06-15 -> 2020-06-21)
- [ ] Shutdown of PHX2 racks
- [ ] Removal of systems from PHX2 and shipment to IAD2
- [ ] Travel of equipment to IAD2
- [ ] PHX2 -- go over remaining hardware to recycle

### Week 16 (2020-06-22 -> 2020-06-28)
- [ ] Most likely time for hardware arrival
- [ ] Racking and stacking of IAD2 equipment
- [ ] Set up mgmt interfaces
- [ ] Do initial hardware installs to RHEL8

### Week 17 (2020-06-29 -> 2020-07-05)
- [ ] Finish initial hardware installs
- [ ] Bring up additional builders

### Week 18 (2020-07-06 -> 2020-07-12)
- [ ] Bring up additional services
- [ ] Move mgmt interfaces back to static

### Week 19 (2020-07-13 -> 2020-07-19)
- [ ] Probably more work at data centre

### Week 20 (2020-07-20 -> 2020-07-26)
- [ ] Sign off on work completed

### Week 21 (2020-07-27 -> 2020-08-02)
- [ ] A miracle occurs *Several in fact*

### Week 22 (2020-08-03 -> 2020-08-09) ???
- [ ] Mass Rebuild for Fedora 33 starts
- [ ] All systems must be up and running
- [ ] Production needs to be normal
- [ ] FlockToFedora 2020