---
tags: storage, curtin
---
# OSS Austin Storage Talk
* [OSS Summit Austin](https://events.linuxfoundation.org/open-source-summit-north-america/)
* [CFP](https://events.linuxfoundation.org/open-source-summit-north-america/program/cfp/) ends 11:59 PM PST on Sunday, February 16. [Submit here](https://linuxfoundation.smapply.io/prog/oss_elc_na_2020/)
* CFP Notifications: March 23, 2020
* Conference Mon June 22 - Wed June 24, 2020.

## Info for Proposal
* **Speaker smoser**
* Full Name: Scott Moser
* Job Title: Software Engineer
* Company: Cisco
* Email: [e-mail]
* Biography (provide a biography that includes your employer (if any), ongoing projects and your previous speaking experience).*
Scott Moser is a software engineer working for Cisco Systems. Scott started Cloud-init, and Curtin, and the CirrOS operating system. Scott has spoken at OSS Summit and OpenStack Developer Summits about cloud-init and containers.
* **Speaker rharper**
* Full Name: Ryan Harper
* Job Title: Staff Software Engineer
* Company: Canonical
* Email: [e-mail]
* Biography (provide a biography that includes your employer (if any), ongoing projects and your previous speaking experience).*

Ryan Harper is a Staff Software Engineer at Canonical as a member of the Ubuntu Server team. Ryan currently maintains the Curtin and Cloud-init opensource projects. Ryan has presented on Virtualization topics at OpenStack Developer Summits and KVM Forums.

* Abstract Title (*will be the Title shown in the conference schedule, often what attendees use as a starting point to determine if they will be interested in the talk*):
* Abstract:
*Provide an abstract that briefly summarizes your proposal. Provide as much information as possible about what the content will include. Do not be vague.*

*This is the description that will be posted on the website schedule if your talk is selected, so be sure to spell check, use complete sentences (and not just bullet points) and to write it in the third person (use your name instead of I).*

*Remember that this description is what will make an attendee decide whether your session would be a good fit for them. Be sure to provide enough information to help attendees make the right choice. Be clear and concise.*

*The presentation selection process is very competitive, with many proposals rejected. A well written abstract will greatly increase the possibility of the proposal being accepted.*

* Key Topic Area (select one):
* **Linux Systems**
* **Filesystems and Storage**
* Linux Kernel Development (Advanced & Beginner)
* Mission-Critical, Real-Time Operating Systems, Real-Time, and Long Life Systems, Scientific & Medical
* Programming Languages and Toolchains
* Networking
* RISC-V
* Tracing
* Scheduler
* VFIO/IOMMU/PCI
* Testing & Fuzzing
* System Boot and Security
* Live Patching
* BPF
* RDMA
* Power Management
* Containers and Checkpoint/Restore
* Distribution Kernels & Distros Considerations for Servers, Desktops, etc.
* Linux on the Desktop
* Open Source Databases
* Embedded Linux Conference (ELC)
* IoT
* Open Source Dependability
* Cloud Infrastructure
* Cloud App Developer Experience
* AI/ML/DL
* Introductory/101 Level - Embedded Essentials
* Introductory/101 Level - Linux Administration Essentials
* Introductory/101 Level - Cloud Administration Essentials
* Diversity Empowerment Summit
* Community & Business Leadership
* Open Source Program Office Management
* Wildcard

* Submission Type:
* **Session Presentation (40-50 minutes in length)**
* Panel Discussion (40-50 minutes in length)
* Birds of a Feather Session (BoFs are typically held in the evenings, 45 minutes to 1 hour in length)
* Tutorial (1.5- 2 hours in length)
* Lightning Talk (5-10 minutes in length; lightning talks are not a valid option for the Embedded Linux Conference or Introductory/101 Levels))

* Technical Talk: yes
* Have you or anyone else given this presentation before?
* If your session is accepted, would you be open to our PR team contacting you about speaking with media & press onsite?

* **Additional Resources**

The Program Committee reviews many proposals, and having extra resources helps to gauge the speaker's experience and expertise.

If you have not given this talk before, and have any links to previous talks showcasing you or any of your speakers, please share below. Other resources, like links to published books, LinkedIn profiles, or relevant blog posts, are also appreciated.

We welcome applications from new speakers, so don't worry if you don't have previous talk recordings to submit here. Even a YouTube video of yourself speaking for a couple of minutes could help the reviewers.

*there are options for Resource 1 and Resource 2. Each has two boxes, 'Type of Resoruce' and 'Details Links, Etc'. We'll ust use Scott as 1 andRyan as 2*

* Resource 1:
* Type of Resource: Previous Talks Scott.
* Details, Links, Etc:
* 2018 OSS Europe talk https://osseu18.sched.com/event/FxX8/
* 2016 Openstack Summit talk: https://youtu.be/SEZI-uw6A9Q

* Resource 2:
* Type of Resource: Previous Talks Ryan.
* Details, Links, Etc:
* 2016 Container Summit talk: https://youtu.be/UZZwP93GwvU
* 2015 Openstack Summit talk: https://youtu.be/90oxad2r8_E
* 2011 KVM Forum talk: https://youtu.be/bhdtvucvZ-4

## Notes

* General Goal of curtin: Go from one layout to another.

* Pain
* Linux has all kinds of magic discover that occurs when block devices are discovered. (multipath Partitions are type=disk block devices.
* lvm
* bcache
* systemd fsck
* dm-cache
* This is generally wonderful when that is what you want. Its not so wonderful if you
* udev discovery
* https://systemd.io/BLOCK_DEVICE_LOCKING/
* buried metadata
* partitioning tools (sgdisk, parted)
* uuids are not unique when you snapshot and re-attach. this goes for partition uuids, and fs uuids. What is a good way to find the right path to this thing. [fstab](https://wiki.ubuntu.com/FSTAB)
* try to make a ~~cloud~~ image that does the right thing... just have it boot and find / (how do you do that)
* ssd/nvme wipe

* Notes:
* `udevadm settle` often used like `sleep(1)`. would be better if udevadm settle could be done recursively.
* lsof/fuser doesn't work.... why can't I open this exclusively
* different kernel versions
* git log of curtin will provide many bug links, a signifcant portion of those are related to block deivce.
* * sysfs holders/slaves

* improvements
* masking devices that are held (dm-1 masks /dev/sdg)
* exclusive open failure returns the current exclusive owner
* udevadm settle --all-events-related-to = /dev/sda
* consistency in sysfs for stopping/releasing devices

* investigate other paas installers
* coreos ignition
* current fedora/anaconda installer
* atomic
* ironic
* Some mention of using ATA secure erase, which may take over 2 hours (it says that.. but it doesn't, my experience is maybe 2 minutes)
* https://ata.wiki.kernel.org/index.php/ATA_Secure_Erase
* wipe_disk.sh (trival)
* https://github.com/RHsyseng/hci/commit/9962912333d44ef43d6c67d7f3dae8771fc6523e#diff-9a6d8e921ad44cb885f3b3f126a9718a
* shred
*
* others

* create dirty disks and throw them to installer. what happens (cause errors in other installers).
* curtin A -> B, B -> C.
* curtin block-meta --config a.yaml # print storage tree
* curtin block-meta --config b.yaml

* Title/Summary (1200 chars for summary)
* IAAS/PAAS Storage Recyclying: Mastering linux block layer dependencies
* *Block Master: Stacking and unstacking the Linux storage layers.*

A. Even a "standard" linux installation now combines disks, partitions, logical volumes, and encryption before providing a simple /home directory. Stacking these layers is often non-trivial, and unstacking them can make unstacking ~~those tiny~~ LEGO pieces look easy. Come listen to developers of Curtin, the Ubuntu Server installer. They will share stacking and unstacking techniques, a war story or two, and some ideas to improve the situation from developers of Ubuntu Server's installer Curtin.

B. Not too long ago block and filesystem layer provisioning was both simple and rare. Hardware was set up once according to a guide and subsequently left untouched. Now we expect to enlist hardware, and stack disks, partitions, logical volumes, encryption, caching and file systems in minutes. The magic that makes all this stacking "just work" can be difficult to untangle when you need to repurpose disks in a system. A simple "Can I please have my disk back, dear linux?" request is anything but simple.
During development of Curtin (The curt installer for Ubuntu Server), Scott and Ryan have learned to stack and unstack and have the gray hair\^H\^H\^H\^H\^H\^H battle wounds to prove it.

The IAAS/PAAS landscape is ever changing requiring server storage configurations to be recycled from layout to the next. Optimizing storage configurations in a price per IOP involves layering caches over layers of aggregated storage. Separating these layers to be reused for the next configuration is complicated and typically involves hours of zeroing.