# CoreOS-Pipelines-Known-Failures

# Common failures

- `cosa build` sometimes tries to fetch packages that were already fetched by `cosa fetch`

14:59:41  Will download: 1 package (733.6?kB)
14:59:41  Downloading from 'fedora-coreos-pool'...done
14:59:41  [0m[31merror: [0mCannot download Packages/t/tzdata-2022g-1.fc37.noarch.rpm: All mirrors were tried; Last error: Curl error (6): Couldn't resolve host name for https://kojipkgs.fedoraproject.org/repos-dist/coreos-pool/latest/x86_64/Packages/t/tzdata-2022g-1.fc37.noarch.rpm [Could not resolve host: kojipkgs.fedoraproject.org]
14:59:41  error: failed to execute cmd-build: exit status 1

- aarch64: network infra flakes for quay.io cdn DNS
- https://github.com/coreos/fedora-coreos-pipeline/issues/852

dial tcp: lookup cdn03.quay.io: no such host

# RHCOS Specific Failures
- `openvswitch-hugetlbfs-groups` test fails on `ppc64le`.
- https://issues.redhat.com/browse/OCPBUGS-13057
- This happens sporadically and just needs to be rebuilt

- [4.15-9.2] legacy-oscontainer builds are failing on `ppc64le`.
- https://github.com/coreos/coreos-assembler/issues/3635

- [4.14-9.2] cloud-replicate job seems to due to image duplication error.
- https://github.com/coreos/coreos-assembler/issues/3634

# FCOS Specific Failures
- bump-lockfile races wrt secex volume
- https://github.com/coreos/fedora-coreos-pipeline/issues/835

- fcos.ignition.* kola tests time out on multiple cloud providers
- https://github.com/coreos/fedora-coreos-tracker/issues/1630

# Emoji Legend

## Jenkins App Emoji
- 🚅: Release Job
- ✨: New Build
- 🔥: Failed Build
- 🟩: Successful Build
- 🟨: Not all architectures succeeded the release job
- 🟥: Failed Build
- :jenkins: : job's Jenkins URL link
- 🌊: jobs's Blue Ocean URL link

## Reaction Emoji
- ❄️: some sort of test flake
- 🔥: The infrastructure is having trouble bringing up instances
- 🗡️: one of us killed the run for a reason (please add reason)
- :tag: koji tagger delay https://github.com/coreos/fedora-coreos-tracker/issues/1185
- ❄️ :rerunning: ✔️: test flake - rerun passed
- 🔒⛔: autolocking failed (inconsistent package set)
- :notebook: known/expected error

# Old issues (haven't seen in a while)
- `cosa buildextend-<platform>` can hit `Resource temporarily unavailable`
- Error: `'OS can't spawn a new worker thread: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }'`
- Issue: https://github.com/coreos/fedora-coreos-pipeline/issues/512
- kernel failure during kola-aws runs
- Error:`harness.go:1359: Found kernel warning (arch/x86/kernel/cpu/mce/intel.c:124 lmce_supported+0x51/0x60
) on machine i-0c4f1b8b0afcfa52f console` for kola-aws
- [Pipeline Link](https://jenkins-fedora-coreos-pipeline.apps.ocp.fedoraproject.org/job/kola-aws/257/)
- kernel failure during kola-gcp runs
- Error: `Found kernel warning (kernel/exit.c:738 do_exit+0x35/0xad0
) on machine kola-dbe0c2c77de11b3f7a80 console`
- [Pipeline Link](https://jenkins-fedora-coreos-pipeline.apps.ocp.fedoraproject.org/blue/organizations/jenkins/kola-gcp/detail/kola-gcp/154/pipeline)