---
# System prepended metadata

title: Untitled

---

## Is it possible to repair a failed drive?
Maybe issues with NVME firmware, motherboard/BIOS or M.2 slot?
We need functional NVME (0) with our data.

## Log files about drives health:

> Failed drive shows:
> FW Rev ERRORMOD
> 
```
root@rescue:~# nvme list
Node             SN                   Model                                    Namespace Usage                      Format           FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1     S437NA0N515103       SAMSUNG MZQLB960HAJR-00007               1           1.07  GB /   1.07  GB    512   B +  0 B   ERRORMOD
/dev/nvme1n1     S437NA0N515105       SAMSUNG MZQLB960HAJR-00007               1         581.61  GB / 960.20  GB    512   B +  0 B   EDA5202Q
```

### Healthy NVME (1)
> Available Spare: 100%
```
root@rescue:~# smartctl -a /dev/nvme1
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-5.10.18-mod-std] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZQLB960HAJR-00007
Serial Number:                      S437NA0N515105
Firmware Version:                   EDA5202Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 960,197,124,096 [960 GB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          960,197,124,096 [960 GB]
Namespace 1 Utilization:            581,611,520,000 [581 GB]
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Sat Dec 18 12:16:19 2021 EST
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f):   Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     87 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +    10.60W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0
 1 -    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:                   0x00
Temperature:                        34 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    146,521,243 [75.0 TB]
Data Units Written:                 69,510,790 [35.5 TB]
Host Read Commands:                 2,923,338,410
Host Write Commands:                1,144,693,075
Controller Busy Time:               4,503
Power Cycles:                       37
Power On Hours:                     13,075
Unsafe Shutdowns:                   30
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               34 Celsius
Temperature Sensor 2:               38 Celsius
Temperature Sensor 3:               42 Celsius
```

### Failed NVME (0)
> Available Spare: 0%, Percentage Used:                    255%
```
root@rescue:~# smartctl -a /dev/nvme0
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-5.10.18-mod-std] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZQLB960HAJR-00007
Serial Number:                      S437NA0N515103
Firmware Version:                   ERRORMOD
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 960,197,124,096 [960 GB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          1,073,741,824 [1.07 GB]
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Sat Dec 18 12:16:03 2021 EST
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f):   Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     87 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +    10.60W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0
 1 -    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:                   0x00
Temperature:                        -
Available Spare:                    0%
Available Spare Threshold:          10%
Percentage Used:                    255%
Data Units Read:                    0
Data Units Written:                 0
Host Read Commands:                 0
Host Write Commands:                0
Controller Busy Time:               0
Power Cycles:                       0
Power On Hours:                     0
Unsafe Shutdowns:                   0
Media and Data Integrity Errors:    0
Error Information Log Entries:      4
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
```
```
root@rescue:~# nvme smart-log /dev/nvme0n1
Smart Log for NVME device:nvme0n1 namespace-id:ffffffff
critical_warning                    : 0
temperature                         : 4294967023 C
available_spare                     : 0%
available_spare_threshold           : 10%
percentage_used                     : 255%
data_units_read                     : 0
data_units_written                  : 0
host_read_commands                  : 0
host_write_commands                 : 0
controller_busy_time                : 0
power_cycles                        : 0
power_on_hours                      : 0
unsafe_shutdowns                    : 0
media_errors                        : 0
num_err_log_entries                 : 4
Warning Temperature Time            : 0
Critical Composite Temperature Time : 0
Temperature Sensor 1                : 0 C
Temperature Sensor 2                : 0 C
Temperature Sensor 3                : 0 C
Temperature Sensor 4                : 0 C
Temperature Sensor 5                : 0 C
Temperature Sensor 6                : 0 C
Temperature Sensor 7                : 0 C
Temperature Sensor 8                : 0 C
```

### LVM errors:
```
root@rescue:~# pvs
  Couldn't find device with uuid qUelxM-FXi6-HduD-m6A4-VVfa-61eF-tJ7h8A.
  PV             VG      Fmt  Attr PSize   PFree
  /dev/nvme1n1p1 Storage lvm2 a--  894.25g 294.25g
  unknown device Storage lvm2 a-m  592.78g 147.78g
```
```
root@rescue:~# vgchange -ay Storage
  Couldn't find device with uuid qUelxM-FXi6-HduD-m6A4-VVfa-61eF-tJ7h8A.
  Refusing activation of partial LV Storage/vsv4765-d0xv6p2u8yc28vqk-gp7socpfcgu1d8rd.  Use '--activationmode partial' to override.
  Refusing activation of partial LV Storage/vsv4770-d1uhqxmvmj6npybe-f6ootr0j5lxaw0yb.  Use '--activationmode partial' to override.
  Refusing activation of partial LV Storage/vsv5015-dmxm0y1ckwmdfsxd-cra9zx8ugpzm0cne.  Use '--activationmode partial' to override.
  Refusing activation of partial LV Storage/vsv5076-djuhho90kmc97glh-k6cjlb53zt30gnze.  Use '--activationmode partial' to override.
  1 logical volume(s) in volume group "Storage" now active
```
```
root@rescue:~# lvdisplay
  /dev/Storage/vsv4765-d0xv6p2u8yc28vqk-gp7socpfcgu1d8rd: read failed after 0 of 4096 at 214748299264: Input/output error
  /dev/Storage/vsv4765-d0xv6p2u8yc28vqk-gp7socpfcgu1d8rd: read failed after 0 of 4096 at 214748356608: Input/output error
  /dev/Storage/vsv4765-d0xv6p2u8yc28vqk-gp7socpfcgu1d8rd: read failed after 0 of 4096 at 0: Input/output error
  /dev/Storage/vsv4765-d0xv6p2u8yc28vqk-gp7socpfcgu1d8rd: read failed after 0 of 4096 at 4096: Input/output error
  /dev/Storage/vsv5015-dmxm0y1ckwmdfsxd-cra9zx8ugpzm0cne: read failed after 0 of 4096 at 21474770944: Input/output error
  /dev/Storage/vsv5015-dmxm0y1ckwmdfsxd-cra9zx8ugpzm0cne: read failed after 0 of 4096 at 21474828288: Input/output error
  /dev/Storage/vsv5015-dmxm0y1ckwmdfsxd-cra9zx8ugpzm0cne: read failed after 0 of 4096 at 0: Input/output error
  /dev/Storage/vsv5015-dmxm0y1ckwmdfsxd-cra9zx8ugpzm0cne: read failed after 0 of 4096 at 4096: Input/output error
  /dev/Storage/vsv5076-djuhho90kmc97glh-k6cjlb53zt30gnze: read failed after 0 of 4096 at 214748299264: Input/output error
  /dev/Storage/vsv5076-djuhho90kmc97glh-k6cjlb53zt30gnze: read failed after 0 of 4096 at 214748356608: Input/output error
  /dev/Storage/vsv5076-djuhho90kmc97glh-k6cjlb53zt30gnze: read failed after 0 of 4096 at 0: Input/output error
  /dev/Storage/vsv5076-djuhho90kmc97glh-k6cjlb53zt30gnze: read failed after 0 of 4096 at 4096: Input/output error
  Couldn't find device with uuid qUelxM-FXi6-HduD-m6A4-VVfa-61eF-tJ7h8A.
```

### fdisk:
```
root@rescue:~# fdisk -l /dev/nvme0n1

Disk /dev/nvme0n1: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
```
```
root@rescue:~# fdisk -l /dev/nvme1n1

Disk /dev/nvme1n1: 894.3 GiB, 960197124096 bytes, 1875385008 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x08755452

Device         Boot Start        End    Sectors   Size Id Type
/dev/nvme1n1p1       2048 1875385007 1875382960 894.3G 8e Linux LVM
```