Category: ubuntu

Encrypted LVM partition on software raid-1 with mdadm

At another post https://www.gonscak.sk/?p=201 I posted how to create raid1 software raid with mdadm in linux. Now I tried to add a crypted filesystem to this.

First, check, that we have working software raid:

sudo mdadm --misc --detail /dev/md0

/dev/md0:
           Version : 1.2
     Creation Time : Wed Aug 22 09:34:23 2018
        Raid Level : raid1
        Array Size : 1953381440 (1862.89 GiB 2000.26 GB)
     Used Dev Size : 1953381440 (1862.89 GiB 2000.26 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent
     Intent Bitmap : Internal
       Update Time : Thu Aug 23 14:18:50 2018
             State : active 
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0
Consistency Policy : bitmap
              Name : gw36:0  (local to host gw36)
              UUID : ded4f30e:1cfb20cb:c10b843e:df19a8ff
            Events : 3481
    Number   Major   Minor   RaidDevice State
       0       8       17        0      active sync   /dev/sdb1
       1       8       33        1      active sync   /dev/sdc1

Now, we synced drives and clean. It is time to encrypt.  If we have not loaded modules for encryption, load it:q

modprobe dm-crypt

Now create the volume with passphrase:

sudo cryptsetup --cipher=aes-xts-plain --verify-passphrase --key-size=512 luksFormat /dev/md0

And we can open it:

sudo cryptsetup  luksOpen /dev/md0 cryptdisk

Now we can create as many times a physical volume, volume group and logical volume.

sudo pvcreate /dev/mapper/cryptdisk
sudo vgcreate raid1 /dev/mapper/cryptdisk
sudo lvcreate --size 500G --name lv-home raid1

sudo pvs
  PV                     VG        Fmt  Attr PSize    PFree
  /dev/mapper/cryptdisk  raid1     lvm2 a--    <1,82t 1,33t
sudo vgs
  VG        #PV #LV #SN Attr   VSize    VFree
  raid1       1   1   0 wz--n-   <1,82t 1,33t
sudo lvs
  LV      VG        Attr       LSize
  lv-home raid1     -wi-ao---- 500,00g            

Next, we create a filesystem on this logical volume:

sudo mkfs.ext4 /dev/mapper/raid1-lv--home

And we can mount it:

sudo mount /dev/mapper/raid1-lv--home crypt-home/

Now we have an encrypted partition (disk) for our home directory.

Total Page Visits: 153350 - Today Page Visits: 17

disk cloning with dd

How to create a disk or usb image, and compress it on the fly? And how to restore it?
I have own operating system on USB key. To create a full-backup and then possible restore to another device, I use linux command dd (dd – convert and copy a file).
Now, we must determine, on which patch we have s source disk. I my case, it is

sudo fdisk -l /dev/sdb
Disk /dev/sdb: 29,5 GiB

First, I install additional  software for monitoring and best compressing on more cores

sudo apt-get install pigz pv

Then, I create a full copy of the usb key. Without compression it takes 30GB, with compression, it take only 3GB. With command “pv” we can watch progress. Pigz compress the source image with multiple threads and cores. With parameter -c it writes all processed output to stdout. So  with operand “>” we write this pigz output to a file:

sudo dd if=/dev/sdb | pv | pigz -c > /home/vasil/Documents/corsair-work.dd.gz

If we had som bad blocks on source disk, and we want to clone it anyway, we can use another conv options. Like:

conv=sync,noerror

This means:

  • noerror – This makes use dd continue even after a read error is encountered;
  • sync – This option has sense especially when used together with noerror.

In such a case the noerror option will make dd continue running even if it a sector cannot be read successfully, and the sync option will make so that the amount of data failed to be read its replaced by NULs, so that the length of the data is preserved even if the actual data is lost (since it’s not possible to read it).

Then, I remove the source usb key and insert new one. It also has a path /dev/sdb. Now, I restore it with this command:

pigz -cdk Documents/corsair-work.dd.gz |pv| sudo dd of=/dev/sdb bs=4M

Parameter -c also write output to stdout and program dd writes it to disk. Parameter -k menas, that keep original file after decompress. And parameter -d means decompress.
Now, we can boot system with new usb key. And this image is identical as the source.
I hope, that this help someone. Have a nice day.

Total Page Visits: 153350 - Today Page Visits: 17

Rescue disk with ddrescue from ubuntu

I have a broken disk, partially working. This is part of dmesg after plug-in USB removable 2,5″ disk, and list from fdisk:

[1448.206941] blk_update_request: I/O error, dev sdb, sector 6293504
fdisk -l /dev/sdb
Disk /dev/sdb: 931,5 GiB, 1000170586112 bytes, 1953458176 sectors
......
Device     Boot Start        End    Sectors   Size Id Type
/dev/sdb1        2048 1953458175 1953456128 931,5G  7 HPFS/NTFS/exFAT

So I will try to rescue some data from it. I will use a gddrescue program:

apt-get install gddrescue

And now, I have mounted a big 3TB nfs storage, where I will save image of this disk:

ddrescue -r1 -v -d /dev/sdb /mnt/nfs/sdb.img /mnt/nfs/sdb.log
  • -r1  means, that ddrescue will try read every block one time before giving it up on this block (reading from it)
  • -v  means verbose mode
  • -d means, that ddrescue use direct disk access and ignore kernel’s cache
  • /dev/sdb is the failing drive
  • /mnt/nfs/sdb.img is the destination image, where we save any data
  • /mnt/nfs/sdb.log is the log file, where is written every bad block and actual position of ddrescue. We can brake this rescue at any time and continue it later with the same command. When ddrescue finish, we can repeat this check only on bad blocks with more retries

 

  • 22.3.2017 – it was stared. post will continue after it finished 😀 maybe it take 3 days to finish, maybe more 🙂 This operation takes a long time to finish…
Total Page Visits: 153350 - Today Page Visits: 17