Scribd Logo
Upload

Welcome to Scribd!

  • 2 views

    Storages

    Uploaded by

    mohammed ahmed

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd

    Storages

    Uploaded by

    mohammed ahmed
    2 views34 pages

    Copyright

    © © All Rights Reserved

    Available Formats

    PPTX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    2 views34 pages

    Storages

    Uploaded by

    mohammed ahmed

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    of 34

    storage devices naming convention

    /dev/nvme0n1 -> advanced SSD


    /dev/sda -> scsi hard disk
    /dev/vda -> virtual hard disk
    /dev/hda -> sata hard disk

    lsblk -> list block devices

    ls -l /dev/nvm* -> list storage devices


    cat /proc/partitions -> Kernel perspective of devices
    MBR / GPT

    BIOS -> MBR -> 4 Partitions -> logical partitions numbering starts with 5

    UEFI -> GPT -> 128 Partitions


    Master Boot Record (MBR) is part of the 1981 PC specification

    512 bytes to store boot information


    64 bytes to store partitions
    Place for 4 partitions only with a max size of 2 TiB
    To use more partitions, extended and logical partions must be used
    GUID Partition Table is newer partition table (2010)

    More space to store partitions


    Used to overcome MBR limitations
    128 partitions max
    parted

    parted /dev/sdb
    print will show if there is a current partition table
    mklabel msdos | gpt
    mkpart part-type name fs-type start end
    part-type: applies to MBR only and sets primary, logical, or extended partition
    name: arbitrary name, required for GPT
    fs-type: does NOT modify the filesystem, but sets some irrelevant file system
    dependent medadata
    start end: specify start and end, counting from the beginning of the disk
    Example: mkpart primary 1024MiB 2048MiB
    Alternatively, use mkpart in interactive mode

    print to verfy creation of the new partition


    quit to exit the parted shell
    udevadm settle to ensure that the new partition device is created
    cat /proc/partitions to verify the creation of the partition
    File System Differences

    XFS is the default file system


     Fast and scalable
     Uses CoW to guarantee data integrity
     size can be increased not decreased
    Ext4 was default in RHEL 6 and is still used
     Backward compatible to Ext2
     Uses Journal to guarantee data integrity
     size can be increased and decreased
    FILE SYSTEMS
    Master Boot Record

    Assembly code boot loader 446 bytes.


    Partition table for 4 primary partitions 16 bytes each.
    sentinel 2 bytes
    0xAA55=Bootable
    parted /dev/sdc print
    parted /dev/sdc mklabel msdos
    dd if=/dev/zero of=/dev/sdc count=1 bs=512
    parted /dev/sdc print
    fdisk vs parted

    fdisk /dev/sdb : If there is no partition table then it will create a partition


    table of msdos. Will work only on MBR disks.
    parted /dev/sdb mklabel msdos : works with gpt as well as MBR disks

    Limitations of MBR
     Limited to 4 primary partitions
     Single byte partition code
     32 bit address limits disk size to 2TB
     Extended partition uses linked logical partitions but one lost link can
    lose all
    making File Systems

    mkfs.xfs creates an XFS file system


    mkfs.ext4 creates an EXT4 file system
    use mkfs.[Tab] [Tab] to show a list of available file systems
    Do Not use mkfs as it will create an Ext2 file system
    After making the file system, you can mount it in runtime using the mount
    command
    /etc/fstab

    /etc/fstab is the main configuration file to persistently mount partitions


    /etc/fstab content is used to generate systemd mounts by the systemd-fstab-
    generator utlity
    To update systemd, make sure to use systemctl daemon-reload after
    editing /etc/fstab
    /etc/fstab
    Persistent Device Naming

    In datacenter environments, block device names may change.


    UUID is automatically generated for each device that contains a file system or
    anything similar
    Label while creating file system, the option -L can be used to set an arbitrary
    name that can be used for mounting the file system
    Example: tune2fs -L Label Device for ext3,ext4
    xfs_admin -L Label Device for xfs
    blkid shows label and UUID
    Unique device names are created in /dev/disk

    Lets us see a demo to show the importance of Persistent Device Naming!


    Systemd Mounts

    /etc/fstab mounts already are systemd mounts


    Mounts can be created using systemd .mount files
    Using .mount files allows you to be more specific in defining dependencies
    Use systemctl cat tmp.mount for an example
    mount | grep tmp
    Use systemctl enable --now tmp.mount
    mount | grep tmp
    cp /usr/lib/systemd/system/tmp.mount /etc/systemd/system/sales.mount
    vim /etc/sales.mount
    To create your Systemd Mounts

    cp /usr/lib/systemd/system/tmp.mount /etc/systemd/system/sales.mount
    Name of the mount unit should be same as that of the directory. If it is a sub-
    directory then the naming of the mount unit will be sales-mydir.mount
    vim /etc/sales.mount
    [Mount]
    What=LABEL=sales
    where=/sales
    Type=ext4
    Options=defaults
    systemctl daemon-reload
    Systemctl --enable now sales.mount
    Xfs File Systems

    The xfsdump utility can be used for creating backups of XFS formatted
    devices and considers specific XFS attributes
    xfsdump only works on a complete XFS device
    xfsdump can make full backups(-l 0) or different levels of incremental
    backups
    xfsdump -l 0 -f /backupfiles/data.xfsdump /data creates a full backup of the
    contents of the /data directory
    The xfsrestore command is used to restore a backup that was made with
    xfsdump using xfsrestore -f /backupfiles/data.xfsdump /data
    xfsrepair command can be manually started to repair broken XFS file
    systems
    GUID Partition Table
    Advantages of GUID Partition Table

    GUID used to identify partition type


    provides disk GUID and partition GUIDs
    128 partitions by default per disk
    64 bit addressing allows for 2 ZB disk
    Backup header and MBR stored at end of the disk for recovery
    CRC32 checksums to verify integrity of partition table
    kernel must support GPT
    Unified Extensible Firmware Interface (UEFI)

    Originally developed by Intel for its Itanium based systems


    New way to boot OS moving away from BIOS and MBR
    Can Include Secure Boot, cryptographic signing of the boot loader
    Requires EFI system partition usually FAT32
    As such UEFI is not limited to reading the 446 byte boot strap code in the
    MBR
    GRUB2 and Syslinux support BIOS and UEFI.
    Gummiboot and Clover support UEFI Only.
    Grub Legacy and LILO support BIOS Only.
    swap

    swap is RAM that is emulated on disk


    All Linux systems should have at least some swap
    The amount of swap depends on the use of the server
    Swap can be created on any block device including swap files
    while creating swap with parted, set file system to linux-swap
    For fdisk mark id 82
    After creating the swap partition, use mkswap to create the swap FS
    Activate using swapon -a
    Note: swap cannot be mounted on / but uses swap kernel interface in fstab
    free -m
    swap storage on existing file system

    dd if=/dev/zero of=/swapfile bs=1024 count=1G generates a file that we


    can overlay a swap file system on of size 1G
    mkswap /swapfile
    swapon –v /swapfile
    swapon –s shows current swap configuration
    swapon -a reads /etc/fstab
    swapoff /dev/device disables swapping on specific device
    free -h displays amount of used and free memory
    Hands-on

    Create a 10GiB hard disk on virtual machine


    Create a primary partition with a size 1GiB. Format it with Ext4 and mount it
    persistently on /mounts/files using its UUID
    Create an extended partition that includes all remaining disk space. In this
    partition, create a 500MiB XFS partition and mount it persistently on
    /mounts/xfs using label myxfs
    Create a 500 MiB swap partition and mount it persistently
    Linux Storage Options

    Partitions the classical solution, used in all cases


    Use to allocate dedicated storage to specific types of data
    LVM Logical Volumes
    Used at default installation of RHEL
    Adds flexibility to storage (resize, snapshots and more)
    Stratis
    Next generation Volume Managing Filesystem that uses thin provisioning
    by default and Implemented in user space, which makes API access possible
    Virtual Data Optimizer
    Focused on storing files in the most efficient way
    Manages deduplicated and compressed storage pools
    LVM

    Physical Volumes can be disks, partitions, Luns of SAN, etc.


    Volume Group is used to create Logical volumes
    There is no direct connection between PVs and LVs.
    Size of the Logical Volumes can be bigger than the actual physical volume
    Logical Volumes Logical Volumes

    Volume Group

    Physical Volumes Physical Volumes


    LVM Steps

    Create a partition from parted use set number lvm on


    pvcreate /dev/sdb1 will create the physical volume
    vgcreate vgdata /dev/sdb1 will create volume group
    lvcreate -n lvdata -L 1G vgdata to create logical volume
    mkfs.fs /dev/vgdata/lvdata to create file system
    /etc/fstab to mount persistently
    UUID or Labels not required as LVM names are device independent
    Device Mapper Names

    Device mapper is the system that the kernel uses to interface storage devices
    Device mapper generates meaning less names e.g. /dev/dm-0 /dev/dm-1
    Meaningful names are provided as symbolic links through /dev/mapper
    e.g. /dev/mapper/vgdata-lvdata
    Alternatively use LVM generated symbolic links /dev/vgdata/lvdata
    LVM Resize

    df -Th queries the file system for disk space


    vgs to verify availability in the volume group
    vgextend vgdata /dev/sdc1 to add one or more PVs to the volume group
    lvextend -r -L +1G to grow the LVM logical volume, including the file system
     e2resize is an independent resize utilty for ext file system
     xfs_growfs can be used to grow an XFS file system
    Shrinking is not possible on XFS volumes

    Note:- While extending use lvextend and then resize2fs for resizing file
    system size and while reducing first reduce file system size and then use
    lvreduce
    LVM Resize Reduce

    umount /mnt/directory
    e2fsck -f /dev/vgdata/lvdata
    resize2fs /dev/vgdata/lvdata 500M
    lvreduce /dev/vgdata/lvdata -L 500M
    mount -a
    LVM Snapshots

    lvcreate -L 30m -s -n backup /dev/vgdata/lvdata


    mount /dev/vg1/backup /mnt -o nouuid,ro
    for xfs file system use nouuid and ro as nothing to be written in snapshot
    tar -cvf /root/backup.tar /mnt
    lvremove /dev/vg1/backup
    setup user and group quotas for filesystems

    Enable quotas on extended file systems like ext4


    Check quotas using mount | grep /dev/sdb1
    umount /dev/sdb1
    vim /etc/fstab
    /dev/sdb1 /data ext4 usrquota, grpquota 0 0
    mount -a
    Create quotadatabase -m to make database, -a for all partitions, -u for users
    quotacheck -maug
    The above command will create .user .group database files at /data
    quotaon /dev/sdb1
    setup and report on ext4 quotas

    repquota -uv /dev/sdb1


    This command will report quota status as root
    setquota -u user1 20000 25000 0 0 /dev/sdb1
    Block limit size is in KB and File limits soft limits & hard limits is unlimited
    Group quotas will be for group and one user can use whole limit in that case
    repquota -uv /dev/sdb1
    edquota -u user1 command is used to edit quota and can be used to copy
    quota settings from one user to another
    edquota -u newuser -p user1
    chmod 777 /data
    su - user1
    setup and report on ext4 quotas

    echo hello >> /data/myfile.txt


    dd if=/dev/zero of=/dev/null count=1 bs=20M
    This generates a warning if soft limit is exceeded for user1
    repquota -uv /dev/sdb1
    This report show + symbol for user1 who exceeds the soft limit
    xfs quotas

    mount | grep sdb2


    some mount options for xfs file system could be gqnoenforce, uquota,
    gquota, uqnoenforce i.e user quota noenforce to only monitor hard limit
    xfs_quota -x for expert mode in interactive way
    xfs_quota -x
     report /data
     quit
    xfs_quota -xc ‘limit -u bsoft=20m bhard=30m user1’ /data
    The above command is to set quota in non interactive mode for user1
    xfs_quota -xc ‘report -h’ /data

    You might also like