Scribd Logo
Upload

Welcome to Scribd!

  • 3 views

    8.20240801 0700 ClassNotes

    Uploaded by

    laxmishanthib

    Copyright:

    © All Rights Reserved
    Download as TXT, PDF, TXT or read online from Scribd

    8.20240801 0700 ClassNotes

    Uploaded by

    laxmishanthib
    3 views4 pages

    Original Title

    8.20240801-0700-ClassNotes

    Copyright

    © © All Rights Reserved

    Available Formats

    TXT, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as TXT, PDF, TXT or read online from Scribd
    Download as txt, pdf, or txt
    3 views4 pages

    8.20240801 0700 ClassNotes

    Uploaded by

    laxmishanthib

    Copyright:

    © All Rights Reserved
    Download as TXT, PDF, TXT or read online from Scribd
    Download as txt, pdf, or txt
    of 4

    GCS service:

    what is GCS?
    what is the purpose of GCS?
    Features of GCS?
    what is Bucket?
    Options we need to choose as part of creation of the bucket?
    - bucketname
    - regions (single, dual, multiregion)
    - storage classes (standard,nearline,coldline,archive)
    - Access control (Uniformed, finegrained)
    - Protect the data
    - Data encryption

    Practicals:

    GCP Console:
    -------------
    1. creation of the bucket
    2. creation of the folder in bucket
    3. uploading a single file into bucket
    4. uploading a single file into bucket into folder
    5. uploading a complete folder

    ***********************************************************************************
    *

    Protect object data in buckets:


    a) object versioning
    b) Retention policy

    Day1: uploaded this file into our bucket


    cusomter.txt

    cid,cname,clocation
    101,srinivas,hyd
    102,quality,viz

    Day2: uploading this file into our bucket ==> overwrite


    cusomter.txt

    cid,cname,clocation
    101,srinivas,hyd
    102,quality,viz
    103,thought,hyd
    104,xx,viz

    Day2 ==> 10Am uploaded customer.txt ==> 11AM you realized that you uploaded a wrong
    file ==> I want to revert my day1 data/file ==> Not possible directly

    As per your requirements if you want to protect data because of deletes or updates
    in such kind of scenarios we can use the concept called as "object versioning"

    if you want to protect your data while cration of the bucket we need to enable
    feature called as "Object versioning"

    Day1: uploaded this file into our bucket


    cusomter.txt
    cid,cname,clocation
    101,srinivas,hyd
    102,quality,viz

    Day2: uploading this file into our bucket ==> it will take backup(version) of your
    day1 file (customer.txt_35235234523) ==> upload day2 file
    cusomter.txt

    cid,cname,clocation
    101,srinivas,hyd
    102,quality,viz
    103,thought,hyd
    104,xx,viz

    Day2 ==> 10Am uploaded customer.txt ==> 11AM you realized that you uploaded a wrong
    file ==> I want to revert my day1 data/file ==> possible directly

    Day3: uploading this file into our bucket ==> it will take backup(version) of your
    day2 file (customer.txt_46346363) ==> upload day3 file
    cusomter.txt

    cid,cname,clocation
    101,srinivas,hyd
    102,quality,viz
    103,thought,hyd
    104,xx,viz
    105,srinivas,hyd
    106,quality,viz

    Like this how many backups/versions we can able to take ?? [N number of


    backups/versions]

    1000 backup files ==> is it really useful these many backup files to protect data
    ==> [NO]

    in real time projects we will take only last 7 or 10 versions of the files

    object versioning:
    - number of versions you want to maintain - 1000
    - Expiry days of the version - 365 days

    Aug 1st ==> version of the file ==> v1 - delete on aug 6th
    Aug 2nd ==> version of the file ==> v2 - delete on aug 7nd
    Aug 3rd ==> version of the file ==> v3
    Aug 4th ==> version of the file ==> v4
    Aug 5th ==> version of the file ==> v5
    Aug 6th ==> version of the file ==> v6
    Aug 7th ==> version of the file ==> v7
    Aug 8th ==> version of the file ==> v8
    Aug 9th ==> version of the file ==> v9
    Aug 10th ==> version of the file ==> v10
    Aug 11th ==> version of the file ==> v11

    Retention policy: For preventing deletions or modifications on top of your bucket


    for a particular duration.

    days - 365
    Day1 ==> 100GB data [10 files or 100 files]
    Day365 ==>

    Data encryption:
    ---------------

    1. Google side defatult data encryption -


    2. Customize data encryption -

    customer.txt

    cid,cname,cmobilenumber,cmailid
    1,srinivas,734572857325,[email protected]

    100,xyzxdfasdfa,235252452345354352

    ***********************************************************************************
    *

    why we need to secure the data ==>

    Data engineering projects ==> data + data + data + data

    banking sectore or telecome domain or insurance domain ==> end user they need to
    perform some analysis on business and they need to decissions ==> Data

    creation of the buckets / how to create the folder / how to upload the folder / how
    to upload a single file /

    copy/move the file ==> bucket to other bucket


    ==> in same bucket from one folder to other folder
    ==> from one bucket folder to other bucket folder

    ***********************************************************************************
    *

    Cloud SDK: [GCS operations]


    ----------

    1. I want to see all available buckets in my project ==> gsutil list


    2. I want to create a new bucket ==> gsutil mb bucketname [mb - make bucket]
    gsutil mb
    gs://srinivascde32sdk
    3. I want to upload/copy a file into bucket from my local system ==> gsutil cp
    sourcepath targetpath
    gsutil cp D:\QT\1_Batches\Batch32\20240723-0800-ClassNotes.txt
    gs://srinivascde32sdk
    gsutil cp D:\QT\1_Batches\Batch32\20240723-0800-ClassNotes.txt
    gs://srinivascde32sdk/srinivas/

    I want to upload/copy a file into from bucet to my loca system ==> gsutil cp
    sourcepath targetpath

    4. I want to upload a folder into my bucket ==>

    gsutil cp -r D:\QT\1_Batches\Batch32\ gs://srinivascde32sdk

    -r - recursive copy operations


    Note: While creation of folders or files in your local system please doint
    give spaces.

    gsutil cp "D:\QT\1 Batches\Batch32\20240723-0800-ClassNotes.txt"


    gs://srinivascde32sdk

    gsutil cp gs://srinivascde32sdk/srinivas/xx.txt gs://srinivascde32

    Whatever questions you are getting in your mind ==> just try to perfrom those
    operations by using our commands

    gsutil cp -r bucketpath localfolderpath


    *****************************************************************************

    You might also like