Scribd Logo
Upload

Welcome to Scribd!

  • 17 views

    A3 EA Airflow Scheduler As A Service

    Uploaded by

    Amgad Alsisi

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    A3 EA Airflow Scheduler As A Service

    Uploaded by

    Amgad Alsisi
    17 views33 pages

    Original Title

    a3-EA-Airflow-Scheduler-as-a-Service

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    17 views33 pages

    A3 EA Airflow Scheduler As A Service

    Uploaded by

    Amgad Alsisi

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 33

    !

    "#$%&'$()*+)*)
    !$(,-"$)

    !"#$%&%'()#*)('+,-./#$('(#0#"1

    !"#"$%&'"(#)*
    +*,,#%"&-./,$%./
    0".)1"/ 2%3
    !"#"$%&'"(#)*
    +),#-.*/&012"1//*&33

    14"(#)*5/.6()7

    © 2020 Electronic Arts, Inc.


    © 2020 Electronic Arts, Inc.
    !"#"$%&'#'(

    !"#$!%&'
    ()&'*+%,-'
    (+'$,./
    5
    !"#$%&'()"*
    +,-./(-#$0)'#$1.,2/&3
    ;<==,-' ><)'%=)2
    $('(#/#"0 *)(12- 32'4,-5 782#928'-()#$('(#
    !"#$%&%'()#*)('+,-. ?(.2#;'<@%,A
    6-(.24,-5 *)('+,-.#:2(. !"#"$%&'(&))*+$,$-.()&#(+#+

    !"#$!%$&'%
    CDA#,+#:2-(B1'2A#,+# IJ#&(.2#A'<@%,A#
    :F,<A(8@A#,+#!:G#
    $('(A2'#%8#*2'(B1'2A @('(#&282-('2@#2E2-1# A=-2(@#(K-,AA#'F2#
    H,BA
    @(1 4,-)@

    ()*+,$*-$%&
    !"#"$"#$%&
    !"#"$"#$%&

    6(3'*73$%)"'"%7*7$3*2$,% <===,%+8%/5>%?+0,%"2)%@A=B%
    !"#$%&'()*+,%-*'.%
    89+#%" :$2'9"3%)"'"% ")C.+:%D($9*$,%$E$94)"4%
    /#0$))$)%12"34,'%5$"#,
    -"9$.+(,$; (,*2F%+(9%73"'8+9#

    G*F.34%,$2,*'*E$%)"'",$', H"9*$)%&>1%9$D(*9$#$2',
    '()*+,-*.$"#$%&
    !"#$%"&'(%#)$* +(,-&'(%#)$* ./,/%"&$#&'(%#)$*

    4)5",67*)/68(*"9#$ 7,/#*.:&;"*<:)= >3:#"9:,&;"*<:)=&$(%,?3:,*

    73$#)@&7)/#"/3)3$&A/#,B*.#")/&C7AD 7)@@)/&+:3B"/$ 8,:<E8,*F,&4*(%,$#*.#")/&8G$#,@

    A/EH)3$,&I,9:)G@,/#&J)): 8#./?.*?&>)/"#)*"/B A@9*)F,?&=)*K:).?&@./.B,@,/#

    L.$"(&@)/"#)*"/B 7A67I

    L.$"(&;3#%&>)?,: 884&"/#,B*.#,?

    8#./?.:)/,&8G$#,@$ >3:#"E#,/./(G
    '()*+,-*./"0/"/'*.12(*$345*(#21*0

    6(3'*C'$2"2:4%-*'.%IJ1K%L&&M% /",$%+8%?+0%)$73+4#$2'%L!*'3"0% /",$%+8%*2'$98":*2F%-*'.%'.$% 6+2*'+9*2F%"2)%13$9',%",%7"9'%


    -*'.%K(,'+#%1('.% KOPKQN )"'",$'%LK(,'+#%R3(F*2,%-*'.% +8%,$9E*:$%
    6*))3$-"9$N 1K>N
    8*//#%"&9.1/$%.1
    +),#-.*/&012"1//*&333

    :2.1/$%.15/.6()7

    © 2020 Electronic Arts, Inc.


    Multi-Tenancy and RBAC

    EADP Data & AI 14


    6,-#2/7*8"8(9$
    :;&< "0$"$'*.12(*
    RBAC as a Service
    Job Deployment

    EADP Data & AI 18


    &,#=$!*1/3>0$"0$"$'*.12(*
    Auto Deployment – S3 Sync Sidecar
    • Need a way to sync DAGs across multiple
    pods whenever there is new or updated
    DAG
    • Sidecar runs alongside Web, Workers and
    Scheduler Pods
    • DAG files and Plugins synced from S3
    location periodically
    '"8+4=B$"0$"$'*.12(*
    ;".)<"1 =%>
    +),#-.*/&012"1//*&333

    ?@%>5/.6()7

    © 2020 Electronic Arts, Inc.


    Custom Plugins
    Ease of accessing the datasets

    EADP Data & AI 23


    C-,D28$E24.".9$"0$"$'*.12(*

    R3(F*2,%"9$%#"2"F$)% I$D(*9$,%#$9F$%9$D($,'% R3(F*2,%"9$%)$73+4$)% O2C.+(,$%R3(F*2,%'+%


    '.9+(F.%"%,*2F3$% "2)%"779+E"3,%'+% '.9+(F.%:+2'*2(+(,% *2'$98":$%-*'.%R9$,'+%
    9$7+,*'+94 :+2'9*0('$ )$73+4#$2' R3"'8+9#
    <,0#=?$C-,D280$F2#)$&<E

    R3(F*2%"::$,,% !*'3"0%KO%5$,',%'+%$28+9:$% T$-%"779+E$)%73(F*2,%


    )$8*2$) 7$9%'$"# *2% "::$,,%:+2'9+3 "))$)%'+%:+28*F
    S&MT
    Monitoring and Alerts

    EADP Data & AI 26


    &-*.#28D$"0$"$'*.12(*

    !"#$%&$'$()*%+,-.$()/) 3+.)0$,$0)4$'%-2()/&.)
    74/-0)#/($.)
    #/($)0$,$0)+1)1/"0') /0$%'()/,/-0/#0$)5-'6)
    &+'-1-2/'-+&(
    '+0$%/&2$ 3%+4$'6$"(

    7?@)/0$%'()/&.)
    8-%10+5)9$%,-2$)0$,$0) :+#;<8=)0$,$0) 4+&-'+%-&>
    4$'%-2()/&.)/0$%'( 4+&-'+%-&>
    6=82#=.28D$"0$"$'*.12(*
    6=82#=.28D$"0$"$'*.12(*
    Architecture and Integration

    EADP Data & AI 30


    &2.G-=F$H8G."0#.,(#,.*
    !"#$%&"'(%)(*+,"-)"."/

    0"$"-&(12"3+.%-(4 5"'6/(,-%7"-

    8%/.9-":;<(,=37")'

    8-%2&(>=/"'(?+.@").63=.6%)

    A+$.64.")=).(/&/."B

    0").-=$6C"'(#$+96)(/&/."B .% 6)."-D=3"(E6.@(
    B+$.6#$"(#-"/.% 3$+/."- "3%/&/."BF
    !"#"$

    You might also like