Final Incident Report: Root Cause Analysis

Download as doc, pdf, or txt
Download as doc, pdf, or txt
You are on page 1of 4

Final Incident Report

ROOT CAUSE ANALYSIS

Title: Cell Broadcast Service config ration C!ile Tic"et: #$ % TT &'$(&)

Proposal for Proc Proposal n. COMSiemens OG COM

Title Date Page 1 of 4

Incident Title Incident Location Service Impact Incident Start: Incident end: Incident Duration Incident %anager

Incident Infor*ation Cell Broadcast Service configuration Chile Cell Broadcast Notifications Date Date 08/0 /!0"# 08/0 /!0"# Time Time 08:!# $% "!:&' (%

! # minutos (edro (ereira

Clarif+ Infor*ation Tic)et $ffected Services $ssigned to "!8 ' Cell Broadcast Notifications ,DC +$S Incident T+pe Service T+pe (!*(" D$T$ +$S ,rd #art+ Origin Users Affected -ua.ei Cell Broadcast Service None

Tec!nical Escalation Level /scalation Level Name Technician Technical Support $dministrative #rd (art2 Tech3 Support #rd part2 Claim ,DC D$T$ +$S Luis Canales -ua.ei Li0ui 1s.aldo Torre -ua.ei

Ticket TT - 2055991
Origination

Trigger

Cell Broadcast Service

'st ac"no-ledge

,DC D$T$+$S C-

O tage Ca se Configuraton lost in regard to the Cell ID and 0NC3

.!+ it !appened Configuration deleted due to .rong manual operation3

.!at aggravated t!e resol tion ti*e N/$3

!" mar4o !0"&

Resol tion 0estore the Data5ase use the 5ac)up previous3

Lessons Learned There should 5e different profile accounts .ith the proper rights on the 5asis of the scope of the areas3 Configuration tas) onl2 can 5e performed 52 e6pertice teams3

.!at is /eing done to prevent t!is fro* !appening in t!e f t re N/$3

Incident Anal+sis Tuesda2 th $ugust at "0:00 am got the Ne6tel report7 the 0NC/Cell Data miss in CBS s2stem3 The service impacted3 Chec) the platform and operation log find out someone login in the s2stem and delete the 0NC/Cell Data manuall2 Immediatel2 start to emergenc2 recovers3 0estore the Data5ase use the 5ac)up previous3 $t "!:#03 service recovered3

Action #lan Create user profiles for according to areas scope

.!o -ua.ei

0 e date " *08*!0"#

Stat s In progress

1 alit+ 2ates E3ENT Net-or" Incident Event detection Tro /le tic"et dispatc! TT ac"no-ledge FL4 Tec!nician 0ispatc! FL4 on Site Spare part on Site Re6 est for Adv7 S pport Incident Resol tion Tic"et Clos re 0ATE 08/0 08/0 08/0 08/0 * * * * 08/0 08/0 /!0"# /!0"# /!0"# /!0"# TI4E Elapsed Ti*e NOC 08:!# $% "0:00 $% "!:#' (% "!:&" (% Elapsed Ti*e FL4 5 ,rd #arties Ne6tel

/!0"# /!0"#

"!:&' (% 0#:#' (%

!" mar4o !0"&

Incident Ti*eline Ti*e 0escription

06-08-2013 12:41 06-08-2013 12:41 06-08-2013 12:41 06-08-2013 13:03 06-08-2013 13:52 06-08-2013 14:05 06-08-2013 15:24

INI Message sent at GNOC Porta IM engage! in t"e case Ce #roa!cast $er%ice con&ig'ration ost #(I)G* is o+en &or t"e case: ON #ri!ge IM ,-n!re .orge/Pe!ro Pereira0/1'is Cana es ,)-T-2-$ C3 TI*(20 3'a4ei sen! Ticket N'56er 72055991 $it'ation is 6ack to nor5a an! esta6 e8 4aiting &or 3'a4ei re+ort &or 5ore &ee!6ack9 $er%ice is reco%ere! since 12:45 :C"i e Ti5e;9 3'a4ei con&ir5 a ser%ices 4orking9 C osing Case 3'a4ei $$- Carrier (es+onsa6i e ,1ir'i0 re+ort a )ata6ase (eco%ere! 4as t"e on < actions taken to reco%er ser%ice9 In 1ir'i o+inion so5eone &ro5 N*=T*1 !e ete! t"e )# - a ar5s are c eare!8 con&ir5e! 6< 3'a4ei 6eca'se )-T-2-$ C"i e oose a access to + at&or5
Affected 8#Is 9After t!e co*plete pro/le* resol tion:

06-08-2013 14:22

06-08-2013 12:45

3AS Cell Broadcast Notifications


S pport Infor*ation

(T& ,N1C (ortal CBC logs

RCA Creator

Luis Canales

RCA Responsi/le RCA Approver

1s.aldo Torre

!" mar4o !0"&

You might also like