WEB LOG

Amazon Web Services provides services and infrastructure to However, as the web application grows and the number of visitors

W S build reliable, fault-tolerant, and highly available web applications

in the cloud. In production environments, these applications can
increases, storing and analyzing web logs becomes increasingly
challenging.
A ce generate huge amounts of log information.
This diagram shows how to use Amazon Web Services to build a
enes
ANALYSIS
This data can be an important source of knowledge for any scalable and reliable large-scale log analytics platform. The core

f e r company that is operating web applications. Analyzing logs can component of this architecture is Amazon Elastic MapReduce, a

Reectur
reveal information such as traffic patterns, user behavior, web service that enables analysts to process large amounts of
marketing profiles, etc. data easily and cost-effectively using a Hadoop hosted framework.

h it 2
Arc
C
E

Ma
n
zo

pR E
a

t
o
m

edlas 4
p
A

Fi Log
S

A
uc tic

m
2

les

a
C

S3

zo
E

a
e

n
zo
R
n

n
M
zo

E
a
m

n
zo
A

3
a

S
m
A

n
zo
a
m

t
n
ro
A

F
d
u
lo

ilesry p
C

oo r
S
on

F
D

g o d
az

Lo posit Hauste
R
m

n
A

zo

Fi Log
Re Cl
a
m

les
A

ro n
F o

b
t
d z

Wervers
u a
lo m

3
C A

Se
EC
A
m
a

2
zo

5
InSpo
n

7
st t
an
1 s ce
l y tice s
a
Antabas
Da 6 RD

A
m
S

a
zo
st

n
HT ly
TP 2 na
Re A
qu
es
ts
e ry
Cl
ou D eliv
dF t
A

tenk
m
a

ro n
zo

Vi
Cotwor
n

sit n
or t
Ne
s

The web front-end servers are running on Amazon An Amazon Elastic MapReduce cluster processes Data processing results are pushed back to a relational
System 1 Elastic Compute Cloud (Amazon EC2) instances. 4 the data set. Amazon Elastic MapReduce utilizes a
hosted Hadoop framework, which processes the data in a
6 database using tools like Apache Hive. The database
can be an Amazon Relational Database Service (Amazon
Overview 2
Amazon CloudFront is a content delivery network that
uses low latency and high data transfer speeds to
parallel job flow. RDS) instance. Amazon RDS makes it easy to set up,
operate, and scale a relational database in the cloud.
distribute static files to customers. This service also When Amazon EC2 has unused capacity, it offers EC2
generates valuable log information. 5 instances at a reduced cost, called the Spot Price. This 7 Like many services, Amazon RDS instances are
price fluctuates based on availability and demand. If your priced on a pay-as-you-go model. After analysis, the
Log files are periodically uploaded to Amazon Simple workload is flexible in terms of time of completion or required database can be backed-up into Amazon S3 as a database
3 Storage Service (Amazon S3), a highly available and capacity, you can dynamically extend the capacity of your snapshot, and then terminated. The database can then be
reliable data store. Data is sent in parallel from multiple web cluster using Spot Instances and significantly reduce the recreated from the snapshot whenever needed.
servers or edge locations. cost of running your job flows.