TCL 2 K
TCL 2 K
TCL 2 K
The Architecture of a Multithreaded High-Performance Web Site Jim Davidson America Online, Inc. February 16, 2000
2/16/2000
You Just
Included 48 dynamic template files Invoked over 5000 Tcl commands:
58 distinct C commands 67 distinct Tcl procs
Locked and unlocked over 800 mutexes Searched a database through a special Tcl proxy process interface Selected promos for over 10 advertising slots And much more all in under 150,000 microseconds (which is actually pretty fast)
2/16/2000
Overview
Digital City, an online service of America Online Inc., is powered by AOLserver, a dynamic, multithreaded Tcl application development platform This talk will describe AOLserver, multithreaded Tcl, and the Digital City web platform
2/16/2000
AOLserver Background
AOLserver is a web application development environment used throughout AOL, e.g.:
Digital City, AOL.com, AOL Search, Shop@AOL, AOL.com/Netscape WebCenters, and many more...
AOLserver Basics
AOLserver is a full featured, multithreaded web application development environment for Unix and NT In addition to static page serving and CGIs, AOLserver includes a rich and flexible C API for:
Dynamically loading user-written C modules (DLLs) Binding functions to arbitrary web requests SQL database services Cron-like background task scheduling Asynchronous socket callbacks And much more...
2/16/2000
AOLserver Tcl
However, the most useful aspect of AOLserver has always been its complete Tcl integration Tcl integration is not unique to AOLserver, other implementations include:
Apaches mod_tcl Binary Evolutions VelociGen Vignettes Story Server
Among other advantages, integrated Tcl generally provides higher performance than CGI-based solutions
2/16/2000
2/16/2000
2/16/2000
10
2/16/2000
11
2/16/2000
12
2/16/2000
13
2/16/2000
14
2/16/2000
15
2/16/2000
16
What is Multithreading?
Multithreading is an environment where more than one thread of execution is active in a program:
Single threaded: Exclusive access to memory, files, etc.
2/16/2000
17
2/16/2000
18
2/16/2000
19
2/16/2000
20
Threads Events
2/16/2000
21
Visual Basic programmers C programmers Why Threads Are Hard C++ programmers Synchronization: Threads programmers Must coordinate access to shared data with locks. Too hard for most programmers to use. Forget a Why Threads Are Hard, cont'd lock? Corrupted data.
u
Even for experts,Deadlock: u development is painful. u Hard to debug: data dependencies, timing dependencies. Circular dependencies among locks. u Threads break abstraction: can't design Each process waits for some other process: system modules independently. 28, 1995, slide 5 Why Thre ads Are A Bad Idea September hangs. u Callbacks don't work with locks. thread 1 lock A T1 lock T2 deadlock! B thread 2 T1
calls Module A deadlock! September 28, 1995, slide 6 Module B sleep wakeup Module B callbacks T2
September 28, 1995, slide 7
Module A
Why Thre ads Are A Bad Idea
2/16/2000
22
Multithreaded Tcl
For better or worse, Tcl was not designed with thread support in mind as the single threaded, event-driven model served most purposes well (e.g., Tk) However, the code was well implemented so that adding thread support was not entirely impossible Lets look at what was done in Tcl 8.2 to support threads...
2/16/2000
23
2/16/2000
24
2/16/2000
25
3) Notifier wakes up individual threads when file of interest is ready Select Loop
2. Notifier watches files for all threads with single select() loop
2/16/2000
26
2/16/2000
27
In addition, AOLserver addresses a basic problem of multithreaded Tcl - nearly seamless multithreaded interpreter initialization
2/16/2000
28
2/16/2000
29
2/16/2000
30
Interp Pool
Interp Pool
Interp Pool
2/16/2000
31
2/16/2000
32
2/16/2000
33
2/16/2000
34
2/16/2000
35
2/16/2000
36
2/16/2000
37
2/16/2000
38
2/16/2000
39
2/16/2000
40
2/16/2000
42
2/16/2000
44
Digital City also inherits AOLs media-company legacy of continuous, instant editorial content publishing
2/16/2000
45
2/16/2000
46
Art01
Update Form
Title
Art02
Front End
Publish
Member
Switch
Front End
Pub
Editor
Member -request s http://home.digitalci ty.com -Swi tch distributes reques t -Front End returns page with updated components
Front End
Poll
SOB/NCB 3 Front End Server -request s update from SOB server -c ac hes updated content
Pub Server -Adds image to Art servers -Updates SOB server -Sends cache flush notic e t o Front End servers
2/16/2000
47
RPC
AOLserver OS (SUN, IRIX, LINUX)
2/16/2000
48
Some Numbers...
The Digital City platform currently comprises:
6000+ ADP templates with 170,000+ lines of HTML and Tcl 100,000+ lines of Tcl in an additional 800+ library files 3,000+ Tcl procs and 450+ C functions, of which about 280 implement Tcl commands Multiple SQL/PLS databases with collectively over 1M records
2/16/2000
49
Examples
Seems like theres probably not enough time to look at all that! So, lets take a look at three interesting examples:
NV: Network shared variable arrays Proxy: Tcl proxy process interface Sob: Small Object server
NCB ROSETTA
POLL
NV
PROXY
ART
SOB NCF
RPC
2/16/2000
NT
50
Example: NV
NV (Network Variable) is an extension of the nsv shared variable interface in AOLserver:
A client (e.g., front end server) can request streaming updates from a backend server Client variable interface remains the same
2/16/2000
51
Example: NV
1. Front end serveres connect to Pub for scores nv array Pub 2. Sports scores sent continuously to Pub by provider in weird format
4. Tcl templates access scores nv array without regard to underlying network updates: <html> Current Scores: <%= [nv.get scores current] %> ...
2/16/2000
52
Example: Proxy
The proxy interface is a modified Tcl-shell capable of communicating via pipes with AOLserver:
Processes are forked on demand and maintained in a pool Pools are initialized with application-specific startup scripts, e.g., open a particular PLS database
2/16/2000
53
Example: Proxy
2. Scripts sent to proxy with proxy.send: <html> Search Results: <% set script pls.search $query foreach doc [proxy.send db $script] { ns_adp_puts [putheadline $doc] } %> ... Proxy Shell
AOLserver Process
2/16/2000
54
Example: Sob
Sob (Small Object) is a highly optimized client/server interface:
Access through pools of persistent socket connections Aggressive caching and optimized I/O
2/16/2000
55
Example: Sob
1. Editor uses Pub tool to send article to SOB
2. Member requests article; template ignores complexity and simply executes: <html> Article: <%= [nsob.get articleid] %> ...
Pub 3. Clients checks for article in client cache and, if necessary, requests from server
SOB Server 4. SOB checks server cache and, if necessary, reads article from disk, and returns article to client
2/16/2000
56
SOB
Templates
2/16/2000
57
2/16/2000
58
MovieGuide
The Digital City MovieGuide provides our members:
Theater, movie, and show time listing Boards for members to provide ratings and reviews My Theaters personalization Top Ten listings by member views and clicks
2/16/2000
59
MovieGuide Features
My Theaters
60
MovieGuide Components
MovieGuide uses many of our platform extensions including:
Fast front end PLS search database through Tcl proxies Network tally system to record site activity NV to broadcast aggregation of site activity Sob for editorial content And much more
2/16/2000
61
MovieGuide 1.0
MovieGuide 1.0, launched summer 1997, utilized a straightforward SQL-based architecture to query listings and update Top-10s
Alamo Drafthouse, Regal Arbor 7... select movies where update views where Where is the English Patient showing?
2/16/2000
62
A more dynamic model which could handle direct traffic was required...
2/16/2000
63
MovieGuide 2.0
MovieGuide 2.0, early 1998, utilized a new file-based caching mechanism and background Top-10 updates:
Access listings through cache Regal Arbor 7, Dobie 4... Where is Titanic showing? On cache miss, select movies where
2/16/2000
64
Cache hit %
60 50 40 30 20 10 0
Increasing data combinations e.g., more cities, with more theaters, with more showtimes
2/16/2000
65
MovieGuide 3.0
Launched in mid-1998, MovieGuide 3.0 utilized a new fast PLS-based proxy database and a special network tally mechanism for Top-10s:
Proxy PLS Front End Proxy Switch PLS Front End Proxy PLS Front End Periodic Tally sweep and broadcast PLS Pub Proxy Sybase DB Asynchronous Tally messages
2/16/2000
66
2/16/2000
67
Monitoring Performance
As demonstrated by the MovieGuide, one can easily fall into some deadly performance traps One of the first lesson learned by Digital City is to watch for trouble by monitoring whats happening under the covers Lets look at a few statistical interfaces...
2/16/2000
68
URL Stats
The ns_server urlstats provides timing statistics of individual URLs, a good first place to look for performance hotspots:
2/16/2000
69
Tcl Statistics
The ns_stats command returns the usage counts of individual commands, a great way to look for fat procedures which could be moved to C:
2/16/2000
70
2/16/2000
71
Cache Statistics
To monitor memory cache hit rate, use the ns_cache_stats command:
2/16/2000
72
Lock Contention
To monitor lock contention, use the ns_info locks command:
2/16/2000
73
Overall Performance
Finally, because multithreaded applications can be so unpredictable, be sure to track overall performance under actual load - Tk is a good choice:
2/16/2000
74
Wrap Up
2/16/2000
76
2/16/2000
77