Assignment 2 Part 3: Asynchronous middleware

1. Illustrate using diagram and briefly describe, How Queues are used in
TP-Monitors?

A TP-Monitor allows building a common interface to several applications

while maintaining or adding transactional properties. Examples: CICS,
Tuxedo, Encina.
A TP-Monitor extends the transactional capabilities of a database beyond the
database domain. It provides the mechanisms and tools necessary to build
applications in which transactional guarantees are provided.
TP-Monitors are, perhaps, the best, oldest, and most complex example of
middleware. Some even try to act as distributed operating systems providing
file systems, communications, security controls, etc.
TP-Monitors have traditionally been associated to the mainframe world. Their
functionality, however, has long since migrated to other environments and has
been incorporated into most middleware tools.

TP-Monitor, generic architecture

Detailed Structure of a TP Monitor Detailed Structure of a TP Monitor

 Queue manager handles incoming messages
 Some queue managers provide persistent or durable message queueing
contents of queue are safe even if systems fails.
 Durable queueing of outgoing messages is important
  application server writes message to durable queue as part of a
transaction
 once the transaction commits, the TP monitor guarantees
message is eventually delivered, regardless of crashes.
 ACID properties are thus provided even for messages sent
outside the database
 Many TP monitors provide locking, logging and recovery services, to
enable application servers to implement ACID properties by
themselves.
2. Illustrate using diagram and briefly describe, How to access a queue?

TP systems offer both queued and direct transaction processing. They offer
both client server and peer-to peer direct processing.
Gray & Reuter offer the following common taxonomy of process-to-process
communication:

Direct
Peer-To-Peer: either process can
initiate request. Responses are not
necessarily paired to requests.
Client-Server: client sends request to
server, server responds.
Queued
Client-Server: client sends request to
queue, server reads queue and places
response in output queue, client reads
output queue.

The shaded boxes in the figure represent a dispatcher that binds client
requests to servers. The dispatcher was traditionally called a TP-monitor.
Today dispatchers are called Object-Request Brokers (ORBs).

In queued processing, clients place request messages in a queue. A pool of

server
processes, managed by a TP-monitor, service these request queues, perhaps
placing
results in other queues. Clients can poll these output queues to see the status
or outcome of their transaction requests. If the client and server on different
computers, the queue may be replicated at both the client and server node so
that either end can generate and process messages even if when disconnected.

Queued processing is the basic mechanism of IBM’s IMS, so we have 30

years
experience with its pros and cons. Advocates of queued processing point out
that, at saturation, a direct system is really a queued system: the TPmonitor
dispatches servers via a queuing mechanism to do load control. When clients
saturate a server pool, the queues become visible. Indeed, it is optimal to
schedule new requests to a server pool via a single global queue.

The difficulty is that queued transaction processing of a request-response is

three ACID
units:
1. Client places request in queue.
2. Server dequeues request, performs task,
enqueues response.
3. Requester dequeues response from output queue.

This tri-ACID unit model has the benefit of decoupling the client from the
server, but has the liability that it makes multi-request (conversational)
transactions impossible. Since each message exchange is three ACID units,
one cannot wrap a multi-step dialog in a single ACID transaction unit.
Implementing distributed transactions, conversational transactions, or multi-
step transactions on top of a queued system requires building a lot of
application-level machinery. IMS customers have invested millions of dollars
in such efforts.

By contrast, direct transaction processing systems can easily add a queuing

mechanism as a new transaction type. They implement a direct transaction
that places requests in queues, have pools of servers that poll these queues,
and have a third transaction that queries the output queues. This is the
approach that CICS, ACMS, Tuxedo, Topend, and Encina take. At last count,
CICS had over six distinct queue managers as part of the regular product --
each with slightly different performance-functionality tradeoffs. I am not
arguing that queued processing is bad -- quite the contrary. Queued
processing has been a common transaction processing style and will continue
to be very important in the future. Queued processing is increasingly
important for workflow, disconnected operation, and replication applications.
It has always been the mainstay of batch and spool operations. Using Bill
Highliman’s acronyms2, the world needs both MOM and
DAD.
3. Why Reliable queuing turned out to be a very good idea?

The other aspect of reliable delivery is assuring that the broker does not lose
messages or delivery information before messages are actually delivered. In
general, messages remain in memory until they have been delivered or they
expire. However, if the broker should fail, these messages would be lost.
A producer client can specify that a message be persistent, and in this case,
the Message Router will pass the message to a Persistence Manager that
stores the message in a database or file system so that the message can be
recovered if the broker fails.

4. Illustrate using diagram and briefly describe Queuing systems

A message queue is a component of messaging middleware solutions that

enables independent applications and services to exchange information.
Message queues store “messages”—packets of data that applications create
for other applications to consume—in the order they are transmitted until the
consuming application can process them. This enables messages to wait
safely until the receiving application is ready, so if there is a problem with the
network or receiving application, the messages in the message queue are not
lost.

This model, known as asynchronous messaging (15:11), prevents data loss

and enables systems to continue to function if processes or connections fail.
This allows developers to keep processes and applications separate, keeping
their communications self-contained and event-driven to make the
architecture more reliable.

Message queues are available in messaging solutions across numerous

deployment options, including optimized physical appliances, cloud services,
mainframes, and as software.
 The main parts of a Message Queue messaging system, shown in Figure , are
the following:
 Message Queue Message Server
 Message Queue Client Runtime
 Message Queue Administered Objects
 Message Queue Administration
5. Why Persistent queues can be implemented as part of a database?

When persistent messages are enqueued, they are stored in database tables.
The performance characteristics of queue operations on persistent messages
are similar to underlying database operations. The code path of
an enqueue operation is comparable to SELECT and INSERT into a
multicolumn queue table with three index-organized tables. The code path of
a dequeue operation is comparable to a SELECT operation on the multi-
column table and a DELETE operation on the dequeue index-organized table.
In many scenarios, for example when Oracle Real Application Clusters
(RAC) is not used and there is adequate streams pool memory, the dequeue
operation is optimized and is comparable to a SELECT operation on a multi-
column table.
Persistent Messaging Basic Tuning Tips
Database table layout is similar to a layout with ordinary database tables and
indexes.
Memory Requirements
Streams pool size should be at least 20 MB for optimal multi-consumer
dequeue performance in a non-RACdatabase. Persistent queuing dequeue
operations use the streams pool to optimize performance, especially under
concurrency situations. This is, however, not a requirement and the code
automatically switches to a less optimal code path.
Using Storage Parameters
Storage parameters can be specified when creating a queue table using
the storage_clause parameter. Storage parameters are inherited by other IOTs
and tables created with the queue table. The tablespace of the queue table
should have sufficient space to accommodate data from all the objects
associated with the queue table. With retention specified, the history table as
well as the queue table can grow to be quite big.
Oracle recommends you use automatic segment-space management (ASSM).
Otherwise initrans, freelists and freelist groups must be tuned for AQ
performance under high concurrency.
Increasing PCTFREE will reduce the number of messages in a queue
table/IOT block. This will reduce block level contention when there is
concurrency.
Storage parameters specified at queue table creation are shared by the queue
table, IOTs and indexes. These may be individually altered by an online
redefinition using DBMS_REDEFINTION.
I/O Configuration
Because Database is very I/O intensive, you will usually need to tune I/O to
remove any bottlenecks.
6. briefly describe the advantages of queues in EAI

Enterprise application integration is an integration framework composed of a

collection of technologies and services which form a middleware or
"middleware framework" to enable integration of systems and applications
across an enterprise.[1]

Many types of business software such as supply chain management

applications, ERP systems, CRM applications for managing customers,
business intelligence applications, payroll, and human resources systems
typically cannot communicate with one another in order to share data or
business rules. For this reason, such applications are sometimes referred to as
islands of automation or information silos. This lack of communication leads
to inefficiencies, wherein identical data are stored in multiple locations, or
straightforward processes are unable to be automated.

The approaches used for EAI have evolved from deploying Point-to-Point
solutions, to Message Broker architectures, and then to Process Broker
architectures (see fig. 1). This line of development clearly reveals the two
main forces driving it, namely:
1) complexity reduction, by moving from a Point-to-Point to a Message
Broker architecture, and 2) process logic extraction and encapsulation,
which is the essence is first in the queue and is therefore consumed, and
the process instance performs a transition to the state
Wait for Event 1. During the transition a message m3 is sent to human actor D
and a timer T1 is started.
Thereafter the message m9 is first in the queue. Since only message m5 can
initiate a further transition from Wait for Event 1, message m9 is discarded
(sent to the back of the queue). The next message in the queue is then m5,
which initiates Automated Business Decision 1 and the process instance
continues following the path satisfying the specified business rule.
7. What is message broker? Briefly, describe benefits and limitations of
message brokers?

Messages can be sent directly between two applications, but this requires the
two applications to know about each other when they are written; it also
means that both applications need to be online at the same time and producing
and consuming data at the same rate to communicate. This hard-wiring of
communication between applications does not scale as more and more
applications become interested in the information being shared.
A message broker provides a decoupling layer. By sending messages to a
third party - the message broker - a message-producing application no longer
has to know about all the applications that are interested in its information.
The message broker can provide queues that carry the messages to interested
message consuming applications. The message broker also provides a buffer
that allows the applications involved to produce and consume data at different
rates.

Benefits of message broker:

Message brokers are software modules within messaging middleware or

message-oriented middleware (MOM) solutions. This type of middleware
provides developers with a standardized means of handling the flow of data
between an application’s components so that they can focus on its core logic.
It can serve as a distributed communications layer that allows applications
spanning multiple platforms to communicate internally.

Message brokers can validate, store, route, and deliver messages to the
appropriate destinations. They serve as intermediaries between other
applications, allowing senders to issue messages without knowing where the
receivers are, whether or not they are active, or how many of them there are.
This facilitates decoupling of processes and services within systems.

In order to provide reliable message storage and guaranteed delivery, message

brokers often rely on a substructure or component called a message queue that
stores and orders the messages until the consuming applications can process
them. In a message queue, messages are stored in the exact order in which
they were transmitted and remain in the queue until receipt is confirmed.

Asynchronous messaging (15:11) refers to the type of inter-application

communication that message brokers make possible. It prevents the loss of
valuable data and enables systems to continue functioning even in the face of
the intermittent connectivity or latency issues common on public networks.
Asynchronous messaging guarantees that messages will be delivered once
(and once only) in the correct order relative to other messages.
 Message brokers may comprise queue managers to handle the interactions
between multiple message queues, as well as services providing data routing,
message translation, persistence, and client state management functionalities.

8. What is SOAP?
SOAP is an acronym for Simple Object Access Protocol. It is an XML-based
messaging protocol for exchanging information among computers. SOAP is
an application of the XML specification.

Points to Note

 SOAP is a communication protocol designed to communicate via

Internet.
 SOAP can extend HTTP for XML messaging.
 SOAP provides data transport for Web services.
 SOAP can exchange complete documents or call a remote procedure.
 SOAP can be used for broadcasting a message.
 SOAP is platform- and language-independent.
 SOAP is the XML way of defining what information is sent and how.
 SOAP enables client applications to easily connect to remote services
and invoke remote methods.
Although SOAP can be used in a variety of messaging systems and can be
delivered via a variety of transport protocols, the initial focus of SOAP is
remote procedure calls transported via HTTP.
Other frameworks including CORBA, DCOM, and Java RMI provide similar
functionality to SOAP, but SOAP messages are written entirely in XML and
are therefore uniquely platform- and language-independent.
9. Illustrate using diagram and briefly describe the structure of SOAP
messages?
SOAP message structure XML isn’t the only reason SOAP is considered
verbose and heavy compared to REST. It’s also the way SOAP messages are
composed. Standard SOAP API requests and responses appear as an
enveloped message that consists of four elements with specific functions for
each one.

Envelope is the core and essential element of every message, which begins
and concludes messages with its tags, enveloping it, hence the name.
Header (optional) determines the specifics, extra requirements for the
message, e.g. authentication.
Body includes the request or response.
Fault (optional) shows all data about any errors that could emerge throughout
the API request and response.
10. Briefly describe The SOAP Header
The optional Header element offers a flexible framework for specifying
additional application-level requirements. For example, the Header element
can be used to specify a digital signature for password-protected services.
Likewise, it can be used to specify an account number for pay-per-use SOAP
services.

Points to Note

 It is an optional part of a SOAP message.

 Header elements can occur multiple times.
 Headers are intended to add new features and functionality.
 The SOAP header contains header entries defined in a namespace.
 The header is encoded as the first immediate child element of the
SOAP envelope.
 When multiple headers are defined, all immediate child elements of
the SOAP header are interpreted as SOAP header blocks.

SOAP Header Attributes

A SOAP Header can have the following two attributes −

Actor attribute
The SOAP protocol defines a message path as a list of SOAP service nodes.
Each of these intermediate nodes can perform some processing and then
 forward the message to the next node in the chain. By setting the Actor
attribute, the client can specify the recipient of the SOAP header.
Must Understand attribute
It indicates whether a Header element is optional or mandatory. If set to true,
the recipient must understand and process the Header attribute according to its
defined semantics, or return a fault.

11. Briefly describe The SOAP Body

The SOAP body is a mandatory element that contains the application-
defined XML data being exchanged in the SOAP message. The body
must be contained within the envelope and must follow any headers
that might be defined for the message.
The body is defined as a child element of the envelope, and the
semantics for the body are defined in the associated SOAP schema.
The body contains mandatory information intended for the ultimate
receiver of the message.

12. Briefly describe SOAP Fault element (v 1.2)

If an error occurs during processing, the response to a SOAP message

is a SOAP fault element in the body of the message, and the fault is
returned to the sender of the SOAP message.
The SOAP fault mechanism returns specific information about the
error, including a predefined code, a description, and the address of
the SOAP processor that generated the fault.

Points to Note

 A SOAP message can carry only one fault block.

 Fault is an optional part of a SOAP message.
 For HTTP binding, a successful response is linked to the 200 to
299 range of status codes.
 SOAP Fault is linked to the 500 to 599 range of status codes.

Sub-elements of Fault
The SOAP Fault has the following sub elements –

Sr.N Sub-element & Description

o

1 <faultCode>
 It is a text code used to indicate a class of errors. See the next Table for a listing
of predefined fault codes.
2 <faultString>
It is a text message explaining the error.
3 <faultActor>
It is a text string indicating who caused the fault. It is useful if the SOAP message
travels through several nodes in the SOAP message path, and the client needs to
know which node caused the error. A node that does not act as the ultimate
destination must include a faultActor element.
4 <detail>
It is an element used to carry application-specific error messages. The detail
element can contain child elements called detail entries.

SOAP Fault Codes

The faultCode values defined below must be used in the faultcode element while

describing faults.
Sr.N Error & Description
o

1 SOAP-ENV:VersionMismatch
Found an invalid namespace for the SOAP Envelope element.
2 SOAP-ENV:MustUnderstand
An immediate child element of the Header element, with the mustUnderstand
attribute set to "1", was not understood.
3 SOAP-ENV:Client
The message was incorrectly formed or contained incorrect information.
4 SOAP-ENV:Server
There was a problem with the server, so the message could not proceed.
13. SOAP specifies in detail how messages must be processed, Briefly
describe.

The beginning of the 21st century is remembered for the internet boom.
Thousands of internet-driven companies were emerging and millions of users
were accessing the web every day. Now imagine that a single server starts
receiving thousands of requests from users (clients) simultaneously. And if
this resource does something more complex than show walls of text, things
can get slow. For instance, if users check the upcoming flights schedule and
must drill down to each flight detail, the server must be aware of what’s
happening with the client, right?

It appears that you can handle this situation in two ways: using stateful and
stateless operations. And SOAP supports both.

Stateful means that the server keeps the information that it receives from a
client across multiple requests. For instance, first it memorizes the flight dates
that you’re looking for and then provides information on the pricing after the
second request. This allows you to chain messages together, making the
server aware of the previous requests. Stateful messaging may be crucial in
operations involving multiple parties and complex transactions, e.g. bank
operations or flight booking. But still, it’s really heavy to a server.

Stateless communication means that each message contains enough

information about the state of the client so that a server doesn’t have to be
bothered with it. Once the server returns requested data, it forgets about the
client. Each request is isolated from the previous one. Stateless operations
helped reduce server load and increase the speed of communication.

Stateful operations is one of the reasons SOAP is used for bank transactions
and other data exchange that requires chaining messages. More on SOAP use
cases below.
14. Illustrate using Example and briefly describe how HTTP is used as a
communication protocol
As a request-response protocol, HTTP gives users a way to interact
with web resources such as HTML files by transmitting hypertext
messages between clients and servers. HTTP clients generally use
Transmission Control Protocol (TCP) connections to communicate
with servers.

HTTP utilizes specific request methods in order to perform various

tasks. All HTTP servers use the GET and HEAD methods, but not all
support the rest of these request methods:

 GET requests a specific resource in its entirety

 HEAD requests a specific resource without the body content
 POST adds content, messages, or data to a new page under an
existing web resource
 PUT directly modifies an existing web resource or creates a new URI
if need be
 DELETE gets rid of a specified resource
 TRACE shows users any changes or additions made to a web
resource
 OPTIONS shows users which HTTP methods are available for a
specific URL
 CONNECT converts the request connection to a transparent TCP/IP
tunnel
 PATCH partially modifies a web resource

15. Illustrate using Example and briefly describe how HTTP is used at the
server side

The Web server

On the opposite side of the communication channel, is the server,
which serves the document as requested by the client. A server appears
as only a single machine virtually: this is because it may actually be a
collection of servers, sharing the load (load balancing) or a complex
piece of software interrogating other computers (like cache, a DB
server, or e-commerce servers), totally or partially generating the
document on demand.

A server is not necessarily a single machine, but several server

software instances can be hosted on the same machine. With HTTP/1.1
and the Host header, they may even share the same IP address.

Proxies
 Between the Web browser and the server, numerous computers and
machines relay the HTTP messages. Due to the layered structure of the
Web stack, most of these operate at the transport, network or physical
levels, becoming transparent at the HTTP layer and potentially making
a significant impact on performance. Those operating at the application
layers are generally called proxies. These can be transparent,
forwarding on the requests they receive without altering them in any
way, or non-transparent, in which case they will change the request in
some way before passing it along to the server. Proxies may perform
numerous functions:

caching (the cache can be public or private, like the browser cache)
filtering (like an antivirus scan or parental controls)
load balancing (to allow multiple servers to serve the different
requests)
authentication (to control access to different resources)
logging (allowing the storage of historical information)

16. Illustrate using Example and briefly describe how Parameter passing is
done using HTTP.

When the GET request method is used, if a client uses

the HTTP protocol on a web server to request a certain resource, the
client sends the server certain GET parameters through
the requested URL. These parameters are pairs of names and their
corresponding values, so-called name-value pairs.

Parameter passing involves passing input parameters into a module (a

function in C and a function and procedure in Pascal) and receiving
output parameters back from the module. For example a quadratic
equation module requires three parameters to be passed to it, these
would be a, b and c.
Parameters are in the form of attribute/value pairs. parameter =
attribute "=" value attribute = token value = token | quoted-string. All
transfer-coding values are case-insensitive. HTTP/1.1 uses transfer-
coding values in the TE header field (section 14.39) and in the
Transfer-Encoding header field.
17. Illustrate using diagram, and briefly describe given two examples how
HTTP is used as a transport protocol to exchange SOAP messages
between service requestor and service provider?

SOAP is a method of transferring messages, or small amounts of

information, over the Internet. SOAP messages are formatted in XML
and are typically sent using HTTP (hypertext transfer protocol).
 WSDL defines contract between client and service and is static by its
nature.
 SOAP builds an XML based protocol on top of HTTP or sometimes
TCP/IP.
 SOAP describes functions, and types of data.
 SOAP is a successor of XML-RPC and is very similar, but describes a
standard way to communicate.
 Several programming languages have native support for SOAP, you
typically feed it a web service URL and you can call its web service
functions without the need of specific code.
 Binary data that is sent must be encoded first into a format such as
base64 encoded.
 Has several protocols and technologies relating to it: WSDL, XSDs,
SOAP, WS-Addressing.

REST (REpresentational State Transfer):

Rest is a simple way of sending and receiving data between client and
server and it doesn’t have very many standards defined. You can send
and receive data as JSON, XML or even plain text. It’s lightweight
compared to SOAP.
 In case of REST contract between client and service is somewhat
complicated and is defined by HTTP, URI, Media Formats and
Application Specific Coordination Protocol. It’s highly dynamic unlike
WSDL.
 REST is very lightweight, it says wait a minute, we don’t need all of
this complexity that SOAP created.
 Typically uses normal HTTP methods instead of a big XML format
describing everything. For example to obtain a resource you use HTTP
GET, to put a resource on the server you use HTTP PUT. To delete a
resource on the server you use HTTP DELETE.
 REST is a very simple in that it uses HTTP GET, POST and PUT
methods to update resources on the server.
 REST typically is best used with Resource Oriented Architecture
(ROA). In this mode of thinking everything is a resource, and you
would operate on these resources.
 As long as your programming language has an HTTP library, and most
do, you can consume a REST HTTP protocol very easily.
 Binary data or binary resources can simply be delivered upon their request.