PEGA Services and Connectors
PEGA Services and Connectors
PEGA Services and Connectors
Services and Connectors
e‐Learning Course Transcript
© Copyright 2008
Pegasystems Inc., Cambridge, MA
All rights reserved.
This document describes products and services of Pegasystems Inc. It may contain trade secrets
and proprietary information. The document and product are protected by copyright and
distributed under licenses restricting their use, copying distribution, or transmittal in any form
without prior written authorization of Pegasystems Inc.
This document is current as of the date of publication only. Changes in the document may be
made from time to time at the discretion of Pegasystems. This document remains the property
of Pegasystems and must be returned to it upon request. This document does not imply any
commitment to offer or deliver the products or services described.
This document may include references to Pegasystems product features that have not been
licensed by your company. If you have questions about whether a particular capability is
included in your installation, please consult your Pegasystems service consultant.
For Pegasystems trademarks and registered trademarks, all rights reserved. Other brand or
product names are trademarks of their respective holders.
Although Pegasystems Inc. strives for accuracy in its publications, any publication may contain
inaccuracies or typographical errors. This document or Help System could contain technical
inaccuracies or typographical errors. Changes are periodically added to the information herein.
Pegasystems Inc. may make improvements and/or changes in the information described herein
at any time.
This document is the property of:
Pegasystems Inc.
101 Main Street
Cambridge, MA 02142‐1590
Phone: (617) 374‐9600
Fax: (617) 374‐9620
www.pega.com
Updated: March 14, 2008
Contents
Introduction to Application Integration ................................................................................................. 1
Application Program Interfaces ............................................................................................................. 1
Service Oriented Architecture................................................................................................................ 2
Application Integration Transports ....................................................................................................... 3
Application Integration Methods ........................................................................................................... 4
Web Services ........................................................................................................................................... 4
Introduction to PRPC Services and Connectors .................................................................................. 6
PRPC Services......................................................................................................................................... 6
PRPC Connectors ................................................................................................................................... 7
Distributed Data Transformation ........................................................................................................... 7
Error Handling in PRPC Services .......................................................................................................... 8
Error Handling in PRPC Connectors ..................................................................................................... 9
Testing and Debugging PRPC Services.............................................................................................. 10
Testing and Debugging PRPC Connectors ........................................................................................ 11
Service and Connector Considerations for System Architects ........................................................ 11
Introduction to Application Integration
At a high level, application integration is the ability of one application to talk to
another application through a series of requests and responses.
The conversation between two programs is as follows:
■ The calling application transforms data to be packaged up in the request
■ The request is sent to another application (the receiving application)
■ The receiving application parses out data from the request
■ The receiving application performs some processing
■ The receiving application transforms data to be packaged up in the response
■ The response is sent back to the calling application
■ The calling application parses out data from the response
As you will see later, in PRPC there are many ways in which you can integrate
your PRPC application to the external world, and they all support this basic
definition of integration.
Application Program Interfaces
Historically, application integration has been achieved by using proprietary
Application Program Interfaces (APIs) which causes software to become brittle.
Why? Because your software code is likely to contain coding that is very specific
to the API.
For example, let’s say you have developed a front end for a benefits enrollment
application. It communicates with an ERP (Enterprise Resource Planning)
solution as the system of record. Your front end will have code strewn
throughout that specifically references the proprietary API.
What if the API signature in your ERP solution changes? What if you want to
dump your current ERP vendor and go with another? You are forced to rewrite
your front‐end solution. And with a proprietary integration solution, the non‐
integration code is likely to be geared toward the integration solution. Tight
coupling reduces the ability for your software to be used in other contexts
reducing business agility. This is often referred to as a point‐to‐point integration
solution since the two pieces of software integrate directly with one another.
2 Services and Connectors
Service Oriented Architecture
Service Oriented Architecture (SOA) is an evolution of distributed computing
and modular programming. The goal of SOA is to allow fairly large chunks of
functionality to be strung together to form ad‐hoc applications that are built
almost entirely from existing software services. Industry standard integration
protocols are used in a SOA, such as web services (though not limited to web
services). Integration becomes more like “plug and play”.
For example, let’s assume we integrated our enrollment system to the ERP
solution via a web service. Your front‐end solution is likely to be only slightly
coupled with the ERP solution. If you want to use a different ERP vendor, fine.
Just bring in the new vendor and modify some of the web service code. You
should not have to rewrite the front‐end solution from scratch.
Even better, a service oriented architecture is composed of an Enterprise
Service Bus or ESB. This is software that manages the integration between
applications. For example, if our front‐end benefits enrollment system wants
to communicate with a system of record to return a list of dependents, the
front‐end system will send its request to the ESB. The ESB will then determine
the application that gets this request. This could be an Oracle application one
day, SAP the next day, and a homegrown application the following day. As long
as the type of operation, the inputs (e.g. employee id) and outputs (e.g. list of
dependents) have not changed, the front‐end application does not need to
change. In an SOA, communication is always to the ESB, so you are never
communicating directly with another application.
One of the prime benefits of an SOA is that legacy applications or portions of
them can be reused across the enterprise. To allow for this reuse, adapters
(software code) are created that turn legacy applications into services.
Implementations of SOA can even support the abstraction of database calls. For
instance, you might communicate with an ESB via a web service, and the ESB
translates the request to a SQL call via JDBC. Some data transformation may be
necessary, and the SOA can perform this as well.
PRPC services may be created, and then called from an SOA. For instance, a SOA
might string together a PRPC service, along with other applications, to form a
composite application. Note that PRPC can also call services indirectly via a
brokering component of an SOA.
Services and Connectors 3
Application Integration Transports
Integration transports are used to provide connectivity to an underlying data
source or message source in a consistent way. Guaranteed message delivery is
a key feature of products that support messaging and is usually achieved by
having the calling application wait until the message has been successfully
placed in a message queue (usually implemented as a database table in a
relational database). This is often used as the backbone of an SOA’s Enterprise
Service Bus. Even though these messaging transports are inherently
asynchronous, the messages can be implemented synchronously.
A synchronous transport means the calling application is blocked until the called
application has completed its processing.
An asynchronous transport means the calling application can continue
processing while the called application is in the middle of its processing.
Some popular integration transports are:
■ HTTP, which stands for Hypertext Transfer Protocol, is the means by which
one communicates using web browsers. Its transmission is inherently
synchronous.
■ SMTP or Simple Mail Transport Protocol is also a synchronous transmission.
It is usually used to support email messages.
■ JMS (Java Messaging Service) and WebSphere MessageQueue support
asynchronous and synchronous messaging. That way, you get synchronous
processing if you need it, but with better guaranteed message delivery than
over HTTP.
PRPC supports all these transports as part of the protocols used in PRPC services
and connectors.
There are two types of messaging:
■ Point‐to‐point: one sender of the message and one receiver (or consumer)
■ Publish‐subscribe: one sender (or publisher) of the message with multiple
receivers (or subscribers)
4 Services and Connectors
Application Integration Methods
There are multiple ways to integrate one application with another.
Web services use SOAP for its messaging. SOAP messages are most commonly
sent using an HTTP transport, but SOAP with other transports, such as JMS, is
gaining traction. The web services recommendation does not identify a specific
transport.
Integration via .NET uses web services as its core means of communication
between applications. Therefore, .NET integration is implemented in the same
manner as other web services.
You can integrate Java applications with different relational database products
in a generic fashion by using a JDBC (Java Database Connectivity) driver. ODBC
(Open Database Connectivity) exists similarly in the Windows world. There are
also ADO.NET drivers to support database integration specifically in .NET
applications.
BPEL (Business Process Execution Language) is used to orchestrate a set of
services together as a business process. BPEL was designed to orchestrate a set
of system to system communications. Human interaction is not defined in the
BPEL standard. Many SOAs support BPEL integration.
There are also several different ways of integrating with Java applications in the
Java world:
■ EJBs (Enterprise Java Beans)
■ POJOs (Plain Old Java Object), which allows one to integrate with simple
java classes
■ JSR‐94 protocol to communicate between java‐based applications
There is support in PRPC for all these integration methods and more.
Web Services
A web service is a way of integrating web based applications using XML, SOAP
and WSDL open standards over an Internet protocol backbone. XML is used to
tag the data, SOAP is used to transfer the data, and WDSL is used for describing
the services available.
Services and Connectors 5
XML
XML…
■ is a widely used format for application integration
■ is tag‐based, similar to html
■ defines a standard structure for data, not presentation
An XML document contains beginning and ending tags, attributes of the tags,
and values associated with the tag. Tags can be and often are nested.
An XML schema is an xml document that serves to validate the structure of
other xml documents. For instance, a SOAP xml document has a particular
structure of tags and attributes that can define the name of a property and its
type. An XML schema can also be used to validate the structure of a SOAP xml
document.
SOAP
SOAP is a protocol for exchanging XML‐based messages over a network and
forms the basic messaging framework for the web service. When you execute a
request to integrate with an application via web services, you are sending a
SOAP message to another application. A SOAP message is a valid XML
document, and is composed of 3 parts:
1. A SOAP envelope encompasses (or wraps) the entire SOAP message
2. A SOAP header can contain varying types of information. One of the more
popular pieces of information to send inside a SOAP header is
userid/password data that can be used in the called application for
purposes of authentication.
3. A SOAP body identifies the operation to be executed (this might include a
URL that identifies the location of the application to be called). It also
identifies data that goes along with the operation or the results if it is the
SOAP response.
WSDL
A WSDL (Web Services Description Language) is a valid XML document that
provides a model for describing a web service. It can define a set of operations,
along with the type of data expected in the request and response. It is often
used by application software to automatically generate some of the code that is
required in order to execute the operations from the calling application. For
example, Amazon.com allows you to integrate with their systems via web
services. You can freely download WSDL files that programmatically describe
how to implement various functionality.
6 Services and Connectors
Introduction to PRPC Services and Connectors
PRPC supports a set of industry standard integration methods. As a service,
PRPC supports SOAP and .NET over HTTP, JAVA, EJB and JSR‐94 via a JAVA API,
and JMS and WebSphere MQ. PRPC can also be integrated into a Portal
according to the JSR‐168 portlet specifications.
Like Services, PRPC Connectors also support a variety of industry standard
integration methods. A sampling of Connector integration methods are JMS,
WebSphere MQ, SOAP, .NET, BPEL (or Business Process Execution Language),
and SQL.
Check the Integration Topics on the Pega Developer Network for a full list of
supported Service and Connector integration methods.
PRPC can act as a server (service) and a client (connector) or both. For example,
the .NET client sends Consumer ID into PRPC by calling a PRPC service rule.
PRPC then takes the Consumer ID and connects to a credit bureau. The credit
bureau returns the Credit Score. PRPC executes SQL against the organization’s
system of record to see if the consumer has an existing account. PRPC
calculates the interest rate from the credit score and whether the consumer has
an existing account, and returns it to the .NET client. In this example, PRPC is
the server to the .NET application and the client to the credit bureau and
account history database.
In a service oriented architecture (SOA), you would have a software layer, such
as a Enterprise Service Bus (ESB), orchestrating the messaging.
PRPC Services
Incoming service requests are associated with the combination of a Service
Package, Service Rule, and service Activity.
A Service Package is an instance of Data‐Admin‐ServicePackage and identifies an
Access Group, thereby granting necessary authorization to the service request.
A Service Package is specific to a particular integration protocol, such as SOAP or
JSR‐94, and contains a list of one or more Service Methods called Service Rules.
Service Rules are specific to the integration protocol, such as Rule‐Service‐SOAP
for SOAP and Rule‐Service‐JSR94 for JSR‐94.
Service Rules always call Activities. Since Activities can do just about anything,
all functionality available in PRPC, whether it be declarative expressions or
creating work, can all be done via a service request. Both the Service Rule and
service Activity can manipulate the clipboard. Configuration of the Request and
Response parameters are done from within the Service Rule. Sometimes the
Services and Connectors 7
packaging of Service Rules within a single Service Package yields additional
functionality.
PRPC contains many wizards that help support it being executed as a service.
PRPC can publish a WSDL file that allows external systems to call a PRPC service
using SOAP. PRPC can import Javabeans, generating data class structures in
support of JSR‐94 integration.
Depending on the integration method chosen, some services can be generated
via a wizard and others are manually created.
PRPC Connectors
Outgoing connector requests are associated with the combination of a
Connector Rule and Connector Activity.
A Connector Rule is specific to the integration protocol, such as Rule‐Connect‐
SOAP for SOAP and Rule‐Connect‐SQL for SQL. A Connector Rule contains
protocol specific configuration, such as the endpoint URL for a SOAP Connector
and SQL statements for the SQL Connector.
A Connector Rule is executed via an Activity method. The Activity executing the
Connector Rule is called the Connector Activity. This activity prepares the
Request parameters and processes the Response parameters. The Activity can
be called from anywhere activities are called and is often configured via
integrator shapes in a flow rule.
Like Services, PRPC contains many wizards that help support a variety of
Connectors. Through the SOAP Connector Wizard, PRPC can inspect a WSDL
from an external system, generating supporting rules in PRPC. Through the SQL
Connector Wizard, PRPC can inspect metadata from external databases,
generating supporting rules, including SQL statements.
Depending on the integration method chosen, some connectors can be
generated via a wizard and others are manually created.
Distributed Data Transformation
Data transformation is key to all system integration since data must be passed
back and forth correctly between applications.
Transformation could be as simple as taking a string parameter and setting it
into an integer property on the clipboard and for turning an incoming string of
XML into a PageList; PRPC provides rules for us to easily accomplish this. To
8 Services and Connectors
make use of data in a Service Request or a Connector Response, the data must
be mapped to properties on the clipboard.
When PRPC is acting as the client, the Connector Activity may contain steps to
transform the request and response.
■ For simple data mapping the Property‐Set method can be used
■ For more complex data, Apply‐Parse‐Structured and Apply‐Parse‐Delimited
methods are used to process structured files in the Connector Response.
■ Apply‐Parse‐XML is used to parse values out of a string containing XML
■ Property‐Set‐Stream can be used to compose a string containing XML in the
Connector Request
You can also create your own custom Activities and Rule‐Utility‐Function rules
to help with data transformation. Also, there are some specialized ways to
perform data transformation that are dependent upon the integration protocol.
For instance, data in Document style SOAP requests is mapped to the clipboard
by using a series of Model rules.
Data transformation needs are similar when PRPC is executed as a service. The
only difference is that you have the capability to configure mapping rules, such
as Parse XML, directly in the Service Rule. Transformation for simple types can
also be handled directly in the Service rule by mapping them directly to the
clipboard. In many cases you can also perform data transformation within the
Service Activity, just as in the Connector Activity, but it is easier and cleaner to
do it in the Service Rule if you can.
Error Handling in PRPC Services
Many service types either have a Faults tab or Exceptions tab that can be
configured on the Service Rule. Through these facilities you can communicate
service errors to the calling application. You can also configure clipboard
property values to be sent back in case of error. It is especially important to
configure the clipboard property values as this is the only clean way to
communicate fatal exceptions, such as service activity not found.
Remember that your PRPC service may be part of a larger application, possibly
orchestrated by a service oriented architecture (SOA). If your service is
performing updates to data repositories, you may need to build in some manual
rollback processing in case other components within the application fail. If you
are doing database updates and your service is being called through a Java API,
such as JSR‐94, then your application server may be set up to handle extended
transactions that cover all database updates occurring in the same JVM, even
when you are accessing different databases. So it very much depends on the
Services and Connectors 9
type of service you are putting together, along with the type of application
server software (for example, Apache Tomcat versus IBM Websphere).
Error Handling in PRPC Connectors
You need to think about any manual rollback processing you need to put in
place if you have a fatal exception during a particular flow action. Let’s say you
have two Integrator shapes in your flow that execute when submitting a flow
action.
■ The first integrator shape calls a SOAP service that inserts rows into a
database.
■ The second integrator shape calls a SOAP service that also inserts rows into
the database.
There is a failure in the second integrator shape.
From a business perspective, the first operation should be cancelled (rollback)
when the second operation fails. There are a couple things that you can do
depending upon your circumstances, which include:
1. Manage the exception in the Connector Activity called by the second
integrator shape, and as part of managing the exception, issue another
SOAP request to reverse the operation of the first one. You can do this by
configuring a Transition section in the activity step that calls a Connector
Rule.
2. Transfer the current business process to an error handling flow.
Developers make whatever fix is necessary, and deploy new rules to
production. A system administrator restarts the real flow, re‐executing at
the second integrator shape.
3. Manage the actual committal of the data from within the services being
called, such that data in the first operation is not committed until the
second operation appears to be successful. An SOA might help in this
effort.
If you are executing two or more SQL connectors as part of a single flow action,
you may be able to use distributed transactions if your database and application
server software both support it. This would mean that if the second SQL
connector fails, you could actually roll back the operation from the first
connector, which would be done automatically by the application server.
Note: You can run connectors in parallel such that two or more connectors are
run asynchronously.
10 Services and Connectors
Testing and Debugging PRPC Services
There are many facilities to help test out your services:
1. Clipboard: Make use of the clipboard throughout your unit testing.
2. Unit Test Service Activities: You can individually unit test your service
activity prior to creating a service package and service rule. It can be unit
tested in the same way that you would unit test other activities.
Remember that once the service rule is created; the service rule will be
doing things like creating a Page, executing a Model, and setting data from
the Request. You can create a wrapper activity (that calls the service
activity) to set up some of these things, and then unit test by running the
wrapper activity.
3. Unit Test Specialized Rules: If you are using specialized rules in your data
transformations, they can be individually unit tested outside of testing
your service rule or service activity. For instance, there is a Run icon
available on a Rule‐Parse‐XML rule. This allows you to enter the text to be
parsed in several different ways, and then see the results of the parse.
4. Unit Test Service Rules: There is a run icon available on a service rule that
allows you to unit test it. By unit testing a service rule you can not only
test the service activity, but the data transformations as well. You will be
prompted to enter in Request parameters, and if execution is successful,
you can see the Response parameter result. When testing, you can also
identify whether you are going to use the logged in Requestor context or a
completely new Requestor context. If you use the logged in Requestor
context, then you can see the results in the Clipboard viewer.
5. Trace Open Rule: For some service rules you can perform a Trace Open
Rule on the service rule. This opens up a tracer session and you can trace
the service rule, not just the service activity. To make this work you need
to select Services in the trace options. The trace output for a service will
show you the data transformations that take place during Request and
Response.
6. Tracer: For some service rule types you can open up a tracer session and
see the results when testing directly from the external client application.
This is very specific to the service type. It works for Soap and .Net because
these communicate via HTTP.
7. PAL: You can run PAL to individually test out your service rules and service
activities. It is also possible to write PAL statistics to a log file using the
PAL API.
8. prlogging.xml: You can log the execution of PRPC classes that support
service processing by configuring the classes in prlogging.xml.
Services and Connectors 11
9. Alerts: There are different options for configuring alerts that are relevant
to services. Configuration is done in prconfig.xml.
Testing and Debugging PRPC Connectors
You can set up a connector simulator prior to having your connector activity and
connector rule created. Often during application development you will stub out
your integration points (i.e. the integration shapes in your process flow that call
connector activities). Early in development you may not have access to the
database table you want to connect with, or you may not have access to the
SOAP service you want to connect with. The same goes for other types of
connectors. A connector simulator allows you to temporarily execute an activity
that acts as a placeholder until you get the “real” connector activity created.
There is the capability on connector rules to test out the connection to an
external system. PRPC makes sure it can ping the external system. For instance,
PRPC makes sure it can:
1. Connect to the URL specified in a Soap connector rule
2. Connect to the database specified in a SQL connector rule
You can configure alerts for elapsed time of database operations. This is
relevant for SQL Connectors. Configuration is done in prconfig.xml.
Service and Connector Considerations for System
Architects
A person acting in the role of a PRPC System Architect should not:
■ Create the client software code that allows an external system to interact
with a PRPC service
Many integration methods come with components that allow a client
application to understand how to connect into PRPC. For instance, a WSDL
file is generated when you create a PRPC Service using SOAP. This describes
the operations, along with inputs and outputs. Just handoff the WSDL to
someone developing the external client, and they can code things up from
there.
Most modern software contains code generators. For instance,
VisualStudio.NET can read in a WSDL, automatically generating much of the
.NET client code.
12 Services and Connectors
■ Define the code that a connector will be calling
If it’s a web service, the web service should exist. If it’s a SQL connector, the
database table should exist.
■ Define deployment files that a connector uses.
For instance, the Connector Wizard for SOAP uses a WSDL to generate much
of the connector code. The person coding up the external system should be
giving you a WSDL file. In fact, modern software such as VisualStudio.Net
should be able to automatically generate a WSDL for you. This is no
different than PRPC generating a WSDL during execution of the service
wizard.
■ Have to understand or be forced to code portions of an organization’s
integration architecture, which may include an SOA (service oriented
architecture).
An understanding of SOA is needed and will allow you to be a productive
part of conversations involving integration. But in most cases, the PRPC
System Architect should not have to be involved in low level integration
details.