FMW Tuning
FMW Tuning
FMW Tuning
Oracle Fusion
Middleware and SOA
Suite 11g
Integration
Architecture
BERN
LAUSANNE
ZRICH
DSSELDORF
FRANKFURT A.M.
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
FREIBURG I.BR.
HAMBURG
MNCHEN
STUTTGART
WIEN
Introduction
Matthias Furrer
Long-standing experience in SOA and ERP Area
Architect, Consultant, Lead-Developer and Project Manager
SOA Certified Professional
Oracle SOA Blackbelt Consultant
Trainer with Trivadis Oracle Middleware und SOA
Reviewer of Technical Publications
Contact: [email protected]
2013 Trivadis
Agenda
Part I: Monitoring Oracle Fusion Middleware 11g
Manual Monitoring
Automated Monitoring
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Manual Monitoring
WebLogic Logging - Overview
Each server instance forwards a subset of its messages to a domainwide log file. By default, only messages with a severity level of
NOTICE or higher are forwarded. The default location for domain
logs is
DOMAIN_HOME\servers\ADMIN_SERVER_NAME\logs\DOMAIN_NAME.log
Detailed Information can be found in Oracle Configuring Log Files and Filtering
Log Messages for Oracle WebLogic Server (E13739).
4
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Manual Monitoring
WebLogic Logging stdout and stderr
Log rotation cannot be set-up for the .out files, on UNIX systems operating
system commands can be used for this.
5
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Manual Monitoring
Fusion Middleware Logging - Overview
You can view log files using Fusion Middleware Control or the WLST
displayLogs command, or you can download log files to your
local client and view them using another tool
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Manual Monitoring
Configuring FMW (Diagnostic) Log Files
You can configure the server log file Fusion Middleware Control under the SOA
Infrastructure -> Log Configuration Menu
Typically you adjust settings about file names, rotation, file retention here after
initial installation
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Manual Monitoring
Configuring FMW (Diagnostic) Log Files Log Levels
There are two types of loggers: Persistent loggers and Runtime loggers
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Manual Monitoring
Selective Tracing
Selective Tracing allows for increasing the trace level in a domain for a
specified subset of activity and for a specified duration in minutes.
Tracing sessions can be started and stopped from either Fusion Middleware
Control or WLST.
Several of the Selective Tracing options are applicable to SOA and one,
Composite Name, is specifically for SOA.
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Manual Monitoring
Selective Tracing - Configuration
You can configure Selective Tracing from the WebLogic Domain Menu
under Logs -> Selective Tracing
10
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Manual Monitoring
Overview available Utilities
Monitoring SOA Suite Infrastructure
Many options are available to monitor SOA Suite Infrastructure,
WebLogic resources and SOA Suite Components. Among these,
there are :
WebLogic Administration Console
WebLogic Administration Console Monitoring Dashboard
Dynamic Monitoring Services - DMS Spy Servlet
.
11
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Monitoring Utilities
WebLogic Administration Console
Monitoring WebLogic Components and Resources
Among others you can monitor the following in WebLogic
Administration Console:
12
Monitoring Utilities
Dynamic Monitoring Service (DMS)
The Oracle Dynamic Monitoring Service (DMS) enables Oracle Fusion
Middleware components to provide administration tools, such as Oracle
Enterprise Manager, with data regarding the component's performance, state
and on-going behaviour. Fusion Middleware Components push data to DMS and
in turn DMS publishes that data through a range of different components..
DMS Spy Servlet
WLDF
WLST
JConsole (MBeans)
13
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Monitoring Utilities
DMS Spy Servlet
The Monitoring Dashboard is available under the URL:
http(s)://<fmwhost>:<soaport>/dms/Spy
You can also monitor your DMS collected metrics here
14
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Monitoring Utilities
Administration Console Monitoring Dashboard
The Monitoring Dashboard provides views and tools for graphically presenting
diagnostic data about servers and applications running on them. The underlying
functionality for generating, retrieving, and persisting diagnostic data is provided
by the WebLogic Diagnostics Framework. The Monitoring Dashboard provides
additional tools for presenting that data in charts and graphs.
Built-In Views
Custom Views
Metrics Browser
Charts
15
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Monitoring Utilities
Monitoring Dashboard
The Monitoring Dashboard is available under the URL:
http(s)://<fmwhost>:<adminport>/console/dashboard
You can create your own views using the metrics browser for your SOA
Composites
16
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Automated Monitoring
Overview
Automating Monitoring
In order to automating monitoring of your infrastructure and SOA
composites with agents or scripting, the options include :
WebLogic Diagnostics Framework (WLDF)
WebLogic ScriptingTool (WLST)
Java Admin Utility
SOA Management Packs
Oracle Business Activity Monitoring (BAM)
17
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Automated Monitoring
WebLogic Diagnostic Framework (WLDF)
The WebLogic Diagnostics Framework (WLDF) is a monitoring and diagnostic
framework that defines and implements a set of services that run within
WebLogic Server processes and participate in the standard server life cycle.
Monitoring Dashboard
Notifications
Metrics
SMTP, SNMP, JMX
and JMS
18
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
19
2.
3.
Use the Dashboard or the DMS Spy Servlet application to find the metrics and
attributes that needs to be monitored
4.
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
The following DMS metrics groups are good candidates to monitor runtime
activity of SOA Suite composites:
The following DMS metrics groups are good candidates to monitor runtime
activity of JVM:
20
soainfra_component
soainfra_mediator_flow
soainfra_JCABinding
soa_infra_bpel_requests
soa_infra_bpel_activity
JRockit (MBean)
weblogic.management.runtime.JRockitRuntimeMBean
JVM
JVM_GC
JVM_ThreadStats
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Automated Monitoring
WLST
Use can use WLST commands to view information about a specific performance
metric, a set of performance metrics, or all performance metrics for a particular
server or component.
DMS Metric Commands
Server Status
21
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Automated Monitoring
Java Admin Utility
The weblogic.Admin utility can access the Administration Server as well as all
active server instances directly. If the Administration Server is down, you can still
use the weblogic.Admin utility to retrieve runtime information from Managed
Servers and invoke administrative commands.
Java Management Extensions (JMX)
Usable from command line or Java Class
while(mbeanIterator.hasNext()) {
serverRuntime = (ServerRuntimeMBean)mbeanIterator.next();
if(serverRuntime.getState().equals(ServerStates.RUNNING)){
serverRuntime.shutdown();
}
22
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Automated Monitoring
Java Admin Utility Command Line
While the Administration Console interacts only with the Administration Server,
the weblogic.Admin utility can access the Administration Server as well as
all active server instances directly..
23
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Automated Monitoring
SOA Management Packs
The Oracle SOA Management Pack Enterprise Edition delivers comprehensive
management capabilities for a Service-Oriented Architecture-based (SOA)
environment.
Centralized Management Console
Infrastructure Management
Configuration Management
Deployment Automation
Adapter Metrics
Service Level Management
Business Transaction Management
24
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Automated Monitoring
SOA Management Packs
Based on Cloud Control /
Grid Control platform
Not included in SOA Suite
license
WebLogic Management
Packs also available
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Automated Monitoring
Oracle Business Activity Monitoring (BAM)
The primary focus of Oracle Business Activity Monitoring (BAM) is to monitor
Business Transactions. However, due to its alerting capabilities it can also be
used to monitor composite instances.
Visualized dashboards of composite instances can be implemented
Alerting when abnormal situations occur
Deviation of expected results can be monitored
JCA BAM-Adapters can be used in BPEL, BPM and Oracle Service Bus
SOAP web service and JMS interface
26
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Automated Monitoring
Oracle Business Activity Monitoring
Integrated component of Oracle SOA Suite
Included in SOA Suite license
27
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Tuning
Top Performance Areas
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Hardware Resources
Operating System
Repository Database
WebLogic Server
Logging Levels
29
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Configuration
Environment and Deployment Architecture Considerations
Hardware
Clustering
Using the current software versions ensures avoidance of any issues that are already addressed
by patches that Oracle has released for issues encountered by other customers.
30
If it is believed that there might be a need for a clustered. configuration eventually, consider
deploying into a cluster from day one, even if it is a cluster with only one node in it.
Versions
Sufficient resources.
Separate architectural components should be deployed into separate tiers the web servers,
application servers and database should be installed in different tiers, since they require different
tuning. If the WebCenter portal and content management components are being used, they
should also be installed on different machines
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Configuration
Environment and Deployment Architecture Considerations
.ADF Applications
Use dedicated managed servers for ADF Applications such as Human Workflow and BPM
Workspaces if used frequently.
SSL Usage
Policy Store
Development Environments
31
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
33
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Connection Pools
When configuring the data source, ensure that the connection pool has enough free
connections.
Statement Caching
Statement caching can eliminate potential performance impacts caused by repeated
cursor creation and repeated statement parsing and creation. Statement caching also
reduces the performance impact of communication between the application server and
the database server.
Initial Capacity
The initial capacity of physical database connections determines the number of
connections to be opened when the server is started. For frequently used databases it
is recommended to configure a higher value to avoid unnecessary open and close of
connections.
Connection Testing
Try to avoid unnecessary connection testing.
34
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Partition Type
Hash partitioning of AC_PK index
Partitioned and Reverse key index
CI_CREATION_DATE
CUBE_SCOPE
Partition by hash (CIKEY)
MEDIATOR_CASE_INSTANCE Partition by hash ("ID")
XML_DOCUMENT
Partition by hash (document_id)
35
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Number
Not Applicable
Not Applicable
Partitions = 200
Partitions = 200
Partitions = 200
COMPOSITE_INSTANCE_CREATED
BRDECISIONINSTANCE_INDX3
MEDIATOR_INSTANCE_INDEX2
MEDIATOR_INSTANCE_INDEX5
MEDIATOR_INSTANCE_INDEX6
MEDIATOR_INSTANCE_INDEX1
MEDIATOR_INSTANCE_INDEX3
MEDIATOR_CASE_INSTANCE_INDEX2
MEDIATOR_CASE_DETAIL_INDEX1
REFERENCE_INSTANCE_CO_ID
CI_NAME_REV_STATE
DOC_DLV_MSG_GUID_INDEX
STATE_TYPE_DATE
36
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
worklistapp
composer
b2bui
OracleAppsAdapter
OracleAq Adapter
MQSeriesAdapter
OracleBAMAdapter
37
Purge instances of SCA processes regularly using the date-based purge scripts or partitioning.
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Audit Level
Level of message tracking infrastructure to be collected by the infrastructure . This information is
collected in repository database. Should not be set to Development if not needed.
Logging Level
The default logging level is "NOTIFICATION". For production environments, consider using the
lowest acceptable logging level, such as "ERROR" or "WARNING" whenever possible.
Property CompositeInstanceStateEnabled
This option enables separate tracking of the running instances. All instances are captured as
either running or not running and should not be enabled in production environments.
Property instanceTrackingAuditTrailThreshold
This parameter is used to limit the audit trail size while it is being built. The default value is 1MB.
This parameter can improve performance, as it prevents huge audit trails to potentially consume a
lot or all the memory.
38
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Higher Settings will allow for more concurrency ( but will increase JVM heap
utilization)
39
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Description
async.persist (Default)
async.cache
sync
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Metrics Level
This property controls DMS metrics tracking level. By default, DMS metrics collections is enabled. If
you do not need to collect DMS metrics data, consider setting the metricsLevel to Disabled to
improve performance.
When Mediator routing rule type is set to 'Parallel', DeferredMaxRowsRetrieved sets the
number of maximum rows (maximum number of messages for parallel routing rule processing)
that are retrieved from Mediator store table (which stores messages for parallel routing rule for
processing.) Default Value is 200 rows.
Audit Level
Configure only the minimum required audit level for the respective environment.
41
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Process Complexity
As a general rule of thumb, it is not desirable to have a BPEL process with more than one hundred
activities in it or a BPMN process with more than about fifty activities.
Consider breaking the process down into smaller sub-processes in order to control the amount of
memory required to maintain state and variables.
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Use of Loops
Avoid extensive use of loops. Use of large loops (i.e. more then ten activities in them) tends to
perform poorly with large number of instances or large variables.
Use of large loops can require the engine too keep large data objects in the JVM heap and perform a
large number of database transactions for dehydration of state and could lead to garbage collection
bottlenecks.
Consider using de-batching strategies for handling large payloads and to control the number of
records handled in a transaction.
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Complex Decisions
If there is a need to make a complex decision, consider implementing the decision in the rules
engine rather than modeling a set of if/then logic in the process itself. Execution of complex logic in
the rules engine is much more efficient in terms of memory footprint, speed of execution and volume
of audit data generated.
*) Source: Oracle , March 2013, Performance Tuning for Oracle Business Process Management Suite 11g
44
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Tuning OSB
Operational Settings
Monitoring
Enable Monitoring only if needed to reduce network traffic.
Tracing
Turn-on Tracing only if needed in order to avoid printing the entire message context including
headers and message body in the server logs.
45
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
*) Source: Oracle Fusion Middleware Performance and Tuning Guide 11g (E10108)
46
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
XQuery Tuning
*) Source: Oracle Fusion Middleware Performance and Tuning Guide 11g (E10108)
47
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Tuning
Top Resources
48
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
Trivadis AG
Thank you.
www.trivadis.com
BASEL
49
BERN
LAUSANNE
ZRICH
DSSELDORF
FRANKFURT A.M.
2013 Trivadis
Monitoring and Tuning Fusion Middleware 11g
April 2013
FREIBURG I.BR.
HAMBURG
MNCHEN
STUTTGART
WIEN