Mining WebLogic Diagnostic Data with XSLT

The following example is based on a blog written by Philip Aston (thanks for the permission)

Using WLDF, you can monitor and record pretty much anything you want to about the behavior of a running WebLogic Server domain. The standard tools can be used to extract harvested data and events from the diagnostic archive but provide little to allow you to build a composite picture from the harvested data. This example uses advanced WebLogic and WLDF techniques to show how to use XSLT to reformat diagnostic data and generate hierarchical HTML reports from the WLDF diagnostic data.

This example shows how to perform a number of useful transformations to the collected data, like reformat diagnostic data as a comma-separated value file so it can be easily imported into spreadsheet tools and recombine raw event data into a hierarchical structure and present that data as an HTML report. In conjunction with the WLDF diagnostic context, this can be used to show the flow of events across multiple WebLogic Server instances.

Process Overview

The example uses a number of stylesheets to process the diagnostic data. The following depicts the overall process for a better understanding.

WebLogic Server stores collected metrics and event data in its diagnostic archive. WLST can then be used to export the data into XML files (events.xml and harvest.xsl). Stylesheets are used afterwards to convert this exported data to comma-separated value (CSV) format to merge two exported data files into one, to build up an XML file that is structured according to the event tree (eventsToTree.xsl). Finally an HTL report will be generated.

First Step: Instrument Your System

First, configure WLDF to instrument your application code to produce event data and to regularly harvest metric data from interesting MBeans. WebLogic Server will store this information in its diagnostic archive.

It is necessary to add instrumentation instructions to your application deployment. To specify the instrumentation, add a weblogic-diagnostics.xml file to the META-INF directory of your application. Make sure you include plenty of delegating monitors or custom monitors with a diagnostic location type of around and a TraceElapsedTimeAction action. These record the time taken by a particular operation when it completes. You'll be needing this data later to build event trees.

This example captures a variety of servlet and EJB operations:

<?xml version="1.0" encoding="UTF-8"?>

<wldf-resource xmlns="http://www.bea.com/ns/weblogic/90/diagnostics">

  <instrumentation>

    <enabled>true</enabled>

    <wldf-instrumentation-monitor>

      <name>Trace_Servlet_Around_Service</name>

      <action>TraceElapsedTimeAction</action>

      <location-type>around</location-type>

      <pointcut>execution(* +javax.servlet.Servlet service(+javax.servlet.ServletRequest, +javax.servlet.ServletResponse))</pointcut>

    </wldf-instrumentation-monitor>

    <wldf-instrumentation-monitor>

      <name>Trace_EJB_Session_Business_Methods</name>

      <action>TraceElapsedTimeAction</action>

      <location-type>around</location-type>

      <pointcut>execution(* +javax.ejb.SessionBean * (...))

        AND NOT execution(* * __WL* (...))

        AND NOT execution(* * ejbCreate (...))

        AND NOT execution(* * ejbRemove ())

        AND NOT execution(* * ejbActivate ())

        AND NOT execution(* * ejbPassivate ())

        AND NOT execution(* * setSessionContext (...))

      </pointcut>        

    </wldf-instrumentation-monitor>

    <wldf-instrumentation-monitor>

      <name>Trace_EJB_Session_Home_Methods</name>

      <action>TraceElapsedTimeAction</action>

      <location-type>around</location-type>

      <pointcut>execution(* +javax.ejb.EJBHome * (...))

            AND execution(* *Impl * (...))

      </pointcut>        

    </wldf-instrumentation-monitor>

    <wldf-instrumentation-monitor>

      <name>Trace_EJB_Stub_Calls</name>

      <action>TraceElapsedTimeAction</action>

      <location-type>around</location-type>

      <pointcut>call(* +wldfexample.SessionEJBRemote * (...))

        AND NOT call(* *_EOImpl* * (...))

       </pointcut>

    </wldf-instrumentation-monitor>

  </instrumentation>

</wldf-resource>

This example specifies monitors with own custom pointcuts, rather than rely on the standard delegating monitors. This provides more control over precisely which events are captured. For example, the last monitor traces only EJB stubs for the test EJB I have been deployed ( wldfexample.SessionEJBRemote).

Add a system diagnostic module to your domains

The instrumentation specified in an application's weblogic-diagnostics.xml file will not be applied unless there is a system diagnostic module with instrumentation enabled, and the system diagnostic module is targeted to the WebLogic Server instance.

If you change the targeting of the system diagnostic module, or change whether instrumentation is enabled, you must fully redeploy your application for the change to take effect. Only a full redeployment will perform the AOP weaving process and change the instrumentation. Stopping and starting the application through the WebLogic console is not sufficient. It is required to either use the WLST redeploy() command or to fully redeploy (untarget, change and retarget) the application. Afterwards restart your servers.

Whenever you change weblogic-diagnostics.xml, you must also do a full redeployment.  You can create a system diagnostic module using the WebLogic console. This results in a file in the config/diagnostics directory of your domain.

The following listing shows an example for such a diagnostic module:

<?xml version='1.0' encoding='UTF-8'?>

<wldf-resource xmlns="http://www.bea.com/ns/weblogic/90/diagnostics"

 xmlns:sec="http://www.bea.com/ns/weblogic/90/security"

 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

 xmlns:wls="http://www.bea.com/ns/weblogic/90/security/wls"

 xsi:schemaLocation="http://www.bea.com/ns/weblogic/90/diagnostics http://www.bea.com/ns/weblogic/920/diagnostics.xsd">

  <name>MySystemDiagnosticModule</name>

  <instrumentation>

    <enabled>true</enabled>

  </instrumentation>

  <harvester>

    <enabled>true</enabled>

    <sample-period>10000</sample-period>

    <harvested-type>

      <name>weblogic.management.runtime.WorkManagerRuntimeMBean</name>

      <harvested-attribute>CompletedRequests</harvested-attribute>

      <harvested-attribute>PendingRequests</harvested-attribute>

      <harvested-attribute>StuckThreadCount</harvested-attribute>

      <harvested-instance>com.bea:ApplicationRuntime=WLDF,Name=default,ServerRuntime=AdminServer,Type=WorkManagerRuntime</harvested-instance>

    </harvested-type>

    <harvested-type>

      <name>weblogic.management.runtime.JTAStatisticsRuntimeMBean</name>

      <harvested-attribute>TransactionCommittedTotalCount</harvested-attribute>

      <harvested-attribute>TransactionRolledBackTotalCount</harvested-attribute>

      <harvested-instance>com.bea:Name=JTARuntime,ServerRuntime=AdminServer,Type=JTARuntime</harvested-instance>

    </harvested-type>

  </harvester>

  <watch-notification></watch-notification>

</wldf-resource>

In addition to the instrumentation the diagnostic module which is defined in the above configuration also configured a number of MBean harvesters.

The Diagnostic Context

An interesting feature of WLDF not commonly found in other tools is its ability to create and propagate a diagnostic "context". This diagnostic context uniquely identifies a particular request, and flows across synchronous calls to other WebLogic Server instances. The diagnostic context includes a unique ID (for example, 583c10bfdbd326ba:-43487c71:112bda0af31:-7ff4-00000000000001b7). You can use this ID to trace event flows across servers.

Related to the diagnostic context is the notion of requesting dyeing. A DyeInjectionMonitor can be used to filter the events that are recorded based on various request characteristics (user identity, protocol type, IP address, and so on). This is a powerful way of focusing what you record and could be a very beneficial extension to this example (not used here to reduce the complexity).

The diagnostic context adds a very small processing overhead: essentially, it's the cost of generating the unique ID, and passing it around with every request. This overhead is negligible. This diagnostic context is generated only if a DyeInjectionMonitor is enabled or if instrumentation is enabled.

WebLogic stores diagnostics in a diagnostic archive local to each server. WLST can be used to extract data from the diagnostic archive

The diagnostic archive contains event and harvested metric data in a binary format. This data can be exported to an XML format using the WebLogic Scripting Tool (WLST), using either the exportDiagnosticData or exportDiagnosticDataFromServer commands.

For example:

exportDiagnosticData(storeDir="/work/domain/servers/server1/data/store/diagnostics",

  logicalName="EventsDataArchive", exportFileName="server1-events.xml")

This example uses two types of data, event data ( logicalName="EventsDataArchive") and harvested metric data ( logicalName="HarvestedDataArchive"). Both exportDiagnosticData and exportDiagnosticDataFromServer have options that allow you to filter data, by time or through a full WLDF query. You should use these filter options to limit the size of the exported XML files.

Here's an example of filtering by a particular diagnostic context ID:

exportDiagnosticData(storeDir="/work/domain/servers/server1/data/store/diagnostics",

  logicalName="EventsDataArchive", exportFileName="events.xml",

  query="CONTEXTID='583c10bfdbd326ba:58f87d21:1126b53f6ec:-7ff3-0000000000000c2a'")

If you look at the exported XML data, you'll see the XML conforms to an abstract schema that models a set of DataRecords consisting of ColumnData values for various columns. The XML file starts with a DataInfo element that contains the names of the columns and their Java types. These columns are slightly different for the event and harvested data files in which we are interested.

How to Run an XSLT Stylesheet

So now we have some XML. Before we get to our first XSLT stylesheet, here's a useful Jython script you can use to process an XML document with a stylesheet:

import sys

from java.io import FileReader, PrintWriter

from java.lang import System

from javax.xml.transform import TransformerFactory, Transformer

from javax.xml.transform.stream import StreamSource, StreamResult

def transform(source, stylesheet, result, parameters):

    transformer = TransformerFactory.newInstance().newTransformer(stylesheet)

    for (p, v) in parameters: transformer.setParameter(p, v)

    transformer.transform(source, result)

if __name__ == '__main__':

    args = sys.argv[1:]

    parameters = []

    while args and args[0].startswith('-'):

        try:

            i = args[0].index('=')

        except ValueError:

            parameters.append((args[0], ""))

        else:

            parameters.append((args[0][1:i], args[0][i+1:]))

        args = args[1:]

    if len(args) == 1:   source = StreamSource(System.in)

    elif len(args) == 2: source = StreamSource(FileReader(args[1]))

    else: raise "Usage: <jython|wlst> process.py -<parameter>=<value> <stylesheetfile> [inputfile]"

    stylesheet = StreamSource(FileReader(args[0]))

    result = StreamResult(PrintWriter(System.out))

    transform(source, stylesheet, result, parameters)

    stylesheet.reader.close()

    source.reader and source.reader.close()

    result.writer.close()

The above WLST script can be called using the following command:

java weblogic.WLST process.py stylesheet.xsl input.xml > output.xml

The output will go to stdout, so we use shell redirection to send it to output.xml. Alternatively, you might use the shell pipe operator to chain multiple process.py processes together.

Some of our stylesheets take parameters that control their behavior. process.py lets you specify stylesheet parameters as follows:

java weblogic.WLST process.py -myparameter=value stylesheet.xsl input.xml > output.xml

Merging Data

The first stylesheet simply merges the exported data from two diagnostic archives. This is a useful way to combine data from multiple WebLogic Server instances. First, extract the data from each server, filtering as appropriate using WLST; then create a combined file for further processing using this stylesheet.

<?xml version="1.0" encoding="utf-8"?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0" xmlns:e="http://www.bea.com/ns/weblogic/90/diagnostics/accessor/Export">

  <xsl:output method="xml"/>

  <xsl:variable name="TIMESTAMP" select="count(//e:ColumnInfo[e:Name='TIMESTAMP']/preceding-sibling::*) + 1"/>

  <xsl:param name="document2"/>

  <xsl:param name="offset" select="0"/>

  <xsl:template match="e:DiagnosticData">

    <xsl:copy>

      <xsl:apply-templates select="@*"/>

      <xsl:apply-templates/>

      <xsl:variable name="d2" select="document($document2)" />

      <xsl:for-each select="$d2/e:DiagnosticData/e:DataRecord">

        <xsl:copy>

          <xsl:apply-templates select="@*" mode="d2"/>

          <xsl:apply-templates mode="d2"/>

        </xsl:copy>

        <xsl:text>

</xsl:text>

      </xsl:for-each>

    </xsl:copy>

  </xsl:template>

  <xsl:template match="e:ColumnData[$TIMESTAMP]" mode="d2">

    <xsl:copy>

      <xsl:value-of select=". + $offset"/>

    </xsl:copy>

  </xsl:template>

  <xsl:template match="node()|@*">

    <xsl:copy>

      <xsl:apply-templates select="@*"/>

      <xsl:apply-templates/>

    </xsl:copy>

  </xsl:template>

  <xsl:template match="node()|@*" mode="d2">

    <xsl:copy>

      <xsl:apply-templates select="@*"/>

      <xsl:apply-templates/>

    </xsl:copy>

  </xsl:template>

</xsl:stylesheet>

This stylesheet takes two parameters: document2 is the name of a document to merge with its main input document; and offset is an optional time offset in milliseconds. If offset is specified, its value is added to all the timestamps in document2. This is useful where the times of the systems that host the WebLogic Server instances are not fully synchronized.

Here is an example of using the stylesheet to merge two event data export files:

java weblogic.WLST process.py -document2=server2-events.xml merge.xsl server1-events.xml > events.xml

It makes sense to merge two event data files or two harvested metric files. It does not make sense to merge an event data file with a harvested metric file, since these file types have different column sets.

Producing a Comma-separated Value (csv) spreadsheet File

The next stylesheet does something more useful. It converts an export file into a comma-separated value file that can be imported into Microsoft Excel and other spreadsheet tools.

The stylesheet takes no parameters.

<?xml version="1.0" encoding="utf-8"?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0" xmlns:e="http://www.bea.com/ns/weblogic/90/diagnostics/accessor/Export">

  <xsl:output method="text"/>

  <xsl:template match="e:DiagnosticData">

    <xsl:apply-templates/>

  </xsl:template>

  <xsl:template match="e:DataInfo"/>

  <xsl:template match="e:DataRecord">

    <xsl:variable name="record" select="."/>

    <xsl:for-each select="/e:DiagnosticData/e:DataInfo[1]/e:ColumnInfo">

      <xsl:variable name="p" select="position()"/>

      <xsl:value-of select="$record/e:ColumnData[$p]"/>

      <xsl:if test="$p != last()">

        <xsl:text>, </xsl:text>

      </xsl:if>

    </xsl:for-each>

    <xsl:text>

</xsl:text>

  </xsl:template>

  <!-- Work around known issue in Java 5 (Sun bug 6413803 inherited

  from XALAN-2230) which means that the xsl:stripspace directive

  doesn't work with XML name spaces. We explicitly trim text nodes.

  -->

  <xsl:template match="text()">

    <xsl:value-of select="normalize-space(.)"/>

  </xsl:template>

</xsl:stylesheet>

Producing an Event Tree File

The exported data files have information about the individual events but don't record the relationships between the events. By using the timing information, we can reconstruct the event tree, which gives us an idea of which parts of the code were called by other parts. To build the event tree, we'll use the difference in times between before and after events recorded by TraceElapsedTimeAction actions. This time is stored in the event payload of the after events. We discard events that do not have a payload, leaving us with just the after events. We can calculate the start timestamp by subtracting the payload time from the after event timestamp, taking care to convert to a common unit (the event timestamps are in milliseconds, but the payload times are in nanoseconds).

The start and end timestamps give us a timespan for the monitored methods. We build the event tree by treating timespans that are directly and wholly included in another timespan as children of those timespans.

The WLDF console extension can produce a similar event tree for the flow of a single request, within a server. While the console extension is great for quick browsing, our approach has a few of advantages. By using merge.xsl to merge events from multiple diagnostic archives, we can trace the flow of a request across multiple servers. The results are captured in an HTML report that can be published and distributed. The event tree is calculated using only timestamps and does not, for example, use the diagnostic context in its correlation. This makes the stylesheet quite general but means that events can be incorrectly marked as causes of other events. To avoid this, you may want to filter by diagnostic context. You can do this by specifying an appropriate WLDF query when you export from the diagnostic archive, or by supplying the stylesheet with a list of diagnostic contexts using its contexts parameter. In practice, you might find the diagnostic contexts of interest first by extracting all the diagnostic data into a CSV file, importing it into a spreadsheet, and looking for unusual (or perhaps typical) transactions.

The stylesheet expects an event data export file as input and takes a single, optional parameter, contexts, which is a list of diagnostic contexts to include from the source. If contexts is not specified, all events recorded by TraceElapsedTimeAction will be included.

<?xml version="1.0" encoding="utf-8"?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0" xmlns:exslt="http://exslt.org/common" xmlns:e="http://www.bea.com/ns/weblogic/90/diagnostics/accessor/Export">

  <xsl:output method="xml" indent="yes"/>

  <xsl:variable name="RECORDID" select="count(//e:ColumnInfo[e:Name='RECORDID']/preceding-sibling::*) + 1"/>

  <xsl:variable name="TIMESTAMP" select="count(//e:ColumnInfo[e:Name='TIMESTAMP']/preceding-sibling::*) + 1"/>

  <xsl:variable name="CONTEXTID" select="count(//e:ColumnInfo[e:Name='CONTEXTID']/preceding-sibling::*) + 1"/>

  <xsl:variable name="TXID" select="count(//e:ColumnInfo[e:Name='TXID']/preceding-sibling::*) + 1"/>

  <xsl:variable name="USERID" select="count(//e:ColumnInfo[e:Name='USERID']/preceding-sibling::*) + 1"/>

  <xsl:variable name="DOMAIN" select="count(//e:ColumnInfo[e:Name='DOMAIN']/preceding-sibling::*) + 1"/>

  <xsl:variable name="SERVER" select="count(//e:ColumnInfo[e:Name='SERVER']/preceding-sibling::*) + 1"/>

  <xsl:variable name="MONITOR" select="count(//e:ColumnInfo[e:Name='MONITOR']/preceding-sibling::*) + 1"/>

  <xsl:variable name="CLASSNAME" select="count(//e:ColumnInfo[e:Name='CLASSNAME']/preceding-sibling::*) + 1"/>

  <xsl:variable name="METHODNAME" select="count(//e:ColumnInfo[e:Name='METHODNAME']/preceding-sibling::*) + 1"/>

  <xsl:variable name="PAYLOAD" select="count(//e:ColumnInfo[e:Name='PAYLOAD']/preceding-sibling::*) + 1"/>

  <xsl:param name="contexts" select=""/>

  <xsl:template match="/">

    <xsl:variable name="sorted-nodes">

      <xsl:for-each select="//e:DataRecord[e:ColumnData[$PAYLOAD] != '' and

                                           ($contexts = '' or contains($contexts, e:ColumnData[$CONTEXTID]))]">

        <xsl:sort select="e:ColumnData[$TIMESTAMP] * 1000000 - e:ColumnData[$PAYLOAD]"/>

        <xsl:copy-of select="."/>

      </xsl:for-each>

    </xsl:variable>

    <xsl:variable name="nodes" select="exslt:node-set($sorted-nodes)/e:DataRecord"/>

    <data>

      <xsl:call-template name="processLevel">

        <xsl:with-param name="nodes" select="$nodes"/>

        <xsl:with-param name="basetime" select="$nodes[1]/e:ColumnData[$TIMESTAMP] * 1000000 - $nodes[1]/e:ColumnData[$PAYLOAD]"/>

      </xsl:call-template>

    </data>

  </xsl:template>

  <xsl:template name="processLevel">

    <xsl:param name="nodes"/>

    <xsl:param name="basetime"/>

    <xsl:for-each select="$nodes">

      <xsl:variable name="position" select="position()"/>

      <xsl:variable name="endMS" select="e:ColumnData[$TIMESTAMP]"/>

      <xsl:variable name="start" select="$endMS * 1000000 - e:ColumnData[$PAYLOAD]"/>

      <!-- Output the nodes that are not fully enclosed by others in $nodes.

           Nodes that overlap partially will all be output at this

           level - their contents may contain duplicates at lower

           levels.

           Where the times match exactly, break the tie using the

           document order. -->

      <xsl:if test="not($nodes[(position() != $position) and

                               ((e:ColumnData[$TIMESTAMP] * 1000000 - e:ColumnData[$PAYLOAD]) &lt; $start and e:ColumnData[$TIMESTAMP] &gt;= $endMS or

                                (e:ColumnData[$TIMESTAMP] * 1000000 - e:ColumnData[$PAYLOAD]) &lt;= $start and e:ColumnData[$TIMESTAMP] &gt; $endMS or

                                (e:ColumnData[$TIMESTAMP] * 1000000 - e:ColumnData[$PAYLOAD]) = $start and e:ColumnData[$TIMESTAMP] = $endMS and position() &lt; $position

                               )]

                       )">

        <record start="{$start}" offset="{$start - $basetime}" duration="{e:ColumnData[$PAYLOAD]}"

                classname="{e:ColumnData[$CLASSNAME]}" methodname="{e:ColumnData[$METHODNAME]}"

                domain="{e:ColumnData[$DOMAIN]}" server="{e:ColumnData[$SERVER]}" recordid="{e:ColumnData[$RECORDID]}"

                userid="{e:ColumnData[$USERID]}" txid="{e:ColumnData[$TXID]}">

          <!-- Recurse to include nodes in $nodes that we enclose. -->

          <xsl:call-template name="processLevel">

            <xsl:with-param name="nodes" select="$nodes[position() != $position and (e:ColumnData[$TIMESTAMP] * 1000000 - e:ColumnData[$PAYLOAD]) &gt;= $start and e:ColumnData[$TIMESTAMP] &lt;= $endMS]"/>

            <xsl:with-param name="basetime" select="$start"/>

          </xsl:call-template>

        </record>

      </xsl:if>

    </xsl:for-each>

  </xsl:template>

</xsl:stylesheet>

Formatting an Event Tree File

The event tree we've produced is just another XML file. Let's make it more comprehensible by rendering it in HTML. This stylesheet expects an event tree document as input, and takes no parameters.

<?xml version="1.0" encoding="utf-8"?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"

                version="1.0">

  <xsl:output method="html"/>

  <xsl:template match="/">

    <html>

      <body>

        <table>

          <tr>

            <th></th>

            <th>Offset</th>

            <th>Method</th>

            <th>Duration</th>

            <th>Record</th>

            <th>User</th>

            <th>Transaction ID</th>

          </tr>

          <xsl:apply-templates/>

        </table>

      </body>

    </html>

  </xsl:template>

  <xsl:template match="record">

    <xsl:param name="indent" select=""/>

    <xsl:param name="level" select="1"/>

    <xsl:variable name="colour">

      <xsl:choose>

        <xsl:when test="$level=1">#F0D0D0</xsl:when>

        <xsl:when test="$level=2">#D0D0F0</xsl:when>

        <xsl:when test="$level=3">#C0C0E0</xsl:when>

        <xsl:when test="$level=4">#B0B0D0</xsl:when>

        <xsl:when test="$level=5">#A0A0C0</xsl:when>

        <xsl:when test="$level=6">#9090B0</xsl:when>

        <xsl:when test="$level=7">#8080A0</xsl:when>

        <xsl:when test="$level=8">#707090</xsl:when>

        <xsl:when test="$level=9">#606080</xsl:when>

        <xsl:otherwise>#505070</xsl:otherwise>

      </xsl:choose>

    </xsl:variable>

    <tr bgcolor="{$colour}">

      <td align="right">

        <font size="-1">

        <xsl:number level="any"/>

        </font>

      </td>

      <td>

        <xsl:value-of select="$indent"/>

        <xsl:text>+</xsl:text>

        <xsl:value-of select="format-number(@offset div 1000000, '0.00')"/>

      </td>

      <td>

        <xsl:value-of select="$indent"/>

        <xsl:value-of select="@classname"/>

        <xsl:text>.</xsl:text>

        <xsl:value-of select="@methodname"/>

        <xsl:text>()</xsl:text>

      </td>

      <td align="right"><xsl:value-of select="format-number(@duration div 1000000, '0.000')"/></td>

      <td>

        <xsl:value-of select="@domain"/>

        <xsl:text>:</xsl:text>

        <xsl:value-of select="@server"/>

        <xsl:text>:</xsl:text>

        <xsl:value-of select="@recordid"/>

      </td>

      <td>

        <xsl:value-of select="@userid"/>

      </td>

      <td>

        <xsl:value-of select="@txid"/>

      </td>

    </tr>

    <xsl:apply-templates>

      <xsl:with-param name="indent" select="concat('&#160;&#160;&#160;&#160;', $indent)"/>

      <xsl:with-param name="level" select="$level + 1"/>

    </xsl:apply-templates>

  </xsl:template>

</xsl:stylesheet>

The stylesheet formats a report like:

The indentation indicates the parent/child relationships in the event tree. The Offset column gives the time in milliseconds since the start of the parent event. The Duration is in milliseconds. By modifying eventsToTree.xsl and treeToHTML.xsl other columns can be included or can be formatted differently.

This example explains how to bring the data collected by WLDF to life using a little bit of effort and some XSLT wizardry.

Summary

WLDF instrumentation and DYE injection provide a very powerful set of features to inject monitoring code into customer applications.  WLDF provides out-of-the-box a large number of standard monitors and options.  This chapter has provided an overview of the features. 

DYE injection is the WebLogic feature which allows administrators to trace call routes throughout the system.  This is a feature which usually is only available in external commercial products. WebLogic provides it as part of WLDF. Surprisingly this feature is very little known and used in the industry.