Microcontainer Inspector Tool

For those of you familiar with the Twiddle application, which allows you to inspect the JMX MBeans in JBoss Application Server, I have created a similar tool for use with the JBoss Microcontainer. This means that you can now manage and inspect ALL the beans in the JBoss Application Server (or in a standalone Microcontainer application). I'll briefly outline the existing functionality below and how to use it. The functionality supported so far is quite basic, but hopefully it will be useful to somebody. I'll be adding more commands to it when I have the time.

Implementing commands to extend the feature set is very easy. If you would like to be a Microcontainer committer, this is a great opportunity to get involved by implementing something simple while you get to know the code and the team. If you are interested please contact me privately or via our forums.

Installation
If you want to run this in a standalone environment you simply create an instance of org.jboss.microcontainer.mcinspector.bean.McInspector and pass it your bootstrapped kernel as shown in the test cases. To install it into a JBoss Application Server installed to ~/jbossas/:

• Check out the source from http://anonsvn.jboss.org/repos/jbossas/projects/mc-tools/mcinspector/trunk/ 
• Build it by going to the checkout directory and run mvn install
• Copy target/mcinspector-server.jar to ~/jbossas/server/default/deploy
• Copy target/mcinspector.jar to ~/jbossas/bin
• Copy src/main/resources/mcinspector.sh to ~/jbossas/bin
• chmod 755 ~/jbossas/bin/mcinspector.sh

That's all, now start up JBoss as normal, and in another terminal: cd ~/jbossas/bin. This last terminal is the one we will be using to try out the commands in the rest of this demo.

Help
To get help you enter: ./mcinspector.sh help, which gives you an overview of the commands available. The command and its output are shown below:
$./mcinspector.sh help
Available commands:
  help   Prints the list of available commands.
  list   Lists all the available contexts.
  change Changes the state of the context.
  bean   Inspect a particular bean.
For more information about a particular command, enter 'help '.
Handled in 1026ms

You can also get more information about a particular command, e.g.:
$./mcinspector.sh help list
Lists all the contexts in the controller.
Usage: list [-o | -v] [ []*]
If no state arguments are passed in all states will be checked for contexts. Otherwise
only the passed in states will be checked
Options:
-o Overview. Show all the states in the controller with the number of contexts in each
   Normally only states with contexts will be shown
-e Error. Show only the contexts which are in the error state. Can not be used in combination  
   with state arguments.
-v Verbose. Show extra information for each context. Contexts with unresolved dependencies
   will have a '*' after their name. Additional information is in the format 
   (ControllerMode, Current state)
Handled in 523ms

List
Let's try out the list command. As promised in the help section, it lists all the states, in ascending order, in the Application Server's main controller and displays the number of beans found in each state:

$./mcinspector.sh list -o
NOT INSTALLED: 0 contexts
PREPARSE: 0 contexts
PARSE: 0 contexts
POSTPARSE: 0 contexts
PREDESCRIBE: 0 contexts
DESCRIBE: 0 contexts
CLASSLOADER: 0 contexts
POSTCLASSLOADER: 0 contexts
PREREAL: 0 contexts
REAL: 0 contexts
PREINSTALL: 0 contexts
DESCRIBED: 33 contexts
INSTANTIATED: 0 contexts
CONFIGURED: 0 contexts
CREATE: 0 contexts
START: 0 contexts
INSTALLED: 997 contexts
Handled in 509ms

To view which beans are in the different states we execute ./mcinspector.sh list. Since there are lot of beans in the Application Server I have deleted most of the output below:

DESCRIBED:
JBossLogManagerContextSelectorService
org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory
org.jboss.netty.channel.socket.nio.NioDatagramChannelFactory
org.jboss.netty.channel.socket.oio.OioClientSocketChannelFactory
org.jboss.netty.channel.socket.oio.OioServerSocketChannelFactory
org.jboss.netty.channel.socket.oio.OioDatagramChannelFactory
org.jboss.netty.channel.local.DefaultLocalClientChannelFactory
org.jboss.netty.channel.local.DefaultLocalServerChannelFactory
ModClusterListener
ModClusterService
...
INSTALLED:
JBossServer
ClassLoaderSystem
ClassLoaderDomain
ClassLoading
bootstrap-classloader:0.0.0$MODULE
bootstrap-classloader:0.0.0
stdio-classloader:0.0.0$MODULE
stdio-classloader:0.0.0
JBossStdioSystemOutStream
JBossStdioSystemErrStream
…
McInspectorCommands
McInspector
org.jboss.kernel:service=McInspector
McInspectorHelpCommand
McInspectorListCommand
McInspectorChangeCommand
McInspectorBeanCommand
file:///Users/kabir/sourcecontrol/jbossas/trunk/subversion/build/target/jboss-6.0.0-SNAPSHOT/server/default/deploy/mcinspector.jar
Handled in 596ms

To get more information about the contexts for a given state, you can do ./mcinspector.sh list -v DESCRIBED. The verbose output currently shows the controller mode of the beans' contexts, along with the current state of the bean. The * means that the context has unresolved dependencies.

$./mcinspector.sh list -v DESCRIBED
DESCRIBED:
JBossLogManagerContextSelectorService* (ON_DEMAND, DESCRIBED)
org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory* (ON_DEMAND, DESCRIBED)
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory* (ON_DEMAND, DESCRIBED)
org.jboss.netty.channel.socket.nio.NioDatagramChannelFactory* (ON_DEMAND, DESCRIBED)
org.jboss.netty.channel.socket.oio.OioClientSocketChannelFactory* (ON_DEMAND, DESCRIBED)
org.jboss.netty.channel.socket.oio.OioServerSocketChannelFactory* (ON_DEMAND, DESCRIBED)
org.jboss.netty.channel.socket.oio.OioDatagramChannelFactory* (ON_DEMAND, DESCRIBED)
org.jboss.netty.channel.local.DefaultLocalClientChannelFactory (ON_DEMAND, DESCRIBED)
org.jboss.netty.channel.local.DefaultLocalServerChannelFactory (ON_DEMAND, DESCRIBED)
ModClusterListener* (ON_DEMAND, DESCRIBED)
ModClusterService* (ON_DEMAND, DESCRIBED)
HAModClusterService* (ON_DEMAND, DESCRIBED)
ModClusterConfig* (ON_DEMAND, DESCRIBED)
DynamicLoadBalanceFactorProvider* (ON_DEMAND, DESCRIBED)
AverageSystemLoadMetric* (ON_DEMAND, DESCRIBED)
OperatingSystemLoadMetricSource (ON_DEMAND, DESCRIBED)
BusyConnectorsLoadMetric* (ON_DEMAND, DESCRIBED)
ThreadPoolLoadMetricSource* (ON_DEMAND, DESCRIBED)
SimpleLoadBalanceFactorProvider* (ON_DEMAND, DESCRIBED)
SystemMemoryUsageLoadMetric* (ON_DEMAND, DESCRIBED)
HeapMemoryUsageLoadMetric* (ON_DEMAND, DESCRIBED)
RequestCountLoadMetric* (ON_DEMAND, DESCRIBED)
ReceiveTrafficLoadMetric* (ON_DEMAND, DESCRIBED)
SendTrafficLoadMetric* (ON_DEMAND, DESCRIBED)
RequestProcessorLoadMetricSource* (ON_DEMAND, DESCRIBED)
ActiveSessionsLoadMetric* (ON_DEMAND, DESCRIBED)
SessionLoadMetricSource* (ON_DEMAND, DESCRIBED)
ConnectionPoolUsageMetric* (ON_DEMAND, DESCRIBED)
ConnectionPoolLoadMetricSource* (ON_DEMAND, DESCRIBED)
MBeanAttributeLoadMetric* (ON_DEMAND, DESCRIBED)
MBeanAttributeRatioLoadMetric* (ON_DEMAND, DESCRIBED)
MBeanQueryLoadMetricSource* (ON_DEMAND, DESCRIBED)
XnioProvider* (ON_DEMAND, DESCRIBED)
Handled in 479ms

Bean
To get more information about one or more beans in particular, you can run the bean command. e.g.:
$./mcinspector.sh bean ModClusterListener ModClusterService
Name='ModClusterListener'
  bean type=org.jboss.modcluster.catalina.CatalinaEventHandlerAdapter
  mode=ON_DEMAND
  state=DESCRIBED
  required state=DESCRIBED
  context type=org.jboss.kernel.plugins.dependency.AbstractKernelControllerContext
  unresolved dependencies=1
Name='ModClusterService'
  bean type=org.jboss.modcluster.ModClusterService
  mode=ON_DEMAND
  state=DESCRIBED
  required state=DESCRIBED
  context type=org.jboss.kernel.plugins.dependency.AbstractKernelControllerContext
  unresolved dependencies=5
Handled in 485ms

To get some more information about the dependencies, you can pass in the -d flag
$./mcinspector.sh bean -d ModClusterListener
Name='ModClusterListener'
  bean type=org.jboss.modcluster.catalina.CatalinaEventHandlerAdapter
  mode=ON_DEMAND
  state=DESCRIBED
  required state=DESCRIBED
  context type=org.jboss.kernel.plugins.dependency.AbstractKernelControllerContext
  unresolved dependencies=1
  Resolved dependencies:
  Unresolved dependencies:
    iDependOn=ModClusterService; whenRequired=INSTANTIATED; dependentState=INSTALLED

The -u flag is the same as -d apart from it does not look for resolved dependencies:

$./mcinspector.sh bean -d ModClusterListener
Name='ModClusterListener'
  bean type=org.jboss.modcluster.catalina.CatalinaEventHandlerAdapter
  mode=ON_DEMAND
  state=DESCRIBED
  required state=DESCRIBED
  context type=org.jboss.kernel.plugins.dependency.AbstractKernelControllerContext
  unresolved dependencies=1
  Unresolved dependencies:
    iDependOn=ModClusterService; whenRequired=INSTANTIATED; dependentState=INSTALLED
Handled in 492ms

Change
Finally there is the change command which attempts to change the state of a bean, so it can for example be used to temporarily disable a bean. Only do this if you know what you are doing! For example, let us try to stop the bean that implements the MC Inspector's bean command:

$./mcinspector.sh change INSTANTIATED McInspectorBeanCommand
Changed 'McInspectorBeanCommand' to INSTANTIATED
Handled in 473ms

This has stopped the bean implementing the bean command, so bean no longer appears in the list of available commands:
$./mcinspector.sh help
Available commands:
  help Prints the list of available commands.
  list Lists all the available contexts.
  change Changes the state of the context.
  For more information about a particular command, enter 'help '.
Handled in 497ms
And it appears in the INSTANTIATED state:
$./mcinspector.sh list INSTANTIATED -v
INSTANTIATED:
McInspectorBeanCommand (AUTOMATIC, INSTANTIATED)
Handled in 488ms

Let us make it installed again:
$./mcinspector.sh change INSTALLED McInspectorBeanCommand
Changed 'McInspectorBeanCommand' to INSTALLED
Handled in 474ms
Now bean appears in the list of available commands again:
$./mcinspector.sh help
Available commands:
  help Prints the list of available commands.
  list Lists all the available contexts.
  change Changes the state of the context.
  bean Inspect a particular bean.
For more information about a particular command, enter 'help '.
Handled in 480ms

Optimizing the JBoss Microcontainer

Ales linked to the ongoing optimization work before I had planned on writing anything, so here is a very rushed overview of what is going on :-) 

For JBoss Application Server 6.0.0.M2 a lot of work went in to optimizing the boot time. Some of the work done by the AS team that really made a difference was optimizing the algorithm to add the deployers, and to make things like the admin console start when first accessed rather than use eager loading.

As part of this, for JBoss Kernel 2.2.0.Alpha6 I took a look at optimizing the dependency resolution algorithm of the Microcontainer, which is used to install and wire together the services the core of the application server. The dependency resolution algorithm is used to determine if the beans implementing the services can be moved through the states on the way from being put into the Microcontainer to being fully up and running. Initially I did work on a fully optimized prototype, but this was too big and risky an undertaking for the short timeframe and did not yield the expected results. This prototype is far from "finished" and there is plenty of room for improvement, so I might come back to it at some later stage.

So instead I went through and used JProfiler to see what is taking the time in the dependency resolution algorithm that we have today. What really made a difference here was that I found that when a bean can not be moved to the next state we were looking up its unresolved dependency three times in the Microcontainer. Wrapping these three calls into one gave a lot less overhead. That was the main optimization we had time for for JBoss Kernel 2.2.0.Alpha6 which is what is used in AS 6.0.0.M2. 

Since then I have been working on quite a few other optimizations that you can see on the Microcontainer Development Forum, more specifically here, which so far significantly improve the boot time of 6.0.0.M2 in my local benchmarks, and I am nowhere not done yet. The fixes are mainly making sure that objects used in maps are properly hashed, and making objects that are expensive to create less expensive to create. All these fixes will all go into the next community release of JBoss AS: 6.0.0.M3.

Another thing is that John Bailey is currently integrating JBoss Virtual File System 3 into AS. VFS is used as an abstraction layer to read files from several different sources, and VFS 3 promises to really speed things up when reading from zipped archives. This is something that happens a lot in the application server, when scanning archives for deployment descriptors and annotations, and loading classes and so on. So it will be very exciting to see this in JBoss AS 6.0.0.M3!

Findbugs Filter Creation Tool

We have started using findbugs on some of the projects I am working on. It seems pretty useful, and it has picked out some bugs already. It does however, report quite a few bugs that are not a problem. Luckily, findbugs provides filters to be able to exclude the bugs you don't want to include.

After spending a day on creating the filter files manually, I found it to be a bit fiddly and time-consuming to create these by hand. I tried looking to see if there was a way to automate this a bit more. I might be wrong, but I could not find a tool to do this. Hopefully the below will help others in the same situation.

When you run a findbugs check it generates an output file. I do this via the maven plugin: mvn findbugs:check which yields a file called findbugsCheck.xml. This file references an xsl stylesheet:

<?xml-stylesheet type="text/xsl" href="http://findbugs.sourceforge.net/xsl/default.xsl"?>

so if you open this file in your browser you will see something like this:


I created another version of default.xsl called filterHelper.xsl. Modifying findbugsCheck.xml to use this stylesheet:

<?xml-stylesheet type="text/xsl" href="http://anonsvn.jboss.org/repos/jbossas/projects/findbugs-filtercreator/trunk/filterHelper.xsl"?>

gives output that looks much the same

Now each bug has a a checkbox next to it. Check the bugs you want to create an exclude filter for, press the "Create Filter" button and the textarea gets populated with the Match elements for each bug.

You can then copy the results into a file, in my projects they are under src/main/resources/findbugs/exclude.xml. For example:

<FindBugsFilter>
<Match>
   <Class name="org.jboss.system.ServiceMBeanSupport"/>
   <Field name="SERVICE_CONTROLLER_SIG"/>
   <Bug pattern="MS_PKGPROTECT"/>
</Match>
<!-- Rest of exclusions here -->
</FindBugsFilter>

Then set maven up to use the filter in the plugins section:

<plugin>
   <groupId>org.codehaus.mojo</groupId>
   <artifactId>findbugs-maven-plugin</artifactId>
   <version>2.0.1</version>
   <configuration>
      <excludeFilterFile>${project.build.outputDirectory}/findbugs/exclude.xml</excludeFilterFile>
       <debug>true</debug>
   </configuration>
</plugin>

Microcontainer Downloads

While getting up to speed after joining the JBoss Microcontainer project, I have updated our downloads section with the latest releases.

The old JBoss Microcontainer 1.0.x releases can still be found on Sourceforge. Since those early releases were uploaded the JBoss Microcontainer has developed a lot, and it is now the umbrella for several sub-projects.

The downloads section has been split into two main categories:

• The Frameworks category contains the downloads most convenient for end users:
  
  
  
  • Kernel: This is what most people think of when they hear "Microcontainer". It contains the core of the Microcontainer and its dependencies. In brief this is the core state machine with the dependency injection framework, with support for bean metadata supplied via xml or annotations.
    
  
  
  • JBoss Virtual Deployment Framework: This contains all the individual projects from the JBoss Microcontainer project. It consists of the Kernel, advanced classloading capabilities and deployment support, as consumed by JBoss Application Server and JBoss Reloaded.
    
  
  
  
  


• The Individual Projects section contains the independent releases of the JBoss Microcontainer sub-projects. Only the binaries and sources are currently contained in the downloads, with no dependencies. The sources are however buildable using maven (version 2.0.9 or later), so you can get all the dependencies and javadoc.
  

The current releases on the downloads pages take the already released latest releases of the MC projects. Over the next few releases the plan is to put in a lot of effort in improving our consumables.