Tuesday, March 22, 2011

Integrating Centralized Logging System to OpenNMS for Alerts

Once the solution is based on SOA, there can be a need to have a centralized logging facility where each of the service would be using log agents to send logs to a cluster of centralized log server. This can be done via various manners using log4J framework by making use of one of the following appenders:

1. SysLogD
2. JMSAppenders

JMSAppender could be used within Log agents to send logs to a messaging bus from where message could be retrieved by store agents on centralized log server.

Further to that, the message once retrieved from the Message bus would be watched for warnings or exceptions for purpose of generating alerts which could be informed to admin via SMS or email.

This can be done by integrating centralized log server with OpenNMS. This is done based on following:

1. The incoming messages is watched for exception/warning messages.
2. Once these messages are found, the event is created and send as an XML document to OpenNMS eventD plugin on a TCP socket. EventD listens on port such as 5817 by default. This port needs to be opened for external system such as centralized log server to send message using Sockets.

Following class could be used to send message using Socket:

he Code

   package com.company.onms.monitor;

 import java.io.IOException;
import java.io.PrintWriter;
import java.net.Socket;
import java.net.UnknownHostException;
import java.text.DateFormat;
import java.util.Calendar;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;
/**
*
* @author apaxson
*/
public class OnmsEvent {
private String uei;
private String source;
private int nodeid;
private String time;
private String host;
private Map parms = new HashMap();
public OnmsEvent(String uei, int nodeid, String host, String source) {
setUei(uei);
setNodeid(nodeid);
setHost(host);
setSource(source);
Date currentTime = Calendar.getInstance().getTime();
DateFormat dformat = DateFormat.getDateTimeInstance(DateFormat.FULL, DateFormat.FULL);
dformat.setTimeZone(TimeZone.getTimeZone("GMT"));
setTime(dformat.format(currentTime));
}
protected String getUei() {
return uei;
}
protected void setUei(String uei) {
this.uei = uei;
}
protected String getSource() {
return source;
}
protected void setSource(String source) {
this.source = source;
}
protected int getNodeid() {
return nodeid;
}
protected void setNodeid(int nodeid) {
this.nodeid = nodeid;
}
protected String getTime() {
return time;
}
protected void setTime(String time) {
this.time = time;
}
protected String getHost() {
return host;
}
protected void setHost(String host) {
this.host = host;
}
public void addParm(String key, String value) {
parms.put(key, value);
}
@SuppressWarnings(value = "unchecked")
protected Map<String, String> getParms() {
return parms;
}
protected String toXml() {
StringBuffer data = new StringBuffer();
data.append("<log>");
data.append("<events>");
data.append("<event>");
data.append("<uei>" + getUei() + "</uei>");
data.append("<source>" + getSource() + "</source>");
data.append("<nodeid>" + getNodeid() + "</nodeid>");
data.append("<time>" + getTime() + "</time>");
data.append("<host>" + getHost() + "</host>");
data.append("<parms>");
// Cycle through each parameter
for (Map.Entry<String, String> e : getParms().entrySet()) {
data.append("<parm>");
data.append("<parmName>");
data.append("<![CDATA[" + e.getKey() + "]]></parmName>");
data.append("<value type=\"string\" encoding=\"text\"><![CDATA[" + e.getValue() + "]]></value>");
data.append("</parm>");
}
data.append("</parms>");
data.append("</event>");
data.append("</events>");
data.append("</log>");
return data.toString();
}
/**
* TODO Remember to change the hardcoded address 2.3.4.5 to your address for onms.
* Preferrably, place this in a config file somewhere
*/
public void sendEvent(String xmlData) {
//TODO check to make sure xmlData isn't null before creating socket
//TODO make some useful logging functions here, rather than println
Socket socket = null;
try {
System.out.println("Sending data");
socket = new Socket("2.3.4.5", 5817);
PrintWriter out = new PrintWriter(socket.getOutputStream(), true);
out.println(xmlData);
System.out.println("Sending data complete");
} catch (UnknownHostException ex) {
System.out.println("Unknown host");
} catch (IOException ex) {
System.out.println("I/O Exception while creating socket");
} finally {
try {
socket.close();
} catch (IOException ex) {
System.out.println("I/O Exception when closing socket");
}
}
}
}

The Usage

Okay, so to use this new class, you create the object, and set the data. As such:

       OnmsEvent event = new OnmsEvent("uei.company.com/application/logFileTableLoadError",26,"test.host.com", "ServerLogMonitor");
event.addParm("file", "logfile001.log");
event.addParm("content","Disregard... testing only");
event.sendEvent(event.toXml());

Friday, March 18, 2011

Lessons learnt for scaling Java EE applications


Check out following link for detailed description:
  1. Use
    share nothing clustering architectures
  2. Use
    scalable session replication mechanisms
  3. Use
    collocated deployment instead of distributed one
  4. Shared
    resources and services
  5. Parallel Processing

Friday, January 28, 2011

funpiper.com - Plan Events and Parties, Broadcast and Share

I am pondering upon funpiper.com, an event management platform to take to next level with GroupOn.com like concept..anyone interested to invest!

Imagine an automated engine helping you to meet your friends with you making very little effort like answering yesy/no for a couple of times!!! Let me know if it sounds interesting to you. Write me at ajitesh at @ at gmail dot com

My Two Cents on SOA & Startups

An organization needs to spend a lot (COST) to achieve a true SOA based architecture just because SOA is not only limited to Service and their interactions. Following are areas which needs to be considered for achieving an “Agile SOA” platform:

- Service

- Governance

- Architecture

- Process

- People

- Engagement & Delivery Operation

Trust me, really speaking, it would not be very SMART & INTELLIGENT if we tell to our clients that we offer SOA based solutions. WE COULD SIMPLY BE CAUGHT UNAWARE! Simply because if we get an SOA expert across the table, we would simply find ourselves struggling with some real tough questions on SOA governance, process, service discovery, Delivery & Operations Management (ITIL) aspects of SOA. So, we rather should take caution (MAKE RIGHT CHOICE OF WORDS) when we speak to our clients that we offer SOA-based solution as it could simply backfire!

And, this is very natural that an organization cannot achieve “Agile SOA” status in first few years as no organization (especially start-ups) could IGNORE COST-EFFECTIVENESS of the solutions. Or, it will simply be DEAD!

Its very natural that in the initial years, organization would want to be happy with SOA based solution rated as “Ad-hoc SOA” status. That said, pl feel free to ask for various transitions for SOA and roadmap that an organization need to take to transition from “Ad-hoc SOA” to “Agile SOA” status.

Tuesday, January 25, 2011

Is the Java Dynasty Seeing the Beginning of the End?

This is very interesting to see that various blogs, articles have started writing about Java and its end. Well, as they say, ALL Good Things come to an end. Would this be applied to Java? I don't want to agree. But, In my past experience, I have found myself moving from C, C++ to Java and then adopt PHP and then started exploring fairly recent JVM languages such as Groovy, Scala, JRuby etc.

Tuesday, January 18, 2011

JBoss AS 6 out with JEE 6 Web Profile Specs

Following are some of key highlights of JBoss AS 6:

1. JEE 6 Web Profile Spec (JSP, EJB3, JPA, JSF, CDI). CDI stands for Contexts and Dependency Injection (JSR 299)

2. RESTEasy 2.1.0, an implementation of JAX-RS 1.1

3. Hornet 2.1.2

4. JBoss Web Services CXF

Also, Infinispan is integrated with JBoss AS 6 and is default distributed cache provider. Cache is JSR 107 compliant.

Also is delivered Mod_Cluster which dynamically discover and register to front-end apache http servers.

Monday, January 17, 2011

Performance management and Monitoring strategy for SOA

Following can be performance management and monitoring strategy that an SOA based solution architecture could adopt. Please share your thoughts.

Performance Management

- Continual performance evaluation (in TPS) of various services by making use of different stress scenarios such as following:
  1. Simple strategy: The Simple Strategy runs the specified number of requests with the specified delay between each run to simulate a "breathing space" for the server.
  2. Fixed rate strategy: Fixed rate strategy guarantees a fixed number of requests in a given time period.
  3. Variable load strategy: This strategy related to vary the load. Following are key strategies for variable load:
  • Variance: This varies the number of requests over time in a “sawtooth” manner as configured; set the Interval to the desired value and the Variance to the how much the number of requests should decrease and increase.
  • Burst: this strategy is specifically designed for Recovery Testing and takes variance to its extreme; it does nothing for the configured Burst Delay, then runs the configured number of requests for the “Burst Duration” and goes back to sleep.
  • Thread: This lets you linearly change the number of requests from one level to another over the run of the stress test. It’s main purpose is as a means to identify at which level certain statistics change or events occur, for example to find at which request count, the maximum TPS or BPS can be achieved or to find at which request count, at which functional testing errors start occurring.
  • Grid: this strategy allows to specifically configure the relative change in number of requests over time, its main use for this is more advanced scenario and recovery testing, where you need to see the services behaviour under varying loads and load changes.
  • Script: the script strategy is the ultimate customization possibility; the script you specify is called regularly and should return the desired number of requests at that current time.
- Tools such as SOAP-UI, GRINDER or Apache JMeter could be used for implementing above stress strategies.
- Java code instrumentation using custom logging framework for collecting timing for various IO (Disk/Network) tasks in any APIs.
- Distributed agents-collector based architecture to store performance metrics on a cluster of centralized log servers. This could make use of framework such as Flume (Cloudera) or Scribe (Facebook) or NIO based SocketServer implementations on centralized log server for log aggregation.
- Integration with monitoring tool such as NAGIOS for active monitoring on service performance, degradation etc.
- Web services to expose the performance metrics to internal and external consumers.
- Usage of performance metrics data for business intelligence. An ETL tool would be used for transferring data from log servers to BI server.

Monitoring

- Performance metrics monitoring based on log messages arriving in centralized log servers. This could make use of framework such as Flume (Cloudera) or Scribe (Facebook) or NIO based SocketServer implementations on centralized log server for log aggregation.
- Services and Server configuration monitoring for tracking configuration changes.
- Custom alerts for sending notifications for scenarios such as configuration changes , service degradation based on performance metrics etc.
- Web services to expose the monitoring information to internal and external consumers
- Integration with monitoring tool such as NAGIOS for active monitoring on service performance, degradation etc.
- Usage of monitoring information for business intelligence. An ETL tool would be used for transferring data from log servers to BI server.