Dynamic JRE: Stressing the machine

When considering whether to use WebLogic Server as a Grails EmbeddableServer, replacing Tomcat or Jetty in the execution of ‘grails run-app’, I came across with some limitations. Certain features of the Java runtime environment are not prepared to be changed after starting the JVM (or are they?). Despite all hype, it’s just software and use it in an unconventional manner won’t trigger the end of the world.

The following solutions (workarounds or tricks, if you prefer) require a SecurityManager or security policies permissive enough. They can be implemented using Groovy or “pure” Java. Basically makes use of AcessibleObject.setAcessible and how some classes of Java core libraries work internally, in particular the implementations available with the Oracle/Sun JDK or JRE and the OpenJDK.

1. Dynamically adding paths to the CLASSPATH (java.class.path)

change_classpath.groovy

// groovy change_classpath.groovy

def changeClasspath = {paths ->
    def systemClassLoader = ClassLoader.systemClassLoader
    assert systemClassLoader.metaClass.respondsTo(systemClassLoader, 'addURL', java.net.URL)
    def pathSeparator = File.pathSeparator
    def classpathExtension = new StringBuilder()
    for (path in paths) {
        def lib = new File(path); assert lib.exists()
        systemClassLoader.addURL(lib.toURI().toURL())
        classpathExtension.append(pathSeparator); classpathExtension.append(lib.canonicalPath)
    }
    // Required under certain conditions
    System.setProperty('java.class.path', System.getProperty('java.class.path', '') + classpathExtension.toString())    
}

def classes = ['Lib1', 'Lib2', 'Lib3', 'Lib4']
for (className in classes) {
    try {Class.forName(className); assert false} catch(ClassNotFoundException e) {assert true}
}

def extraPaths = ['lib1.jar', 'lib2classes']
extraPaths.addAll((new File('extraLibs').list() as List).collect {'extraLibs/' + it})
changeClasspath(extraPaths)

for (className in classes) {
    def lib = Class.forName(className)
    lib.doSomething()
    println "${className}.classLoader ${lib.classLoader}"
}

ChangeClasspath.java

// java -ea ChangeClasspath

import java.lang.reflect.Method;
import java.io.File;
import java.util.ArrayList;
import java.util.List;

public class ChangeClasspath {

    public static void main(String [] args) throws java.net.MalformedURLException, NoSuchMethodException, IllegalAccessException, java.lang.reflect.InvocationTargetException, ClassNotFoundException {
        String [] classes = {"Lib1", "Lib2", "Lib3", "Lib4"};
        for (String className : classes) {
            try {Class.forName(className); assert false;} catch(ClassNotFoundException e) {assert true;}
        }

        List <String> extraPaths = new ArrayList();
        extraPaths.add("lib1.jar"); extraPaths.add("lib2classes");
        for (String path : new File("extraLibs").list()) {
            extraPaths.add("extraLibs/" + path);
        }
        
        changeClasspath(extraPaths);

        for (String className : classes) {
            Class lib = Class.forName(className);
            Method doSomethingMethod = lib.getDeclaredMethod("doSomething");
            doSomethingMethod.invoke(null);
            System.out.println(className + ".classLoader " + lib.getClassLoader());
        }
    }

    private static void changeClasspath(List <String> paths) throws NoSuchMethodException, IllegalAccessException, java.net.MalformedURLException, java.lang.reflect.InvocationTargetException{
        ClassLoader systemClassLoader = ClassLoader.getSystemClassLoader();
        assert systemClassLoader instanceof java.net.URLClassLoader;

        String pathSeparator = File.pathSeparator;
        
        StringBuilder classpathExtension = new StringBuilder();
        Method addURLMethod = java.net.URLClassLoader.class.getDeclaredMethod("addURL", java.net.URL.class);
        addURLMethod.setAccessible(true);

        for (String path : paths) {
            File lib = new File(path); assert lib.exists();
            addURLMethod.invoke(systemClassLoader, lib.toURI().toURL());
        }
        // Required under certain conditions
        System.setProperty("java.class.path", System.getProperty("java.class.path", "") + classpathExtension.toString());
    }

}

Continue reading

Groovy Closure: Thoughts on the evolution

Each one of the following constructions has its own characteristics, with benefits and limitations. However, it is hard not to notice certain similarities about the concepts or the implementation mechanisms between them.

C – function_pointer.c

#include

/* gcc --ansi function_pointer.c -o function_pointer ; ./function_pointer */
int add(int a, int b);
int multiply(int a, int b);

typedef int (*math_operation)(int, int);

int main(int argc, char *argv[]) {
  math_operation ops[2] = {add, multiply};
  int i = 0;
  int result = -1;

  for (i = 0; i < 2; i++) {
    result = ops[i](4, 7);
    printf("%i\n", result);
  }

  return 0;
}

int add(int a, int b) {
  return a+b;
}

int multiply(int a, int b) {
  return a*b;
}

C++ – virtual_function.cpp
Continue reading

Groovy SQL and batch updates (microbenchmark)

Before saying that a tool is not appropriate for a particular task, you should learn as much as possible about it.

The following tests were performed using the snapshot of version 1.8.1 of Groovy, which includes the improvement GROOVY-4798. When analyzing the results, keep in mind that:

  • Microbenchmarks are just microbenchmarks;
  • The values should be compared with each other and not considered in absolute terms;
  • The corresponding servers of different RDMBS were at distinct geographic locations, making it impossible to compare values ​​that do not belong to the same group/RDBMS: while the PostgreSQL server was on the same computer on which the tests were run, the Oracle server was several miles away.


OS: x86 Windows XP 5.1
JVM: Sun Microsystems Inc. 1.6.0_23

=============================
== Testing oracle10g ==
=============================
== without batch (Statement)
1385 of 1385 rows inserted in 24375 (ms)
== without batch (PreparedStatement)
1385 of 1385 rows inserted in 22141 (ms)
== with batch (Statement)
1385 of 1385 rows inserted in 22312 (ms)
== with batch (PreparedStatement)
1385 of 1385 rows inserted in 140 (ms)
Continue reading

GORM/Hibernate performance tuning for batch processing

For cases in which the GORM is not essential, you can use Groovy SQL directly:

class BookService {
  static transactional = true

  def dataSource

  def list = {
    def sql = new groovy.sql.Sql(dataSource)
    // (...)
    sql.eachRow('select * from book')
    // (...)
    sql.withBatch(1000) {stmt -> /* (...) */}
    // (...)
    sql.withBatch(1000, 'insert into catalog values (?)') {
       stmt.addBatch('Main catalog') // ***
       stmt.addBatch('Presale catalog')
       // (...)
    }
  }
  // (...)
}

*** Preview of GROOVY-4798.

For the remaining cases, based on experience gained during the migration of a Java “batch”  (batch processing without GUI) app using Oracle as RDBMS, the following list was prepared. The applicability of each item depends on the occasion and should be examined carefully.

1. Enable, during development, logging of SQL statements executed by Hibernate

// grails-app/config/Config.groovy
log4j = {
 /* (...) */
 debug  'org.hibernate.SQL' /*,
 'org.hibernate.transaction',
 'org.hibernate.jdbc',
 trace 'org.hibernate.type'*/
}

or modify the DataSource.groovy.

The log of other components can also be enabled:

  • org.hibernate.transaction – Demarcation of the transaction;
  • org.hibernate.type – Parameters used in the DML statements;
  • org.hibernate.jdbc – Demarcation of batch updates and other information.

This item helps to identify areas for improvement or attention.

2. Disable the second level cache of Hibernate

// grails-app/config/DataSource.groovy
hibernate {
  /* (...) */
  cache.use_second_level_cache = false
  cache.use_query_cache = false
  cache.provider_class = 'net.sf.ehcache.hibernate.EhCacheProvider'
}

3. Avoid redundant checks, especially those that imply in additional access to the database

DomainClass.save(validate: false)

and

// grails-app/conf/Config.groovy
grails.gorm.failOnError=true

4. Use batch updates, adjusting the amount of statements

// grails-app/config/DataSource.groovy
hibernate {
  /* (...) */
  jdbc.batch_size=50
}

Continue reading

Investigating and solving problems: Divide and Conquer

Nothing is more direct and effective, when exposing a problem, than writing a simple application that can be re-executed in any environment. Nothing.

If the problem is simple enough, illustrate it using an executable source code in the Groovy web console.

When requesting help, through Groovy and Grails mailing lists, make such application or executable source available to anyone. Do not assign to another person the thankless task of helping you to assemble a huge puzzle, with access limited to one or two pieces of the puzzle.

If the original application is complex, isolate the problem. Extrapolate the idea of ​​divide and conquer: Write a basic application and add more components, until you be able to Continue reading

Preamble

Time is a valuable resource. So principles as KISS and DRY are important and applicable here.

I apologize in advance by extreme simplification and excessive use of hyperlinks. Feel free to discuss whatever you consider relevant, using appropriate means.

Why Groovy and Grails?

Groovy = Java++

Grails = (Spring Framework + Hibernate)++

With time and maturity of the developer and the technology itself:

Groovy = Java **

Grails = (Spring Framework + Hibernate) **

You write the same kind of application, in significantly less time.

And if I discover that some piece of code in Groovy is affecting the performance of my application? Continue reading