Google Tech Talks

Google tech talks is a series of video recorded presentations hosted in 'google video'. It is about technologies involved in google products and google apis. Of course, Java and related technologies are an important subject in the series :)

Here is a Joshua Bloch's presentation about good api design. Joshua is a popular software engineer known from his best-selling book "Effective Java" . Enjoy! ...

Real time applications in the java platform

This article is about the Real Time Specification for Java (RTSJ). We will cover the following areas:

• What is real time and how it can be used to solve mission critical financial applications
• Is java a suitable language for a real time development and what keeps it away from being real time
• What is RTSJ and what features are provided in the Sun RTSJ implementation

Financial markets are getting more complicated by the addition of new products and services or by the application of new regulations. IT systems serving the industry need to cope with the new factor of complexity. High level languages such as Java is used extensively to tackle the new evolving requirements in the financial area. Java provides facilities to developers to focus on the business problems ignoring common problems like memory management, etc. However the new advantageous benefits come with some trade-offs. The common burdens are now handled by the Java virtual machine which adds an unpredictable/unexpected behaviour to java IT systems. Real time behaviour has been gradually lost from financial applications. The implication of that can be probably measured in thousands of dollars damage having an impact to the further development of the financial IT infrastructure. Adding a real time aspect to java banking systems could minimise these loses.

Algorithmic trading system is a classic example of a financial application that could benefit from real time support. In particular, a market maker system where users place buy and offer limits to benefit from the bid-ask spread, could provide a stable trading performance. Lets assume a high volatile market where a market maker application operates. When the system's quote monitor thread reports the last quote price a potential garbage collector invocation could delay the execution of a trade for unknown time. Such delays cause losses because of the unpredictable latencies. These losses can be illustrated in the following diagrams:

In a real time system these losses could have been minimised or even eliminated. As many financial applications are built using the java platform it becomes apparent that a real time approach should be followed together with the java software design. Most of us might think that this is impossible taking to account the nature of the java platform. Fortunately though, Sun and other top IT vendors have published the 'real time Java specification' (RTSJ) which tries to address the aforementioned issues. Before entering into more details about the RTSJ and its reference implementation let us define some properties of a real time application.

A general misconception about real time is that it means "real fast". This is wrong! It actually means "respond within a predictable time". There are two important measurable variables of real time applications: latency and jitter. Latency is defined as the average time it takes the system to respond to a higher priority interrupt. Short latencies result in fast execution times. Jitter is a measure of the variability of the latency number. Short jitter numbers increase the predictability of the system responses to events. The two variables are depicted below:

A common attribute of real time applications is that they are mission critical. A potential failure in the system can have fatal consequences. For instance a DVD player could be considered as real time however there are no significant effects on the users when the system fails to render the frames promptly. On the contrary an execution delay in a auto-pilot system could cause the loss of many lives. Therefore the design of real time software is a very difficult process. RTSJ - the real time specification for Java - tries to define a clear to grasp real time platform and ease the software design of such systems. A brief analysis to the java language and platform will allow us to view the benefits of using a high level language in real time systems and what keeps java away from being real time.

Java as a high level language provides helpful features that allow them to focus on the solution of the problem comparing to traditional real time languages like C and C++. Below I provide the strong points of Java:

• Java is a type safe language. Its designed to catch as many errors as possible during compilation time.
• As a pure object oriented language, it allows developers to write software in a modular way. Its access control to methods and fields help developers to hide information about the implementation.
• The memory model is reliable and safe. References to object could not lead to protection or segmentation faults. These errors are very often in native language like C and C++
• Threads are first class citizens in the language. Concurrency can be tackled better and easier using Java.
• Synchronisation is again a language feature and is not depended to external non standard libraries. Besides java 5 introduced the concurrency api which removes some burden of designing multi-threaded applications.

The above features of java gives developers higher productivity and improves quality of software systems. However Java has not been designed initially to solve real time problems. Java SE applications cannot be predictable for the following reasons:

• Garbage collection (GC) is a good and a bad thing. It removes the burden from developers of clearing the memory but it lowers the predictability of the system. GC has the ability to stop the applications threads to do its own task in the heap memory. It does not care if the system entered a critical state. The duration of its invocation is also unknown. This is because the GC implementation is vendor specific rather than part of the java specification.
• Although java supports threads, it does not provide a scheduling control. The priority property of java threads is optional and jvm vendors are not forced to implement it.
• The use of standard SE synchronisation can lead to concurrency problems like priority of inversion. This is the state of an application where a higher priority thread waits the release of an object lock from a low priority thread.
• The standard event processing model in java is not predictable.
  
• The Just in Time compilation which adds many benefits to the java programs (compilation optimization at the deployed machine, etc) adds some unpredictable behaviour to the java systems. Code paths which belong to critical tasks could be compiled during run-time resulting to a higher latency and jitter.

The Real-Time Specification for Java (RTSJ) was specifically created to address these problems. Strange enough, RTSJ is the first JSR (JSR 0001) of the java community process (JCP). It provides the specification of a real time JVM and the sun's reference implementation. The specification does not alter the semantics of the java language and can be thought as an API extension of the Java SE and its current version RTSJ 2.0 is java 5 compatible. The specification designers have defined three real time levels of Java. The "non real time" the "soft real time" and the "hard real time". The non real time java is the level of the Java SE which every developer programs. The new RTSJ features could be used in the soft and hard real time levels. The following diagram show the relation of the 3 levels in an application which is developed using all:

The soft real time level is used for moderate critical application processes. The key point of this level is the ease of conversion of existing non real time tasks to soft real time. The hard real time level is used for mission critical tasks. The developers who write hard real time threads have to do some extra work regarding memory management and inter thread communication. Developers could consider that a hard real time thread runs in a special container.

The RTSJ new features can be summarised below:

• There are two new types of thread the RealTimeThread and the NoHeapRealTimeThread in addition to the standard thread. The new real time threads are schedulable compared to standards threads. Therefore the added thread scheduler in a rtsj compliant jvm will respect the priorities defined in the working threads. Developers in soft level can use the RealTimeThread while in hard level can use the NoHeapRealTimeThread.
• RTSJ deals with memory allocation. It gives developers more option to manage memory. There are 2 new kind of memories in addition to heap memory, immortal and scoped memory. The scoped memory is a memory space which is attached to a code block. During the code execution, objects which have been created will be allocated in the scoped memory rather than the standard heap memory. At the end of the code block execution the scoped memory is cleared without any checks to object references. As you can imagine GC is not allowed to work in scoped memory. The immortal memory as its name implies is a memory which its object are not managed by the GC and are not deleted by an external process till the end of the application. Therefore developers should be responsible of managing the immortal memory space. The RTSJ api defines classes and methods which deal with memory management. We have to note here that the memory management is similar to C++ but the whole management is monitored by security jvm components. The developers cannot allocate memory wherever they desire. soft real time threads have access to heap and scoped memory while NoHeapRealTime thread have access to scoped and immortal memory.
• The RTSJ jvm give the ability to the application developers to define their own thread scheduler.
• The reference implementation includes a new real time garbage collector which is more predictable than the standard one. It belongs to the time-based family of garbage collectors and it uses the Henriksson's (Lund) algorithm. That is the GC operates in the lowest priority of the critical threads avoiding any delays to soft real time threads with higher priority. Memory allocated by a soft real time thread in the heap has higher priority of de-allocation by the real time GC.
• Priority inversion in RTSJ is tackled with the priority inheritance and priority sealing mechaninsm. Both features are provided by the jvm and its transparent to the users. The specification forces vendors to implement the priority inheritance and has the priority sealing as optional.
• The event mechanism in RTSJ includes asynchronous events. They can be triggered by Unix signals, timers or by your own application calling the fire method. Other part of the code can register handlers to deal with the event objects.
• Asynchronous transfer of control is a new RTSJ feature. Threads can asynchronously raise an interrupt exception to a target thread. The target thread will stop what he is doing and will transfer control to an exception handler which can do many things like recovering from the intermediate state of the thread or shutting down the application.
• The api provides high resolution precision of nano-second level to timers. The actual resolution is implementation defined.
• To avoid JIT compilation latencies the RTSJ sun implementation allows users to define a list of methods used in NoHeapRealTimeThreads. The methods will be compiled at the start up of the application and not during their first invocation. Because of the fact that NoHeapRealTimeThread cannot access the heap memory any checks for object references to ensure that they don't point to the heap can be avoided by defining a list of classes of which their objects will be only created in scoped or immortal memory.

Sun provides a jvm for the Solaris operating system and support for 'x86' chips and Linux is expected in the near future. The reference implementation is not free however a evaluation license of 90 days can be obtained from Sun. Lastly, it has to be noted that the current J2EE application servers are not RTSJ compliant yet. Commercial RTSJ compliant servers are already released in the market.

My opinion is Java real time design allows developers to write more code in the soft real time level where they can have all the java SE facilities (memory management, etc) and less code in the hard real time level. According to Sun, the best thing is all this gives us higher predicatability but at a cost of lower throughput. The RTSJ specification and its reference Sun implementation have a large number of features which cannot be listed all together. More details can be found in the 'Links and References' section of this article.

Links and References:

• RTSJ official website
• Bollella presentation on RTSJ
  
• Bruno presentation on RTSJ in finance
• Locke presentation on application design with RTSJ
• Presentation about RTSJ specifications
• Simon Ritter about real time java
  
• http://en.wikipedia.org/wiki/Priority_inheritance
• Priority Inversion
• Priority Ceiling
• Algorithmic Trading
• Real time java introduction