Let’s talk about the reports

Another long time until last post. I’m a lazy writer after all. 😉

Working on something interesting recently, very interesting indeed, but in this article, the part that I want to speak, is the report part.

1. The requirements of the report

Let’s start with the requirements part.

For the reports, I have these requirements:

1. The data of the report should be load from the database, but, will allow the report be generated using Java’s Map and POJO.
2. It should respect the privileges, the privileges should get from some authentication service, and enforced by using AOP, for example, the Spring Security
3. The report should have different kind of devices to view, for example, PDF for printers, HTML for web interface
4. The report engine’s API should be as easy as possible to develop
5. It should be easy to design and implement the report template
6. It should be easy to test and render the report template
7. Some report will need precision print, for example, every cell’s in the grid must have the width and height of 1mm exactly. Because this is the standard for these reports

And, as usual, I’ll explain all the requirements in more details as below.

1.1. The data of the report should be load from the database, but, will allow the report be generated using Java’s Map and POJO.

This seems the most useless requirement. Yes, nearly all the report engine support this. So, I’ll turn to next requirement.

1.2. It should respect the privileges, the privileges should get from some authentication service, and enforced by using AOP, for example, the Spring Security

There will be lots of users in this system, in a tree style of customer privilege model. For example, the organization level user will have more privileges than the department level user. The system can support these privileges through the container’s AOP security support(such as Spring Security). But most of the report engine will know nothing about the security, or even support AOP by default.

But, if the report service didn’t respect the privileges management of the system. Anyone know about the report API will have the ability to access the reports and data that he didn’t have authority to view. And for this system, this is a big security flaw.

Though, the user may have the authority to view the report, for example, it is the department’s manager, so he will have the authority to view the report of his department, but, he’ll have no authority to view other department’s data.

So, even though, the user may have the authority to view the report, the data retrieving part, must consider the privileges too.

Let’s make it short:

• The report engine should have the support for restrict the access to the report template by using the privileges to filter the access
• The report engine should have the support for restrict the data access for the report by respect the privileges to filter the data access

1.3. The report should have different kind of devices to view, for example, PDF for printers, HTML for web interface

This requirement is quite crucial. The user of this system should be able to view the reports by Phone, Tablet and PC, so the basic HTML support is a must have.

But, then they’ll need to have the report pretty printed in the paper. And the report should be pretty printed(This will have a nice report template design included)

To create the same function in different implements is a foolish idea. The report engine must have the render engine for each device, and even more, for example, can render the report to Image.

1.4. The report engine’s API should be as easy as possible to develop

This is the most basic programmer’s requirement, I’ll turn to next requirement.

1.5. It should be easy to design and implement the report template

Yes, this is an very important requirement. As a programmer, I didn’t afraid to code, but I’m not a good designer, and believe me, all the report designers sucks.

They really sucks, not only stupid and hard to use. And they all invent some kind of foolish concepts, that they thought this should be understand by the novice user, but not.

Just think of the Visual Designer based on Swing for Swing, you’ll understand what I’ve said. I prefer writing code than use these tools. They are all slow and stupid, will do nothing instead of slowing you down.

1.6. It should be easy to test and render the report template

I don’t believe WYSIWYG, and I suggest you should not believe that either.

What I believe is how can I get the result as soon as possible when I developing the report template. And render it as it should be in the live environment, so that I can understand what to do, what to change.

Let’s take an estimate.

Say, if it’ll take about 1000 operations (create, modify and delete the components in the template). And you’ll need to render the report once every 10 operations.

So, you’ll need to render the template 100 times to complete(for the real situation, it should be more, since you may tweaking some small details of the component to make it prettier).

And this time, you can see the efficiency of the report rendering.

Say, for each render, you’ll take 2 seconds more.

Then 2 * 100 = 200 seconds, you’ll take at least half an hour to complete this report(and you’ll know it will be at least 4 times larger, since 2 second is an awkward time, you’ll take other tasks just when you start the render operation, and after you switch back, it’ll be lots of seconds later).

Don’t underestimate these seconds, it’ll may effectly delay your launch and make you frastrated very quickly. You should have a better life somewhere, instead of wasting the time struggling with the report template.

1.7. Some report will need precision print, for example, every cell’s in the grid must have the width and height of 1mm exactly. Because this is the standard for these reports

This is the requirement of the user of my system. And this is an sane requirements, that’s what reports are for. And we can indeed make this possible by using nearly all the common report engine.

This is the foundation requirements for the report engine.

2. The solution for the report

So, after the requirements, let’s face the solution for the reports.

2.1. The report engine

For the report engine, I choosed JasperReports. Because it is written by Java(the language that I wrote my system). It supports JDBC’s ResultSet, Map and Java’ POJO as its DataSource.

And Jasper is very stable, it based its function on some very stable libraries, and it is used by lots people, and it is opensourced.

This is my choice, you can use your choice, but, it must have the support for dynamic report.

2.2. The dynamic report support

Like what I said above. I didn’t expect that Jasper Reports will respect my application’s privileges management, but, I’ll need to code it myself, so how can I implement that?

By writting my wrapper for the dynamic report interface above Jasper.

I’ll explain the details here:

1. All the report request and API is created by myself, and all the service will go through the AOP security, so that I can apply the security filters as I want to make the system secure, and disable them in the development mode to let my testing more easier, since all the API is done by myself, I have the complete control of what to do and how to do
2. My code is based on the dynamic report library above the Jasper’s engine API. It is a wrapper layer to generate the Jasper’s report template by API, so I can create and modify Jasper’s report template on the fly using the code. This is quite useful for my requirements, so that the system will have the ability of modify the appearence of the report in the live by the request parameters or the configurations, this will make the report more intelligent and I can write less reports for the user since some simillar reports can be combined into one report
3. The dynamic report library will let me write the report by code, not by the FUCKING report designer. Believe me, I prefer to write code than using that XML based FUCKING designer, it is stupid and useless

2.3. The all in one device support solution of Jasper Report

Jasper is using a very brilliant way to handle the reports. It uses Java’s Graphics2D layer as a standard layer of abstraction for rendering.

And Java’s Graphics2D is a good abstraction for rendering after all(and a defacto standard of Java, the foundation of Swing).

So, when Jasper based it’s rendering on Graphics2D, it have all the type of Image rendering by default since JRE has already has them in ImageIO.

Then, Jasper is using the splendid iText Library to create the PDF files. The iText library implements the Graphics2D library to render the result in PDF.

By using this, Jasper has the ability to render the report into beautiful PDF files.

And, not only this, Jasper can use Apache Batik to render the report into SVG. So, we can render the report into the vector document using the same appearance.

Jasper provide its implement for the HTML too, and it will only support the logic elements, all the rending components that will need the graphics, Jasper will render them into Image, and have the image servlet to serve them.

I don’t want to write too much about Jasper in this document, but I really like Jasper’s way of doing reports.

Why? Since Jasper is using Graphics2D as the rendering engine. that means, that Jasper’s report can be render in Swing natively and perfectly, since Swing is based on Graphics2D too. And Jasper is using this for it’s report reviewer. I’ll talk about it later.

2.4. The easy to write API of DynamicReports

For me, I’m using DynamicReports to create the report templates.

Yes, I know DynamicJasper, you can choose the library you like, for me DynamicReports is good enough for me to use, and the API of it is quite pleasant to use.

So, I won’t compare about the APIs of these libraries, for me, I can write the report template by code, and the API is nice to use, is enough. You can pick anyone you like.

2.5. The work flow for me to develop the report templates

1. You should mock the data to render the report instead of make the report reading the data from the DataBase this will save at least 0.5 seconds or more for rendering your template, depends on how you connect to your databas

2. You should use the Swing report reviewer to review the report instead render it into PDF or HTML. Since if you want to render the report to HTM, you’ll need to setup the JavaEE server and the web application, or you won’t see any charts(that’ll need be rendered as Image, and if you didn’t configure the Image servlet — yes, even after you configure it, you’ll need to put the JasperPrint object into the session, so, this means, that you must open this HTML by using the web server, or there is no session at all). And the PDF one is slower than the Swing review, since PDF is using iText library and the Swing one is using the Graphics2D directly, the render performance is better.

3. You should not start the Jasper Engine every time you start the template, Jasper Engine’s startup time is quite slow, will need about 2 seconds before rendering.

But, we got into a dead end here, the report template is dynamic, and is written using Java code, so we can’t really deploy it into JasperReportServer, since JasperReportServer will only support static XML report templates.

And the report template is Java class, we’ll change it every time we update the code, and need to reload it into the running report and render it out again. How can we get this done?

Here is my solution:

3. The solution for writting and review the report template faster

This part is the most valuable part of this blog, and this is the purpose of this blog, if you have read this long, please read this section carefully, this part really constains some of my adivces for writing the reports by the dynamic report API to design and debug the report templates.

1. Use JVM’s Hotswap function to load the changed report code automaticly Using an IDE that support JVM’s Hotswap support, Eclipse for example. That will swap the code in the JVM to the new code that you have changed, so that, you can keep the report reviewer opening, without restart the Jasper engine or the View form, to view the change of your report template. But, the default Hotswap support of JVM is quite lame, it only support method level’s change, it won’t support class reloading, so, you’ll need to patch it. By using this HotSwapAgent‘s support, you’ll make the Hotswap as perfect as you needed. With this help, you can update the report template’s code in the reviewer anytime you update and save the code(it must compiled successfully).

2. Write your report reviewer to watch the code change then update the report view automaticly. Jasper’s report reviewer didn’t support the dynamic report reload. So, even the code is changed, you can’t see the update unless what you changed is in the repaint area, and not the logical part. To avoid this, you should write your own report viewer, and then watch the code change using Java 7 NIO’s file system event support. For me, I just watch all the Java code file in the source folder, and if any file is changed, I’ll reload the report.

It’s a pity that HotSwapAgent didn’t have any JVM level event to tell me what class is reloaded, or I won’t need to watch the source file change, just listen to its event is fine.

So, if HotSwapAgent can have some API to support this, this will be quite good for me. 😀

After all this, it is quite fine to write the report by using DynamicReport.

Here is my workflow.

1. Open Eclipse’s project
2. Start the report preview application to preview the report in debug mode(in this mode, Eclipse will enable the JVM’s hotswap)
3. Write and change the java code for the report, then click save, and wait the report preview application get the file system event, and reload the event(and force repaint the whole component to avoid the stale paint)

These steps only takes few miliseconds, compare to the original 2 seconds or more, it is fast like lightning.

And since the report that I saw, is the final result, this is truely what you see is what you get for me, now.

So, this is how I create reports in my project, I hope this will help you about these problems, and make your life easier.