Support GC log file rotation

[lifey]

Java 7 supports Rolling GC logs files with the parameters
-XX:+UseGCLogFileRotation
-XX:GCLogFileSize=M
-XX:NumberOfGCLogFiles=
the files will be suffexed with .1 .2 .3 etc....

GCViewer should be able to read all the files at once and merge them in correct order.
Add a checkbox to load all related files or add a popup dialog to ask if you want to load all related log files.

[chewiebug]

Hi Haim,

That looks like an interesting option - I'll have to look into it.

Regards, Jörg

[lifey]

Hi Jörg , really sorry
I forgot I opened this issue and I have opened the new one as a preliminary action before I start solving this issue

[chewiebug]

Hi Haim,

No problem. It is a very good idea to open an issue as a preliminary action before solving it!

I am looking forward to seeing your pull request!

Regards, Jörg

[geld0r]

Any progress on this?
I'd also be interested in this feature and could - depending on implementation effort - also try to work on this.

[chewiebug]

It looks, like Haim never got round to implement this issue. If you have some time to work on this one, I'll happily assign it to you. Would you like to have some hints on where to start?

By the way: Please pull the latest commits from the "develop" branch. I have just merged a feature, that included quite some refactoring to GCViewer.

[geld0r]

Yes, hints are very welcome. I'll make sure to pull the develop-branch.

What I have been thinking about: How should non-consecutive logfiles be handled?
In contrast to log4j/logbacks rolling mechanism, the logfiles simply start with 0 after a JVM restart. In such a case the files from a previous run still remain, but must not be appended.

Edit: I already looked into this.
For actually merging the files I'd use IOUtils (from apache-commons-io). Its not a dependency of the project (in fact nothing from apache-commons is). Is that on purpose?

[chewiebug]

I think, I'd start here:

• extend GCResource with a flag, to say, if rotated log files should be
  loaded
• extend DataReaderFacade.loadModel() to load the whole list of the
  files and return them inside the same GCModel object

To tackle your question concerning non-consecutive logfiles: The class
GCModel has an inner class "FileInformation" that is able to hold size +
lastmodified timestamp of a file or an url. Maybe, we can use this class
to check, if a next file in the sequence is younger, than the current
and only then load the next file. If it is not younger, stop loading.

Probably, the DataReader interface should be extended, so that an
existing GCModel can be passed in. So that with those consecutive files
loaded, all events can be put into the same GCModel object.

Maybe, as a second step, a menu item could be added to be able to turn
loading of the consecutive files on and off. If we even extend
GCPreferences, we can also save the item in the properties file.

To complete the whole thing, a command line parameter could be added to
allow to choose the behaviour in that respect as well.

What do you think? Can you start with this information? If you would
like to have more hints / discuss ideas, please just tell. If you can
implement the first step, I'd say the main part of the solution is
there. If you still have time, you may attack more steps.

[geld0r]

If already started in the meantime and followed the apporach that I use when doing this by hand:

• Add new menu option to load several files as a series
• files are (attempted to be) sorted ascendingly. If possible this is done by timestamps in the log (e.g. when either -XX:+PrintGCTimeStamps or -XX:+PrintGCDateStamps was active). As a fallback this could also sort by file by creation date as you suggested.
• The file is then merged and then loaded as if it was a single file.

What do you think of this approach?
Your other suggestions can of course be integrated, e.g. when the GcModel was already built for sorting it could of course be reused for building the merged model/file.
Adding a commandline option sounds nice as well.

[chewiebug]

I like the idea to have a new menu option for the new feature. Like this, it is very clear for the user, if he uses the new / different functionality or still the one he was used to. Your approach with first merging the files and then parsing them will also work. As long as you keep the logic of how to find out, which files to merge and parse under control of the DataReaderFacade, extending the implementation for usage with commandline will be very easy. Implementing the logic inside DataReaderFacade will probably proove to be more difficult, because more classes need to be changed. If you prefer to implement a more straightforward approach, just go ahead. Refactoring it later should not be overly difficult. By the way: The current implementation of GCModel relies on the parser to add the events sorted by timestamp. It contains no sorting logic of its own.

[geld0r]

You can have a look at a current WIP state here:
https://github.com/geld0r/GCViewer/tree/feature/%2361

As far as my testing goes this works fine. I'll start with the refactoring you mentioned next.

[chewiebug]

Looks good! I very much like your unittesting! The FileMerger class, especially #getFirstTimeStamp() is yet inefficient, but you obviously know that. I am looking forward to your refactoring. Would it help your refactoring, if there was a method DataReader#read(GCModel model) that would append to an existing GCModel?