OOM termination on very large calendar

[marsianer]

Problem scope

• I'm sure that this is a DAVx⁵ problem.

App version

• I'm using the latest available DAVx⁵ version.

Android version and device/firmware type

Android 11

Steps to reproduce

I synchronize my calendar with DavX5 and Radicale.
Unfortunately I get an OOM termination.

Failed to allocate a 91922592 byte allocation with 25165824 free bytes and 80 MB until OOM, target footprint 209636936, growth limit 268435456.

Total RAM: 6.0 GB
Free RAM: 3.5 GB

Note:
I record my running routes and attach them to the calendar.

DavX5:
v4.4.4-ose

Radicale:
v3.3.1

Actual result

The calendar cannot be synchronized.

Expected result

The calendar is fully synchronized.

Further info

The calendar has a file size of 394.6 MiB.

[rfc2822]

Unfortunately I get an OOM termination.

I guess on DAVx5 side?

DAVx5 has to keep the whole iCalendar at once in RAM, and it can't just allocate 400 MB.

Also there would still be an 1 MB IPC limit in communication with the calendar provider, meaning that event data > 1 MB couldn't be synchronized.

So the question is how it should handle this situation – probably show a sync error, which is exactly what it did, I guess?

Are the routes attached inline?

Can you provide a sample file and more detailled steps to reproduce? I'd also need the debug info with the sync error.

[marsianer]

I guess on DAVx5 side?

Yes, the error is on DAVx5 side.

DAVx5 has to keep the whole iCalendar at once in RAM, and it can't just allocate 400 MB.

The calendar file has a size of 394.6 MiB.
The size increases after each training session.

Are the routes attached inline?

All routes are attached inline.

How about splitting up the components inside the calendar file?

• VEVENT
• VTODO
• VJOURNAL
• VFREEBUSY
• VTIMEZONE
• VAVAILABILITY

This way, each calendar entry would be its own file, which can then be imported one by one. The memory limit then no longer exists.

[rfc2822]

As far as I know, ical4j (the library we use) doesn't support streamed parsing. We get the whole iCalendar as one object.

Also it's possible that even with streaming we would have to keep the whole iCalendar in memory to sort and associate the RECURRENCE-IDs and things like that.

I'd say that such large events are just not supported because of the limited resources (heap size, not RAM) of the mobile environment.

The question is only whether we should ignore the whole event or how to handle it, given that it can't be processed.

[marsianer]

I forwarded the question:
iCal4j / Streaming API

[rfc2822]

How about splitting up the components inside the calendar file?

But one file still contains one big EVENT, right? Or do you have a recurring event where every route is attached to an exception?

[marsianer]

Many events include attachments.
However, parsing them individually shouldn't be a problem.

File file = new File("calendar.ics");
// unfolding reader handles split lines..
UnfoldingReader calendarReader = new UnfoldingReader(file);

CalendarParser parser = CalendarParserFactory.getInstance().get();
ContentHandler handler = ...

parser.parse(calendarReader, handler);

Technically it's the low level of DefaultContentHandler.

[camlafit]

Hello

I catch a similar error with notes part. I use JtxBorad to took my note and add some media as picture to my notes.
After some pictures added, davx complain with an OOM :
"Failed to allocate 174... byte allocation with 100... free bytes and 104MB until OOM, target footprint 259..., growth limit 268..."
Related dump is available, if required;

Thanks

[rfc2822]

Yes, attachments should ideally be in separate files, not inline. This is however not yet supported.