-
Notifications
You must be signed in to change notification settings - Fork 0
Memory management problem specification
“Resource management refers to techniques for managing resources (components with limited availability)”
It’s important to admit that we have limits. Any application in the world has limited resources, and dataClay is not an exception. However, having limited resources does not imply that we have to stop providing our services. The forest might have a limited number of trees, but the trees can be planted again. This document tries to explain what are dataClay limits in detail and how to face them in order to be functional (our service does not stop) and to offer quality of service.
We split our problem into two main sections: Resource leaks and Resource Contention. Formally, Resource management seeks to prevent both situations, per each resource.
The act of refusing to release a resource when a process has finished using it is known as a resource leak
That’s the formal definition of a resource leak. Next sections will specify the leak itself (the problem) in order to define the solution.
Memory in dataClay is a limited resource. In Java or Python, all objects in memory produce a memory leak since processes must be finished before they are collected. This is know as reclaiming resources or Garbage Collection. Garbage collector is the responsible for releasing memory. Java Garbage Collector releases the occupied memory of an object if that object is not reachable.
An object is unreachable if and only if there is no path from a GC root to the object
There are four kinds of GC roots in Java:
- Local variables kept alive by the stack of a thread. It is important to know that Java does not consider local variables as objects referenced by threads.
- Active Java threads are always live objects and therefore GC roots.
- Static variables are referenced by their classes. However, new versions of JVM introduced the ability to collect classes and therefore remove static variables.
- JNI references are Java objects but they must be treated specially because JVM does not know if any native code is referencing them.
The Garbage Collector solves almost all memory leaks in our application. Almost. Let’s define which memory leaks can be produced in any application:
- Unreachable object: Any object that is unreachable produces a memory leak. It is solved by the GC. It includes: local variables out of scope and threads that are no longer alive. In the case of local variables, "scope" refers to the lifetime and accessibility of a variable. How large the scope is depends on where a variable is declared. For example, if a variable is declared at the top of a class then it will accessible to all of the class methods. If it’s declared in a method then it can only be used in that method.
- Mutable static fields: As explained before, static fields are de facto GC roots, and if the class they belong to is not collected, they are going to remain in memory. It is a problem if the static field is a mutable field (collection) with no limits (a cache) since they could remain in memory forever.
- JNI references: Programmers must clean JNI references manually since JVM does not know if any native code is referencing them.
- Alive threads: Since threads are GC roots, programmers must control that there are not too many threads (since they cannot be cleaned) and that are properly cleaned from memory.
- Reference Retention: This leak is produced when the garbage collector is not able to do its job because there is always some path from any GC root that makes objects reachable. This retention is caused by the application code. For example, any local variable of the main thread.
Now we have some background and we know that any application should solve the memory leak cases explained before. In this section we will explain the problem in detail. For each memory leak case, we will study if there is a problem and explain them so:
- Unreachable object: Solved by the GC. ✔
- Mutable static fields: Currently, dataClay only allows users to register ‘static final’ fields, it means that the value of the field is not mutable and is collected when the class of the user is collected (the class of the user is collected when no object is using it). In the future, we want to design a solution for user-defined mutable static fields: like an object associated to the class that contains the fields in a not static way. But that’s future work. On the other hand, dataClay static fields can last forever if its class is always referenced by the service (main) thread. dataClay has limited mutable static fields by using LRU caches or similar. ✔
- JNI references: dataClay is not using JNI references. But if needed, we will free them when necessary. ✔
- Alive threads (starvation problem): Since dataClay is a service there will always be some alive threads. It is our responsibility to control that threads and the variables reachable from them are not producing any memory leak. We should split this case into two problems:
- dataClay core: the code of dataClay itself, any thread created here are under our control so we are responsible for managing them. For instance, we could state that dataClay core cannot use more than 10% of available memory. ✔
- Requests: Per each user request, there is a thread and a memory consumption. dataClay must solve this problem. ✗
- Reference retention: GC is not able to do its job in dataClay because there are some threads that are ‘retaining’ objects in memory. The GC root here is the local variables kept alive by their stacks. dataClay must solve this problem. ✗
At this point, we know that we must design a solution to solve starvation caused by ‘reference retention’ and ‘alive threads’. We define now the predicates to be satisfied by our solutions:
Remember that local variables that are kept alive by stack are GC roots. When a method exits, they are out of scope (the stack is not keeping them alive), then they become unreachable and can be cleaned up. However, if there are too many local variables the memory can be filled. Java prevents this from happening with ‘StackOverflow’ exceptions. For dataClay, this might not be enough due to:
- Remote executions: If a thread requires waiting for a remote execution, those variables are going to stay in memory. Threads and local variables cannot be cleaned up by the GC and can produce a memory leak. Finishing the remote call does not mean that the resources are going to be released, but it is important to notice that remote calls could cause memory leaks.
- All memory filled by objects kept alive by stacks: N threads retain a certain number of objects that use the whole memory available. We could have: Big objects and a few threads, Many threads and small objects or the combination of both.
Remember that a Thread is actually an object and therefore it consumes memory. In this case, N requests are working and not finishing their job (or taking too long) so there are too many threads and GC cannot release them and new requests cannot be processed. There are two reasons the threads are not able to finish their job:
- Iterative execution: Threads are running loops that take too long or not finish and therefore they are retaining memory (at least, the memory occupied by the Thread object itself) and cannot be released.
- Recursive execution: Methods might need to call other methods that are not in the present node. We specify that if the method to call is actually in the present node it is an ‘iterative execution alike problem’. However, if the method is in another node the thread must wait for it to finish, and cannot be released.
If our solution satisfies both predicates, our solution is valid.
In conclusion our problems are memory leaks caused by alive threads and reference retention explained here.
We have specified the problem, next sections explain the solution. Our problem does not have only one possible solution. Actually, we have found two kinds of solutions:
- Solutions based on limited access to resources: we establish some limits to users, threads, requests, … in order to ensure that there is no leak.
- Solutions based on unlimited access to resources: we modify our core design, our use cases (dataClay threads, execution classes, models) or which services we provide to ensure that there are no leaks. Each solution must explain how it satisfies each predicate detailed in next sections.