-
Notifications
You must be signed in to change notification settings - Fork 0
Disk management problem specification
“Resource management refers to techniques for managing resources (components with limited availability)”
It’s important to admit that we have limits. Any application in the world has limited resources, and dataClay is not an exception. However, having limited resources does not imply that we have to stop providing our services. The forest might have a limited number of trees, but the trees can be planted again. This document tries to explain what are dataClay limits in detail and how to face them in order to be functional (our service does not stop) and to offer quality of service.
We split our problem into two main sections: Resource leaks and Resource Contention. Formally, Resource management seeks to prevent both situations, per each resource.
The act of refusing to release a resource when a process has finished using it is known as a resource leak
That’s the formal definition of a resource leak. Next sections will specify the leak itself (the problem) in order to define the solution.
All objects in dataClay are persistent and therefore they consume disk space. Disk space is not unlimited and if there are too many objects we could have a disk leak.
First, we should specify the problem: in which situations disk leaks can happen in dataClay.
We should analyze all possible ways that an object is persisted:
- Make Persistent: The user requires some object to be persisted.
- Replica: The user creates a replica of the object
- Volatile created: Any object created by any method in dataClay is persisted.
- Volatile received: Any object sent by the user to dataClay is persisted.
Volatiles system summary If you are asking why volatile objects are persisted there is another document explaining it in detail. Let’s explain an example to show you the decision.
We could set a volatile object (created or received) as a field of a persistent object and unset it later. If the volatile is associated to a persistent object it should be persistent, otherwise it should be removed. But, even if we know the volatile is not assigned to a persistent object anymore, we cannot remove the volatile if the client is using it! Is the client the only one that could remove the volatile object?
As you can see, this simple example (one of many situations) shows us that we cannot delegate the responsibility of deleting volatiles to the client and even if the volatile is not assigned to any object inside dataClay it must be persisted since there might be some client using it (one could think about ‘retaining it in memory’ until the clients have finished but this is a memory leak, so it MUST be persisted).
Since volatiles must be persisted anyway, we decided that all objects in dataClay are persistent.
Now, we define the predicates of our problem:
Too many make persistents could cause a disk leak if the objects to persist consume all disk space available.
A replica is also an object and it also consumes disk space. Therefore too many replicas could consume all disk space available.
First, we should define what is an unreachable object for us. Unreachable objects are objects in disk that cannot be used anymore by any client or request and must be deleted. If they are not deleted as soon as possible we could have a disk leak. For example, if a certain method creates N volatile objects but they are not assigned to any object, they go to disk but they are never going to be used again.