Export EARLYOOM_ENVIRON of killed processes along with related info #340
Conversation
With this change, along with info such as cmdline, PID, UID and so on, extra information (its environment variables) will be available to allow to identify the process being killed due to memory overuse. This is not captured by the existing methods, and it can be useful to identify processes which are invoked in the same generic way and whose runtime behavior or the data that it processes changes depending on the environment at the time of launch, or that simply by looking at the content of the variables in its environment gives extra info about a single process from others using the same binary. This can be used in combination with -N to run a script with info about the process just killed, and use the information on the desired environment variables to log/notify about the problem.
|
Hi, what would be an example process where this is needed? It's somehow problematic becaue there may be secret data (passwords...) in the environment. |
For example:
Yes, this is true. Would adding an option to only export specific variables (e.g. |
|
I meant a concrete example. Postgresql? Microsoft access 3.9? |
|
The ones with this in use and other examples that I was thinking about are actually purpose-built programs, mostly, not like PostgreSQL or other DBs. For the second kind, a wrapper that launches a different application and contains identifiers in the environment about the app in question, for different purposes. In this case this app-id is used in reports if an OOM or other conditions happen (to correlate the actual application using too much memory, or having other problems). For the first kind, I was not using it at the time, but in retrospective it would have been very handy. That was in lab environments processing large amounts of data, and the jobs sometimes killed also due to OOM. They were generic scripts and programs working on different datasets; and not always, but at least often, defined in the environment. So basically my example in a previous message. |
|
No, sorry, looking at the additional capabilities this requires, in addition to running as root, AND the requirement (and headache) to filter out secret variable, this is too risky. |
I see, I understand, thanks for considering it. Would you think that it's acceptable and worth adding the option to run another external program before killing? In this case, it would be the external program doing the filtering of env-variables, or whatever data-gathering actions deemed necessary. This program would also need special capabilities for some of them, so spawning from the earlyoom systemd-service as is would not help (out-of-the-box), but the added capabilities could be local configuration in given systems, not affecting the default service file that you ship. (Sure, in near-OOM situations this is risky, but in many modern devices the threshold can be when there are still hundreds of MBs still available, so the risk is small IMO. And in any case, any problem that it might cause, like getting deeper into OOM territory and lock-ups, it would be the users'/admins' responsibility). |
|
Yes, I'd call it a pre-kill hook. Sounds good. |
2 participants
With this change, along with info such as cmdline, PID, UID and so on, extra information (its environment variables) will be available to allow to identify the process being killed due to memory overuse.
This is not captured by the existing methods, and it can be useful to identify processes which are invoked in the same generic way and whose runtime behavior or the data that it processes changes depending on the environment at the time of launch, or that simply by looking at the content of the variables in its environment gives extra info about a single process from others using the same binary.
This can be used in combination with -N to run a script with info about the process just killed, and use the information on the desired environment variables to log/notify about the problem.