ENH: Print failure logs from logfiles for ERT log by slangeveld · Pull Request #138 · equinor/runrms

slangeveld · 2026-07-14T11:32:35Z

Resolves #124

Add failure logs found in RMS logfiles as part of the output that is printed to ERT GUI after running RMS.

In this way we hope to improve the current error logging situation for users where they are left a bit in the dark if RMS fails during an ERT run. With this change, the users should in some situations (f.ex if a python script is failing) get a better understanding of what is actually failing and needs to be fixed.

Today, runrms prints the path to the logfiles so that the users can go and look for errors in the files themselves. This points them in the right direction, but actually finding the errors in these files can be challenging as the files can be large and it is difficult to know what to look for.

With this change, runrms will now:

Look after failing jobs in the logfiles found in the project
Print the job failures that are found as part of what is returned to ERT and displayed to the user

🔍 An example of how the job failures will be displayed in ERT gui can be seen in the comment below.

💬 Some comments are added below explaining assumptions that are made when looking for failures.

Checklist

slangeveld · 2026-07-16T10:26:59Z

Example of the new logging added for a failing job (what is within the red circle is what is added in this change)

slangeveld · 2026-07-17T06:37:36Z

+
+        # Find the line number where the logging of the last job starts
+        with open(logfile_path) as logfile:
+            for line_no, line in enumerate(logfile, start=1):


The assumption is that the logs from the failing job we are interested in, will be at the end of the file as that will break the workflow (and stop logging)

Therefore we look for the log section for the last job that was ran before the workflow breaks

slangeveld · 2026-07-17T06:39:12Z

+        # Find the line number where the logging of the last job starts
+        with open(logfile_path) as logfile:
+            for line_no, line in enumerate(logfile, start=1):
+                if line.strip().startswith("<pre>"):


Each new job starts with this <pre> tag

slangeveld · 2026-07-17T06:41:08Z

+            if last_job_line_no == 0:
+                return ""
+
+        # Collect the logs for the last job and look for failures


Once we know which line the log section for the last job starts, we can iterate through this section line for line, parse the logs and look for failures.

slangeveld · 2026-07-17T06:54:08Z

+                    if cleaned_text != "":
+                        job_failed_msg += cleaned_text
+
+        return job_failed_msg if "failed" in job_failed_msg else ""


When a job fails it is logged "Job: failed", "Failed job " or "Workflow failed" so the assumption is that as long as the log section for the last job contains the word "failed", the job has failed and we want to return those log messages.

Can probably improve a bit on how we evaluate if a job has failures, but maybe it is good enough as a first iteration

slangeveld · 2026-07-17T06:55:17Z

        * rms/model/workflow.log
        * Other named log files in rms/model, e.g. workflow_sim2seis.log
-        * rms/model/YYYYMMDD-HHMMSS-XXXXX-RMS.log corresponding to your run
+        * rms/model/YYYYMMDD-HHMMSS-XXXXXX-RMS.log corresponding to your run


From checking the logs on scratch after an ERT run, the given format appeared to be wrong. The XXXXXX part of the file name has 6 characters, not 5.

slangeveld · 2026-07-17T07:00:03Z

            )
            fail_msg += "\n".join([f"* {f}" for f in log_files])

+            for log_file in log_files:


Loop through the logfiles and look for failures. If any failures are found we break and return as we only want to print the most relevant log files to not pollute the logs of the users.

Due to the sorting of the logfiles, we assume the most relevant logfile (workflog.log) will appear first in the list so that will be checked first. If no failures are found in workflow.log, we examine the next file.

slangeveld · 2026-07-17T08:04:33Z

+                if re.match(
+                    r"^\d{8}-\d{6}-[A-Za-z0-9]{6}-RMS", os.path.basename(log_file)
+                ):
+                    # These logfiles are unstructured and will not give any results


Logfiles on the format "YYYYMMDD-HHMMSS-XXXXXX-RMS.log" are skipped as they are on a different format and does not contain job specific logs: As the workflow.log files are easier to parse and gives a more detailed overview of the failure, we have chosen to focus on these workflow.log files in this first iteration.

slangeveld · 2026-07-17T08:12:13Z

+                try:
+                    job_failed_msg = self._find_job_failures(Path(log_file))
+                except Exception:
+                    job_failed_msg = ""


To make sure an exception in the search after job failures does not affect the rest of the error logging, we simply catch all exception that might occur during this search and return an emtpy string instead.

In this way the error message is kept as before if no failures are found or any exceptions occur while searching, with the addition that any discovered failures will be appended at the end of the fail_msg and displayed to the user.

slangeveld · 2026-07-17T08:24:08Z

@perolavsvendsen feel free to add your feedback here as well

tnatt

This looks like a nice first iteration, and it will be very useful and welcomed by users 🥳

We should remove all the rms/model/YYYYMMDD-HHMMSS-XXXXXX-RMS.log from the list of logs in the stderr though. They are making noise in the print and makes the failure message less prominent. I've never found anything useful in there that I could not use the workflow log for. This could be a separate issue.

Maybe in another iteration we will make a more general parser of the workflow log 🙂 For the user it would be very useful to have a structured log of what was run in RMS, the order of the jobs/workflows and the duration of them.

krhart

Very nice!
I agree with Therese, that long list of log files is probably not needed and create some noise.
I also agree with the comment about making the workflow.log more available to the users, as it has a good overview of what is run and where it has failed.

I've tested it both with failing python script and with failing RMS standard job. Same result.

slangeveld · 2026-08-10T09:17:43Z

This looks like a nice first iteration, and it will be very useful and welcomed by users 🥳

We should remove all the rms/model/YYYYMMDD-HHMMSS-XXXXXX-RMS.log from the list of logs in the stderr though. They are making noise in the print and makes the failure message less prominent. I've never found anything useful in there that I could not use the workflow log for. This could be a separate issue.

Maybe in another iteration we will make a more general parser of the workflow log 🙂 For the user it would be very useful to have a structured log of what was run in RMS, the order of the jobs/workflows and the duration of them.

Good point regarding the rms/model/YYYYMMDD-HHMMSS-XXXXXX-RMS.log files. I created #150 for this.

slangeveld changed the title ~~ENH: Print failure logs in logfiles to display in ERT~~ ENH: Print failure logs from logfiles for ERT log Jul 14, 2026

slangeveld force-pushed the 124-improve-logging-from-rms branch 2 times, most recently from 3a42e0f to eba6567 Compare July 17, 2026 06:32

slangeveld commented Jul 17, 2026

View reviewed changes

slangeveld marked this pull request as ready for review July 17, 2026 08:16

ENH: Print failure logs in logfiles to display in ERT

2baf347

slangeveld force-pushed the 124-improve-logging-from-rms branch from eba6567 to 2baf347 Compare July 17, 2026 08:21

slangeveld requested review from krhart and tnatt July 17, 2026 08:22

tnatt approved these changes Aug 4, 2026

View reviewed changes

krhart approved these changes Aug 6, 2026

View reviewed changes

slangeveld merged commit 818219d into equinor:main Aug 10, 2026
7 checks passed

slangeveld deleted the 124-improve-logging-from-rms branch August 10, 2026 09:21

Uh oh!

Conversation

slangeveld commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

slangeveld commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

slangeveld Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slangeveld Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

slangeveld Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

slangeveld Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

slangeveld Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

slangeveld Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slangeveld Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

slangeveld Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slangeveld commented Jul 17, 2026

Uh oh!

tnatt left a comment

Choose a reason for hiding this comment

Uh oh!

krhart left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slangeveld commented Aug 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

slangeveld commented Jul 14, 2026 •

edited

Loading

slangeveld commented Jul 16, 2026 •

edited

Loading

slangeveld Jul 17, 2026 •

edited

Loading

slangeveld Jul 17, 2026 •

edited

Loading

slangeveld Jul 17, 2026 •

edited

Loading

krhart left a comment •

edited

Loading