Skip to content

output cycle timing to file#1613

Open
BenWibking wants to merge 2 commits intodevelopmentfrom
BenWibking/print-cycle-timing-file
Open

output cycle timing to file#1613
BenWibking wants to merge 2 commits intodevelopmentfrom
BenWibking/print-cycle-timing-file

Conversation

@BenWibking
Copy link
Collaborator

@BenWibking BenWibking commented Jan 26, 2026

Description

This outputs the per-cycle timing to a text file. This is essential for debugging performance issues at scale.

Related issues

N/A

Checklist

Before this pull request can be reviewed, all of these tasks should be completed. Denote completed tasks with an x inside the square brackets [ ] in the Markdown source below:

  • I have added a description (see above).
  • I have added a link to any related issues (if applicable; see above).
  • I have read the Contributing Guide.
  • I have added tests for any new physics that this PR adds to the code.
  • (For quokka-astro org members) I have manually triggered the GPU tests with the magic comment /azp run.

@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jan 26, 2026
@BenWibking BenWibking enabled auto-merge January 26, 2026 18:14
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @BenWibking, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the simulation's diagnostic capabilities by adding functionality to record the wall-clock time taken for each computational cycle. This data is written to a user-configurable text file in a buffered manner, providing a valuable tool for identifying and analyzing performance bottlenecks, particularly in high-performance computing environments.

Highlights

  • New Configuration Options: Introduced cycleTimingFlushInterval_ and cycleTimingFile_ member variables in AMRSimulation to control the buffered write interval and the output filename for cycle timing data, respectively. These can be configured via input parameters.
  • Buffered File Output for Cycle Timing: Implemented a buffered file writing mechanism for per-cycle timing. If printCycleTiming_ is enabled, the simulation now writes cycle number and elapsed wall-clock time to a specified file (cycle_timing.txt by default), flushing the buffer at a configurable interval and at the end of the simulation.
  • Performance Debugging Aid: The primary purpose of these changes is to provide a persistent record of per-cycle timing, which is crucial for debugging performance issues in large-scale simulations.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a valuable feature for performance analysis by outputting per-cycle timing data to a file. The implementation is straightforward, using a buffered approach to manage I/O. My review includes two main suggestions to enhance the robustness of this new functionality. First, to handle simulation restarts correctly, the timing file should be opened in append mode to avoid losing historical data. Second, I've recommended adding error handling to the file-writing process to gracefully manage potential I/O issues. These improvements will ensure the feature is reliable for large-scale and long-running simulations.

Comment on lines +1327 to +1335
if (printCycleTiming_ == 1 && amrex::ParallelDescriptor::IOProcessor()) {
cycle_timing_file.open(cycleTimingFile_, std::ios::out);
if (cycle_timing_file.is_open()) {
cycle_timing_file << "# cycle elapsed_sec\n";
cycle_timing_buffer.reserve(static_cast<size_t>(cycle_timing_flush_interval));
} else {
amrex::Print() << "[WARNING] Failed to open cycle timing file '" << cycleTimingFile_ << "' for writing.\n";
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current implementation opens the cycle timing file in truncation mode (std::ios::out), which overwrites the file's contents upon simulation restart. This results in the loss of timing history from the previous run, which is critical for performance analysis across restarts. To preserve the history, the file should be opened in append mode when restarting a simulation.

    if (printCycleTiming_ == 1 && amrex::ParallelDescriptor::IOProcessor()) {
		const bool is_restarting = !restart_chkfile.empty();
		auto open_mode = is_restarting ? std::ios::app : std::ios::out;
		cycle_timing_file.open(cycleTimingFile_, open_mode);
		if (cycle_timing_file.is_open()) {
			if (!is_restarting) {
				cycle_timing_file << "# cycle elapsed_sec\n";
			}
			cycle_timing_buffer.reserve(static_cast<size_t>(cycle_timing_flush_interval));
		} else {
			amrex::Print() << "[WARNING] Failed to open cycle timing file '" << cycleTimingFile_ << "' for writing.\n";
		}
	}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point. I will change it to open in append mode with a restart message, like the history.txt output.

@BenWibking BenWibking disabled auto-merge January 26, 2026 18:20
@sonarqubecloud
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant