Add Timestamps to CSV

[kdvalin]

User description

Description

This PR does two things,

1. Convert CSVs to use commas as a delimeter instead of colons
2. Adds start and end timestamps to new CSV columns

Due to the way that the pyperformance suite presently is impelmented, we cannot get the
start/end timestamps for each benchmark individually. The timestamps are of the entire benchmark suite.

Before/After Comparison

Before

No timestamps whatsoever in the CSV. CSV use colons to delineate fields.

After

Suite start/end timestamps are in the last 2 columns in CSV. The CSV uses commas to delineate fields.

Clerical Stuff

Closes #60

Relates to JIRA: RPOPC-814
run.log
pyperf_out_2026.02.04-19.53.23.csv
pmrep.txt

---

PR Type

Enhancement

---

Description

• Convert CSV delimiter from colons to commas

• Add suite start/end timestamps to CSV output

• Capture timestamps before and after benchmark execution

• Pass timestamps to CSV generation function

---

Diagram Walkthrough

flowchart LR
  A["Benchmark Execution"] -->|"capture start_time"| B["Run pyperformance"]
  B -->|"capture end_time"| C["Generate CSV"]
  C -->|"comma-delimited with timestamps"| D["CSV Output"]

Loading

File Walkthrough

Relevant files

Enhancement

pyperf_run CSV delimiter conversion and timestamp integration              pyperf/pyperf_run Modified generate_csv_file() function to accept start_time and end_time parameters Changed CSV field delimiter from colons to commas in printf statements Updated CSV header to use comma-separated format via --field_header flag Added timestamp capture before and after pyperformance execution Pass captured timestamps to generate_csv_file() function call +8/-6

---

[github-actions]

This relates to RPOPC-814

[qodo-code-review]

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢	No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🟡	🎫 #60 🟢 Change the CSV delimiter from `:` to `,` because datetime formats contain colons. Add start and end times to the CSV output so it is clear when a run was performed. ⚪ Validate the exact timestamp format returned by retrieve_time_stamp to ensure it produces CSV-safe values (e.g., no unquoted spaces/commas) and that downstream consumers correctly parse the new timestamp columns.
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
🔴	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Missing timestamp checks: The new start_time/end_time capture and use is not validated (e.g., empty/failed retrieve_time_stamp results are written to CSV without error handling). Referred Code start_time=$(retrieve_time_stamp) $python_exec -m pyperformance run --output ${pyresults}.json $pyperf_flags if [ $? -ne 0 ]; then exit_out "Failed: $python_exec -m pyperformance run --output ${pyresults}.json" 1 fi end_time=$(retrieve_time_stamp) Learn more about managing compliance generic rules or creating your own custom rules
⚪	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Unquoted CSV fields: New CSV-writing code passes unquoted variables (e.g., printf ... $test_name ... $start_time $end_time) which may allow malformed CSV output or unintended interpretation if values contain spaces/commas/newlines. Referred Code printf "%s,%.2f,%s,%s,%s\n" $test_name $results $unit $start_time $end_time >> ${1}.csv Learn more about managing compliance generic rules or creating your own custom rules

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

[qodo-code-review]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Correct CSV header column mismatch Update the CSV header to include StartTime and EndTime columns to match the five columns of data being written to each row. pyperf/pyperf_run [133] -$TOOLS_BIN/test_header_info --front_matter --results_file "${1}.csv" --host $to_configuration --sys_type $to_sys_type --tuned $to_tuned_setting --results_version "py${PYTHON_VERSION}_$PYPERF_VERSION" --test_name $test_name_run --field_header "Test,Avg,Unit" +$TOOLS_BIN/test_header_info --front_matter --results_file "${1}.csv" --host $to_configuration --sys_type $to_sys_type --tuned $to_tuned_setting --results_version "py${PYTHON_VERSION}_$PYPERF_VERSION" --test_name $test_name_run --field_header "Test,Avg,Unit,StartTime,EndTime" Apply / Chat Suggestion importance[1-10]: 9 __ Why: This suggestion correctly identifies a bug where the number of columns in the CSV header does not match the number of columns in the data rows, which would result in a malformed CSV file.	High
General	Quote timestamps in CSV rows Quote the variables in the printf statement to prevent word-splitting and ensure correct CSV formatting, especially for timestamp values. pyperf/pyperf_run [146] -printf "%s,%.2f,%s,%s,%s\n" $test_name $results $unit $start_time $end_time >> ${1}.csv +printf '%s,%.2f,%s,"%s","%s"\n' "$test_name" "$results" "$unit" "$start_time" "$end_time" >> "${1}.csv" Apply / Chat Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies that unquoted variables in the printf command can lead to corrupted CSV output if they contain spaces, and proposes a robust fix by quoting them.	Medium
	Validate timestamp arguments presence Add a check at the beginning of the generate_csv_file function to validate that the start_time and end_time arguments are provided. pyperf/pyperf_run [130-131] +if [ $# -lt 3 ]; then + echo "Usage: generate_csv_file <prefix> <start_time> <end_time>" >&2 + exit 1 +fi start_time=$2 end_time=$3 Apply / Chat Suggestion importance[1-10]: 6 __ Why: This suggestion improves the script's robustness by adding a check to ensure that the required timestamp arguments are passed to the generate_csv_file function, preventing potential errors.	Low
More

[frival]

The code looks good to me, but you need to add PCP output to the PR. I can't see anything that would break it, but it's better safe than sorry.

[kdvalin]

Added pmrep output @frival

[frival]

Wow that is some ugly pmrep output. I can't see anything wrong though, so LGTM.