Skip to content

Fix arraylist import #454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
a34e593
Jules was unable to complete the task in time. Please review the work…
google-labs-jules[bot] May 22, 2025
7aab209
feat: Add Excel and ExcelMulti data providers and refactor parsers
google-labs-jules[bot] May 22, 2025
0973de1
fix: Resolve compilation errors and JSR 305 warnings
google-labs-jules[bot] May 22, 2025
aa4419c
fix: Resolve all compilation errors related to ReportDto methods
google-labs-jules[bot] May 22, 2025
3c03c0a
fix: Correct findItem calls in unit tests
google-labs-jules[bot] May 22, 2025
72325a3
fix: Address test failures, add diagnostics, and improve robustness
google-labs-jules[bot] May 23, 2025
697a043
Fix: Add missing ArrayList import in Item.java
google-labs-jules[bot] May 23, 2025
7026e9e
Fix: Address various test failures and initial compilation error
google-labs-jules[bot] May 23, 2025
f06a94b
Partial Fix: Address ID prefixing and log issues in CsvCustomParserTest
google-labs-jules[bot] May 23, 2025
9b85311
Attempt to fix compilation errors in ExcelReportParser
google-labs-jules[bot] May 23, 2025
d25b5bf
I've attempted to fix compilation errors and test failures.
google-labs-jules[bot] May 23, 2025
df4a17d
Fix: Resolve 'cannot find symbol: method parseNumericValue' in ExcelR…
google-labs-jules[bot] May 23, 2025
46943db
Final Attempt: Fix CSV structure detection and summarize session
google-labs-jules[bot] May 23, 2025
48a2c81
add_exceland ExcelMultiTab provider
google-labs-jules[bot] May 24, 2025
ddf9dba
Final Analysis of Remaining Test Failures
google-labs-jules[bot] May 24, 2025
013076c
I'm making a final attempt to correct the ID expectation in `CsvCusto…
google-labs-jules[bot] May 24, 2025
db63e71
Final Analysis of Test Failures (End of Session)
google-labs-jules[bot] May 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 39 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Whether you're tracking metrics, analyzing trends, or monitoring performance, th
## Key Features

- **Visualize Nested Data**: Display hierarchical data structures in pie charts, trend charts, and tables.
- **Multiple File Formats**: Supports JSON, YAML, XML, and CSV files.
- **Multiple File Formats**: Supports JSON, YAML, XML, CSV, and Excel (.xls, .xlsx) files.
- **Dynamic UI**: Interactive charts and tables that update based on your data.
- **Customizable Colors**: Define custom colors for your data points or use predefined color schemes.
- **Trend Analysis**: Track data trends over multiple builds with history charts.
Expand Down Expand Up @@ -85,6 +85,34 @@ The plugin supports the following file formats for data input:
#### YAML and XML
- Similar hierarchical structures as JSON are supported.

#### Excel (`excel` provider)
- This provider parses a single Excel sheet from an `.xls` or `.xlsx` file. By default, it processes the **first sheet** in the workbook.
- **Structure Expectation:**
- The parser automatically detects the header row (the first non-empty row).
- Columns *before* the first column containing predominantly numeric data are treated as hierarchy levels.
- Columns *from* the first numeric-looking column onwards are treated as data values, with their respective header names as keys for the results.
- **Example Data (conceptual view of a sheet):**
```
(Sheet1 in an .xlsx or .xls file)
Category, SubCategory, Metric1, Value2
Alpha, X, 10, 100
Alpha, Y, 15, 150
Beta, Z, 20, 200
```
In this example:
- "Category" and "SubCategory" would form the hierarchy (e.g., Alpha -> X).
- "Metric1" and "Value2" would be the data keys with their corresponding numeric values.
- Empty rows before the header or between data rows are typically ignored.

#### Multi-Sheet Excel (`excelmulti` provider)
- This provider parses **all sheets** in an Excel workbook (.xls or .xlsx).
- **Header Consistency Requirement:**
- The header from the *first successfully parsed sheet* (first non-empty sheet with a valid header) is used as a reference.
- Subsequent sheets **must have an identical header** (same column names in the same order) to be included in the report.
- Sheets with headers that do not match the reference header will be skipped, and a warning will be logged.
- **Data Structure per Sheet:** Within each sheet, the data structure expectation is the same as for the `excel` provider (auto-detected header, hierarchy based on pre-numeric columns, values from numeric columns onwards).
- Item IDs are generated to be unique across sheets, typically by internally prefixing them with sheet-specific information.

---

## Color Management
Expand All @@ -95,7 +123,7 @@ The plugin allows you to customize the colors used in the visualizations. You ca

To customize colors, add a `colors` object to your JSON, YAML, or XML file. The `colors` object should map metric keys or category names to specific colors. Colors can be defined using **HEX values** or **predefined color names**.

> **Note**: Color customization is **not supported for CSV files** due to the format does not allow color attribute definition. For now, colors are attributed aleatory.
> **Note**: Color customization is **not supported for CSV or Excel files** as these formats do not have a standard way to define color attributes within the data file itself for this plugin's use. For CSV and Excel, colors are attributed automatically by the charting libraries.

#### Example in JSON:
```json
Expand Down Expand Up @@ -146,9 +174,17 @@ You can interact with the charts and tables to drill down into specific data poi
- `relative`: Show percentage values.
- `dual`: Show both absolute and relative values.
- **`provider`**: Specify the file format and pattern for the data files.
- **`id`**: (Required for CSV) A unique identifier for the report.
- **`id`**: (Optional, but recommended for CSV, Excel, and ExcelMulti if multiple reports of the same type are used) A unique identifier for the report instance. This helps in creating distinct report URLs and managing history, especially if you have multiple CSV or Excel reports in the same job.
- **`pattern`**: An Ant-style pattern to locate the data files.

**Examples for `provider`:**
- JSON: `provider: json(pattern: 'reports/**/*.json')`
- CSV: `provider: csv(id: 'my-csv-report', pattern: 'reports/data.csv')`
- Excel (single sheet): `provider: excel(pattern: 'reports/data.xlsx')`
- Excel (multi-sheet): `provider: excelmulti(pattern: 'reports/multi_sheet_data.xlsx')`
- You can also add an `id` to `excel` and `excelmulti` if needed:
`provider: excel(id: 'my-excel-report', pattern: 'reports/data.xlsx')`


## Examples

Expand Down
19 changes: 19 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,25 @@
<artifactId>jackson2-api</artifactId>
</dependency>

<!-- Apache POI -->
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
<version>5.4.1</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>5.4.1</version>
</dependency>

<!-- JSR 305 for annotations -->
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
<version>3.0.2</version>
</dependency>

<!-- Workflow dependencies -->
<dependency>
<groupId>org.jenkins-ci.plugins.workflow</groupId>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
package io.jenkins.plugins.reporter.model;

import java.io.Serializable;

public class ExcelParserConfig implements Serializable {

private static final long serialVersionUID = 1L;

// Future configuration options can be added here, for example:
// private int headerRowIndex = 0; // Default to the first row
// private int dataStartRowIndex = 1; // Default to the second row
// private String sheetName; // For single sheet parsing, if specified
// private boolean detectHeadersAutomatically = true;

public ExcelParserConfig() {
// Default constructor
}

// Add getters and setters here if fields are added in the future.
private boolean skipNonNumericValues = false; // Default value

public boolean isSkipNonNumericValues() {
return skipNonNumericValues;
}

public void setSkipNonNumericValues(boolean skipNonNumericValues) {
this.skipNonNumericValues = skipNonNumericValues;
}

Check warning on line 28 in src/main/java/io/jenkins/plugins/reporter/model/ExcelParserConfig.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Not covered lines

Lines 23-28 are not covered by tests
}
15 changes: 12 additions & 3 deletions src/main/java/io/jenkins/plugins/reporter/model/Item.java
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import java.io.Serializable;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
Expand Down Expand Up @@ -39,7 +40,7 @@
@Nullable
@JsonProperty(value = "items", required = false)
@JsonInclude(JsonInclude.Include.NON_NULL)
List<Item> items;
List<Item> items = new ArrayList<>();

public String getId() {
return id;
Expand Down Expand Up @@ -72,9 +73,14 @@
return result;
}

return getItems()
// NPE fix: check if items list is null or empty before streaming
if (items == null || items.isEmpty()) { // items is the List<Item> field

Check warning on line 77 in src/main/java/io/jenkins/plugins/reporter/model/Item.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Partially covered line

Line 77 is only partially covered, one branch is missing
return new LinkedHashMap<>(); // Return empty map if no sub-items to aggregate from
}

return items // Now items is guaranteed not to be null and not empty
.stream()
.map(Item::getResult)
.map(Item::getResult) // Recursive call
.flatMap(map -> map.entrySet().stream())
.collect(Collectors.groupingBy(Map.Entry::getKey, LinkedHashMap::new, Collectors.summingInt(Map.Entry::getValue)));
}
Expand Down Expand Up @@ -114,6 +120,9 @@
}

public void addItem(Item item) {
if (this.items == null) {

Check warning on line 123 in src/main/java/io/jenkins/plugins/reporter/model/Item.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Partially covered line

Line 123 is only partially covered, one branch is missing
this.items = new ArrayList<>();

Check warning on line 124 in src/main/java/io/jenkins/plugins/reporter/model/Item.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Not covered line

Line 124 is not covered by tests
}
this.items.add(item);
}
}
12 changes: 12 additions & 0 deletions src/main/java/io/jenkins/plugins/reporter/model/ReportDto.java
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ public class ReportDto extends ReportBase {
@JsonInclude(JsonInclude.Include.NON_NULL)
private Map<String, String> colors;

@JsonProperty(value = "parserLogMessages")
@JsonInclude(JsonInclude.Include.NON_EMPTY) // Only include in JSON if not empty
private List<String> parserLogMessages;

public String getId() {
return id;
}
Expand All @@ -42,6 +46,14 @@ public Map<String, String> getColors() {
public void setColors(Map<String, String> colors) {
this.colors = colors;
}

public List<String> getParserLogMessages() {
return parserLogMessages;
}

public void setParserLogMessages(List<String> parserLogMessages) {
this.parserLogMessages = parserLogMessages;
}

@JsonIgnore
public Report toReport() {
Expand Down
Loading
Loading