Skip to content

Implement uncovered branch highlight #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 17, 2025
Merged

Conversation

xpirad
Copy link
Collaborator

@xpirad xpirad commented Feb 12, 2025

In current implementation, a if without else, when the cond always taken, the code coverage report shows 100% code coverage and 50% branch coverage.
But the html doesn't show the line missing branch coverage. Something it makes app developer difficult to improve branch coverage.
Refer to GNU user experience, the missing branch coverage line need to be marked as yellow

@xpirad xpirad changed the title Implement uncovered branch highlight [WIP] Implement uncovered branch highlight Feb 12, 2025
@xpirad
Copy link
Collaborator Author

xpirad commented Feb 12, 2025

UI preview
image

@xpirad xpirad added the enhancement New feature or request label Feb 14, 2025
@xpirad xpirad self-assigned this Feb 14, 2025
@xpirad
Copy link
Collaborator Author

xpirad commented Feb 17, 2025

test case source code

export function ifWithoutElse(x: i32): i32 {
  if (x > 1) {
    return 1;
  }
  return 0;
}

export function whileLoop(x: i32): i32 {
  let i = 0;
  while (i < x) {
    i++;
  }
  return i;
}

export function forLoop(n: i32): i32 {
  let i = 0;
  for (let j = 0; j < n; j++) {
    // does not enter
    i++;
  }
  return i;
}

export function threeOperandOperator(x: i32): i32 {
  return x > 0 ? 1 : 0;
}

@xpirad
Copy link
Collaborator Author

xpirad commented Feb 17, 2025

test code

import { CovInfo, FunctionCoverageResult } from "../../../src/interface.js";
import { SingleFunctionCoverageAnalysis } from "../../../src/parser/singleFunctionAnalysis.js";

describe("singleFunctionAnalysis", () => {
  test("function with two if-else statement", () => {
    const covInfo: CovInfo = {
      branchInfo: [
        [1, 2],
        [1, 6],
        [2, 3],
        [2, 4],
      ],
      lineInfo: new Map([
        [1, new Set([6, 7])],
        [2, new Set([8, 9])],
        [3, new Set([10])],
        [4, new Set([12])],
        [5, new Set([])],
        [6, new Set([15, 16])],
      ]),
    };
    const traceInfo = [1, 2, 3, 5, 6];
    const analyzer = new SingleFunctionCoverageAnalysis(covInfo, "main");
    const result = analyzer.update(traceInfo);
    expect(result.lineRange).toEqual([6, 16]);
    expect(result.branchCoverageRate.toString()).toEqual("2/4");
    expect(result.branchCoverageRate.getRate()).toBeCloseTo(50);
    expect(result.sourceUsedCount).toEqual(
      new Map([
        [6, 1],
        [7, 1],
        [8, 1],
        [9, 1],
        [10, 1],
        [12, 0],
        [15, 1],
        [16, 1],
      ])
    );
  });

  test("function with only one basic block", () => {
    const covInfo: CovInfo = {
      branchInfo: [],
      lineInfo: new Map([[1, new Set([3, 4, 5])]]),
    };
    const traceInfo = [1, 1, 1];
    const analyzer = new SingleFunctionCoverageAnalysis(covInfo, "main");
    const result = analyzer.update(traceInfo);
    expect(result.lineRange).toEqual([3, 5]);
    expect(result.branchCoverageRate.toString()).toEqual("0/0");
    expect(result.branchCoverageRate.getRate()).toBeCloseTo(100);
    expect(result.sourceUsedCount).toEqual(
      new Map([
        [3, 3],
        [4, 3],
        [5, 3],
      ])
    );
  });

  test("forLoop", () => {
    const covInfo: CovInfo = {
      branchInfo: [
        [1, 2],
        [1, 3],
      ],
      lineInfo: new Map([
        [0, new Set([17, 18])],
        [1, new Set([18])],
        [2, new Set([18, 20])],
        [3, new Set([])],
        [4, new Set([22])],
      ]),
    };
    const traceInfo = [0, 1, 3, 4];
    const analyzer = new SingleFunctionCoverageAnalysis(covInfo, "main");
    const result = analyzer.update(traceInfo);
    expect(result.uncoveredlines).toEqual(new Set([18]));
  });

  test("ifWithoutElse", () => {
    const covInfo: CovInfo = {
      branchInfo: [
        [0, 1],
        [0, 2],
      ],
      lineInfo: new Map([
        [0, new Set([2])],
        [1, new Set([3])],
        [2, new Set([5])],
      ]),
    };
    const traceInfo = [0, 1];
    const analyzer = new SingleFunctionCoverageAnalysis(covInfo, "main");
    const result = analyzer.update(traceInfo);
    expect(result.uncoveredlines).toEqual(new Set([2]));
  });

  test("threeOperandOperator", () => {
    const covInfo: CovInfo = {
      branchInfo: [
        [0, 1],
        [0, 2],
      ],
      lineInfo: new Map([
        [0, new Set([26])],
        [1, new Set([26])],
        [2, new Set([26])],
        [3, new Set([26])],
      ]),
    };
    const traceInfo = [3, 0, 1];
    const analyzer = new SingleFunctionCoverageAnalysis(covInfo, "main");
    const result = analyzer.update(traceInfo);
    expect(result.uncoveredlines).toEqual(new Set([26]));
  });

  test("whileLoop", () => {
    const covInfo: CovInfo = {
      branchInfo: [
        [1, 2],
        [1, 3],
      ],
      lineInfo: new Map([
        [0, new Set([9])],
        [1, new Set([10])],
        [2, new Set([11])],
        [3, new Set([])],
        [4, new Set([10, 13])],
      ]),
    };
    const traceInfo = [0, 1, 3, 4];
    const analyzer = new SingleFunctionCoverageAnalysis(covInfo, "main");
    const result = analyzer.update(traceInfo);
    expect(result.uncoveredlines).toEqual(new Set([10]));
  });
});

test("mergeFromGeneric()", () => {
  const genericType_i64: CovInfo = {
    branchInfo: [],
    lineInfo: new Map([[1, new Set([2, 5, 8, 9])]]),
  };
  const trace_i64 = [1, 1, 1, 1];
  const analyzer_i64 = new SingleFunctionCoverageAnalysis(genericType_i64, "A<i64>");
  const result_i64 = analyzer_i64.update(trace_i64);

  const genericType_array: CovInfo = {
    branchInfo: [
      [2, 3],
      [2, 4],
    ],
    lineInfo: new Map([
      [1, new Set([68])],
      [2, new Set([68])],
      [3, new Set([71, 72])],
      [4, new Set([68])],
      [5, new Set([71, 74, 75, 76])],
    ]),
  };
  const trace_array = [1, 2, 4, 1, 4, 5];
  const analyzer_array = new SingleFunctionCoverageAnalysis(genericType_array, "A<~lib/array/Array<i32>|null>");
  const result_array = analyzer_array.update(trace_array);

  const result = FunctionCoverageResult.mergeFromGeneric("A", [result_i64, result_array]);
  expect(result.lineRange).toEqual([2, 76]);
  expect(result.branchCoverageRate.toString()).toEqual("1/2");
  expect(result.branchCoverageRate.getRate()).toBeCloseTo(50);
  expect(result.sourceUsedCount).toEqual(
    new Map([
      [2, 4],
      [5, 4],
      [8, 4],
      [9, 4],
      [68, 5],
      [71, 1],
      [72, 0],
      [74, 1],
      [75, 1],
      [76, 1],
    ])
  );
});

@xpirad
Copy link
Collaborator Author

xpirad commented Feb 17, 2025

Print basic blocks and trace && uncoveredlines

// in start_unit_test
for (const [funcName, covInfoObj] of parser.functionCovInfoMap.entries()) {
    if (funcName.startsWith("assembly/coverUncoveredLines/")) {
      const branchInfoString = JSON.stringify(covInfoObj.branchInfo);

      const lineInfoEntries: string[] = [];
      for (const [blockIndex, lineSet] of covInfoObj.lineInfo.entries()) {
        const linesArray = [...lineSet].join(", ");
        lineInfoEntries.push(`[${blockIndex}, new Set([${linesArray}])]`);
      }
      const lineInfoString = `new Map([${lineInfoEntries.join(", ")}])`;
      console.log(`Function Name: ${funcName}`);
      console.log(`const covInfo: CovInfo = {
        branchInfo: ${branchInfoString},
        lineInfo: ${lineInfoString},
      };`);
      console.log();
    }
  }
  console.log("\n");

  for (const [funcName, traceData] of parser.functionCovTraceMap.entries()) {
    if (!funcName.startsWith("assembly/coverUncoveredLines/")) {
      continue;
    }

    console.log(`Function Name: ${funcName}`);
    console.log(`const traceInfo = [${traceData.join(", ")}];`);

    const coverageResult = parser.functionCoverageResults.find((res) => res.functionName === funcName);
    if (coverageResult) {
      const uncovered = [...coverageResult.uncoveredlines].join(", ");
      console.log(`expect(result.uncoveredlines).toEqual([${uncovered}]);`);
    } else {
      console.log(`expect(result.uncoveredlines).toEqual([]);`);
    }
    console.log();
  }

@xpirad xpirad changed the title [WIP] Implement uncovered branch highlight Implement uncovered branch highlight Feb 17, 2025
@XMadrid XMadrid merged commit 541736b into wasm-ecosystem:main Feb 17, 2025
3 checks passed
@HerrCai0907 HerrCai0907 linked an issue Feb 20, 2025 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

mark not fully taken branch as yellow
3 participants