Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tools] console log issue from filter log #33075

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 71 additions & 2 deletions eng/tools/spec-gen-sdk-runner/src/commands.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import fs from "node:fs";

Check failure on line 1 in eng/tools/spec-gen-sdk-runner/src/commands.ts

View workflow job for this annotation

GitHub Actions / Protected Files

File 'eng/tools/spec-gen-sdk-runner/src/commands.ts' should only be updated by the Azure SDK team. If intentional, the PR may be merged by the Azure SDK team via bypassing the branch protections.
import path from "node:path";
import { fileURLToPath } from "node:url";
import {
Expand All @@ -8,8 +8,8 @@
getAllTypeSpecPaths,
resetGitRepo,
} from "./utils.js";
import { LogLevel, logMessage, vsoAddAttachment } from "./log.js";
import { SpecGenSdkCmdInput } from "./types.js";
import { LogLevel, logMessage, vsoAddAttachment, vsoLogIssue } from "./log.js";
import { CommandLog, SpecGenSdkCmdInput } from "./types.js";
import { detectChangedSpecConfigFiles } from "./change-files.js";

export async function generateSdkForSingleSpec(): Promise<number> {
Expand Down Expand Up @@ -43,6 +43,9 @@
statusCode = 1;
}
logMessage("ending group logging", LogLevel.EndGroup);

logIssuesToADO(commandInput, specConfigPath || '');

Check failure on line 47 in eng/tools/spec-gen-sdk-runner/src/commands.ts

View workflow job for this annotation

GitHub Actions / spec-gen-sdk-runner / test (ubuntu, 20)

Prefer using nullish coalescing operator (`??`) instead of a logical or (`||`), as it is a safer operator

return statusCode;
}

Expand Down Expand Up @@ -105,6 +108,8 @@
statusCode = 1;
}
logMessage("ending group logging", LogLevel.EndGroup);

logIssuesToADO(commandInput, changedSpecPath || '');

Check failure on line 112 in eng/tools/spec-gen-sdk-runner/src/commands.ts

View workflow job for this annotation

GitHub Actions / spec-gen-sdk-runner / test (ubuntu, 20)

Prefer using nullish coalescing operator (`??`) instead of a logical or (`||`), as it is a safer operator
}
return statusCode;
}
Expand Down Expand Up @@ -180,6 +185,8 @@
statusCode = 1;
}
logMessage("ending group logging", LogLevel.EndGroup);

logIssuesToADO(commandInput, specConfigPath || '');
}
if (failedCount > 0) {
markdownContent += `${failedContent}\n`;
Expand Down Expand Up @@ -325,3 +332,65 @@
}
return specConfigPaths;
}

/**
* Extract and format the prefix from tspConfigPath or readmePath.
* @param {string | undefined} tspConfigPath The tspConfigPath to extract the prefix from.
* @param {string | undefined} readmePath The readmePath to extract the prefix from.
* @returns {string} The formatted prefix.
*/
function extractPathFromSpecConfig(tspConfigPath: string | undefined, readmePath: string | undefined): string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems we still need this logic to find the log file name in case of batch run. Can you add a comment saying "this function is copied from 'spec-gen-sdk'" and add the source code link?

let prefix = '';
if (tspConfigPath) {
const match = tspConfigPath.match(/specification\/(.+)\/tspconfig\.yaml$/);

Check failure on line 345 in eng/tools/spec-gen-sdk-runner/src/commands.ts

View workflow job for this annotation

GitHub Actions / spec-gen-sdk-runner / test (ubuntu, 20)

Use the `RegExp#exec()` method instead
if (match) {
const segments = match[1].split('/');
prefix = segments.join('-').toLowerCase().replace(/\./g, '-');

Check failure on line 348 in eng/tools/spec-gen-sdk-runner/src/commands.ts

View workflow job for this annotation

GitHub Actions / spec-gen-sdk-runner / test (ubuntu, 20)

Prefer `String#replaceAll()` over `String#replace()`
}
} else if (readmePath) {
const match = readmePath.match(/specification\/(.+?)\/readme\.md$/i);

Check failure on line 351 in eng/tools/spec-gen-sdk-runner/src/commands.ts

View workflow job for this annotation

GitHub Actions / spec-gen-sdk-runner / test (ubuntu, 20)

Use the `RegExp#exec()` method instead
if (match) {
const segments = match[1].split('/');
prefix = segments.join('-').toLowerCase().replace(/\./g, '-');

Check failure on line 354 in eng/tools/spec-gen-sdk-runner/src/commands.ts

View workflow job for this annotation

GitHub Actions / spec-gen-sdk-runner / test (ubuntu, 20)

Prefer `String#replaceAll()` over `String#replace()`
}
}
return prefix;
}

/**
* Generates file paths for different log files based on the given command input.
* @param {SpecGenSdkCmdInput} commandInput
* @returns An object containing the full log file path, filtered log file path, and HTML log file path.
*/
function getLogsPath(commandInput: SpecGenSdkCmdInput): { fullLogFileName: string, filteredLogFileName: string, htmlLogFileName: string } {
const fileNamePrefix = extractPathFromSpecConfig(commandInput.tspConfigPath, commandInput.readmePath)
const logFolder = path.join(commandInput.workingFolder, 'out/logs');
const fullLogFileName = path.join(logFolder, `${fileNamePrefix}-full.log`);
const filteredLogFileName = path.join(logFolder, `${fileNamePrefix}-filtered.log`);
const htmlLogFileName = path.join(logFolder, `${fileNamePrefix}-${commandInput.sdkRepoName.substring("azure-sdk-for-".length)}-gen-result.html`);

Check failure on line 370 in eng/tools/spec-gen-sdk-runner/src/commands.ts

View workflow job for this annotation

GitHub Actions / spec-gen-sdk-runner / test (ubuntu, 20)

Prefer `String#slice()` over `String#substring()`
return { fullLogFileName, filteredLogFileName, htmlLogFileName };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove the unused code, such as other logfiles?

}

/**
* Logs SDK generation issues to Azure DevOps (ADO)
* @param commandInput
* @param specConfigPath
*/
function logIssuesToADO(commandInput: SpecGenSdkCmdInput, specConfigPath: string): void {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
function logIssuesToADO(commandInput: SpecGenSdkCmdInput, specConfigPath: string): void {
function logIssuesToPipeline(commandInput: SpecGenSdkCmdInput, specConfigPath: string): void {

const logPath = getLogsPath(commandInput).filteredLogFileName;
let logIssues:CommandLog[] = []
try {
const log = JSON.parse(fs.readFileSync(logPath, "utf8"));
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
logIssues = log.logIssues;
} catch (error) {
logMessage(`Error reading log at ${logPath}:${error}`, LogLevel.Error);
}

if(logIssues.length > 0) {
logMessage(`Error generating SDK from ${specConfigPath}`, LogLevel.Group);
const logIssuesList = logIssues.flatMap(entry => [`Get log issues from script ${entry.command} `, ...entry.logIssues]);;
vsoLogIssue(logIssuesList.join('%0D%0A'));
logMessage("ending group logging", LogLevel.EndGroup);
}
}
5 changes: 5 additions & 0 deletions eng/tools/spec-gen-sdk-runner/src/types.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/**

Check failure on line 1 in eng/tools/spec-gen-sdk-runner/src/types.ts

View workflow job for this annotation

GitHub Actions / Protected Files

File 'eng/tools/spec-gen-sdk-runner/src/types.ts' should only be updated by the Azure SDK team. If intentional, the PR may be merged by the Azure SDK team via bypassing the branch protections.
* Represents the input parameters required for spec-gen-sdk command execution.
*/
export interface SpecGenSdkCmdInput {
Expand All @@ -16,3 +16,8 @@
headRepoHttpsUrl?: string;
headBranch?: string;
}

export type CommandLog = {
command: string;
logIssues: string[];
};
Loading