Skip to content

fix(security): remediate 35 BLOCKER severity SonarQube issues #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
devin-ai-integration wants to merge 1 commit into
base: master
Choose a base branch
from
Open

fix(security): remediate 35 BLOCKER severity SonarQube issues #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 31 additions & 18 deletions .github/workflows/sonarcloud.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,16 +86,21 @@ jobs:

- name: Check if Devin Remediation Commit
id: check-devin
env:
EVENT_NAME: ${{ github.event_name }}
HEAD_REF: ${{ github.head_ref }}
COMMIT_AUTHOR_NAME: ${{ github.event.head_commit.author.name }}
COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
run: |
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
git fetch origin ${{ github.head_ref }}
COMMIT_AUTHOR=$(git log -1 --format='%an' origin/${{ github.head_ref }})
COMMIT_MSG=$(git log -1 --format='%s' origin/${{ github.head_ref }})
if [[ "$EVENT_NAME" == "pull_request" ]]; then
git fetch origin "$HEAD_REF"
COMMIT_AUTHOR=$(git log -1 --format='%an' "origin/$HEAD_REF")
COMMIT_MSG=$(git log -1 --format='%s' "origin/$HEAD_REF")
echo "Checking PR commit - Author: $COMMIT_AUTHOR"
echo "Commit message: $COMMIT_MSG"
else
COMMIT_AUTHOR="${{ github.event.head_commit.author.name }}"
COMMIT_MSG="${{ github.event.head_commit.message }}"
COMMIT_AUTHOR="$COMMIT_AUTHOR_NAME"
COMMIT_MSG="$COMMIT_MESSAGE"
echo "Checking push commit - Author: $COMMIT_AUTHOR"
fi

Expand All @@ -109,11 +114,14 @@ jobs:

- name: Collect Sonar Issue Summary
id: extract-issues
env:
EVENT_NAME: ${{ github.event_name }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
echo "馃搳 Collecting SonarCloud issue summary..."

if [[ "${{ github.event_name }}" == "pull_request" ]]; then
PR_PARAM="&pullRequest=${{ github.event.pull_request.number }}"
if [[ "$EVENT_NAME" == "pull_request" ]]; then
PR_PARAM="&pullRequest=$PR_NUMBER"
else
PR_PARAM=""
fi
Expand Down Expand Up @@ -191,6 +199,11 @@ jobs:
env:
DEVIN_API_KEY: ${{ secrets.DEVIN_API_KEY }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_REF: ${{ github.head_ref }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
run: |
echo "馃 Triggering Devin AI Quality Gate Remediation Specialist..."

Expand All @@ -210,7 +223,7 @@ jobs:
4. Tag commit with [devin-remediation] prefix
5. DO NOT create a new PR - push to current branch only"

USER_PROMPT="# SonarCloud Issue Remediation - PR #${{ github.event.pull_request.number }}
USER_PROMPT="# SonarCloud Issue Remediation - PR #${PR_NUMBER}

## Issue Summary
- **Total Issues:** ${TOTAL_ISSUES}
Expand All @@ -219,8 +232,8 @@ jobs:
- **Code Smells:** ${CODE_SMELLS} 馃挩
- **Security Hotspots:** ${SECURITY_HOTSPOTS} 馃敟
- **Project:** uspto-patent-public-data
- **Branch:** ${{ github.head_ref }}
- **PR:** #${{ github.event.pull_request.number }}
- **Branch:** ${HEAD_REF}
- **PR:** #${PR_NUMBER}

## Detailed Issue Data
\`\`\`json
Expand All @@ -229,22 +242,22 @@ jobs:

## Resources
- 馃搳 [SonarCloud Dashboard](https://sonarcloud.io/dashboard?id=uspto-patent-public-data)
- 馃敡 [GitHub Actions Run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
- 馃敡 [GitHub Actions Run](${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID})

## Instructions
1. Query SonarCloud API for detailed issue information
2. Fix all vulnerabilities, bugs, and code smells
3. Run mvn test to verify fixes
4. Commit with message: \`fix: [devin-remediation] Resolve SonarCloud issues\`
5. Push to branch: \`${{ github.head_ref }}\`"
5. Push to branch: \`${HEAD_REF}\`"

DEVIN_SESSION=$(curl -s -X POST "https://api.devin.ai/v1/sessions" \
-H "Authorization: Bearer $DEVIN_API_KEY" \
-H "Content-Type: application/json" \
-d "$(jq -n \
--arg name "SonarCloud Issue Remediation - PR #${{ github.event.pull_request.number }}" \
--arg repo "${{ github.server_url }}/${{ github.repository }}" \
--arg branch "${{ github.head_ref }}" \
--arg name "SonarCloud Issue Remediation - PR #${PR_NUMBER}" \
--arg repo "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}" \
--arg branch "${HEAD_REF}" \
--arg system_prompt "$SYSTEM_PROMPT" \
--arg user_prompt "$USER_PROMPT" \
'{
Expand Down Expand Up @@ -282,12 +295,12 @@ jobs:
### 馃敆 Links
- [馃憖 Monitor Devin Session]($SESSION_URL)
- [馃搳 SonarCloud Dashboard](https://sonarcloud.io/dashboard?id=uspto-patent-public-data)
- [馃敡 GitHub Actions Run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
- [馃敡 GitHub Actions Run](${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID})

---

**Note:** Devin will push a fix to this branch. The workflow will automatically re-run after the commit.
EOF

gh pr comment ${{ github.event.pull_request.number }} --body-file devin-comment.md
gh pr comment ${PR_NUMBER} --body-file devin-comment.md
fi
66 changes: 41 additions & 25 deletions BulkDownloader/src/main/java/gov/uspto/bulkdata/cli/ExtractPatent.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,43 +182,59 @@ public static void main(String[] args) throws IOException, PatentReaderException

FileFilterChain filters = new FileFilterChain();

DumpReader dumpReader;
if (aps) {
dumpReader = new DumpFileAps(inputFile);
//filter.addRule(new SuffixFileFilter("txt"));
try (DumpFileAps dumpReader = new DumpFileAps(inputFile)) {
dumpReader.setFileFilter(filters);
dumpReader.open();
dumpReader.skip(skip);

if (options.has("id")) {
String docid = (String) options.valueOf("id");
extract.run(dumpReader, docid, outdir, aps);
} else {
extract.run(dumpReader, limit, outdir, aps);
}
}
} else {
PatentDocFormat patentDocFormat = new PatentDocFormatDetect().fromFileName(inputFile);
switch (patentDocFormat) {
case Greenbook:
aps = true;
dumpReader = new DumpFileAps(inputFile);
//filters.addRule(new PathFileFilter(""));
//filters.addRule(new SuffixFilter("txt"));
try (DumpFileAps dumpReader = new DumpFileAps(inputFile)) {
dumpReader.setFileFilter(filters);
dumpReader.open();
dumpReader.skip(skip);

if (options.has("id")) {
String docid = (String) options.valueOf("id");
extract.run(dumpReader, docid, outdir, aps);
} else {
extract.run(dumpReader, limit, outdir, aps);
}
}
break;
default:
DumpFileXml dumpXml = new DumpFileXml(inputFile);
if (PatentDocFormat.Pap.equals(patentDocFormat) || addHtmlEntities) {
dumpXml.addHTMLEntities();
}
dumpReader = dumpXml;
filters.addRule(new SuffixFileFilter("xml"));
try (DumpFileXml dumpReader = new DumpFileXml(inputFile)) {
if (PatentDocFormat.Pap.equals(patentDocFormat) || addHtmlEntities) {
dumpReader.addHTMLEntities();
}
filters.addRule(new SuffixFileFilter("xml"));
dumpReader.setFileFilter(filters);
dumpReader.open();
dumpReader.skip(skip);

if (options.has("id")) {
String docid = (String) options.valueOf("id");
extract.run(dumpReader, docid, outdir, aps);
} else {
extract.run(dumpReader, limit, outdir, aps);
}
}
}
}

dumpReader.setFileFilter(filters);

dumpReader.open();
dumpReader.skip(skip);

if (options.has("id")) {
String docid = (String) options.valueOf("id");
extract.run(dumpReader, docid, outdir, aps);
} else {
extract.run(dumpReader, limit, outdir, aps);
}

System.out.println("--- Finished ---");

}

}
}
97 changes: 60 additions & 37 deletions BulkDownloader/src/main/java/gov/uspto/bulkdata/cli/Look.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -268,34 +268,6 @@ public static void main(String[] args) throws PatentReaderException, IOException

FileFilterChain filters = new FileFilterChain();

DumpReader dumpReader;
if (aps) {
dumpReader = new DumpFileAps(inputFile);
//filter.addRule(new SuffixFileFilter("txt"));
} else {
PatentDocFormat patentDocFormat = new PatentDocFormatDetect().fromFileName(inputFile);
switch (patentDocFormat) {
case Greenbook:
aps = true;
dumpReader = new DumpFileAps(inputFile);
//filters.addRule(new PathFileFilter(""));
//filters.addRule(new SuffixFilter("txt"));
break;
default:
DumpFileXml dumpXml = new DumpFileXml(inputFile);
if (PatentDocFormat.Pap.equals(patentDocFormat) || addHtmlEntities) {
dumpXml.addHTMLEntities();
}
dumpReader = dumpXml;
filters.addRule(new SuffixFileFilter("xml"));
}
}

dumpReader.setFileFilter(filters);

dumpReader.open();
dumpReader.skip(skip);

Writer writer = null;
if (options.has("out")) {
String outStr = (String) options.valueOf("out");
Expand All @@ -306,16 +278,67 @@ public static void main(String[] args) throws PatentReaderException, IOException
writer = new BufferedWriter(new OutputStreamWriter(System.out, Charset.forName("UTF-8")));
}

try {
if (options.has("id")) {
String docid = (String) options.valueOf("id");
look.look(dumpReader, docid, writer, fields);
} else {
look.look(dumpReader, limit, writer, fields);
if (aps) {
try (DumpFileAps dumpReader = new DumpFileAps(inputFile)) {
dumpReader.setFileFilter(filters);
dumpReader.open();
dumpReader.skip(skip);

try {
if (options.has("id")) {
String docid = (String) options.valueOf("id");
look.look(dumpReader, docid, writer, fields);
} else {
look.look(dumpReader, limit, writer, fields);
}
} finally {
writer.close();
}
}
} else {
PatentDocFormat patentDocFormat = new PatentDocFormatDetect().fromFileName(inputFile);
switch (patentDocFormat) {
case Greenbook:
aps = true;
try (DumpFileAps dumpReader = new DumpFileAps(inputFile)) {
dumpReader.setFileFilter(filters);
dumpReader.open();
dumpReader.skip(skip);

try {
if (options.has("id")) {
String docid = (String) options.valueOf("id");
look.look(dumpReader, docid, writer, fields);
} else {
look.look(dumpReader, limit, writer, fields);
}
} finally {
writer.close();
}
}
break;
default:
try (DumpFileXml dumpReader = new DumpFileXml(inputFile)) {
if (PatentDocFormat.Pap.equals(patentDocFormat) || addHtmlEntities) {
dumpReader.addHTMLEntities();
}
filters.addRule(new SuffixFileFilter("xml"));
dumpReader.setFileFilter(filters);
dumpReader.open();
dumpReader.skip(skip);

try {
if (options.has("id")) {
String docid = (String) options.valueOf("id");
look.look(dumpReader, docid, writer, fields);
} else {
look.look(dumpReader, limit, writer, fields);
}
} finally {
writer.close();
}
}
}
} finally {
dumpReader.close();
writer.close();
}

System.out.println("--- Finished ---");
Expand Down
6 changes: 6 additions & 0 deletions PatentDocument/src/main/java/gov/uspto/parser/dom4j/Dom4JParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ public Patent parse(File file) throws PatentReaderException, IOException {
public Patent parse(InputStream inputStream) throws PatentReaderException {
try {
SAXReader sax = new SAXReader(false);
sax.setFeature("http://xml.org/sax/features/external-general-entities", false);
sax.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
sax.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
Document document = sax.read(inputStream);
return parse(document);
Expand All @@ -82,6 +84,8 @@ public Patent parse(InputStream inputStream) throws PatentReaderException {
public Patent parse(Reader reader) throws PatentReaderException {
try {
SAXReader sax = new SAXReader(false);
sax.setFeature("http://xml.org/sax/features/external-general-entities", false);
sax.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
sax.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
return parse(sax.read(reader));
} catch (DocumentException | SAXException e) {
Expand Down Expand Up @@ -111,6 +115,8 @@ public static Document fixTagsJDOM(String badXml) throws IOException, PatentRead
doc = "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n" + doc;
try {
SAXReader sax = new SAXReader(false);
sax.setFeature("http://xml.org/sax/features/external-general-entities", false);
sax.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
sax.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
return sax.read(new StringReader(doc));
} catch (DocumentException | SAXException e) {
Expand Down
8 changes: 4 additions & 4 deletions PatentDocument/src/main/java/gov/uspto/patent/TransformerCli.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ public class TransformerCli {

private final DocumentBuilder<Patent> fileBuilder;
private Path outputDir;
private boolean stdout = false;
private boolean stdoutEnabled = false;
private boolean insertHtmlEntities = false;
private boolean outputBulkFile;
private File inputFile;
Expand All @@ -77,7 +77,7 @@ public TransformerCli(DocumentBuilder<Patent> fileBuilder, Path outputDir, boole

public TransformerCli(DocumentBuilder<Patent> fileBuilder) {
this.fileBuilder = fileBuilder;
this.stdout = true;
this.stdoutEnabled = true;
}

/**
Expand Down Expand Up @@ -187,7 +187,7 @@ private void processDumpFile(DumpReader dumpReader) throws IOException {
String patentId = patent.getDocumentId() != null ? patent.getDocumentId().toText() : "";
MDC.put("DOCID", patentId);

if (!stdout && !outputBulkFile || outputBulkFile && currentWriter == null) {
if (!stdoutEnabled && !outputBulkFile || outputBulkFile && currentWriter == null) {
if (!outputBulkFile) {
currentFileName = patentId + ".json";
if (currentWriter != null) {
Expand Down Expand Up @@ -224,7 +224,7 @@ private void write(Patent patent, Writer writer) throws IOException {
fileBuilder.write(patent, writer);
if (outputBulkFile) {
writer.write("\n");
} else if (stdout) {
} else if (stdoutEnabled) {
//System.out.println("JSON: " + writer.toString());
STDOUT.println("JSON: " + writer.toString());
}
Expand Down
2 changes: 2 additions & 0 deletions PatentDocument/src/main/java/gov/uspto/patent/doc/cpc/masterfile/CpcMasterReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ public MasterClassificationRecord read(Reader reader) throws PatentReaderExcepti

try {
SAXReader sax = new SAXReader(false);
sax.setFeature("http://xml.org/sax/features/external-general-entities", false);
sax.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
sax.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
Document document = sax.read(reader2);
return parse(document);
Expand Down
2 changes: 2 additions & 0 deletions PatentDocument/src/main/java/gov/uspto/patent/doc/cpc/scheme/CpcXmlParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ public ClassificationItem parse(File file) throws PatentReaderException, FileNot
public ClassificationItem parse(Reader reader) throws PatentReaderException {
try {
SAXReader sax = new SAXReader(false);
sax.setFeature("http://xml.org/sax/features/external-general-entities", false);
sax.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
sax.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
Document document = sax.read(reader);
return parse(document);
Expand Down
Loading