Skip to content

Commit b6e9f38

Browse files
SindiBuklajisindiibbPredixx
authored
Development: Add AI enabled commit classifier (#44)
* feat: add commit classifier service for AI integration * chore: update AI configuration * fix: exclude AI services from openapi profile * style: use explicit imports instead of wildcard * feat: implement anomaly detection service and related DTOs * feat: refactor AI DTOs and services for anomaly detection and commit classification * feat: refactor anomaly and commit classification DTOs, update service logic for improved clarity * feat: add prompt templates for anomaly detection and commit classification * fix: improve confidence parsing logic in AnomalyDetectorService * fix: clean up formatting in AnomalyDetectorService and CommitClassifierService, update AiResource documentation * fix: remove baseUrl configuration from tsconfig.app.json --------- Co-authored-by: Sindi Buklaji <ge43yif@mytum.de> Co-authored-by: Predixx <liam.patrice.berger@gmail.com> Co-authored-by: Liam Berger <63974342+Predixx@users.noreply.github.com>
1 parent fd6bfa0 commit b6e9f38

File tree

13 files changed

+768
-8
lines changed

13 files changed

+768
-8
lines changed

docs/ai-anomaly-detection.md

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Anomaly Detection
2+
3+
## Purpose
4+
Detects suspicious collaboration patterns in team commit history. **Does NOT affect CQI score** - only displays warning flags for instructor review.
5+
6+
## Anomaly Types
7+
8+
| Flag | Threshold | Meaning |
9+
|------|-----------|---------|
10+
| **LATE_DUMP** | >50% commits in last 20% of time | Cramming before deadline |
11+
| **SOLO_DEVELOPMENT** | One person >70% of commits | Unbalanced workload |
12+
| **INACTIVE_PERIOD** | Gap >50% of assignment period | Long period with no activity |
13+
| **UNEVEN_DISTRIBUTION** | LLM-detected | Commits in short bursts |
14+
15+
## Detection Method
16+
**AI-first with validation:**
17+
1. **LLM** (GPT-4o-mini) - analyzes patterns, detects anomalies
18+
2. **Rule-based validation** - verifies findings with exact math, corrects percentages
19+
3. **Best of both** - LLM catches complex patterns, rules ensure accuracy
20+
21+
## Testing
22+
23+
**Endpoint:** `GET /api/ai/detect-anomalies?scenario={type}`
24+
25+
### Test Scenarios
26+
27+
**1. Late Dump + Solo Development (default)**
28+
```
29+
http://localhost:8080/api/ai/detect-anomalies
30+
```
31+
Expected: `[LATE_DUMP, SOLO_DEVELOPMENT]`
32+
- Alice: 7/8 commits (87.5%)
33+
- 650/800 lines in last 6 days (last 20% of 30-day period)
34+
35+
**2. Solo Development Only**
36+
```
37+
http://localhost:8080/api/ai/detect-anomalies?scenario=solo
38+
```
39+
Expected: `[SOLO_DEVELOPMENT]`
40+
- Alice: 9/10 commits (90%)
41+
- Work spread throughout period
42+
43+
**3. Inactive Period**
44+
```
45+
http://localhost:8080/api/ai/detect-anomalies?scenario=inactive
46+
```
47+
Expected: `[INACTIVE_PERIOD]`
48+
- 16-day gap between Day 6 and Day 22 (53% of 30-day period)
49+
- Balanced contributions otherwise
50+
51+
**4. Good Collaboration**
52+
```
53+
http://localhost:8080/api/ai/detect-anomalies?scenario=good
54+
```
55+
Expected: `[]` (no anomalies)
56+
- Balanced: Alice 50%, Bob 50%
57+
- Spread evenly over time
58+
59+
## Configuration
60+
61+
```yaml
62+
harmonia:
63+
ai:
64+
enabled: true
65+
anomaly-detector:
66+
enabled: true
67+
confidence-threshold: 0.7
68+
```
69+
70+
## Example Output
71+
72+
```json
73+
{
74+
"flags": ["LATE_DUMP", "SOLO_DEVELOPMENT"],
75+
"confidence": 1.0,
76+
"reasons": [
77+
"Alice has 83.3% of commits (5/6)",
78+
"66.7% of commits (4/6) in last 2 days"
79+
]
80+
}
81+
```
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
package de.tum.cit.aet.ai.dto;
2+
3+
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
4+
import java.time.LocalDateTime;
5+
import java.util.List;
6+
7+
/**
8+
* Request for anomaly detection analysis.
9+
*
10+
* @param teamId team identifier
11+
* @param commits list of commit summaries
12+
* @param assignmentStart assignment start date
13+
* @param assignmentEnd assignment end date (deadline)
14+
*/
15+
@JsonIgnoreProperties(ignoreUnknown = true)
16+
public record AnomalyDetectionRequestDTO(
17+
String teamId,
18+
List<CommitSummary> commits,
19+
LocalDateTime assignmentStart,
20+
LocalDateTime assignmentEnd
21+
) {
22+
@JsonIgnoreProperties(ignoreUnknown = true)
23+
public record CommitSummary(String author, LocalDateTime timestamp, int linesChanged) {}
24+
}
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
package de.tum.cit.aet.ai.dto;
2+
3+
/**
4+
* Anomaly flags for collaboration pattern detection.
5+
*/
6+
public enum AnomalyFlag {
7+
/** More than 50% of commits in the last 20% of the assignment period. */
8+
LATE_DUMP,
9+
/** One person has more than 70% of all commits. */
10+
SOLO_DEVELOPMENT,
11+
/** Gap of more than 50% of the assignment period with no commits. */
12+
INACTIVE_PERIOD,
13+
/** Commits clustered in short bursts rather than spread out. */
14+
UNEVEN_DISTRIBUTION
15+
}
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
package de.tum.cit.aet.ai.dto;
2+
3+
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
4+
import java.util.List;
5+
6+
/**
7+
* Anomaly detection report.
8+
*
9+
* @param flags detected anomaly flags
10+
* @param confidence overall confidence (0.0-1.0)
11+
* @param reasons explanations for each detected anomaly
12+
*/
13+
@JsonIgnoreProperties(ignoreUnknown = true)
14+
public record AnomalyReportDTO(
15+
List<AnomalyFlag> flags,
16+
double confidence,
17+
List<String> reasons
18+
) {
19+
}
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
package de.tum.cit.aet.ai.dto;
2+
3+
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
4+
5+
/**
6+
* Commit classification result.
7+
*
8+
* @param label the classification label
9+
* @param confidence confidence score (0.0-1.0)
10+
* @param reasoning explanation for the classification
11+
*/
12+
@JsonIgnoreProperties(ignoreUnknown = true)
13+
public record CommitClassificationDTO(
14+
CommitLabel label,
15+
double confidence,
16+
String reasoning
17+
) {
18+
}
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
package de.tum.cit.aet.ai.dto;
2+
3+
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
4+
import java.util.List;
5+
6+
@JsonIgnoreProperties(ignoreUnknown = true)
7+
public record CommitClassificationRequestDTO(
8+
String sha,
9+
String message,
10+
List<String> filePaths,
11+
String diffContent
12+
) {
13+
public CommitClassificationRequestDTO(String sha, String message, List<String> filePaths) {
14+
this(sha, message, filePaths, null);
15+
}
16+
}
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
package de.tum.cit.aet.ai.dto;
2+
3+
/**
4+
* Labels for commit classification.
5+
*/
6+
public enum CommitLabel {
7+
/** New functionality or significant enhancement. */
8+
FEATURE,
9+
/** Fixes a bug or error. */
10+
BUG_FIX,
11+
/** Adds or modifies tests only. */
12+
TEST,
13+
/** Code restructuring without behavior change. */
14+
REFACTOR,
15+
/** Formatting, comments, docs, whitespace, typos. */
16+
TRIVIAL
17+
}

0 commit comments

Comments
 (0)