Skip to content

Communication: Add Iris Tutor Suggestions #10666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 80 commits into from
May 19, 2025

Conversation

alexjoham
Copy link
Member

@alexjoham alexjoham commented Apr 12, 2025

Checklist

General

Server

  • Important: I implemented the changes with a very good performance and prevented too many (unnecessary) and too complex database calls.
  • I strictly followed the principle of data economy for all database calls.
  • I strictly followed the server coding and design guidelines.
  • I added multiple integration tests (Spring) related to the features (with a high test coverage).
  • I added pre-authorization annotations according to the guidelines and checked the course groups for all new REST Calls (security).
  • I documented the Java code using JavaDoc style.

Client

  • Important: I implemented the changes with a very good performance, prevented too many (unnecessary) REST calls and made sure the UI is responsive, even with large data (e.g. using paging).
  • I strictly followed the principle of data economy for all client-server REST calls.
  • I strictly followed the client coding and design guidelines.
  • Following the theming guidelines, I specified colors only in the theming variable files and checked that the changes look consistent in both the light and the dark theme.
  • I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.
  • I added authorities to all new routes and checked the course groups for displaying navigation elements (links, buttons).
  • I documented the TypeScript code using JSDoc style.
  • I added multiple screenshots/screencasts of my UI changes.
  • I translated all newly inserted strings into English and German.

Motivation and Context

The communication section gets a big upgrade: The tutor suggestion. This adds a new tool for all tutors to use when answering student questions. When opening a thread, Iris creates a suggestion for tutors to use when answering questions from students. This means the communication section needs to add this feature. For that, a first version is needed that already includes: Settings to toggle this feature, Authentification to only allow tutors and higher to see the suggestions, and a database migration that saves sessions and history. This basic version lays the ground for further improvements to be added in follow-up PRs.

Description

!! Important: This is work in progress! With this PR a first version of the tutor suggestion feature is introduced. More changes are added with upcoming PRs !!

This PR introduces the Iris Tutor Suggestion functionality, enabling the generation of Iris-assisted tutor suggestions in response to student posts within conversation threads.
Domain & Session Layer:

  • Introduced IrisTutorSuggestionSession, a new session type that links a tutor’s Iris chat session to a specific Post.
  • Updated IrisMessageSender to include ARTIFACT as a new message source.
  • Extended IrisSessionService to handle the new session type and associated access control.

Settings Infrastructure:

  • Added IrisTutorSuggestionSubSettings with support across global, course, and exercise-level configurations.
  • Incorporated tutor suggestion settings into IrisCombinedSettingsDTO.
  • Updated IrisSettingsService and IrisSubSettingsService to manage these new settings and enable cascading logic.

Pipeline Execution

  • Implemented TutorSuggestionJob and PyrisTutorSuggestionPipelineExecutionDTO to define the tutor suggestion pipeline in Pyris.
  • Integrated execution logic into PyrisPipelineService and status handling in PyrisStatusUpdateService.

REST API:
Added IrisTutorSuggestionSessionResource with endpoints to:

  • Create or fetch the current tutor suggestion session for a post.
  • Extended IrisMessageResource to allow ARTIFACT messages to be saved and processed appropriately.

Frontend:
New Angular component TutorSuggestionComponent:

  • Dynamically loads AI suggestions for a post.
  • Displays formatted LLM messages and pipeline execution status.
  • Hooked into the existing ConversationThreadSidebar to render suggestions in line.
  • Extended settings management UI to include tutor suggestion settings.
  • Fully tested with component and service unit tests.

Persistence:
Liquibase changes to:

  • Add post_id to iris_session.
  • Support foreign key constraints for the new subsettings.

Steps for Testing

Prerequisites:

Activate the feature:

  1. Log in to Artemis as an Instructor
  2. Navigate to Course Administration
  3. Activate the Tutor suggestions in the Artemis Iris settings globally and also for the course

Test for students:

  1. Login as a student and ask any question in a course channel
  2. Check if anything related to Tutor suggestions is displayed if you open a thread. Nothing should happen

Test for tutors:

  1. Login as a Tutor
  2. Navigate to the Communication section and open the previously asked question in a thread
  3. Check if the tutor suggestion div is displayed with its stages and a summary is generated

Check settings:

  1. log in as a instructor and disable the feature
  2. Check if a tutor suggestion is still generated in the thread of a post

To roll back the migration, execute this SQL statement:

-- Drop foreign key constraints
ALTER TABLE iris_session DROP CONSTRAINT FK_IRIS_SESSION_ON_POST;
ALTER TABLE iris_settings DROP CONSTRAINT FK_IRIS_SETTINGS_ON_IRIS_TUTOR_SUGGESTION_SETTINGS;

-- Drop added columns
ALTER TABLE iris_session DROP COLUMN post_id;
ALTER TABLE iris_settings DROP COLUMN iris_tutor_suggestion_settings_id;

-- Delete the migration from the databasechangelog
DELETE FROM databasechangelog WHERE id = '20250326145226'

Testserver States

You can manage test servers using Helios. Check environment statuses in the environment list. To deploy to a test server, go to the CI/CD page, find your PR or branch, and trigger the deployment.

Review Progress

Performance Review

  • I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance even for very large courses with more than 2000 students.
  • I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance even for very large courses with more than 2000 students.

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

Test Coverage

Client

Class/File Line Coverage Confirmation (assert/expect)
conversation-thread-sidebar.component.ts 88.88% ✅ ❌
tutor-suggestion.component.ts 96.15% ✅ ❌
iris-settings-update.component.ts 90.9% ✅ ❌
iris-chat-http.service.ts 90% ✅ ❌
iris-chat.service.ts 88.41% ✅ ❌
iris-status.service.ts 91.66% ✅ ❌
iris-message.model.ts 100%
iris-settings.model.ts 90.9% ✅ ❌
iris-sub-settings.model.ts 100%

Server

Class/File Line Coverage Confirmation (assert/expect)
IrisMessageSender.java 100%
IrisSession.java 100%
IrisTutorSuggestionSession.java 100%
IrisCourseSettings.java 100%
IrisExerciseSettings.java 85% ✅ ❌
IrisGlobalSettings.java 100%
IrisSettings.java 100%
IrisSubSettings.java 100%
IrisSubSettingsType.java 100%
IrisTutorSuggestionSubSettings.java 100%
IrisCombinedSettingsDTO.java 100%
IrisCombinedTutorSuggestionSubSettingsDTO.java 100%
IrisTutorSuggestionSessionRepository.java 89% ✅ ❌
IrisSessionService.java 81% ✅ ❌
PyrisJobService.java 93% ✅ ❌
PyrisPipelineService.java 82% ✅ ❌
PyrisStatusUpdateService.java 61% ✅ ❌
PyrisTutorSuggestionPipelineExecutionDTO.java 100%
PyrisAnswerPostDTO.java 96% ✅ ❌
PyrisPostDTO.java 100%
TutorSuggestionJob.java 46% ✅ ❌
IrisTutorSuggestionSessionService.java 65% ✅ ❌
IrisSettingsService.java 87% ✅ ❌
IrisSubSettingsService.java 89% ✅ ❌
IrisMessageResource.java 95% ✅ ❌
IrisTutorSuggestionSessionResource.java 93% ✅ ❌
PublicPyrisStatusUpdateResource.java 46% ✅ ❌

Screenshots

Running pipeline

Screenshot 2025-04-13 at 08 14 17

Finished pipeline with result

Screenshot 2025-04-13 at 08 14 29

Failed pipeline

Screenshot 2025-04-13 at 08 15 17

Summary by CodeRabbit

  • New Features

    • Introduced tutor suggestion functionality with dedicated session management, messaging, and pipeline execution.
    • Added UI components for tutor suggestions visible to users with tutor roles in conversation threads.
    • Enabled configuration of tutor suggestion settings in global, course, and exercise settings.
    • Enhanced chat service and backend to support tutor suggestion message types and workflows.
    • Added REST endpoints for creating and retrieving tutor suggestion sessions.
    • Implemented rate limiting and access control specific to tutor suggestion sessions.
    • Added pipeline execution and status update handling for tutor suggestion jobs.
    • Integrated tutor suggestion settings into settings management and UI updates.
    • Added new DTOs and data models to support tutor suggestion features.
  • Bug Fixes

    • Improved message creation logic to differentiate between tutor suggestion system messages and user messages, enhancing access control.
  • Tests

    • Added extensive unit and integration tests covering tutor suggestion sessions, message handling, pipeline execution, and access permissions.

@github-project-automation github-project-automation bot moved this to Work In Progress in Artemis Development Apr 12, 2025
@github-actions github-actions bot added tests server Pull requests that update Java code. (Added Automatically!) client Pull requests that update TypeScript code. (Added Automatically!) database Pull requests that update the database. (Added Automatically!). Require a CRITICAL deployment. labels Apr 12, 2025
Copy link

End-to-End (E2E) Test Results Summary

TestsPassed ✅Skipped ⚠️FailedTime ⏱
End-to-End (E2E) Test Report201 ran198 passed3 skipped0 failed49m 55s 717ms
TestResultTime ⏱
No test annotations available

coderabbitai[bot]
coderabbitai bot previously approved these changes May 17, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (4)
src/test/java/de/tum/cit/aet/artemis/iris/IrisTutorSuggestionIntegrationTest.java (4)

51-80: Use service-based testing approach rather than direct repository access.

According to your coding guidelines (avoid_db_access: true), tests should use service methods instead of direct repository access.

The test injects multiple repositories directly, which violates the guidelines. Consider using service methods for repository operations.


90-102: Include database query counting in test setup for performance tracking.

Your coding guidelines specify db_query_count_tests: track_performance, but the test setup doesn't include query count tracking.

Add query count tracking setup in your @BeforeEach method as mentioned in past review comments.


104-112: Use descriptive test method name and test service methods instead of repositories.

The test method name lacks descriptiveness and uses direct repository access.

Follow the descriptive test naming pattern (testX_shouldY) used in other methods and use service methods instead of repository access.


133-146: 🛠️ Refactor suggestion

Inconsistent usage of service vs repository methods for message handling.

This test uses a service method for saving but direct repository access for retrieval.

Make the approach consistent by using service methods for both operations:

var irisSession = request.postWithResponseBody(tutorSuggestionUrl(post.getId()), null, IrisSession.class, HttpStatus.CREATED);
var message = new IrisMessage();
message.addContent(new IrisTextMessageContent("Test tutor suggestion request"));
message.setSender(IrisMessageSender.LLM);
message.setSession(irisSession);
irisMessageService.saveMessage(message, irisSession, IrisMessageSender.LLM);
-var messages = irisMessageRepository.findAllBySessionId(irisSession.getId());
+var messages = irisMessageService.findAllBySessionId(irisSession.getId());
assertThat(messages).hasSize(1);
🧹 Nitpick comments (3)
src/test/java/de/tum/cit/aet/artemis/iris/IrisTutorSuggestionIntegrationTest.java (3)

1-47: Add class-level documentation to describe the purpose of the integration test.

This integration test class lacks JavaDoc documentation explaining its purpose and relationship to the Iris Tutor Suggestion feature.

Add a JavaDoc comment at the class level:

@ActiveProfiles(PROFILE_IRIS)
+/**
+ * Integration tests for the Iris Tutor Suggestion feature.
+ * Tests the creation of tutor suggestion sessions, messaging, pipeline execution,
+ * and access control for different user roles.
+ */
class IrisTutorSuggestionIntegrationTest extends AbstractIrisIntegrationTest {

230-271: Refactor complex test method for better readability.

This test method is quite long and complex, containing token generation, request authentication, and status verification.

Extract helper methods to improve readability:

@Test
@WithMockUser(username = TEST_PREFIX + "tutor1", roles = "TA")
void testTutorSuggestionStatusUpdateShouldBeHandled() throws Exception {
    var post = createPostInExerciseChat(exercise, TEST_PREFIX);
    var conversation = post.getConversation();
    conversation.setCourse(course);
    post = postRepository.save(post);

    var irisSession = request.postWithResponseBody(tutorSuggestionUrl(post.getId()), null, IrisTutorSuggestionSession.class, HttpStatus.CREATED);
    irisSession.setPostId(post.getId());

-   String token = pyrisJobService.addTutorSuggestionJob(post.getId(), course.getId(), irisSession.getId());
-
-   HttpHeaders headers = new HttpHeaders();
-   headers.add("Authorization", "Bearer " + token);
-   // Manually authenticate the job to verify token registration
-   var mockRequest = new org.springframework.mock.web.MockHttpServletRequest();
-   mockRequest.addHeader("Authorization", "Bearer " + token);
-   var job = pyrisJobService.getAndAuthenticateJobFromHeaderElseThrow(mockRequest, de.tum.cit.aet.artemis.iris.service.pyris.job.TutorSuggestionJob.class);
-   assertThat(job).isNotNull();
-   assertThat(job.jobId()).isEqualTo(token);
+   String token = createAndVerifyTutorSuggestionJob(post.getId(), course.getId(), irisSession.getId());
    
    List<PyrisStageDTO> stages = List.of(new PyrisStageDTO("Test stage", 0, PyrisStageState.DONE, "Done"));
    var statusUpdate = new TutorSuggestionStatusUpdateDTO("Test suggestion", "Test result", stages, null);
    var mockRequestForStatusUpdate = new org.springframework.mock.web.MockHttpServletRequest();
    mockRequestForStatusUpdate.addHeader("Authorization", "Bearer " + token);
    publicPyrisStatusUpdateResource.setTutorSuggestionJobStatus(token, statusUpdate, mockRequestForStatusUpdate);

-   // Remove the job and assert that accessing it throws an exception
-   var requestAfterRemoval = new org.springframework.mock.web.MockHttpServletRequest();
-   requestAfterRemoval.addHeader("Authorization", "Bearer " + token);
-   assertThatThrownBy(
-           () -> pyrisJobService.getAndAuthenticateJobFromHeaderElseThrow(requestAfterRemoval, de.tum.cit.aet.artemis.iris.service.pyris.job.TutorSuggestionJob.class))
-           .isInstanceOf(de.tum.cit.aet.artemis.core.exception.AccessForbiddenException.class).hasMessageContaining("No valid token provided");
+   verifyJobRemovalThrowsException(token);

    // Check if the messages were saved
    var messages = irisMessageRepository.findAllBySessionId(irisSession.getId());
    assertThat(messages).hasSize(2);
    assertThat(messages.getFirst().getContent().getFirst().toString()).contains("Test suggestion");
    assertThat(messages.getFirst().getSender()).isEqualTo(IrisMessageSender.ARTIFACT);
    assertThat(messages.get(1).getContent().getFirst().toString()).contains("Test result");
    assertThat(messages.get(1).getSender()).isEqualTo(IrisMessageSender.LLM);
}

+/**
+ * Creates a tutor suggestion job and verifies it was registered correctly
+ */
+private String createAndVerifyTutorSuggestionJob(Long postId, Long courseId, Long sessionId) {
+    String token = pyrisJobService.addTutorSuggestionJob(postId, courseId, sessionId);
+    
+    HttpHeaders headers = new HttpHeaders();
+    headers.add("Authorization", "Bearer " + token);
+    // Manually authenticate the job to verify token registration
+    var mockRequest = new org.springframework.mock.web.MockHttpServletRequest();
+    mockRequest.addHeader("Authorization", "Bearer " + token);
+    var job = pyrisJobService.getAndAuthenticateJobFromHeaderElseThrow(mockRequest, de.tum.cit.aet.artemis.iris.service.pyris.job.TutorSuggestionJob.class);
+    assertThat(job).isNotNull();
+    assertThat(job.jobId()).isEqualTo(token);
+    return token;
+}
+
+/**
+ * Verifies that attempting to access a removed job throws the expected exception
+ */
+private void verifyJobRemovalThrowsException(String token) {
+    var requestAfterRemoval = new org.springframework.mock.web.MockHttpServletRequest();
+    requestAfterRemoval.addHeader("Authorization", "Bearer " + token);
+    assertThatThrownBy(
+            () -> pyrisJobService.getAndAuthenticateJobFromHeaderElseThrow(requestAfterRemoval, de.tum.cit.aet.artemis.iris.service.pyris.job.TutorSuggestionJob.class))
+            .isInstanceOf(de.tum.cit.aet.artemis.core.exception.AccessForbiddenException.class).hasMessageContaining("No valid token provided");
+}

Also, fix the comment typo: "where" should be "were".


230-264: Use Spring's MockHttpServletRequestBuilder instead of manual MockHttpServletRequest creation.

The test creates and configures mock HTTP request objects manually.

Use Spring's MockMvc request builders for creating test requests:

-var mockRequest = new org.springframework.mock.web.MockHttpServletRequest();
-mockRequest.addHeader("Authorization", "Bearer " + token);
+import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
+
+var mockRequest = post("/api/mock-endpoint")
+    .header("Authorization", "Bearer " + token)
+    .buildRequest(null);
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge Base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between efafac4 and a67e779.

📒 Files selected for processing (1)
  • src/test/java/de/tum/cit/aet/artemis/iris/IrisTutorSuggestionIntegrationTest.java (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`src/test/java/**/*.java`: test_naming: descriptive; test_size: small_specific; fixed_data: true; junit5_features: true; assert_use: assertThat; assert_specificity: true; archunit_...

src/test/java/**/*.java: test_naming: descriptive; test_size: small_specific; fixed_data: true; junit5_features: true; assert_use: assertThat; assert_specificity: true; archunit_use: enforce_package_rules; db_query_count_tests: track_performance; util_service_factory_pattern: true; avoid_db_access: true; mock_strategy: static_mocks; context_restart_minimize: true

  • src/test/java/de/tum/cit/aet/artemis/iris/IrisTutorSuggestionIntegrationTest.java
⏰ Context from checks skipped due to timeout of 90000ms (9)
  • GitHub Check: Build and Push Docker Image / Build Docker Image for ls1intum/artemis
  • GitHub Check: Build and Push Docker Image / Build Docker Image for ls1intum/artemis
  • GitHub Check: Build .war artifact
  • GitHub Check: server-style
  • GitHub Check: client-style
  • GitHub Check: client-tests
  • GitHub Check: server-tests
  • GitHub Check: client-tests-selected
  • GitHub Check: Analyse

Copy link

End-to-End (E2E) Test Results Summary

TestsPassed ☑️Skipped ⚠️Failed ❌️Time ⏱
End-to-End (E2E) Test Report201 ran197 passed3 skipped1 failed55m 9s 167ms
TestResultTime ⏱
End-to-End (E2E) Test Report
e2e/exercise/programming/ProgrammingExerciseStaticCodeAnalysis.spec.ts
ts.Static code analysis tests › Configures SCA grading and makes a successful submission with SCA errors❌ failure1m 46s 336ms

Copy link
Contributor

@isabellagessl isabellagessl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good to me!

Copy link
Contributor

@Anishyou Anishyou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally works as expected. re-approve

Copy link

End-to-End (E2E) Test Results Summary

TestsPassed ✅Skipped ⚠️FailedTime ⏱
End-to-End (E2E) Test Report201 ran198 passed3 skipped0 failed54m 31s 578ms
TestResultTime ⏱
No test annotations available

Copy link
Contributor

@sebastianloose sebastianloose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reapprove! Code looks still good

@helios-aet helios-aet bot temporarily deployed to artemis-test3.artemis.cit.tum.de May 19, 2025 09:32 Inactive
Copy link

@armanhabibiTUM armanhabibiTUM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on test server 3. Works as expected.

Bildschirmfoto 2025-05-19 um 11 41 40 AM

@bassner bassner added this to the 8.1.0 milestone May 19, 2025
@bassner bassner changed the title Communication: Add first Iris Tutor Suggestion Interface Communication: Add Iris Tutor Suggestions May 19, 2025
@bassner bassner merged commit 59ff09f into develop May 19, 2025
48 of 53 checks passed
@bassner bassner deleted the feature/iris/iris-tutor-suggestions-communication branch May 19, 2025 13:46
@github-project-automation github-project-automation bot moved this from Ready For Review to Merged in Artemis Development May 19, 2025
@github-project-automation github-project-automation bot moved this from Todo to Done in Communication Webclient May 19, 2025
@coderabbitai coderabbitai bot mentioned this pull request Jul 7, 2025
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client Pull requests that update TypeScript code. (Added Automatically!) communication Pull requests that affect the corresponding module core Pull requests that affect the corresponding module database Pull requests that update the database. (Added Automatically!). Require a CRITICAL deployment. iris Pull requests that affect the corresponding module ready to merge server Pull requests that update Java code. (Added Automatically!) tests
Projects
Archived in project
Status: Done
Development

Successfully merging this pull request may close these issues.

7 participants