fix: Add proper role to human feedback messages for LiteLLM

[devin-ai-integration]

Fixes #2111

When human_input is set to True, feedback messages were not properly formatted with a role for LiteLLM. This caused LiteLLM calls to fail. This PR:

• Updates _handle_human_feedback to set user role for feedback messages
• Adds test coverage for message format
• Fixes [BUG] LiteLLM call fails, when human_input set to True #2111

Link to Devin run: https://app.devin.ai/sessions/75d754d9da88477784b62db499274f6c
Requested by: Joe

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add "(aside)" to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[joaomdmoura]

Disclaimer: This review was made by a crew of AI Agents.

Code Review Comment for PR #2113 - LiteLLM Message Role Fix

Overview

This PR addresses the feedback handling mechanism in crew_agent_executor.py, ensuring that message roles are properly defined for LiteLLM compatibility. The restructuring of feedback message processing is a commendable improvement, showing responsiveness to issue #2111.

Positive Aspects

1. Role Attribution: The introduction of message roles enhances clarity and interaction quality, visibly designating the source of the feedback as "user".
2. Code Organization: The extraction of feedback processing logic promotes maintainability.
3. Backward Compatibility Maintained: Changes do not disrupt existing functionality, ensuring a smooth transition.

Issues and Recommendations

1. Inconsistent Message Formatting

Location: _handle_regular_feedback method

• Issue: The direct dictionary creation for feedback messages lacks consistency.
• Recommendation: Implement the existing message formatter to standardize:

  self.messages.append(self._format_msg(
      f"Feedback: {feedback}",
      role="user"
  ))

2. Missing Type Hints

Location: Various methods

• Issue: Insufficient type hints for parameters and return types hinder readability.
• Recommendation: Include comprehensive type hints:

  def _handle_regular_feedback(
      self,
      feedback: str,
      current_answer: AgentFinish
  ) -> AgentFinish:

3. Method Documentation Gaps

Location: _process_feedback_iteration method

• Issue: The absence of detailed docstrings leads to ambiguity.
• Recommendation: Add robust documentation:

  def _process_feedback_iteration(self, feedback: str) -> AgentFinish:
      """Process a single feedback iteration.
      
      Args:
          feedback (str): The feedback to process
          
      Returns:
          AgentFinish: The processed agent response after feedback
      """

4. Lack of Error Handling

Location: Throughout feedback processing

• Issue: No designated error management to capture processing failures.
• Recommendation: Implement error handling in feedback processing:

  try:
      # feedback processing logic
  except Exception as e:
      logger.error(f"Error processing feedback: {str(e)}")
      raise FeedbackProcessingError(f"Failed to process feedback: {str(e)}")

Security Considerations

• Input Sanitization: Ensure feedback is sanitized to protect against injection risks.
• Validation: Implement input validation to confirm feedback length and content safety.

Performance Impact

The changes present minimal performance impact as they mainly adjust message structures.

Testing Recommendations

1. Incorporate unit tests for various feedback scenarios.
2. Validate edge cases, including empty or overly long feedback.
3. Ensure message role consistency across different LLM providers.

Example Test Case:

def test_feedback_message_formatting():
    executor = CrewAgentExecutor()
    feedback = "This needs improvement"
    executor._process_feedback_iteration(feedback)
    
    last_message = executor.messages[-1]
    assert last_message["role"] == "user"
    assert "feedback" in last_message["content"].lower()

Summary of Changes Needed

1. Standardize message formatting across feedback handling.
2. Add thorough type hints throughout the codebase.
3. Enhance method documentation.
4. Implement practical error handling measures.
5. Validate feedback input for security.
6. Expand testing coverage to encompass multiple scenarios.

The modifications present a significant step toward resolving the LiteLLM compatibility issue while simultaneously enhancing code quality. Implementing the suggested improvements will further solidify the resilience and maintainability of the application's feedback handling capabilities.