|
| 1 | +--- |
| 2 | +description: |
| 3 | +globs: |
| 4 | +alwaysApply: false |
| 5 | +--- |
| 6 | +# AI-Native Interview Evaluation Rubric Generator |
| 7 | + |
| 8 | +## Purpose |
| 9 | + |
| 10 | +This subprompt generates a comprehensive evaluation rubric for assessing candidates on their AI-native technical skills. The rubric focuses on three key areas: technical implementation, AI integration & workflow, and communication & documentation. |
| 11 | + |
| 12 | +## Input Parameters |
| 13 | + |
| 14 | +The generator requires the following inputs: |
| 15 | + |
| 16 | +- `roleTitle`: The specific role being interviewed for (e.g., "Senior Frontend Engineer") |
| 17 | +- `domainFocus`: Technical domain focus (e.g., "web-frontend", "backend", "fullstack") |
| 18 | +- `aiMaturityLevel`: Expected AI proficiency level (beginner, intermediate, advanced, expert) |
| 19 | +- `assessmentFormat`: Format of the challenge (take-home, live-coding, pair-programming, architecture-review) |
| 20 | +- `teamFluencyLevel`: Team's overall AI fluency (novice, familiar, proficient, expert) |
| 21 | + |
| 22 | +## Output Format |
| 23 | + |
| 24 | +```md |
| 25 | +## Evaluation Rubric |
| 26 | + |
| 27 | +### Technical Implementation (40%) |
| 28 | +- **Excellent (35-40)**: [Criteria for excellent implementation] |
| 29 | +- **Good (25-34)**: [Criteria for good implementation] |
| 30 | +- **Satisfactory (15-24)**: [Criteria for satisfactory implementation] |
| 31 | +- **Needs Improvement (0-14)**: [Criteria for needs improvement] |
| 32 | + |
| 33 | +### AI Integration & Workflow (30%) |
| 34 | +- **Excellent (25-30)**: [Criteria for excellent AI integration] |
| 35 | +- **Good (18-24)**: [Criteria for good AI integration] |
| 36 | +- **Satisfactory (10-17)**: [Criteria for satisfactory AI integration] |
| 37 | +- **Needs Improvement (0-9)**: [Criteria for needs improvement] |
| 38 | + |
| 39 | +### Communication & Documentation (30%) |
| 40 | +- **Excellent (25-30)**: [Criteria for excellent communication] |
| 41 | +- **Good (18-24)**: [Criteria for good communication] |
| 42 | +- **Satisfactory (10-17)**: [Criteria for satisfactory communication] |
| 43 | +- **Needs Improvement (0-9)**: [Criteria for needs improvement] |
| 44 | +``` |
| 45 | + |
| 46 | +## Technical Implementation Rubric Guidelines |
| 47 | + |
| 48 | +Focus on domain-specific technical excellence: |
| 49 | + |
| 50 | +### Frontend Focus |
| 51 | +- Component architecture and reusability |
| 52 | +- State management approach |
| 53 | +- Performance optimization |
| 54 | +- Accessibility implementation |
| 55 | +- Responsive design implementation |
| 56 | +- Testing coverage and approach |
| 57 | + |
| 58 | +### Backend Focus |
| 59 | +- API design and implementation |
| 60 | +- Data modeling and database design |
| 61 | +- Authentication and authorization |
| 62 | +- Error handling and logging |
| 63 | +- System scalability considerations |
| 64 | +- Testing coverage and approach |
| 65 | + |
| 66 | +### Fullstack Focus |
| 67 | +- End-to-end architecture |
| 68 | +- Frontend-backend integration |
| 69 | +- Cross-layer concerns (e.g., validation, error handling) |
| 70 | +- Full-stack performance considerations |
| 71 | +- Testing across the stack |
| 72 | + |
| 73 | +### Data Science/ML Focus |
| 74 | +- Data processing pipeline design |
| 75 | +- Model selection and implementation |
| 76 | +- Feature engineering approach |
| 77 | +- Model evaluation methodology |
| 78 | +- Production integration considerations |
| 79 | + |
| 80 | +### DevOps/Platform Focus |
| 81 | +- Infrastructure design |
| 82 | +- CI/CD implementation |
| 83 | +- Monitoring and observability |
| 84 | +- Security implementation |
| 85 | +- Resource optimization |
| 86 | + |
| 87 | +## AI Integration & Workflow Rubric Guidelines |
| 88 | + |
| 89 | +Assess the candidate's ability to effectively leverage AI as a force multiplier: |
| 90 | + |
| 91 | +### Beginner Level |
| 92 | +- Basic prompt engineering |
| 93 | +- Understanding of AI limitations |
| 94 | +- Ability to verify and correct AI outputs |
| 95 | +- Simple workflow integration |
| 96 | + |
| 97 | +### Intermediate Level |
| 98 | +- Strategic prompt design |
| 99 | +- Context management |
| 100 | +- Tool selection and orchestration |
| 101 | +- Iterative refinement process |
| 102 | + |
| 103 | +### Advanced Level |
| 104 | +- Complex AI workflow design |
| 105 | +- Sophisticated prompt engineering |
| 106 | +- Tool chaining and multi-step processes |
| 107 | +- Novel application of AI capabilities |
| 108 | + |
| 109 | +### Expert Level |
| 110 | +- System-level AI integration |
| 111 | +- Original AI workflow patterns |
| 112 | +- Performance optimization of AI processes |
| 113 | +- Knowledge-building and continuous improvement |
| 114 | + |
| 115 | +## Communication & Documentation Rubric Guidelines |
| 116 | + |
| 117 | +Evaluate the candidate's ability to communicate their thought process and decisions: |
| 118 | + |
| 119 | +### Take-Home Format |
| 120 | +- Code documentation |
| 121 | +- README quality |
| 122 | +- Architectural explanation |
| 123 | +- Decision justification |
| 124 | +- Process transparency |
| 125 | + |
| 126 | +### Live-Coding Format |
| 127 | +- Verbal communication during coding |
| 128 | +- Question handling |
| 129 | +- Problem-solving narration |
| 130 | +- Code organization and clarity |
| 131 | + |
| 132 | +### Pair-Programming Format |
| 133 | +- Collaboration approach |
| 134 | +- Ideas communication |
| 135 | +- Receptiveness to feedback |
| 136 | +- Joint problem-solving |
| 137 | + |
| 138 | +### Architecture-Review Format |
| 139 | +- Clarity of architectural diagrams |
| 140 | +- System explanation |
| 141 | +- Trade-off articulation |
| 142 | +- Question responses |
| 143 | + |
| 144 | +## Example Output (Frontend, Intermediate AI Level) |
| 145 | + |
| 146 | +```md |
| 147 | +## Evaluation Rubric |
| 148 | + |
| 149 | +### Technical Implementation (40%) |
| 150 | +- **Excellent (35-40)**: Complete implementation with advanced features like real-time validation, proper state management patterns, and optimized performance. Component architecture shows clear separation of concerns and reusability. Code demonstrates deep understanding of React patterns and best practices. Includes thoughtful error handling, loading states, and edge cases. |
| 151 | +- **Good (25-34)**: Solid implementation with all core features working correctly. Good component structure with reasonable separation of concerns. Effective state management, though may have room for optimization. Handles common error cases and includes basic loading states. |
| 152 | +- **Satisfactory (15-24)**: Basic implementation with core functionality working. Component structure is functional but may lack optimal organization. State management works but may have inconsistencies. Limited error handling and loading states. |
| 153 | +- **Needs Improvement (0-14)**: Incomplete implementation or significant issues with core functionality. Poor component structure with inappropriate coupling. Ineffective state management leading to bugs or inconsistencies. Missing error handling and loading states. |
| 154 | + |
| 155 | +### AI Integration & Workflow (30%) |
| 156 | +- **Excellent (25-30)**: Demonstrates sophisticated AI prompt engineering that significantly enhances productivity. Shows clear evidence of iterative refinement of AI outputs. Uses AI strategically for different aspects of development (architecture planning, component design, testing). Can articulate clear reasoning for when and how AI was used in the workflow. Final code shows deep understanding and ownership. |
| 157 | +- **Good (18-24)**: Effectively uses AI to accelerate development while maintaining quality. Shows good prompt construction and ability to guide AI toward useful outputs. Applies appropriate review and refinement to AI-generated code. Can explain AI usage strategy and how it enhanced workflow. |
| 158 | +- **Satisfactory (10-17)**: Basic AI usage that assists with some development tasks. Some evidence of prompt engineering but may rely on basic prompts. Minimal refinement of AI outputs. Limited explanation of AI integration into workflow. Some generated code may be used without full understanding. |
| 159 | +- **Needs Improvement (0-9)**: Ineffective AI usage or over-reliance on AI without critical evaluation. Poor prompt construction leading to low-quality outputs. Direct copying of AI-generated code without adaptation or understanding. Unable to explain AI integration strategy. Final implementation shows lack of ownership. |
| 160 | + |
| 161 | +### Communication & Documentation (30%) |
| 162 | +- **Excellent (25-30)**: Exceptional documentation with clear explanations of component architecture, state management, and key decisions. Includes detailed README with setup instructions, usage examples, and architectural diagrams. Code has meaningful comments that explain "why" not just "what." Provides thorough explanation of AI integration strategy and workflow process. Commit history tells a clear story of development progression. |
| 163 | +- **Good (18-24)**: Good documentation covering main architectural decisions and setup process. README includes necessary information for running the application. Code is generally well-organized and readable. Some explanation of AI workflow included. Commit messages are informative. |
| 164 | +- **Satisfactory (10-17)**: Basic documentation that covers setup but lacks depth on design decisions. Limited code comments focused on obvious functionality. Minimal discussion of AI usage or workflow. Commit history provides basic tracking but limited insight. |
| 165 | +- **Needs Improvement (0-9)**: Poor or missing documentation. Unclear setup instructions. Code lacks organization and meaningful comments. No explanation of AI integration or workflow. Commit history is sparse or uninformative. |
| 166 | +``` |
| 167 | + |
| 168 | +## Example Output (Backend, Advanced AI Level) |
| 169 | + |
| 170 | +```md |
| 171 | +## Evaluation Rubric |
| 172 | + |
| 173 | +### Technical Implementation (40%) |
| 174 | +- **Excellent (35-40)**: Exceptional API architecture showing deep understanding of backend principles. Clean, modular code structure with proper separation of concerns. Robust error handling with appropriate status codes and informative messages. Thorough data validation and security implementation. Optimized database queries and proper indexing. Comprehensive test coverage with unit, integration, and potentially end-to-end tests. |
| 175 | +- **Good (25-34)**: Well-structured API with good adherence to RESTful principles. Modular code organization with clear boundaries. Effective error handling for common cases. Solid data validation and basic security measures. Efficient database operations. Good test coverage of critical paths. |
| 176 | +- **Satisfactory (15-24)**: Functional API that meets basic requirements. Reasonable code organization but may have some structural issues. Basic error handling for major cases. Some data validation but may miss edge cases. Functional database operations. Limited test coverage focused on happy paths. |
| 177 | +- **Needs Improvement (0-14)**: Poorly structured API with significant architectural issues. Disorganized code with improper separation of concerns. Inadequate error handling. Missing or ineffective data validation. Inefficient or problematic database operations. Minimal or no testing. |
| 178 | + |
| 179 | +### AI Integration & Workflow (30%) |
| 180 | +- **Excellent (25-30)**: Demonstrates advanced AI workflow integration with complex, multi-step processes. Shows sophisticated prompt engineering techniques including chain-of-thought, few-shot learning, and context management. Evidence of AI tool chaining for different aspects of development. Can articulate advanced strategies for leveraging AI in backend development. Final implementation shows expert refinement and enhancement of AI outputs. AI usage significantly accelerates development without compromising quality. |
| 181 | +- **Good (18-24)**: Employs effective AI workflow with strategic prompt engineering. Shows good understanding of context management and refinement processes. Uses AI for multiple aspects of development with appropriate tool selection. Can explain reasoning behind AI integration choices. Implementation shows thoughtful adaptation of AI outputs. AI usage enhances productivity while maintaining code quality. |
| 182 | +- **Satisfactory (10-17)**: Basic AI workflow with functional prompt construction. Some evidence of iterative refinement. Limited use of advanced AI techniques. Can provide basic explanation of how AI was used. Implementation shows some adaptation of AI outputs but may rely too heavily on unrefined generations. AI usage provides some productivity benefits. |
| 183 | +- **Needs Improvement (0-9)**: Ineffective AI workflow with poor prompt engineering. Little or no refinement of AI outputs. Inability to explain AI integration strategy. Implementation shows direct use of AI-generated code without understanding or adaptation. AI usage either provides minimal benefit or actually detracts from solution quality. |
| 184 | + |
| 185 | +### Communication & Documentation (30%) |
| 186 | +- **Excellent (25-30)**: Comprehensive API documentation with clear endpoint descriptions, request/response examples, and error handling information. Detailed README with architecture overview, setup instructions, and design rationale. Thorough code comments explaining complex logic and design decisions. Clear explanation of database schema and relationships. Detailed description of AI integration approach and workflow optimization. Well-structured commit history showing logical progression. |
| 187 | +- **Good (18-24)**: Solid API documentation covering all endpoints and major use cases. Good README with setup instructions and basic architectural information. Helpful code comments for non-obvious logic. Database schema documentation. Some explanation of AI workflow and usage. Informative commit messages. |
| 188 | +- **Satisfactory (10-17)**: Basic API documentation covering main endpoints. Simple README with minimal setup instructions. Limited code comments focused on complex sections only. Basic database information. Minimal discussion of AI integration. Basic commit history. |
| 189 | +- **Needs Improvement (0-9)**: Incomplete or missing API documentation. Insufficient README lacking setup instructions. Few or no code comments. Missing database documentation. No explanation of AI workflow. Poor commit history with uninformative messages. |
| 190 | +``` |
| 191 | + |
| 192 | +## Adaptation Guidelines |
| 193 | + |
| 194 | +Adjust rubric based on: |
| 195 | +- Role seniority (adjust expectations accordingly) |
| 196 | +- Domain-specific technical requirements |
| 197 | +- Team's AI fluency level (set appropriate AI integration expectations) |
| 198 | +- Assessment format (different communication expectations) |
0 commit comments