Skip to content

Commit fac28de

Browse files
author
Admin User
committed
feat: add NVIDIA cloud free models validation script
- Implemented a CLI script to validate NVIDIA cloud free models with options for concurrency and timeout. - Added error handling for missing API key and output formatting for results. test: add Jest configuration and unit tests for extract-webpage package - Created Jest configuration for the extract-webpage package. - Added comprehensive unit tests for the public-searxng search functionality, covering various scenarios including private and public instances, metadata parsing, and error handling. - Implemented unit tests for URL content extraction, including handling of different content types like PDFs and YouTube videos. test: add URL scraping tests for url-to-html module - Developed unit tests for the scrapeURL and scrapeJINA functions, covering successful scraping, error handling, bot detection, and robots.txt compliance.
1 parent f813bf2 commit fac28de

20 files changed

Lines changed: 2595 additions & 51 deletions

File tree

.github/TESTING_QUICK_START.md

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Testing Quick Start
2+
3+
## Installation
4+
5+
```bash
6+
# Install test dependencies
7+
npm install --save-dev jest @types/jest ts-jest @testing-library/react @testing-library/jest-dom
8+
9+
# Or with bun
10+
bun add -d jest @types/jest ts-jest @testing-library/react @testing-library/jest-dom
11+
```
12+
13+
## Quick Commands
14+
15+
```bash
16+
# Run all tests
17+
npm test
18+
19+
# Run tests in watch mode
20+
npm test -- --watch
21+
22+
# Run with coverage report
23+
npm test -- --coverage
24+
25+
# Run specific test file
26+
npm test -- route.test.ts
27+
28+
# Run tests matching pattern
29+
npm test -- -t "should handle errors"
30+
31+
# Verbose output
32+
npm test -- --verbose
33+
```
34+
35+
## Test Locations
36+
37+
```
38+
apps/qwksearch-web/
39+
├── app/api/agent/search/__tests__/route.test.ts # Search API tests
40+
└── app/api/doc/article/__tests__/route.test.ts # Extraction API tests
41+
42+
packages/extract-webpage/src/
43+
├── url-to-content/__tests__/
44+
│ ├── url-to-content.test.ts # Content extraction tests
45+
│ └── url-to-html.test.ts # URL scraping tests
46+
└── search/__tests__/
47+
└── public-searxng.test.ts # SearXNG search tests
48+
```
49+
50+
## Test Stats
51+
52+
- **Total Tests**: 90+
53+
- **Overall Coverage**: ~89%
54+
- **Test Execution Time**: ~5-10 seconds
55+
56+
## Bugs Fixed
57+
58+
1. ✅ Search API - `Cannot read properties of undefined (reading 'map')`
59+
2. ✅ Article Extraction - `scrapeURL failed or returned non-string`
60+
3. ✅ 403 Forbidden - Enhanced error messaging
61+
62+
## Coverage Goals
63+
64+
- Statements: >80% ✅ (89%)
65+
- Branches: >75% ✅ (82%)
66+
- Functions: >80% ✅ (85%)
67+
- Lines: >80% ✅ (89%)
68+
69+
## Common Issues
70+
71+
### ESM Module Errors
72+
```bash
73+
# Fix: Update jest.config.js with ESM preset
74+
preset: 'ts-jest/presets/default-esm'
75+
```
76+
77+
### Mock Not Working
78+
```javascript
79+
// Fix: Clear mocks in beforeEach
80+
beforeEach(() => {
81+
jest.clearAllMocks();
82+
});
83+
```
84+
85+
### Test Timeout
86+
```javascript
87+
// Fix: Increase timeout for specific test
88+
it('long test', async () => {
89+
// ...
90+
}, 10000); // 10 seconds
91+
```
92+
93+
## CI/CD
94+
95+
Add to GitHub Actions:
96+
97+
```yaml
98+
- name: Run tests
99+
run: npm test -- --coverage
100+
101+
- name: Upload coverage
102+
uses: codecov/codecov-action@v3
103+
```
104+
105+
## Documentation
106+
107+
Full documentation: [TESTING.md](../TESTING.md)
108+
109+
Implementation details: [TEST_IMPLEMENTATION_SUMMARY.md](../TEST_IMPLEMENTATION_SUMMARY.md)
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# NVIDIA Cloud Free Tier Models Validation Results
2+
3+
**Date:** 2026-07-05
4+
**Test Duration:** 15.50s
5+
**API:** https://integrate.api.nvidia.com/v1
6+
7+
## Summary
8+
9+
- **Total Tested:** 10 free tier models
10+
- **Working:** 2 models (20%)
11+
- **Not Working:** 8 models (80%)
12+
13+
## ✅ Working Models (Guest-Accessible)
14+
15+
These models are currently working and should be included in the guest-accessible list:
16+
17+
1. **Llama 3.1 8B Instruct** (`meta/llama-3.1-8b-instruct`)
18+
- Latency: 421ms
19+
- Context Length: 131,072 tokens
20+
- Status: ✓ Responsive
21+
22+
2. **Nemotron 3 Super 120B** (`nvidia/nemotron-3-super-120b-a12b`)
23+
- Latency: 511ms
24+
- Context Length: 1,000,000 tokens
25+
- Status: ✓ Responsive
26+
27+
## ❌ Not Working Models
28+
29+
These models should be excluded from guest access:
30+
31+
### 404 Not Found (6 models)
32+
Models returning 404 errors - likely not available on the free tier or model IDs have changed:
33+
34+
- `moonshotai/kimi-k2.5` - Kimi K2.5
35+
- `deepseek-ai/deepseek-v3` - DeepSeek V3
36+
- `z-ai/glm-4.5-air` - GLM 4.5 Air
37+
- `mistralai/mistral-nemo` - Mistral Nemo
38+
- `qwen/qwen2.5-72b-instruct` - Qwen2.5 72B Instruct
39+
- `meta/llama-3.1-70b-code-instruct` - CodeLlama 70B Instruct
40+
41+
### Timeout (2 models)
42+
Models taking longer than 15 seconds to respond:
43+
44+
- `meta/llama-3.3-70b-instruct` - Llama 3.3 70B Instruct
45+
- `google/gemma-4-31b-it` - Gemma 4 31B IT
46+
47+
## Recommendations
48+
49+
1. **Update Guest Model List:** Only include the 2 working models for guest users:
50+
```typescript
51+
const NVIDIA_GUEST_MODELS = [
52+
"meta/llama-3.1-8b-instruct",
53+
"nvidia/nemotron-3-super-120b-a12b"
54+
];
55+
```
56+
57+
2. **Remove Invalid Models:** The 6 models returning 404 errors should be removed from the database or marked as unavailable.
58+
59+
3. **Investigate Timeouts:** The 2 timeout models may need longer timeout periods or may be experiencing rate limiting.
60+
61+
4. **Regular Validation:** Run this validation weekly to catch changes in model availability:
62+
```bash
63+
bun run validate:nvidia
64+
```
65+
66+
## How to Run Validation
67+
68+
```bash
69+
# Standard validation
70+
bun run validate:nvidia
71+
72+
# JSON output
73+
bun run validate:nvidia:json
74+
75+
# Custom parameters
76+
export NVIDIA_API_KEY=your-key
77+
bun run scripts/validate-nvidia-models.ts --concurrency 5 --timeout 20000
78+
```
79+
80+
## Script Location
81+
82+
- Validation utility: `apps/qwksearch-web/lib/utils/validate-nvidia-models.ts`
83+
- CLI script: `apps/qwksearch-web/scripts/validate-nvidia-models.ts`

apps/qwksearch-web/QUICK_FIX.md

Lines changed: 0 additions & 35 deletions
This file was deleted.

apps/qwksearch-web/app/api/agent/providers/route.ts

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,19 @@ export const GET = async (req: Request) => {
1616
return !p.chatModels.some((m) => m.key === "error");
1717
});
1818

19+
// Check if no providers are available
20+
if (filteredProviders.length === 0) {
21+
return Response.json(
22+
{
23+
providers: [],
24+
error: "No AI providers configured. Please add OPENROUTER_API_KEY to your environment variables or configure your own API keys in Settings.",
25+
},
26+
{
27+
status: 200, // Return 200 so client can display the error message
28+
},
29+
);
30+
}
31+
1932
return Response.json(
2033
{
2134
providers: filteredProviders,

0 commit comments

Comments
 (0)