Skip to content

Commit 52dafd0

Browse files
authored
Fix import/export API endpoints and improve CSV upload reliability (#458)
* refactor(shared): fix zod schemas for partial clusters and lint errors Changes: - Replace z.record with z.object for zodPolisClusters and zodPolisClustersMetadata to allow partial clusters (z.record with enum key requires ALL values present, but conversations can have fewer than 6 clusters) - Wrap numeric constants in String() for template literal error messages to fix TypeScript lint errors about implicit string coercion - Remove unused imports (zodConversationBodyOutput, zodOpinionContentOutput) from dto.ts Deploy: agora, api, math-updater * refactor(api): convert import/export endpoints from GET/DELETE to POST Standardize all API endpoints to use POST method following the codebase's JSON-RPC-like pattern. This ensures consistent UCAN authentication handling and prevents issues with URL parameter limitations. Backend changes: - Convert GET /conversation/import/active to POST - Convert GET /conversation/import/status/:importSlugId to POST with body - Convert GET /conversation/export/status/:exportSlugId to POST with body - Convert GET /conversation/export/history/:conversationSlugId to POST with body - Convert GET /conversation/export/readiness/:conversationSlugId to POST with body - Convert DELETE /conversation/export/:exportSlugId to POST /export/delete with body - Fix verifyUcanAndKnownDeviceStatus to properly spread expectedKnownDeviceStatus Frontend changes: - Update all API wrapper functions to use new POST endpoints with body params - Add file-upload timeout profile (90s) for large CSV uploads - Add configurable UCAN lifetime for file uploads (120s vs default 30s) - Update generated OpenAPI client to match new endpoints Deploy: agora, api * fix(nginx): increase client_max_body_size for CSV import uploads Set client_max_body_size to 150M on /api location to support importing 3 CSV files (summary, comments, votes) up to 50MB each. Deploy: nginx * fix(agora): continue polling import status on 404 for read replica lag When using database read replicas, newly created import records may not be immediately visible due to replication lag. This causes the import status query to return 404 briefly after creating an import. Continue polling on 404 errors instead of showing an error state, giving the read replica time to catch up with the primary. Deploy: agora * chore: remove debug logging and reduce import noise - Remove console.log debug statements from PreLoginIntentionDialog - Reduce noise in comment import logging by treating null vote counts as 0 (only log warnings when Polis provides a non-null value that differs from calculated) Deploy: agora, api * fix(agora): improve AsyncStateHandler and ImportStatusView components AsyncStateHandler: - Fix error message priority: show config.error.message before errorMessage - Add bottom padding to asyncStateMessage for better visual spacing ImportStatusView: - Use computed statusData accessor for cleaner template bindings - Remove redundant isEmpty prop (AsyncStateHandler handles empty state) - Simplify enabled condition (isGuestOrLoggedIn implies auth initialized) Deploy: agora * feat(agora): improve CSV upload error formatting and display Add comprehensive error formatting for CSV upload failures: - Format Axios errors with HTTP status, response body, and stack trace - Handle both string and JSON response bodies (nginx HTML vs API JSON) - Include full error object for debugging - Better error details for generic Error instances This helps users report and debug upload failures more effectively. Deploy: agora * feat(api): add unique constraint on conversation_import.conversationId Add unique constraint to prevent duplicate conversation references in the import table. The constraint allows multiple NULL values (imports in progress or failed) while ensuring each completed import points to a unique conversation. Includes Drizzle schema update and Flyway migration script. Deploy: api (requires migration) * docs: document POST-only API pattern in CLAUDE.md Add comprehensive documentation for the codebase's POST-only API design pattern: - All endpoints use POST (never GET, PUT, PATCH, DELETE) - Request data goes in body, never URL parameters - All "delete" operations are soft-deletes - SSE endpoints are the only exception (must use GET) Include code examples showing correct vs incorrect patterns. Deploy: none * fix(agora): fix export enabled check for undefined env var The VITE_EXPORT_CONVOS_ENABLED env var was using .default("true") in the Zod schema, but defaults only apply during Zod parsing. Since processEnv is a direct type cast of import.meta.env without runtime parsing, the default never applied when the var was unset. Changes: - Make VITE_EXPORT_CONVOS_ENABLED optional instead of using .default() - Change visibility check from === "true" to !== "false" so export is enabled by default when the env var is not set Deploy: agora
1 parent 1c73503 commit 52dafd0

35 files changed

Lines changed: 7520 additions & 929 deletions

File tree

CLAUDE.md

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -254,6 +254,67 @@ When adding API endpoints:
254254
- Run `make generate` to update frontend client
255255
- Never manually edit generated API client code
256256

257+
### REST API Design: POST-Only Pattern
258+
259+
This codebase uses a **POST-only API pattern** similar to JSON-RPC but with free-form endpoint naming. This is NOT a traditional RESTful API.
260+
261+
**Rules:**
262+
1. **ALL endpoints use POST** - Never use GET, PUT, PATCH, or DELETE HTTP methods
263+
2. **Request data goes in the body** - Never use URL parameters or query strings for data
264+
3. **Soft-delete only** - All "delete" operations are soft-deletes (set `deletedAt` timestamp), never hard-deletes
265+
266+
**Why POST-only:**
267+
-**Consistent authentication**: UCAN tokens work reliably in POST request bodies/headers
268+
-**No URL length limits**: Complex queries with many parameters work without issues
269+
-**Simpler caching control**: No accidental browser/CDN caching of sensitive data
270+
-**Uniform request handling**: All requests follow the same pattern
271+
272+
**Exception:** Server-Sent Events (SSE) endpoints MUST use GET (protocol requirement). Example: `/api/v1/notification/stream`
273+
274+
**Example endpoint patterns:**
275+
```typescript
276+
// ✅ GOOD: POST with body parameters
277+
server.route({
278+
method: "POST",
279+
url: `/api/${apiVersion}/conversation/export/status`,
280+
schema: {
281+
body: Dto.getConversationExportStatusRequest, // { exportSlugId: string }
282+
response: { 200: Dto.getConversationExportStatusResponse },
283+
},
284+
handler: async (request) => {
285+
const { exportSlugId } = request.body;
286+
// ...
287+
},
288+
});
289+
290+
// ✅ GOOD: "Delete" operation uses POST + soft-delete
291+
server.route({
292+
method: "POST",
293+
url: `/api/${apiVersion}/conversation/export/delete`,
294+
schema: {
295+
body: Dto.deleteConversationExportRequest, // { exportSlugId: string }
296+
},
297+
handler: async (request) => {
298+
// Sets deletedAt, does NOT remove from database
299+
await softDeleteExport({ exportSlugId: request.body.exportSlugId });
300+
},
301+
});
302+
303+
// ❌ BAD: GET with URL parameters
304+
server.route({
305+
method: "GET",
306+
url: `/api/${apiVersion}/conversation/export/status/:exportSlugId`,
307+
// ...
308+
});
309+
310+
// ❌ BAD: DELETE method
311+
server.route({
312+
method: "DELETE",
313+
url: `/api/${apiVersion}/conversation/export/:exportSlugId`,
314+
// ...
315+
});
316+
```
317+
257318
### Background Jobs (pg-boss)
258319

259320
Math-updater uses PostgreSQL-based job queue:

script/vhosts/nginx.conf

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,7 @@ server {
9696
proxy_buffering off;
9797
}
9898
location /api {
99+
client_max_body_size 150M; # 50MB × 3 CSV files for import
99100
proxy_set_header Host $host;
100101
proxy_set_header X-Real-IP $remote_addr;
101102
proxy_pass http://api:8080;

0 commit comments

Comments
 (0)