Encoding error « UTF8 » : 0xe9 0x65 0x73

[JulienPivard]

We have an encoding error when we try to regenerate index.

Context
We have run a curl with a script and cron.
curl command :

 curl 'https://doc.ganil.local/graphql' \
  -X POST \
  -H 'content-type: application/json' \
  -H 'Authorization: Bearer a_very_long_key_I_don_t_share ' \
  --data-raw '[{"operationName":null,"variables":{},"extensions":{},"query":"mutation {\n  search {\n    rebuildIndex {\n      responseResult {\n        succeeded\n        errorCode\n        slug\n        message\n        __typename\n      }\n      __typename\n    }\n    __typename\n  }\n}\n"}]'

when I run this command 3 times, the third time cmd never finish, and in log we have this error :

déc. 03 13:41:10 doc node[3725466]: 2025-12-03T12:41:10.717Z [MASTER] warn:
déc. 03 13:41:10 doc node[3725466]:             INSERT INTO "pagesVector" (path, locale, title, description, "tokens", content) VALUES (
déc. 03 13:41:10 doc node[3725466]:               $1, $2, $3, $4, (setweight(to_tsvector('french', $5), 'A') || setweight(to_tsvector('french', $6), 'B'>
déc. 03 13:41:10 doc node[3725466]:             )
déc. 03 13:41:10 doc node[3725466]:            - séquence d'octets invalide pour l'encodage « UTF8 » : 0xe7 0x61 0x20

I add logs to know on which page we have the error (in file ./server/modules/search/postgres/engine.js), and file is never the same, error always happen on first character with accent. Characters in errors are between 0x0080 and 0x00FF.

After error :

• We cannot update page with characters between 0x0080 and 0x00FF;
• If a page start with a double byte character (like 0x200B) this page have no problem even if we have characters between 0x0080 and 0x00FF;
• We need to restart wikijs to "solve" problem until the next time we run 3 times curl to regenrate index.

I think this is a configuration problem, but I can't find it. Any Idea ?

Wiki JS version : Wiki.js 2.5.307

[JulienPivard]

A new symptom, even if we don't regenerate index, character with accent are broken after a time (don't know when exactly when this happen this night)

déc. 03 17:46:29 doc node[3758965]: Loading configuration from /path/to/our/install/ofwikijs/config.yml... OK
déc. 03 17:46:30 doc node[3758965]: 2025-12-03T16:46:30.201Z [JOB] info: Rendering page ID 198...
déc. 03 17:46:31 doc node[3758965]: 2025-12-03T16:46:31.419Z [JOB] info: Using database driver pg for postgres [ OK ]
déc. 03 17:46:36 doc node[3758965]: 2025-12-03T16:46:36.383Z [JOB] info: Rendering page ID 198: [ COMPLETED ]
déc. 03 17:46:36 doc node[3752209]: 2025-12-03T16:46:36.494Z [MASTER] info: (STORAGE/DISK) Updating file [fr] G2CA/projets/cyren/pocs_ihm/pyqt...
déc. 03 17:55:23 doc node[3752209]: 2025-12-03T16:55:23.055Z [MASTER] info: Purging orphaned upload files...
déc. 03 17:55:23 doc node[3752209]: 2025-12-03T16:55:23.064Z [MASTER] info: Purging orphaned upload files: [ COMPLETED ]
déc. 03 18:10:23 doc node[3752209]: 2025-12-03T17:10:23.066Z [MASTER] info: Purging orphaned upload files...
déc. 03 18:10:23 doc node[3752209]: 2025-12-03T17:10:23.076Z [MASTER] info: Purging orphaned upload files: [ COMPLETED ]
déc. 03 18:25:23 doc node[3752209]: 2025-12-03T17:25:23.078Z [MASTER] info: Purging orphaned upload files...
déc. 03 18:25:23 doc node[3752209]: 2025-12-03T17:25:23.082Z [MASTER] info: Purging orphaned upload files: [ COMPLETED ]
déc. 03 18:40:23 doc node[3752209]: 2025-12-03T17:40:23.083Z [MASTER] info: Purging orphaned upload files...
déc. 03 18:40:23 doc node[3752209]: 2025-12-03T17:40:23.084Z [MASTER] info: Purging orphaned upload files: [ COMPLETED ]
déc. 03 18:55:23 doc node[3752209]: 2025-12-03T17:55:23.086Z [MASTER] info: Purging orphaned upload files...
déc. 03 18:55:23 doc node[3752209]: 2025-12-03T17:55:23.093Z [MASTER] info: Purging orphaned upload files: [ COMPLETED ]

[JulienPivard]

I happen logs in all place I could, but now I'm stuck and don't know what I could try to give you more information to find where is our problem.

Arround WIKI.models.knex.raw in server/modules/search/postgres/engine.js line 170 I add a try catch to see stack trace of the exception

déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.897Z [MASTER] info: (DEBUG/DEBUT) line 167 [server/modules/search/postgres/engine.js] [fr] [G2I/Systeme/Linux/proxmox_tags]                       
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.898Z [MASTER] info: line 118 [node_modules/knex/lib/util/make-knex.js]... [raw() {            
déc. 04 16:45:35 doc node[3897969]:           WIKI.logger.info('line 128 [node_modules/knex/lib/client.js]...')                                       
déc. 04 16:45:35 doc node[3897969]:     return new Raw(this).set(...arguments);                                                                       
déc. 04 16:45:35 doc node[3897969]:   }]                                                                                                              
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.898Z [MASTER] info: line 118 [node_modules/knex/lib/util/make-knex.js]... [function apply() { [native code] }]
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.898Z [MASTER] info: line 128 [node_modules/knex/lib/client.js]...                             
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.898Z [MASTER] info: line 18 [node_modules/knex/lib/raw.js]...                                 
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.899Z [MASTER] info: line 37 [node_modules/knex/lib/raw.js]...                                 
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.899Z [MASTER] info: line 75 [node_modules/knex/lib/raw.js]...                                 
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.899Z [MASTER] info: line 128 [node_modules/knex/lib/raw.js]...                                
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.899Z [MASTER] info: line 142 [node_modules/knex/lib/raw.js] [?]                               
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.900Z [MASTER] info: line 142 [node_modules/knex/lib/raw.js] [?]                               
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.900Z [MASTER] info: line 142 [node_modules/knex/lib/raw.js] [?]                               
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.900Z [MASTER] info: line 142 [node_modules/knex/lib/raw.js] [?]                               
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.900Z [MASTER] info: line 142 [node_modules/knex/lib/raw.js] [?]                               
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.900Z [MASTER] info: line 142 [node_modules/knex/lib/raw.js] [?]                               
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.900Z [MASTER] info: line 142 [node_modules/knex/lib/raw.js] [?]                               
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.900Z [MASTER] info: line 142 [node_modules/knex/lib/raw.js] [?]                               
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.901Z [MASTER] info: line 117 [node_modules/knex/lib/raw.js]...                                
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.911Z [MASTER] error:                                                                          
déc. 04 16:45:35 doc node[3897969]:             INSERT INTO "pagesVector" (path, locale, title, description, "tokens", content) VALUES (              
déc. 04 16:45:35 doc node[3897969]:               $1, $2, $3, $4, (setweight(to_tsvector('french', $5), 'A') || setweight(to_tsvector('french', $6), 'B') || setweight(to_tsvector('french', $7), 'C')), $8
déc. 04 16:45:35 doc node[3897969]:             )                                                                                                     
déc. 04 16:45:35 doc node[3897969]:            - séquence d'octets invalide pour l'encodage « UTF8 » : 0xe9 0x63 0x72                                 
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.911Z [MASTER] error: error:                                                                   
déc. 04 16:45:35 doc node[3897969]:     at Object.raw (/path/to/our/install/ofwikijs/node_modules/knex/lib/util/make-knex.js:117:18)                          
déc. 04 16:45:35 doc node[3897969]:     at Function.value (/path/to/our/install/ofwikijs/node_modules/knex/lib/util/make-knex.js:90:29)                       
déc. 04 16:45:35 doc node[3897969]:     at Transform.transform [as _transform] (/path/to/our/install/ofwikijs/server/modules/search/postgres/engine.js:170:34)
déc. 04 16:45:35 doc node[3897969]:     at Transform._write (node:internal/streams/transform:171:8)                                                   
déc. 04 16:45:35 doc node[3897969]:     at doWrite (node:internal/streams/writable:596:12)                                                            
déc. 04 16:45:35 doc node[3897969]:     at clearBuffer (node:internal/streams/writable:781:7)                                                         
déc. 04 16:45:35 doc node[3897969]:     at onwrite (node:internal/streams/writable:651:7)                                                             
déc. 04 16:45:35 doc node[3897969]:     at node:internal/streams/transform:191:7                                                                      
déc. 04 16:45:35 doc node[3897969]:     at Transform.transform [as _transform] (/path/to/our/install/ofwikijs/server/modules/search/postgres/engine.js:180:11)
déc. 04 16:45:35 doc node[3897969]:     at process.processTicksAndRejections (node:internal/process/task_queues:105:5)                                
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.912Z [MASTER] info: (DEBUG/CB) line 179 [server/modules/search/postgres/engine.js][fr] [G2I/Systeme/Linux/proxmox_tags]                          
déc. 04 16:45:35 doc node[3897969]: 2025-12-04T15:45:35.925Z [MASTER] info: (DEBUG/FIN) line 181 [server/modules/search/postgres/engine.js][fr] [G2I/Systeme/Linux/proxmox_tags]