[BUG] File upload via API does not work

[bendadaniel]

Problem:
Hello,
I have a chatflow with Retrieval Agent + Pinecone + File loader. When user uploads file via chat embed, everything works fine. File is uploaded, loaded by File loader, auto upsert to Pinecone and then Agent knows context of file.

However, in my case, I need to upload file to conversation via API first. When I call send message and upload base64, it does not work, Pinecone is not updated and Agent does not know anything about the document. There is no error in response on this API call or in LangSmith. I used the upload code from the Flowise docs example so it should work.

My general goal for context:
In our system, we have a lot of dynamic files and I want to make Agent than can retriavel from specific one file that user select. So I cant use some batch upsert of all files at one or something like that. So my idea is that when user want to "chat with file", embed is loaded and inicialised chat and my component send API call with this file automatically, with file I send same info like sessionId, chatId same as embed component. File is uploaded via API and then user can talk to agent about this specific file freshly loaded in Pinecone. I think there is alternative solution to create some other upsert flow where I can upsert single file with some metadata and then connect it to this my original flow, but Idea with api seems more clean and more compact for me.

Code
Api call:

const sendMessage = async (data: any) => {
    const response = await fetch(
        `${apiHost}/api/v1/prediction/${chatflowid}`,
        {
            method: "POST",
            headers: {"Content-Type": "application/json"},
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}
  sendMessage({
          "question": "",
          "chatId": chatId,
          "overrideConfig": {
            "sessionId": sessionId,
          },
          "uploads": [
              {
                  "data": "data:application/pdf;base64,JVBERi0xLjcNCiW1tbW1DQoxIDAgb2J........."
                  "type": 'file',
                  "name": 'VOP_EN_2023-06-26.pdf',
                  "mime": 'application/pdf'
              }
          ]
      }).then((response) => {
          console.log(response);
      });

React embed:

<FullPageChat
    chatflowid={chatflowid}
     apiHost={apiHost}
     chatflowConfig={
     { 
        sessionId: sessionId,
           vars: {
              customerId: chatId
            }
          }
     }
/> 

Expected behavior
Works same as manual user upload.

Screenshots

Setup

• Render instance
• Flowise Version 2.1.1

Thank you for your time.

[bendadaniel]

I found out, that when user manually upload file via UI then there are 2 api calls in background, first call is 'upsert api' call and then "my" prediction api call. So my problem would be solved if I replicate it, right?

[bendadaniel]

Okey it is also not posible because I found out that internal chatId in embed is generated as:

FlowiseAI/FlowiseChatEmbed:Bot.tsx::260

const [chatId, setChatId] = createSignal(
  (props.chatflowConfig?.vars as any)?.customerId ? `${(props.chatflowConfig?.vars as any).customerId.toString()}+${uuidv4()}` : uuidv4(),
);

So there is no way how to get the chatId of the current conversation outside of embed. So I cant programmatically write/upload/upsert anything to this specific conversation via direct API call.

Do you have any idea how I can achieve my goal?

[tijani2020]

I face the same issue

[bendadaniel]

I temporary solved this problem byt forking FlowiseChatEmbed and adding a new 'chatId change' observer. Something like:

Bot.tsx::292

typeof observeChatId === 'function' &&
  createMemo(() => {
     observeChatId(chatId());
   });

this allows to get chatId outside of embed when chatId is initiated or changed. So now I can do normal upsert to specific chat outside of embed. I think this similar observer should be in embed's code by default, because it opens new oportunity for more custom features. @HenryHengZJ

[vitusan]

Facing the same issue here

[HenryHengZJ]

We have updated the docs here: https://docs.flowiseai.com/using-flowise/uploads#files

Try following the API examples from the docs, if its still not working, feel free to reopen the issue.

Make sure you are on the latest version of FW

[marcochang1028]

I've encountered an issue when I tried to follow the url below to call full file upload api
https://docs.flowiseai.com/using-flowise/uploads#full-file-uploads

When uploading a file by providing its base64 encoded content, the API does not seem to convert the base64 string into a file or extract its content before passing it to the language model. As a result, the api response always indicates that it received a base64 text string and is unable to read the file content.

Even after changing the prefix from data:text/plain to data:application/pdf, the response is the same.