Perpetual queue for uploading a document to the knowledge base

[larkinmd7]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.13.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

previously uploaded documents, they were uploaded immediately and were available within 1-2 minutes. Now for some reason the documents are loaded immediately, but are constantly in the queue, without changes, although the document is the same, the settings are the same, I do not understand why this happens, what should I check to fix this bug?

✔️ Expected Behavior

❌ Actual Behavior

[dosubot]

I found some similar threads that might help you resolve the issue of documents remaining perpetually queued in the Dify knowledge base:

1. Documents in queue: This issue was resolved by modifying the .env file to set the LOG_FILE path, which helped in diagnosing the problem ^[1].

2. Queuing doesn't stop at Dify: This is an open discussion where users are experiencing similar issues with documents being stuck in the queue ^[2].

3. 我添加的知识库的文件一直是在排队中是怎么回事啊？: Another open discussion about files remaining in the queue ^[3].

To address your issue, consider the following steps:

• Rate Limit for Embedding Model API Key: Check if the rate limit for your Embedding model API key has been reached, as this can cause documents to remain queued ^[4].

• Celery Worker Service: Ensure the Celery worker service is running correctly. You can start it with docker compose up -d worker and check the logs for any errors ^[4].

• Service Status: Verify that all necessary services are running using docker compose ps. Ensure services like api, worker, web, weaviate, db, redis, and nginx are all up ^[4].

• Pending Processes and Configuration: Check for any pending segmentation or embedding processes, and ensure all dependencies and services like db and redis are running smoothly ^[4].

These steps should help resolve the issue of documents being stuck in the queued state.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[crazywoola]

Can you attach those documents?