Skip to content

feat: add GET upload file API endpoint to dataset service api#11899

Merged
JohnJyong merged 2 commits intolanggenius:mainfrom
euxx:service-api-dataset-upload-file
Jan 9, 2025
Merged

feat: add GET upload file API endpoint to dataset service api#11899
JohnJyong merged 2 commits intolanggenius:mainfrom
euxx:service-api-dataset-upload-file

Conversation

@euxx
Copy link
Contributor

@euxx euxx commented Dec 20, 2024

Summary

It's related to issues #9547 and #10296

When an app references a Knowledge resource, we want to download the source files linked in the citations section of the resulting chat message:

image

I known Dify team has permission concerns, and this is not a high priority from #9547.

So to keep it simple, this PR adds an API to retrieve upload file infos ( including download URL) from related documents.

The chat-messages API already provides document infos within the metadata - retriever_resources field.

In addition to this PR, we can achieve source file downloads within our custom chat app.

This API also supports broader use cases related to uploaded files, such as downloading individual files or even batch downloading files from a knowledge base.

So this PR could be beneficial to other users.

For code style and consistency, I referenced existing code snippets:

`api/controllers/service_api/dataset/segment.py:72` # Fetch document
`api/controllers/console/datasets/datasets_document.py:373` # Fetch upload_file
`api/controllers/console/remote_files.py:72` # Return fields

Screenshots

Local API test result:

image

Doc preview:

image

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

euxx added 2 commits December 20, 2024 14:48
- Introduced a new UploadFileApi class to handle file uploads associated with datasets.
- Implemented GET method to retrieve upload file details, including validation for dataset and document existence.
- Updated service_api initialization to include the new upload_file module.
- Added documentation for the new GET upload file API endpoint in both English and Chinese.
- Included details on request parameters, example cURL request, and expected JSON response format.
- Enhanced the clarity of API usage for users in both languages.
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. 💪 enhancement New feature or request 📚 documentation Improvements or additions to documentation labels Dec 20, 2024
@crazywoola crazywoola requested a review from laipz8200 December 20, 2024 09:18
@euxx
Copy link
Contributor Author

euxx commented Jan 8, 2025

@laipz8200 Hi, could you please take a look the PR, and let me know if you have any feedback? Thank you.

@crazywoola crazywoola requested a review from JohnJyong January 8, 2025 11:01
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jan 9, 2025
@JohnJyong JohnJyong merged commit 20f0905 into langgenius:main Jan 9, 2025
jsincorporated pushed a commit to jsincorporated/asaAi that referenced this pull request Jul 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📚 documentation Improvements or additions to documentation 💪 enhancement New feature or request lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants