Skip to content

refactor: Move file upload and compression to backend#1383

Merged
edgehog-app[bot] merged 2 commits intoedgehog-device-manager:mainfrom
ArnelaL:move-file-upload-to-backend
Apr 27, 2026
Merged

refactor: Move file upload and compression to backend#1383
edgehog-app[bot] merged 2 commits intoedgehog-device-manager:mainfrom
ArnelaL:move-file-upload-to-backend

Conversation

@ArnelaL
Copy link
Copy Markdown
Collaborator

@ArnelaL ArnelaL commented Apr 17, 2026

What this PR does / why we need it:

Previously the frontend would:

  • request presigned PUT/GET URLs,
  • upload directly to storage,
  • then mark records uploaded

Now the backend:

  • receives an Upload (GraphQL Upload) and uploads it to bucket
    storage itself
  • computes digests on the backend
  • generates compressed variants (.gz and .lz4) and stores them too
  • exposes direct URLs (not presigned) via baseFile/gzFile/lz4File
    in GraphQL

based on #1347 #1357 #1398

Additional documentation e.g. usage docs, diagrams, reviewer notes, etc.:


Thanks for sending a pull request! If this is your first time, here are some tips for you:
  1. You can take a look at our developer guide for an introduction on Edgehog development!
  2. Make sure to read CONTRIBUTING.md and CODE_OF_CONDUCT.md
  3. If the PR is unfinished or you're actively working on it, mark it as draft

When fixing existing issues, use github's syntax to link your pull request to it

fixes #<issue number>

We also have a syntax to signal dependencies to other open pull requests

depends on #<pr number>
depends on https://github.com/...

In case of stacked PRs, you may add the PR number in the last commit's title instead:

gitGraph
    commit id: "Current master"
    branch feat1
    checkout feat1
    commit id: "feat: add something"
    commit id: "feat: add something else (#100)"
    branch feat2
    checkout feat2
    commit id: "refactor: do something"
    commit id: "fix: solve issue"
    commit id: "feat: add a feature (#101)"
    branch feat3
    checkout feat3
    commit id: "feat: feat without pr number"
Loading

@ArnelaL ArnelaL force-pushed the move-file-upload-to-backend branch 30 times, most recently from 98ae65d to dccc5bd Compare April 22, 2026 11:27
@ArnelaL ArnelaL force-pushed the move-file-upload-to-backend branch 8 times, most recently from 4c2e281 to 7890975 Compare April 23, 2026 07:34
@ArnelaL ArnelaL changed the title Move file upload to backend refactor: Move file upload and compression to backend Apr 23, 2026
@ArnelaL ArnelaL force-pushed the move-file-upload-to-backend branch 3 times, most recently from 5c23ba4 to 4626646 Compare April 24, 2026 06:47
@ArnelaL ArnelaL marked this pull request as ready for review April 24, 2026 06:56
Copy link
Copy Markdown
Collaborator

@lusergit lusergit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this is great 🦸‍♀️

I have a couple of nits, but once these are solved this sounds good to me

Comment thread backend/lib/edgehog/files/file/changes/handle_file_upload.ex Outdated
Comment thread backend/lib/edgehog/files/file/changes/handle_file_upload.ex Outdated
@ArnelaL ArnelaL force-pushed the move-file-upload-to-backend branch 2 times, most recently from 1422b80 to 9118037 Compare April 27, 2026 10:55
Copy link
Copy Markdown
Collaborator

@OmarBrbutovic OmarBrbutovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍽️

After updating ash dependencies, new functionality was added to
automatically clean up old snapshots and extensions

Used command:
```
mix ash.codegen automated_cleanup
```

Signed-off-by: Omar <omar.brbutovic@secomind.com>
@ArnelaL ArnelaL force-pushed the move-file-upload-to-backend branch from 9118037 to 8171e55 Compare April 27, 2026 12:31
Previously the frontend would:

- request presigned PUT/GET URLs,
- upload directly to storage,
- then mark records uploaded

Now the backend:

- receives an `Upload` (GraphQL `Upload`) and uploads it to bucket
storage itself
- computes digests on the backend
- generates compressed variants (`.gz` and `.lz4`) and stores them too
- exposes direct URLs (not presigned) via `baseFile/gzFile/lz4File`
in GraphQL

Signed-off-by: ArnelaL <arnela.lisic@secomind.com>
@ArnelaL ArnelaL force-pushed the move-file-upload-to-backend branch from 8171e55 to 1c47da0 Compare April 27, 2026 12:32
@lusergit
Copy link
Copy Markdown
Collaborator

/fast-forward

@edgehog-app edgehog-app Bot merged commit 1c47da0 into edgehog-device-manager:main Apr 27, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants