Extract progress threshold as configurable parameter in MemoryStreamUploadHandler#1328
Merged
Merged
Conversation
…treamUploadHandler Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Address feedback on upload system overhaul
Extract progress threshold as configurable parameter in MemoryStreamUploadHandler
Nov 3, 2025
rorychatt
pushed a commit
that referenced
this pull request
Nov 4, 2025
…ess tracking, and chunked audio support (#1320) * feat(upload): add FileUpload type and enhance file metadata Refactored upload handling to use a new FileUpload record, enabling access to file metadata (name, content type, size, stream) instead of just raw bytes. Updated FileInput and related APIs for richer file info and upload state tracking. Updated docs, tests, and backward-compatible overloads. * feat(file-input): add progress and state to FileInput model Update FileInput and FileBase to include progress and state properties, making them mutable for upload tracking. Sync backend and frontend models, and enhance the UI with a progress bar during file uploads. * feat(file-input): add clear handler support to FileInput Add `OnClear` event and `HandleClear` extensions for FileInput. UI now shows a clear button when a clear handler is provided. Refactor server control to manage file state; remove OnChange. Update sample apps and tests for new clear semantics. * refactor(file): remove LastModified from FileInput/FileBase Eliminates the LastModified property from FileBase and FileInput models, updates constructors and usage accordingly, and removes related test cases and frontend references. Also renames FuncBuilder to FuncViewBuilder for clarity. Adds FuncBuilder for table customizations. * feat(file-input): add support for file delete event Add OnDelete event handler to FileInput and UI. Allows users to remove individual files from multi-file upload, with backend and frontend support for the new delete action. Also improves file input validation and layout. * refactor(state): add atomic Set and Default methods Implement thread-safe Set(Func) and Default methods for IState<T> and ConvertedState<TFrom, TTo>. Refactor usages to ensure atomic state updates, improving reliability in concurrent scenarios. * refactor(file-input): replace Clear with Delete and State with Status Rename FileInputState to FileInputStatus and remove clear event/handler in favor of delete. Update related usages in backend and frontend. * feat(upload): add cancellation support for file uploads Enable file upload cancellation via CancellationToken and abort handling. FileInput now supports cancellation and implements IDisposable. * refactor(file): replace FileInput with FileUpload model Replaces FileInput and FileBase with new FileUpload record and FileUploadStatus enum. Updates all usages, validation, and APIs to use FileUpload. Refactors file upload handling to use async delegates with stream and cancellation support. Updates docs and tests for new model. * refactor(file-input): use Guid for FileUpload.Id and OnDelete Update FileUpload.Id to Guid and update related OnDelete handlers and usages to use Guid instead of object. Cleans up type safety and clarifies ownership of file identifiers. * refactor(docs): remove async keyword from UseUpload handlers Replaces unnecessary async lambdas with synchronous handlers returning Task.CompletedTask in UseUpload examples for clarity and consistency. Also removes commented-out unused upload handler code. * feat(upload): add IUploadHandler for pluggable upload logic Introduce IUploadHandler interface and MemoryStreamUploadHandler to enable custom upload logic. Refactor UseUpload to accept IUploadHandler, improving extensibility and code clarity. * feat(services): add MultiFileUploadHandler stub implementation Introduce MultiFileUploadHandler class with a stubbed HandleUploadAsync method to support future multi-file uploads. * feat(upload): add detailed upload progress and logging Add console logs for upload start, progress, finish, abort, and failures. Improve State<T> to avoid unnecessary notifications on unchanged state. Reset file input after upload/delete to allow re-selecting the same file. * feat(uploads): add generic FileUpload<T> and upload cancelation Introduce FileUpload<T> for typed file content, unify IFileUpload usage, and add upload cancelation support in UploadService. Refactor FileInput and validation logic to support generic and non-generic uploads. * refactor(file-input): rename delete handlers to cancel Rename file input delete event handlers and related methods to cancel for clarity and consistency in handling file upload cancellation. * feat(core): add per-session event dispatch queues Introduce dedicated EventDispatchQueue for each app session to handle UI and event bursts without consuming ThreadPool threads. Refactor upload APIs to use UploadContext for cancellation support. Improve server scalability and responsiveness under heavy load. * fix(file-input): handle undefined events in cancel handler Prevent runtime errors by checking if events is an array before calling includes in FileInputWidget. * feat(upload): support multi-file state in MemoryStreamUploadHandler Refactor MemoryStreamUploadHandler to support both single and multiple file upload states using an internal sink abstraction. Adds Create factory methods for easy usage with single or ImmutableArray state. Stream reading logic is now centralized, avoiding duplication and preserving compatibility with existing IUploadHandler and UseUpload interfaces. Also updates UI to add dialog-based file upload and fixes motion component usage in frontend. * fix(upload): improve file cancel logic and tab event handling Enhances file input cancelation to properly update state for multi-file uploads, and refactors Tabs widget to use a safe event handler for tab actions. Also adds variant support for UploadApp tabs. * refactor(file-input): require UploadContext in ToFileInput BREAKING CHANGE: ToFileInput now requires an UploadContext parameter for file upload handling. The previous overload is obsolete and will throw an exception. Update usages to provide an UploadContext from UseUpload. * feat(widgets): add upload context support to FileInput demos Refactor FileInput demos to use UseUpload context, enabling custom upload handling for both single and multiple file inputs. Also remove unused multi-file upload planning docs. * Add ScaffoldColumnAttribute filtering to DataTable and Form builders * feat(upload): improve progress reporting & update UI details Refine upload progress reporting to trigger updates every 5%. Remove file content from upload details display. Add debug logging for file upload state changes. Annotate Content property to hide from UI scaffolding. * feat(file-upload): add max file size validation support Add maxFileSize prop to FileInput components and widgets, enabling file size validation on client, server, and widget layers. Update UploadService and validation logic to enforce maximum file size. Remove debug logging from upload handlers. * feat(upload): set file type and size limits on uploads Apply Accept(*/*) and MaxFileSize(10MB) to upload handlers for single, dialog, and multiple file uploads. Removes redundant Accept on file inputs. * Refactor event handling to support asynchronous invocation * feat(upload): add file upload validation tab and logic Introduce file upload validation with configurable max size, file count, and accepted types. Refactor event trigger to async in WidgetTree. * feat(upload): add max files support to file upload Add MaxFiles property and validation to allow limiting the number of files that can be uploaded. Also add new Form tab and model to demonstrate file upload in forms. * feat(upload): improve file input UX and validation settings Refactor file upload validation to support optional accept types, enforce max files, and update frontend UI for better empty state and file list display. * refactor(upload): generalize upload sinks and handler Refactors MemoryStreamUploadHandler to support generic content types and moves IFileUploadSink, SingleFileSink, and MultipleFileSink to UploadService. Also renames CancelUpload to Cancel for consistency. * refactor(upload): split handler into static factory and impl Separate MemoryStreamUploadHandler into a static factory class and an internal implementation class for improved structure and clarity. * fix(upload): rethrow OperationCanceledException after abort Ensure OperationCanceledException is rethrown after calling _sink.Aborted to preserve proper cancellation handling. Also remove unused using directive. * feat(forms,upload): improve file upload UX and error handling - Add upload context support to FormBuilder and ToForm extension - Require UploadContext for forms with FileUpload fields - Show toast notifications for file size/type errors (server & client) - Limit file selection and show toast if max files exceeded in FileInputWidget - Refactor file upload validation for better feedback and consistency * refactor(upload): make UploadContext state non-nullable Refactor all usages of IState<UploadContext?> to IState<UploadContext> to simplify upload state management and remove unnecessary null checks. * fix(article): update nielsbosma title to Founder Changed team member title from CEO to Founder for accuracy. * fix(file-input): enforce maxFiles limit across uploads Ensure maxFiles limit accounts for already uploaded files in file input widget. Improves user feedback and prevents exceeding allowed file uploads. * Enhance file upload handling and details representation in forms * refactor(forms): update input factory to accept view context Refactors form input factory signatures to accept both state and view context, enabling context-aware field customization. Removes UploadContext from FormBuilder constructor and ToForm extension. Updates related interface and builder methods for improved flexibility and future extensibility. * feat(docs,widgets): add chunked audio upload & enhance file upload docs - Add `ChunkedMemoryStreamUploadHandler` for accumulating audio stream chunks - Refactor `AudioRecorder` to take an `UploadContext` for automatic uploads - Update docs for FileInput and AudioRecorder with chunked upload, progress, validation, and new usage patterns - Add new examples and best practices for advanced upload scenarios - Update samples to demonstrate chunked audio and delayed file upload handling * feat(forms): block form submission while file uploads are in progress - Add CheckForLoadingUploads() method to recursively detect uploading files - Disable submit button when uploads are in progress - Show toast notification if user tries to submit during upload - Add FormUploadBlockingDemo sample app to demonstrate feature - Automatically tracks FileUpload and FileUpload<T> status across model * fix(apphub): remove unnecessary async keyword from OnWidgetTreeChanged The method doesn't actually await anything, so the async keyword was causing CS1998 warning. Changed from 'async void' to 'void'. * test(ConvertJsonNodeTests): update JSON keys for file fields Renamed JSON properties in test from name/size/type to fileName/length/contentType to match updated schema. * Update Ivy/Core/Hooks/State.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update Ivy/Core/EventDispatchQueue.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Replace Thread.Sleep with Task.Delay in event queue update coalescing (#1327) * Initial plan * refactor(apphub): replace Thread.Sleep with Task.Delay in update coalescing Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com> * Extract progress threshold as configurable parameter in MemoryStreamUploadHandler (#1328) * Initial plan * feat: extract progress threshold as configurable parameter in MemoryStreamUploadHandler Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com> * Refactor file upload handlers and update labels for attachments * fix(server): inject ivy-host meta tag for file uploads in dev mode Completes the implementation from PR #766 which added frontend support for the ivy-host meta tag but never added the backend code to inject it. Without this meta tag, the FileInputWidget and AudioRecorderWidget send POST requests to the Vite dev server instead of the backend API, causing 405 Method Not Allowed errors. The tag is only injected in DEBUG mode since production serves frontend and backend from the same origin. Fixes file uploads broken since commit d6adb2b (Sept 5, 2025). * fix(upload): add /upload to excluded paths to prevent routing interference The PathToAppIdMiddleware was rewriting upload requests from: /upload/{connectionId}/{uploadId} to: /?appId=upload/{connectionId}/{uploadId} This caused the UploadController endpoint to never be reached, resulting in 405 Method Not Allowed errors. By adding /upload to the excludedPaths in routing-constants.json, the middleware now skips /upload routes and allows them to reach the UploadController properly. Fixes file upload 405 errors that have existed since the upload endpoint was introduced. * Remove debug ivy-host meta tag injection from Server.cs * Remove delayed upload handler for file uploads --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com>
rorychatt
pushed a commit
that referenced
this pull request
Dec 18, 2025
…ess tracking, and chunked audio support (#1320) * feat(upload): add FileUpload type and enhance file metadata Refactored upload handling to use a new FileUpload record, enabling access to file metadata (name, content type, size, stream) instead of just raw bytes. Updated FileInput and related APIs for richer file info and upload state tracking. Updated docs, tests, and backward-compatible overloads. * feat(file-input): add progress and state to FileInput model Update FileInput and FileBase to include progress and state properties, making them mutable for upload tracking. Sync backend and frontend models, and enhance the UI with a progress bar during file uploads. * feat(file-input): add clear handler support to FileInput Add `OnClear` event and `HandleClear` extensions for FileInput. UI now shows a clear button when a clear handler is provided. Refactor server control to manage file state; remove OnChange. Update sample apps and tests for new clear semantics. * refactor(file): remove LastModified from FileInput/FileBase Eliminates the LastModified property from FileBase and FileInput models, updates constructors and usage accordingly, and removes related test cases and frontend references. Also renames FuncBuilder to FuncViewBuilder for clarity. Adds FuncBuilder for table customizations. * feat(file-input): add support for file delete event Add OnDelete event handler to FileInput and UI. Allows users to remove individual files from multi-file upload, with backend and frontend support for the new delete action. Also improves file input validation and layout. * refactor(state): add atomic Set and Default methods Implement thread-safe Set(Func) and Default methods for IState<T> and ConvertedState<TFrom, TTo>. Refactor usages to ensure atomic state updates, improving reliability in concurrent scenarios. * refactor(file-input): replace Clear with Delete and State with Status Rename FileInputState to FileInputStatus and remove clear event/handler in favor of delete. Update related usages in backend and frontend. * feat(upload): add cancellation support for file uploads Enable file upload cancellation via CancellationToken and abort handling. FileInput now supports cancellation and implements IDisposable. * refactor(file): replace FileInput with FileUpload model Replaces FileInput and FileBase with new FileUpload record and FileUploadStatus enum. Updates all usages, validation, and APIs to use FileUpload. Refactors file upload handling to use async delegates with stream and cancellation support. Updates docs and tests for new model. * refactor(file-input): use Guid for FileUpload.Id and OnDelete Update FileUpload.Id to Guid and update related OnDelete handlers and usages to use Guid instead of object. Cleans up type safety and clarifies ownership of file identifiers. * refactor(docs): remove async keyword from UseUpload handlers Replaces unnecessary async lambdas with synchronous handlers returning Task.CompletedTask in UseUpload examples for clarity and consistency. Also removes commented-out unused upload handler code. * feat(upload): add IUploadHandler for pluggable upload logic Introduce IUploadHandler interface and MemoryStreamUploadHandler to enable custom upload logic. Refactor UseUpload to accept IUploadHandler, improving extensibility and code clarity. * feat(services): add MultiFileUploadHandler stub implementation Introduce MultiFileUploadHandler class with a stubbed HandleUploadAsync method to support future multi-file uploads. * feat(upload): add detailed upload progress and logging Add console logs for upload start, progress, finish, abort, and failures. Improve State<T> to avoid unnecessary notifications on unchanged state. Reset file input after upload/delete to allow re-selecting the same file. * feat(uploads): add generic FileUpload<T> and upload cancelation Introduce FileUpload<T> for typed file content, unify IFileUpload usage, and add upload cancelation support in UploadService. Refactor FileInput and validation logic to support generic and non-generic uploads. * refactor(file-input): rename delete handlers to cancel Rename file input delete event handlers and related methods to cancel for clarity and consistency in handling file upload cancellation. * feat(core): add per-session event dispatch queues Introduce dedicated EventDispatchQueue for each app session to handle UI and event bursts without consuming ThreadPool threads. Refactor upload APIs to use UploadContext for cancellation support. Improve server scalability and responsiveness under heavy load. * fix(file-input): handle undefined events in cancel handler Prevent runtime errors by checking if events is an array before calling includes in FileInputWidget. * feat(upload): support multi-file state in MemoryStreamUploadHandler Refactor MemoryStreamUploadHandler to support both single and multiple file upload states using an internal sink abstraction. Adds Create factory methods for easy usage with single or ImmutableArray state. Stream reading logic is now centralized, avoiding duplication and preserving compatibility with existing IUploadHandler and UseUpload interfaces. Also updates UI to add dialog-based file upload and fixes motion component usage in frontend. * fix(upload): improve file cancel logic and tab event handling Enhances file input cancelation to properly update state for multi-file uploads, and refactors Tabs widget to use a safe event handler for tab actions. Also adds variant support for UploadApp tabs. * refactor(file-input): require UploadContext in ToFileInput BREAKING CHANGE: ToFileInput now requires an UploadContext parameter for file upload handling. The previous overload is obsolete and will throw an exception. Update usages to provide an UploadContext from UseUpload. * feat(widgets): add upload context support to FileInput demos Refactor FileInput demos to use UseUpload context, enabling custom upload handling for both single and multiple file inputs. Also remove unused multi-file upload planning docs. * Add ScaffoldColumnAttribute filtering to DataTable and Form builders * feat(upload): improve progress reporting & update UI details Refine upload progress reporting to trigger updates every 5%. Remove file content from upload details display. Add debug logging for file upload state changes. Annotate Content property to hide from UI scaffolding. * feat(file-upload): add max file size validation support Add maxFileSize prop to FileInput components and widgets, enabling file size validation on client, server, and widget layers. Update UploadService and validation logic to enforce maximum file size. Remove debug logging from upload handlers. * feat(upload): set file type and size limits on uploads Apply Accept(*/*) and MaxFileSize(10MB) to upload handlers for single, dialog, and multiple file uploads. Removes redundant Accept on file inputs. * Refactor event handling to support asynchronous invocation * feat(upload): add file upload validation tab and logic Introduce file upload validation with configurable max size, file count, and accepted types. Refactor event trigger to async in WidgetTree. * feat(upload): add max files support to file upload Add MaxFiles property and validation to allow limiting the number of files that can be uploaded. Also add new Form tab and model to demonstrate file upload in forms. * feat(upload): improve file input UX and validation settings Refactor file upload validation to support optional accept types, enforce max files, and update frontend UI for better empty state and file list display. * refactor(upload): generalize upload sinks and handler Refactors MemoryStreamUploadHandler to support generic content types and moves IFileUploadSink, SingleFileSink, and MultipleFileSink to UploadService. Also renames CancelUpload to Cancel for consistency. * refactor(upload): split handler into static factory and impl Separate MemoryStreamUploadHandler into a static factory class and an internal implementation class for improved structure and clarity. * fix(upload): rethrow OperationCanceledException after abort Ensure OperationCanceledException is rethrown after calling _sink.Aborted to preserve proper cancellation handling. Also remove unused using directive. * feat(forms,upload): improve file upload UX and error handling - Add upload context support to FormBuilder and ToForm extension - Require UploadContext for forms with FileUpload fields - Show toast notifications for file size/type errors (server & client) - Limit file selection and show toast if max files exceeded in FileInputWidget - Refactor file upload validation for better feedback and consistency * refactor(upload): make UploadContext state non-nullable Refactor all usages of IState<UploadContext?> to IState<UploadContext> to simplify upload state management and remove unnecessary null checks. * fix(article): update nielsbosma title to Founder Changed team member title from CEO to Founder for accuracy. * fix(file-input): enforce maxFiles limit across uploads Ensure maxFiles limit accounts for already uploaded files in file input widget. Improves user feedback and prevents exceeding allowed file uploads. * Enhance file upload handling and details representation in forms * refactor(forms): update input factory to accept view context Refactors form input factory signatures to accept both state and view context, enabling context-aware field customization. Removes UploadContext from FormBuilder constructor and ToForm extension. Updates related interface and builder methods for improved flexibility and future extensibility. * feat(docs,widgets): add chunked audio upload & enhance file upload docs - Add `ChunkedMemoryStreamUploadHandler` for accumulating audio stream chunks - Refactor `AudioRecorder` to take an `UploadContext` for automatic uploads - Update docs for FileInput and AudioRecorder with chunked upload, progress, validation, and new usage patterns - Add new examples and best practices for advanced upload scenarios - Update samples to demonstrate chunked audio and delayed file upload handling * feat(forms): block form submission while file uploads are in progress - Add CheckForLoadingUploads() method to recursively detect uploading files - Disable submit button when uploads are in progress - Show toast notification if user tries to submit during upload - Add FormUploadBlockingDemo sample app to demonstrate feature - Automatically tracks FileUpload and FileUpload<T> status across model * fix(apphub): remove unnecessary async keyword from OnWidgetTreeChanged The method doesn't actually await anything, so the async keyword was causing CS1998 warning. Changed from 'async void' to 'void'. * test(ConvertJsonNodeTests): update JSON keys for file fields Renamed JSON properties in test from name/size/type to fileName/length/contentType to match updated schema. * Update Ivy/Core/Hooks/State.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update Ivy/Core/EventDispatchQueue.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Replace Thread.Sleep with Task.Delay in event queue update coalescing (#1327) * Initial plan * refactor(apphub): replace Thread.Sleep with Task.Delay in update coalescing Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com> * Extract progress threshold as configurable parameter in MemoryStreamUploadHandler (#1328) * Initial plan * feat: extract progress threshold as configurable parameter in MemoryStreamUploadHandler Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com> * Refactor file upload handlers and update labels for attachments * fix(server): inject ivy-host meta tag for file uploads in dev mode Completes the implementation from PR #766 which added frontend support for the ivy-host meta tag but never added the backend code to inject it. Without this meta tag, the FileInputWidget and AudioRecorderWidget send POST requests to the Vite dev server instead of the backend API, causing 405 Method Not Allowed errors. The tag is only injected in DEBUG mode since production serves frontend and backend from the same origin. Fixes file uploads broken since commit d6adb2b (Sept 5, 2025). * fix(upload): add /upload to excluded paths to prevent routing interference The PathToAppIdMiddleware was rewriting upload requests from: /upload/{connectionId}/{uploadId} to: /?appId=upload/{connectionId}/{uploadId} This caused the UploadController endpoint to never be reached, resulting in 405 Method Not Allowed errors. By adding /upload to the excludedPaths in routing-constants.json, the middleware now skips /upload routes and allows them to reach the UploadController properly. Fixes file upload 405 errors that have existed since the upload endpoint was introduced. * Remove debug ivy-host meta tag injection from Server.cs * Remove delayed upload handler for file uploads --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: nielsbosma <4034492+nielsbosma@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses code review feedback to remove magic number
0.05fby making progress reporting granularity configurable.Changes:
progressThresholdparameter toMemoryStreamUploadHandlerImpl<T>constructor (default:0.05f)Create()factory methods to accept optionalprogressThresholdparameterReadAllWithProgressAsync()to use parameter instead of local constantUsage:
Maintains backward compatibility while allowing callers to customize progress reporting frequency based on their UX requirements.
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.