Skip to content

Comments

Feature/table personalization#223

Merged
darinkelkhoff merged 29 commits intodevelopfrom
feature/table-personalization
Sep 9, 2025
Merged

Feature/table personalization#223
darinkelkhoff merged 29 commits intodevelopfrom
feature/table-personalization

Conversation

@darinkelkhoff
Copy link
Contributor

@darinkelkhoff darinkelkhoff commented Sep 9, 2025

Pull Request

📝 Description

What does this PR do?
This PR adds a new concept of "Table Meta Data Personalization" to QQQ.

The concept here is, that an application can provide a code reference on the QInstance, which implements a new TableMetaDataPersonalizerInterface, and all "user-facing" actions executed in the application are then subject to using a cloned QTableMetaData object, which has gone through a level of personalization (customization) specific for the user/session of the request. An implementation is being developed along with this framework capability in qbit-customizable-table-views.

The business goal of this is so that an application's various user roles can be given different views of tables, e.g., "show all fields to admins", "hide some fields from end-users", "make users in Role X be required to put a value in field Foo", "don't allow users in Role Y to edit values in field Bar", and/or, "don't show a given Widget to users in some role".

The goal of the implementation is to remove fields entirely from users - so that they may not insert, update, or query/get values from fields that aren't in their personalized version of a table - and supporting actions (such as column stats, child record list widgets, audits, etc) all respect this personalization as well, in theory, to completely block fields from users who aren't allowed access to them. Yet - at the same time - if a user with a personalized view of a table, for example, runs a process, and that process works with a table (say to insert records with values that the user themselves can't see), the process should still have access to all field. Generally this is expected to be handled within the implementation of TableMetaDataPersonalizerInterface, where personalization would only be applied if inputSource.equals(QInputSource.USER) - but this is not enforced by QQQ.

Secondary changes

  • To remove hidden fields from audits, a new process was written to fetch audits (GetAuditsForRecordProcess).
  • To allow the new GetAuditsForRecordProcess to use the same permission as the audit table's read permission, CustomPermissionChecker and PermissionsHelper were updated (basically custom permissions were less than half-implemented before, but are more complete now).
  • The Filesystem module's file importer process previously relied on the memory backend's ignoring of unrecognized fields. As a mongoDB backend may still work this way (as per the original intent of that process), it is now a setting on processes of this type whether they expect to store their record values in unstructured fields, or in a "values" JSON-blob.

Related Issue:
Closes #[issue_number]

🔍 Type of Change

  • Bug Fix - Fixes an existing issue
  • New Feature - Adds new functionality
  • Breaking Change - Fix or feature that would cause existing functionality to not work as expected
    • This feature does make some subtle changes to some of the core actions of the framework, such as specifically:
      • QueryAction throwing an exception if given an unrecognized criteria fieldName
      • MemoryBackendModule stripping values from records if they aren't in the (active) QTableMetaData.
      • Table action input objects now store a QTableMetaData object, rather than always getting it from the QInstance. This can cause subtle changes if a table meta data object is modified on a re-used input object (an anti-pattern, but one that some tests were doing).
  • Documentation Update - Updates documentation
  • Refactoring - Code change that neither fixes a bug nor adds a feature
  • Performance Improvement - Code change that improves performance
  • Test Addition - Adding missing tests or correcting existing tests

🧪 Testing

How has this been tested?

  • Unit Tests: All unit tests pass
  • Integration Tests: Integration tests pass
  • Coverage: Meets QQQ's coverage requirements (80% instructions, 95% classes)
  • [x\ Manual Testing: Tested manually in development environment

Test Commands:

# Commands used for testing
mvn test
mvn verify

📋 Checklist

Before submitting this PR, please ensure:

  • Code Style: Follows QQQ's Code Review Standards
  • Tests: All tests pass with required coverage
  • Documentation: Updated relevant wiki pages and code comments
  • Breaking Changes: Documented any breaking changes
  • Commit Messages: Follow conventional commit format
  • Self Review: Code has been reviewed by the author

🔗 Related Resources

Documentation:

📊 Additional Information

Any additional information that reviewers should know:

  • Screenshots (if applicable)
  • Performance impact
  • Migration notes
  • Dependencies added/removed

Thank you for contributing to QQQ! 🚀

…roughout core actions; tests on the same, along with a new ExamplePersonalizer that tests can use
…ng values for fields that aren't in the table.

BREAKING CHANGE:  Previously the memory backend was very loose, and just stored QRecords without regard to what fields were in them, which applications may have taken advantage of, to store values that aren't fields in the table.  But, with the advent of table personalization, and the notion that a backend should not return values that aren't part of the active user's table, the memory backend now removes any field/value pairs from records that are not valid fields in the table.
…personalization - namely, around stricter behavior in memory backend re: unrecognized fields.
… validation to CountAction

BREAKING CHANGE:  application code that supplied unrecognized field names to count actions in the past used to be up to the specific backend module what to do with them (e.g., RDBMS would fail, but memory would silently ignore).  Now, before we get into the backend module, we will validate field names in criteria and order by elements of the filter.  This feels like the right thing to always do, and good to be standardized - but it is a (breaking) change in behavior.
BREAKING CHANGE:  application code that supplied unrecognized field names to query actions in the past used to be up to the specific backend module what to do with them (e.g., RDBMS would fail, but memory would silently ignore).  Now, before we get into the backend module, we will validate field names in criteria and order by elements of the filter.  This feels like the right thing to always do, and good to be standardized - but it is a (breaking) change in behavior.
…ware-api module.

Includes:
- applying table personalization throughout api implementation class (e.g., for application-api calls made by users)
- applying table personalization to api-aware middleware executors for count, query, and tableMeta actions (the only table-based ones that exist at this point) - for application-api-version-aware calls made by qqq frontends.
- this involved refactoring GetTableApiFieldsAction's static (caching) wrapper methods to take GetTableApiFieldsInput, which now has an InputSource field - expected to be used personalization implementations (e.g., to only personalize USER-sourced actions).
…hroughout actions called by middleware-javalin, in general, by setting USER inputSource on actions that didn't have it.
…robably should have been used all along.

It looks like it was previously only used for processes - now it will correctly be used for tables, apps, widgets, reports - all the things.
…stance's available permissions.

This involves adding 3 new default methods in CustomPermissionChecker:  handlesBuildAvailablePermission, buildAvailablePermission, and buildBaseAvailablePermission
…Checker interface: UseOtherPermissionNameCustomPermissionChecker and UseTablePermissionCustomPermissionChecker
…able personalization, to not show fields to a user who isn't allowed to see them.
…from feature/improve-bulk-edit-dependencies)
…unknown fields and to re-set table object in QueryInput after table meta data is modified.
…ends re: unrecognized field names (pursuant to table personalization), make FilesystemImporter process expose a field that specifies if unstructured values should remain in record, or only be stashed in values json blob.
@github-actions
Copy link

github-actions bot commented Sep 9, 2025

🔒 Security Scan Complete

Dependency vulnerability scan completed for this PR. View full report

… field name handling) to AggregateAction

This included refactoring getUnrecognizedFieldNames into its own reusable method in QueryAction, and marking AggregateInput as implementing QueryOrCountInputInterface
…sorting, in case a PossibleValue has null label.
@github-actions
Copy link

github-actions bot commented Sep 9, 2025

🔒 Security Scan Complete

Dependency vulnerability scan completed for this PR. View full report

@github-actions
Copy link

github-actions bot commented Sep 9, 2025

🔒 Security Scan Complete

Dependency vulnerability scan completed for this PR. View full report

@darinkelkhoff darinkelkhoff merged commit 0946212 into develop Sep 9, 2025
15 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant