Bump jsoup from 1.8.3 to 1.15.3 in /dora/underfs/web #17386
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![dependabot[bot]](https://avatars.githubusercontent.com/in/29110?s=60&v=4){kind=small}
### What changes are proposed in this pull request? Remove the restriction of UFS for SDK. ### Why are the changes needed? To meet the business requirements. ### Does this PR introduce any user facing changes? No, it doesn't. pr-link: #16570 change-id: cid-cca2be339f42d6035968999f258c04ad76d1b85e
### What changes are proposed in this pull request? Add property key to enable dora client read location polic ### Why are the changes needed? enable dora client read location polic ### Does this PR introduce any user facing changes? addition of property keys pr-link: #16601 change-id: cid-30229fa9aac78359ebdd12b1cb317245016c370c
### What changes are proposed in this pull request? Implement the skeleton of dora work ### Why are the changes needed? ### Does this PR introduce any user facing changes? no pr-link: #16615 change-id: cid-e48f5bd1e4611d5e6b300cf9851fce18adb808c1
### What changes are proposed in this pull request? Add method to get file status. ### Why are the changes needed? We need to get file status in Dora. ### Does this PR introduce any user facing changes? No, it doesn't. pr-link: #16627 change-id: cid-d877d3d1240663afc89bc6d6d079bf75a8751711
### What changes are proposed in this pull request? Please outline the changes and how this PR fixes the issue. ### Why are the changes needed? Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug. ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs 2. addition or removal of property keys 3. webui pr-link: #16632 change-id: cid-6affd9d6cb0a01ea8352bfa7af7731dab49333ae
### What changes are proposed in this pull request? Please outline the changes and how this PR fixes the issue. ### Why are the changes needed? Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug. ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs 2. addition or removal of property keys 3. webui pr-link: #16637 change-id: cid-99bc5802f28799347181313efb9e37b2cd46b186
Use mdtest to get file status of 1000 files, by removing the isFile call, the get status performance can double. pr-link: #16678 change-id: cid-31edf9b34080ddd5595c9ce613ddaa8e29901a66
### What changes are proposed in this pull request? Remove extra get status call in fuse.open() ### Why are the changes needed? Reduce one unneeded get metadata call ### Does this PR introduce any user facing changes? NA pr-link: #16664 change-id: cid-5a9abb7673491737ba376cdf2eeb2a13a1baf63e
### What changes are proposed in this pull request? Remove some overhead in big file read (cache hit/ cache miss) in fuse sdk. ### Why are the changes needed? Improve overall read performance ### Does this PR introduce any user facing changes? NA pr-link: #16670 change-id: cid-d0d32eeebf9039753881b160443791325ccf5dad
### What changes are proposed in this pull request?
Enable default client-side metadata cache (20,000 entries with 40MB
memory).
### Why are the changes needed?
Each fuse.open() has two extra get file status calls which cannot be
provided via kernel metadata cache but can via user space metadata
cache. Enable user space metadata cache can largely improve the overall
fuse FIO small file read performance. S3FS-FUSE has similar settings of
enable user space metadata cache (100,000 entries with 40MB memory) by
default.
FIO benchmarking FUSE SDK against s3-fuse in terms of small file read:
fio --name=sequentialread --rw=read --bs=100k --numjobs=1
--filesize=100k --direct=1 --group_reporting --nrfiles 1000
<google-sheets-html-origin><style type="text/css"><!--td {border: 1px
solid #cccccc;}br {mso-data-placement:same-cell;}--></style>
| S3 FUSE | ALLUXIO FUSE
-- | -- | --
Original | 9000 KiB/s | 1500 KiB/s
Kernal metadata cache (attr_timeout=7200, entry_timeout=7200) | 9000
KiB/s | 2400 KiB/s
User metadata cache | 9000 KiB/s | 1150 MiB/s
### Does this PR introduce any user facing changes?
Yeah, alluxio-fuse mount <..> metadata cache option with default value
changed
-o metadata_cache_size=<size> (Default=\"20000\" (around 40MB memory))
-o metadata_cache_expire=<timeout> (Default=\"no expire\")
pr-link: #16662
change-id: cid-5423f1ab7f37ed89cdaa1ac0e85328478fbd1ce4
Implement basic functionalities for Dora worker. pr-link: #16644 change-id: cid-522ecaf0f7f39915fc8d55a2aa83b5b114355458
### What changes are proposed in this pull request? Fix an NPE in PermissionCheckTest. ### Why are the changes needed? The `mMetricsMaster` was initialized to `null` and the setup was adding a null pointer to the registry. A null pointer should have been rejected by the registry but it was not the case until refactoring was done in #16644. ### Does this PR introduce any user facing changes? No. pr-link: #16734 change-id: cid-7c61bd50e97dba6c211d6fea29f1555d0fa91e16
Set default fuse version to 3 because of easy to use and better performance. One can sudo yum install fuse3 and then alluxio-fuse to be able to enjoy the better performance brought by MAX_IDLE_THREADS=64 (64 threads instead 10 idle threads to execute fuse operations for better performance). pr-link: #16682 change-id: cid-bfc194d628a887b2508e249a2bb583c64540d8f9
### What changes are proposed in this pull request? Add mission proto lock content. ### Why are the changes needed? #16578 Modified the proto file but not the `proto.lock` file. ### Does this PR introduce any user facing changes? No user facing changes. pr-link: #16651 change-id: cid-adbcdf461ceb5f8c43379ff510c3c9ffabcd992f
### What changes are proposed in this pull request? Dora client side affinity ### Why are the changes needed? skip grpc call of the get block location from master ### Does this PR introduce any user facing changes? new configruations pr-link: #16726 change-id: cid-bfda07a61fc0a9ed21e9f47652e973cc4e59ca9b
### What changes are proposed in this pull request? Enable data transmission with Netty. User is able to enable Netty data transmission by setting the following configurations: alluxio.user.netty.data.transmission.enabled=true alluxio.worker.network.netty.channel=nio ### Why are the changes needed? Support Netty data transmission that has higher performance. ### Does this PR introduce any user facing changes? No, it doesn't. pr-link: #16599 change-id: cid-0af83e5128edee32c1512beea6ad79c545ada424
Implement `getChannel`. The implementation does not yet enable zero-copy transfer with netty. pr-link: #16742 change-id: cid-a2ecf8d06c0c82f56c259f45e2348f6a90d0eeb1
### What changes are proposed in this pull request? A temporary solution for metadata caching on worker ### Why are the changes needed? reduce the api call to ufs ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including add properties pr-link: #16732 change-id: cid-28a01ea4081202b80a31393e097df3f59530e821
In the async page store, when worker threads run out, the caller thread should perform the action. The test had a race condition where all the threads could block, including the calling thread. pr-link: #16731 change-id: cid-81d809c343968d68b57988978575d8b9e4ace63a Original commit 7a4e0a5 pr-link: #16797 change-id: cid-c0113006a76824e1cef41c6f44820e344e047ebe
Add a property key to enable or disable dora metadata cache `alluxio.dora.client.metadata.cache.enabled=true`. Added full fuse tests with dora data cache only under `alluxio.client.fuse.dor`a. Added read only from UFS FUSE tests with dora data cache and metadata cache `alluxio.client.fuse.dora.readonly`. Currently the fuse related tests are duplicated copies with small modification, will refactor the test cases to avoid code copying in the future pr-link: #16809 change-id: cid-95f08387cf221656811a503f3cfeb73fc7dd07c7
Remove Alluxio-fuse script. Rename alluxio-fuse-sdk script to alluxio-fuse script. Modify the generate tarball logics to match the changes pr-link: #16814 change-id: cid-4f5274c276bb75f1f353a6f36af7263eb86f3ed8
### What changes are proposed in this pull request? Implement local cache invalidate method ### Why are the changes needed? support ttl ### Does this PR introduce any user facing changes? no pr-link: #16783 change-id: cid-2b1b606be2b4e1b756a813b0757144d6175f7b58
Signed-off-by: Huang Hua <[email protected]> ### What changes are proposed in this pull request? Basic framework of metadata store for Dora ### Why are the changes needed? Dora needs to persist some file/dir metadata ### Does this PR introduce any user facing changes? A new PropertyKey "alluxio.dora.worker.meta.store.rocksdb.dir" is added. pr-link: #16735 change-id: cid-e39150d1ea202ccd926e29aaa94d6f3844aaa47d
### What changes are proposed in this pull request? Fix caching not working in dora worker. ### Why are the changes needed? Two bugs are fixed: 1. Page store in Dora worker was not created with the worker specific property keys. Keys for client local cache was used, so the settings like cache dir location and sizes did not have effect. 2. `DoraCacheFileSystem.open` did not honor client side read default options, and was using hard-coded defaults from gRPC definition, which is `NO_CACHE`. ### Does this PR introduce any user facing changes? No. pr-link: #16837 change-id: cid-f2614d63e05a2f616bc09ab907e26f2e233934d4
### What changes are proposed in this pull request? Fix bug that paged block store can neither write nor cache after reading. Please outline the changes and how this PR fixes the issue. Fix the bug by using PagedBlockStore's method to build BlockReader. ### Why are the changes needed? Paged block store cannot be used if we don't fix this bug. ### Does this PR introduce any user facing changes? No, it doesn't. pr-link: #16785 change-id: cid-8f51e0eb2acf59ecd49872ed86abd643b7cd72db
### What changes are proposed in this pull request? Please outline the changes and how this PR fixes the issue. ### Why are the changes needed? Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug. ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs 2. addition or removal of property keys 3. webui pr-link: #16859 change-id: cid-084a3d5e779bda5bad932a5af331d78ee50222d2
This reverts commit b391d2d.
### What changes are proposed in this pull request?
```
re-org the dora directories:
/common/transport
/dora/integration
/job
/shaded
/stress
/table
/generated
/underfs
/pom.xml
/core
/examples
/shell
/minicluster
/tests
/lib
/microbench
```
### Why are the changes needed?
Re-org directories to indicate dora explicit
### Does this PR introduce any user facing changes?
NA
pr-link: #17355
change-id: cid-e762b1ce623383e02a327f8da9ff2662f9f2d9b2
move non-java integration folders to alluxio/integration from alluxio/dora/integration pr-link: #17364 change-id: cid-58206656f527b4067c15c2ef533f5b5c3e13b62d
Cherry-pick of existing commit. orig-pr: #16597 orig-commit: 00da77c orig-commit-author: Tyler Crain <[email protected]> pr-link: #17361 change-id: cid-64723be309bdb14b05613864af3b6a1bb30cba6d
### What changes are proposed in this pull request? Make some changes for extension. ### Why are the changes needed? Make extending PropertyKeys doable ### Does this PR introduce any user facing changes? NA pr-link: #17371 change-id: cid-d98ee5814929be71bff6d2a405871691ebeffa01
### What changes are proposed in this pull request? Prepare default options for ListStatus and merge it with the existing one from user. ### Why are the changes needed? So the configurations can be passed from client/user to worker. For example, user can use the following command to refresh the root dir: ``` bin/alluxio fs -Dalluxio.user.file.metadata.sync.interval=0 ls / ``` the `alluxio.user.file.metadata.sync.interval=0` must be added to the listStatus option. ### Does this PR introduce any user facing changes? N/A pr-link: #17374 change-id: cid-21fb461d32bc42b7885f94e56cddb163a0c44530
the dockerfile at dev/github/Dockerfile-jdk11 defines the build environment with maven 3.8.6. update the `maven.version` property in pom.xml to match pr-link: #17367 change-id: cid-3253ac2ad693318a9c378745b1c4ce1d6d3be0f1
not sure how this `java.version` property in the pom.xml affects the build after changing everything else to java 11. referenced as a configuration value for the `maven-compiler-plugin` and `maven-enforcer-plugin` pr-link: #17366 change-id: cid-41575a9cf0b7a80ec67c0915400c5086a56a8528
### What changes are proposed in this pull request? Create interface for security ### Why are the changes needed? Create interface for security ### Does this PR introduce any user facing changes? NA pr-link: #17375 change-id: cid-6c8b02ba5b492e249c3d1a72436b17fba212c96f
Bumps [jsoup](https://github.com/jhy/jsoup) from 1.8.3 to 1.15.3. - [Release notes](https://github.com/jhy/jsoup/releases) - [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES) - [Commits](jhy/jsoup@jsoup-1.8.3...jsoup-1.15.3) --- updated-dependencies: - dependency-name: org.jsoup:jsoup dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]>
dependabot
bot
added
dependencies
|
Thank you for your pull request. |
apc999
force-pushed
the
main
branch
2 times, most recently
from
2fec0ec to
b597c61
Compare
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in two weeks if no further activity occurs. Thank you for your contributions. |
github-actions
bot
added
the
stale
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Bumps jsoup from 1.8.3 to 1.15.3.
Release notes
Sourced from jsoup's releases.
Changelog
Sourced from jsoup's changelog.
... (truncated)
Commits
c596417[maven-release-plugin] prepare release jsoup-1.15.3d2d9ac3Changelog for URL cleaner improvement4ea768dStrip control characters from URLs when resolving absolute URLs985f1feInclude help link for malformed URLs6b67d05Improved Validate error messages653da57Normalized API doc link5ed84f6Simplified the Test Server startupc58112aSet the read size correctly when cappedfa13c80Added jar manifest default implementation entries.5b19390Bump maven-resources-plugin from 3.2.0 to 3.3.0 (#1814)You can trigger a rebase of this PR by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot mergewill merge this PR after your CI passes on it@dependabot squash and mergewill squash and merge this PR after your CI passes on it@dependabot cancel mergewill cancel a previously requested merge and block automerging@dependabot reopenwill reopen this PR if it is closed@dependabot closewill close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)You can disable automated security fix PRs for this repo from the Security Alerts page.