Skip to content

[To dev/1.3] Fixed the schema cache calculation 2 #16832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 937 commits into
base: master
Choose a base branch
from
Open

[To dev/1.3] Fixed the schema cache calculation 2 #16832

wants to merge 937 commits into from
+169,058 −46,507

Conversation

@Caideyipi
Copy link
Collaborator

@Caideyipi Caideyipi commented Nov 28, 2025

Description

As the title said.

This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.
Key changed/added classes (or packages if there are too many classes) in this PR

Wei-hao-Li and others added 30 commits June 16, 2025 21:59
@Wei-hao-Li
[To dev/1.3]Support order by time for Agg align by device query with …
236c098 
…cross region data
@JackieTien97
[To dev/1.3] Fix timeout didn't pass to client bug (#15743)
b33fc38 
(cherry picked from commit 232e0c9)
@HxpSerein
fix datanode check (#15730) (#15737)
0dd2f8c
@CRZbulabula @liyuheng55555
[To dev/1.3] Enhance "remove region" sql to handle the case of DataNo…
d799098 
…de not exist (#15744)

* Enhance "remove region" sql to handle the case of DataNode not exist (#15728)

* finish

---------

Co-authored-by: Li Yu Heng <[email protected]>
@Wei-hao-Li
[To dev/1.3]Fix TypeProvider and AggregationDescriptor serde in Templ…
dc5a36e 
…ated Query
@JackieTien97
[To dev/1.3] Avoiding error log by removing ThreadLocal for Mqtt Client
f68b0e8 
(cherry picked from commit 8a8a7be)
@Caideyipi
Pipe: Fixed the insert node EMA logic for degrading (#15741) (#15758)
3a61340
@luoluoyuyu
Pipe: Delete the extra tags created by the PipeWALInsertNodeCacheMetr…
30fb194 
…ics (#15761) (#15760)
@Pengzna
Conf Template: Fix wrong symbol (#15748)
d976703
@Caideyipi
Pipe: Set WAL to uncompressed when using real-time sync (#15733) (#15757
2699ed1 
)
@liyuheng55555
[to dev/1.3] Fix replication factors of root.__system (#15759)
4a1cf06
@shuwenwei
[to dev/1.3] Add metric for aggregator
9f3972f
@JackieTien97
[To dev/1.3] Fix thread leak in session while connected DN is down
4f6f4c8
@RkGrit
update version ratis from 3.1.3 to 3.2.0-3247c7f-SNAPSHOT (#15763)
85ecde9
@hongzhi-gao
[To dev/1.3] User password encryption upgraded from MD5 to SHA256 (#1…
e6c524e 
…5762)

* Implement high availability for the C++ client and support tsblock

* fix issues on windows

* User password encryption upgraded from MD5 to SHA256. Existing users' passwords will be upgraded upon next login.

* spotless apply
@Caideyipi
Pipe: Fixed the NPE when closing sync connector (#15772) (#15776)
5ec14a0
@DanielWang2035
Pipe: Add retry for tablet batch req to avoid retransmission when mem…
d4605c4 
…ory is insufficient (#15715) (#15771)
@DanielWang2035
Load: Convert to tablets when node is read-only (#15693) (#15774)
25d51fe
@luoluoyuyu
Pipe: Fix the inconsistency between schema and values columns in the …
e480abc 
…process of building tsfile (#15625) (#15778)
@VGalaxies
Subscription: return immediately when unsubscribe empty complete topi…
c77c9bb 
…cs (#15770) (#15781)
@Caideyipi
Pipe: Fixed the NPE caused by the compression timer (#15782) (#15783)
4687f7d
@JackieTien97
[To dev/1.3] Correct Block Queue Time metric
5f6c2e6 
(cherry picked from commit e7d009c)
@shuwenwei
[to dev/1.3] Prevent the contents of log_datanode_sample_query.log an…
ee8b8ef 
…d log_explain_ananlyze.log from being output to log_datanode_all.log (#15801)
@liyuheng55555
[to dev/1.3] Fix the returning state of NotifyRegionMigrationProcedure (
b3deabb 
#15802)
@JackieTien97
[To dev/1.3] Throw CANNOT_FETCH_FI_STATE(722) instead of 301/305 whil…
0626d34 
…e DN restarting
@JackieTien97
[To dev/1.3] Correct status code from 301 to 719 while memory not eno…
84f0465 
…ugh & Record release resource blocked too long by driver (#15829)
@shuwenwei
[to dev/1.3] Parallel disptach read fragment instance
d7c05fa
@HTHou
Reduce default value of rpcMaxConcurrentClientNum (#15260) (#15848)
75ae9cb
@VGalaxies
Subscription: add config to enable or disable subscription (#15820) (#…
1a33299 
…15822)
@XNX02 @SteveYurongSu
[To dev/1.3] Pipe: Add upper bound check for Pipe request decompressi…
b53b1a6 
…on buffer (#15699) (#15806)

Co-authored-by: Steve Yurong Su <[email protected]>
Caideyipi and others added 30 commits November 13, 2025 14:21
@Caideyipi
Pipe: Optimized the default behavior in meta sync for retries to wait…
7c01dfb 
… for other regions (#16740) (#16746)

* partial

* cfg

* core

* fix

* fix
@luoluoyuyu
Load: Fix the Session exit issue after type conversion (#16037) (#16743)
de1649a 
(cherry picked from commit 8a79fa3)

# Conflicts:
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/session/SessionManager.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/converter/LoadTsFileDataTypeConverter.java
@luoluoyuyu
[To dev/1.3] Load: Fixed parameter passing error in MoveFile function (…
583f0fa 
…#16748) (#16749)

* [To dev/1.3] Load: Fixed parameter passing error in MoveFile function

* update
@Caideyipi
[To dev/1.3] Pipe: Trimmed some of the pipe logs (#16747) (#16754)
72a3f45
@luoluoyuyu
[To dev/1.3] feat: encode load attributes in active load directories (#…
f66794b 
…16722) (#16758)
@luoluoyuyu
[To dev/1.3] Active Load: Fixes errors caused by Load copying resourc…
df13612 
…e files. (#16753) (#16757)

(cherry picked from commit db7c801)
@VGalaxies
setup (#16752)
d1270de
@VGalaxies
[To dev/1.3] Pipe: support path exclusion under tree model (#16632) (#…
3d79e36 
…16763)

* setup

* FIN
@shuwenwei
[to dev/1.3] reduce error log when compaction service is stopped & fi…
a60d00e 
…x bugs (#16755)

* reduce error log when compaction service is stopped

* fix writer for compaction tests

* fix npe
@Caideyipi
fix (#16702) (#16772)
f0cdfe0
@luoluoyuyu
[To dev/1.3] Pipe: Modify the Pipe configuration item log name to be …
403cf64 
…consistent with the Properties name. (#16732) (#16759)

* Pipe: Modify the Pipe configuration item log name to be consistent with the Properties name. (#16732)

* Pipe: Modify the Pipe configuration item log name to be consistent with the Properties name.

* update

(cherry picked from commit a6191d9)

* update

* update
@Caideyipi
Set the parameter names from default_storage_group_level to default_d…
78630df 
…atabase_level (#16626) (#16646)
@Caideyipi
[To dev/1.3] Load: Fixed the bug that mods is not deleted in load tsF…
460a570 
…ile when there are exceptions & Fixed the potential NPE in air gap agent close() method #16775 (#16776)
@VGalaxies
[To dev/1.3] Pipe IT: always throw exception with failure when execut…
a719e49 
…ing non-queries & Fixed some semantic errors of IT (#16331) (#16771)

* setup

* spotless

* cont

* try fix

* reset

* reset

* remove TODO
@shuwenwei
[to dev/1.3] Fix slow query threshold & fix a bug for lastQuery & opt…
db77455 
…imize encodeBatch for multi tvlist scene (#16766)
@Caideyipi
[To dev/1.3] Fixed the bugs related to device auto-create alignment i…
90331ae 
…gnorance (#16780) (#16782)

* Fixed the bugs related to device auto-create alignment ignorance (#16780)

* plan-first

* refact

* refa

* fux

* fix

* ref

* partial-fix

* fix

* may-final

* fix

* PBTree

* fix

* gras

* err

* bugfix

(cherry picked from commit 9fd9d7e)

* fix

* Fixed the bug related to "Fixed the bugs related to device auto-create alignment ignorance" (#16781)

* gsa

* ff

* ff

* fix

* fix

* fix

* f9ix
@Caideyipi
[To dev/1.3] Load: Fixed the issue of TSFile parent directory being n…
c7ee924 
…ull and TSFile resource being updated during the Load process. (#16751) (#16790)
@Caideyipi @luoluoyuyu
[To dev/1.3] Load: Remove the getParentFile and exists checks from Lo…
65ebd7c 
…ad New File (#16787)

Co-authored-by: Zhenyu Luo <[email protected]>
@luoluoyuyu
[To dev/1.3] Load : Fixed parameter passing error in MoveFile function (
1db0bc8 
#16748) (#16792)

* Load: Fixed the issue of TSFile parent directory being null and TSFile resource being updated during the Load process. (#16751)

* fix

* fix
@jt2594838
Bump ratis version to 3.2.1 (#16800)
1d267aa
@jt2594838 @HTHou
Avoid connection reset error log printed (#16797) (#16802)
752313e 
Co-authored-by: Haonan <[email protected]>
@HTHou
[To dev/1.3] Update ratis-thirdparty-misc.version to 1.0.10 (#16810)
beb979e
@HTHou
[To dev/1.3] Cherry pick session connection leak fixes (#16808)
05aeab1 
* Fix Session reconnection increase connection number (#15677)

* Fix Session reconnection increase connection number

* Fix Session reconnection increase connection number

* Optimize the implement

* Fix SessionConnection opens too much transport without closing (#15649)
@Caideyipi
Pipe: Adjusted the waiting time of temporary unavailable exceptions (#…
480fc99 
…16798) (#16805)

* fff

* fix

* fix

* fix

* fix

* f
@jt2594838 @zerolbsony
Add a judgment in order to avoid occur NPE when one value of measurem…
eb063f1 
…entSchemas is null (#15574) (#16813)

Co-authored-by: libo <[email protected]>
@Caideyipi
[To dev/1.3] Pipe: Fixed the check for no permission 2 (#16804) (#16815)
9ec13a8 
* Pipe: Fixed the check for no permission 2 (#16804)

* edge

* sonar

* emperor-cloth

* emperor-patch

* cp

* sptls
@Caideyipi
Pipe: Fixed the bug of missing parallel conflict check and receiver l…
50563b6 
…og (#16814) (#16820)
@Caideyipi
Fixed the calculation error of schema cache (#16818) (#16822)
be82810 
* ff

* back-roll
@jt2594838 @HxpSerein
[To dev/1.3] Cherry-pick some IoTConsensus related commits to 1.3 (#1…
f0ec9c4 
…6812)

* more accurate mermory size (#15713)

* Fix stuck when stopping a DataNode with large unremovable WAL files (#15727)

* Fix stuck when stopping a DataNode with large unremovable WAL files

* spotless

* add shutdown hook watcher

* Fix logDispatcher stuck

* add re-interrupt

* Add a multiplier to avoid receiver OOM in IoTConsensus (#16102)

* Fix negative iot queue size & missing search index for deletion & missed request when performing empty table deleting (#16022)

* Fix double memory free of iotconsensus queue request during region deletion

* Fix missing searchIndex and lost deletion when no TsFile is involved.

* Fix ref count of IoTConsensus request not decreased in allocation failure (#16169)

* fix IoTConsensus memory management

* Fix ref count of IoTConsensus request not decreased in allocation failure

* fix log level

* remove irrelevant codes from 2.0

* Remove a table test

* Interrupt wal-delete thread when WALManager is closed (#15442)

---------

Co-authored-by: Xiangpeng Hu <[email protected]>
@Caideyipi
[To dev/1.3] Fixed the concurrency issue of region migrate and load (#…
bab1937 
…16796) (#16823)

* Fixed the concurrency issue of region migrate and load (#16796)

* rq

* gra

* fix

* fix

* coverage

* fix

* fix

* fix

* cp

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.