[Feature] Sink fluss,the checkpoint execution is too slow

[GangYang-HX]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Deploying method: Deploying Distributed Cluster


The write traffic is small，but the sink operator has not yet trigger checkpoint

Solution

No response

Anything else?

No response

Willingness to contribute

• I'm willing to submit a PR!

[GangYang-HX]

There are two key pieces of information for reference。
1.There are a lot of warn logs:"Get error write response on table bucket"

2.com.alibaba.fluss.client.write.RecordAccumulator.awaitFlushCompletion Execution time is too long.

[swuferhong]

Hi @GangYang-HX, did you start this job immediately after you create the fluss table? I suspect the possible cause is that you started writing data immediately after creating the table. At this point, the bucket on the tabletServer is not yet ready, so the client will retry, and it's reasonable for a warning to be logged at this time.

[GangYang-HX]

Hi @GangYang-HX, did you start this job immediately after you create the fluss table? I suspect the possible cause is that you started writing data immediately after creating the table. At this point, the bucket on the tabletServer is not yet ready, so the client will retry, and it's reasonable for a warning to be logged at this time.

@swuferhong Hi,Thanks for your reply
I didn't pay attention to the time between creating the table and starting the task, but it should have taken a few minutes.
As of today, this warn log still exists, and it has been running for more than 18 hours.

[GangYang-HX]

@swuferhong Hi,how long after creating the table is it appropriate to start the task?

[swuferhong]

@swuferhong Hi,how long after creating the table is it appropriate to start the task?
It's strange that this problem is still occurring after such a long time theoretically. Unless there is a bug in the code, could you give me a rough description of your table creation ddl and the Flink SQL? I would like to try to reproduce it locally.

[GangYang-HX]

@swuferhong Hi,how long after creating the table is it appropriate to start the task?
It's strange that this problem is still occurring after such a long time theoretically. Unless there is a bug in the code, could you give me a rough description of your table creation ddl and the Flink SQL? I would like to try to reproduce it locally.

[GangYang-HX]

@swuferhong
Create table script：

CREATE TABLE test_fluss_tb_02 (
  device_id VARCHAR,
  device_name varchar,
  version VARCHAR,
  meta_id int,
  response_id VARCHAR,
  ads_id bigint,
  client_time bigint,
  server_time  bigint,
  PRIMARY KEY (device_id) NOT ENFORCED
) WITH (
  'bucket.num' = '4'
);

insert script:

INSERT INTO `fluss_catalog`.`test_db`.`test_fluss_tb_02`
(SELECT `device_id`, `device_name`, `version`, `meta_id`, `response_id`, `ads_id`, `client_time`, `server_time`
FROM `src_xdcs`)