ElasticJob add static sharding strategy, add completed flag and split job create trigger API plan

[Lucas-307]

Reason

Scaling module dependent on ElasticJob module to sharding and execute scaling job, there are some issues like:

1. OneOff job can't do average sharding even if one-off job start in a short interval.
2. Sharding item sometimes exexute in different server, which may cause some problems.
3. OneOff job don't have a complete flag, which may cause job execute more than once.

Plan

So, we decide to refactor it with this plan:

1. add static sharding strategy
2. add completed flag
3. split job create trigger API

Steps

1. Before refactor

   • Optimize JobInstance
     • add server field(value is current server ip), and replace it with split by jobInstanceId.
     • add labels field which match JobConfiguration's label field(one to more).
   • Optimize JobConfiguration
     • add label field which match JobInstance's labels field.

2. Add static sharding

   • Optimize JobConfiguration

     • add boolean staticSharding field to mark current JobConfiguration is use static sharding strategy or not.

   • ShardingListenerManager and ReconcileService

     • if staticSharding is true and have been done sharding before, do not create necessary flag again.

3. Add completed flag

   • for one-off job, if any item completed, create a completed flag in sharding/item/completed

   • failover or reboot JobInstance do not do it again.

4. Register job instance

   Add JobInstanceRegistry.register(jobInstanceId, labels) API to start a JobInstance and execute:

   • listen /jobs/*/config and start all uncompleted job

     uncompleted job check by

     • config.enable = true
     • config.label match JobInstance's labels
     • all schedule job
     • one-off job which sharding/item/completed < config.shardingTotalCount

   • start job by old API

     • new ScheduleJobBootstrap().schedule()
     • new OneOffJobBootstrap().execute()

5. Add trigger API

   add JobOperateAPI.trigger(jobName, class, config) API to create a job

   • create /jobs/jobName node to persist job class.

   • create /jobs/jobName/config to persist job config.

API changes

1. new API: split register and trigger

   • register job instance

   JobInstanceRegistry.register(jobInstanceId, labels)
   

   • trigger job

   JobOperateAPI.trigger(jobName, class, jobConfig)
   

2. old API: create and execute

   • one-off job

   new OneOffJobBootstrap(regCenter, elasticJob, jobConfig).execute()
   

   • schedule job

   new ScheduleJobBootstrap(regCenter, elasticJob, jobConfig).schedule()