[bugfix] update results of state_dict loading, embedding resizing to secondary partitions (hpz)

[cyr0930]

After this commit (#4906), secondary partitioned tensors are updated only after optimizer.step().

When loading state_dict or resizing embedding after init, secondary partitioned tensors should be updated.

e.g., https://github.com/huggingface/transformers/blob/1c4b62b219323a31011bac3bd3cece7675d9e4c3/src/transformers/integrations/deepspeed.py#L344

[tjruwase]

@cyr0930, thanks for this PR. Can you provide more context about the failure this fixes? For example, did you encounter convergence issues after checkpoint loading?

[cyr0930]

Partitioned parameters are updated only when ds_secondary_partition_tensor is None by this commit (#4906).
And ds_secondary_partition_tensors only become None after optimizer.step function is called (that function contains logic that invalidate secondary tensor)
But there's other cases that partitioned parameters should be updated such as loading state_dict or resizing embeddings.
(Above explanation is based on transformers implementation)

For now, after parameter initialization, ds_secondary_partition_tensors are created and existed for each params, so they are not updated when we perform state_dict loading or embedding resizing.
Maybe we can add a function that invalidate ds_secondary_partition and call it after every parameter changes.
But IMO it is quite messy and I decided to revert has_been_updated part of the previous commit.