add async api for kv-transfer

[niqi-lyu]

Motivation

This PR introduces asynchronous support for the KV manager in the disaggregation module to improve performance and reduce latency in PD disaggregation inference scenarios. The key goals include:

• Implementing asynchronous transfer mechanisms for KV cache management
• Adding support for layer callback tracking and batch preparation
• Enhancing the scheduler to handle async operations
• Optimizing block alignment calculations for efficient memory transfers

Modifications

1. StreamAsyncSubmitter:

• use the ring-buffer and async copy to check the gpu compute is finished. ideas from mscclpp.

2. BaseKVManager enhancements:

• Added is_support_async property flag
• Implemented prepare_batch() for async batch preparation
• Added mark_layer_ready() for layer completion tracking
• Introduced insert_layer_callbacks() for model layer hooking

3. New async components:

• Created MooncakeAsyncKVManager class in conn_async.py
• Modified SchedulerDisaggregationPrefillMixin to support async apis.

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[ShangmingCai]

Thank you for this PR. Since we have PD + chunked prefill, and the compute and transfer are already fully overlapping with each other, can you provide a benchmark serving result to present how the async kv manager bring performance gain and reduce latency?