-
Notifications
You must be signed in to change notification settings - Fork 1
2025 07 21 webex
AI-generated content may be inaccurate or misleading. Always check for accuracy.
The team discussed various aspects of resource management and communication optimization in MPI, including the concept of persistent optimization features and the handling of request objects in library implementations. They explored the design of a communication channel with client-server characteristics and its state transitions, while also discussing the importance of state management and resource allocation in MPI. The conversation ended with discussions about process management, dynamic MPI models, and the need for comprehensive support of dynamic process groups, along with plans for an upcoming presentation in late August.
-
Dominik to create a Google Presentation for the slide deck and share it with the team for collaborative editing and feedback.
-
Dominik to add specific examples or use case scenarios to the slides showing where current MPI specification is insufficient for dynamic resource management.
-
Dominik to include examples from both resource management and application perspectives in the use cases.
-
Dominik to specify a minimal example of the call object in the slides for better understanding.
-
Team to collect and formulate concrete questions for the larger MPI audience and add them to the end of the slide deck.
-
Team to review and provide feedback on Dominik's slides via email or comments on the shared presentation.
-
Team to prepare for potential presentation of the dynamic resource management approach in September .
The team discussed the concept of a "persistent" or "ongoing" optimization feature for resource management, where Martin explained that it would allow the resource manager to continue optimization steps even after receiving a response from the application. Anthony advised against hiding parameters in info objects, emphasizing that explicit parameters are preferred in MPI for language bindings, while info objects should be reserved for implementation-specific or future-proofing features. The team agreed that while the persistent notion could be encoded in the call info object, it might be better to keep it as an explicit parameter in the API call itself.
Dominik and Sonja discussed the handling of request objects in an MPI library implementation. Dominik suggested that the application does not need to use the same request object for different operations, as this information is primarily for the resource manager. Sonja agreed, noting that the request would need to be passed through to the resource manager, which would depend on the environment. They also discussed the potential for the MPI library to optimize the call object, which might require caching. The conversation highlighted the complexity of managing requests that are neither truly MPI persistent nor through normal requests, as mentioned by Howard in a previous meeting.
Anthony discussed the design of a communication channel similar to a TCP/IP session, explaining its client-server nature and state transitions. He emphasized the importance of defining the channel's semantics, including how it handles multiple MPI processes and supports one-to-one or many-to-many communication patterns. Anthony also described the channel's lifecycle, which includes an initialization phase, an active state for ongoing communication, and a closing phase, highlighting the need for precise semantic definitions for these stages.
Anthony discussed the importance of carefully planning state transitions and resource management in MPI, emphasizing the need for clear expectations and event-driven models. He suggested using state diagrams to inform API design and highlighted the potential for these concepts to enhance distributed computing capabilities. Dominik agreed and mentioned the need for a sketch of the interaction between the application and resource manager, suggesting the inclusion of state transition diagrams and interaction sketches in the slides. Both Anthony and Dominik acknowledged the need for further discussion on whether the function should be called by one process or a group of processes.
Anthony and Dominik discussed the use of MPI communication in process management and the potential for implementing a publish-subscribe model or distributed computing approach. Anthony suggested building a concurrent publish-subscribe database to support object sharing and synchronization, emphasizing the need for fault tolerance and consensus among processes. They also touched on the dictionary functionality and its separation from the resource manager. Dominik confirmed that while interaction with the resource manager is not required for the dictionary operations, it can be combined with other processes for information sharing.
Dominik and Sonja discussed the need to improve understanding of a dynamic MPI model and its implementation. Sonja suggested expanding the "State of the Art" section to include specific use cases and examples where the current MPI specification falls short. Anthony emphasized the importance of highlighting the limitations of current MPI features, such as the inability to include non-member processes in groups. Dominik agreed to include both resource management and application perspectives in the use case scenarios, emphasizing the need for more comprehensive support from MPI for dynamic process management.
Anthony shared historical insights on group operations in MPI, highlighting the challenges and considerations in managing dynamic process groups, including the need for join and unjoin operations. Dominik and Anthony discussed the differences between group operations in MPI and Pset operations, emphasizing the importance of process order and visibility to the resource manager. Dominik suggested creating a minimal example of the call object to illustrate intended usage and planned to include this in future presentations. Sonja proposed comparing the information required for a normal spawn operation in a call-up Jags to an MPI spawn operation to help clarify the new interface.
The team discussed their presentation slides and agreed to collect and add concrete questions for the audience at the end of the deck. They tentatively planned to present in late August, though the exact date was still to be confirmed after discussions with Howard, Martin Schultz, and Wesley. Dominik offered to share the slide deck via Google Presentation for feedback, as they would need to skip some meetings in the coming weeks due to vacations and cancellations.