Buffer Data During Network Outages

[Tom-McQuillan]

Description

During network connection loss, the Nominal LabVIEW Client currently discards data generated while the connection is down. When the connection is restored, the data stream resumes but the intermediate data is lost.

User Feedback

"If the network connection is lost and then restored, the data stream will pick back up where it left off. Data that is generated during this buffer is lost however (which makes sense since the queue is flushed every attempt)."

Proposed Solutions

Questions for User

Before implementing a solution, need to gather:

1. Typical network drop duration: Is ~15 seconds typical for your environment?
2. Sample rate: What is your typical data acquisition rate?
3. Channel count: How many channels are being monitored?

Option 1: Memory-Monitored Buffer

• Implement in-memory buffer with available memory monitoring
• Flush buffer once network communications restored
• Pros: Fast, no disk I/O overhead
• Cons: Limited by available RAM, data lost if application crashes

Option 2: Persistent Storage Buffer

• Stream data to disk during outage (SQLite DB or TDMS file)
• Flush persisted data once network communications restored
• Pros: Survives application crashes, handles longer outages
• Cons: Disk I/O overhead, slightly more complex implementation

Option 3: Hybrid Approach

• Use memory buffer up to a threshold
• Spill to disk if outage exceeds threshold or memory limit
• Pros: Best of both approaches
• Cons: Most complex to implement