Why do we need to calculate for expectation before sum

       Hello, I've read your wonderful paper published on ICLR, and I'd like to consult you some questions. 
       The two summation symbols in the objective function sum the positive and negative samples and find the average value. Why do you need to calculate the expectation before sum?
       thank you!