Accessibility for a virtual list could be lacking. ARIA provides a way to mitigate this; even if it may not be supported by accessibility user agents, it's worth having and doesn't break anything else.
See http://www.w3.org/TR/wai-aria/states_and_properties#aria-setsize - by defining setSize and posInset, an accessibility user agent can know how large the list is despite the entire representation not being in the DOM.
This should only require aria properties on the list items in the html template, but it may also require the ul element to be an aria live region.