Stats Page V2

[maxpatiiuk]

Re-posting @beach53 wish list for next gen stats. This is just a list of potential features, not commitments. Before implementing, need to discuss some of them more and make changes/see if they still make sense.

• ability to run very large queries quickly, within a few seconds, RBGE large queries (multiple simultaneously on stats page) take 10-15-20 mins.
• Need a place to put notes on stats queries, a remarks field that says what they are, free text, who made them date, how to use them or modify them. (Add descriptions for stats #3333)
• Need to be able to reorder query lines in stats page within categories, also auto-alphabetize
• Reordering of categories or pages (Reorder and clone categories in the stats page #2672)
• Need to be able to drag stats queries from one category to another
• Support ranked lists results, "group by" subgroup totals, e.g. Top ten families by CO count
• Specify Network. stats based on external (specify cache) operations
• Maybe a kill switch to stop stats calculations that hose server and stop processing
• Maybe a different way to handle crippling queries, like computing them in different threads, different workers, only at times when interactivity is not needed, at the end of sessions, when no one is logged in?
• Perhaps a way to set long stats to DO NOT REFRESH for each particular stat, or for those in certain categories, or for those that take very long times, or automatically when a server crushing query is detected we automatically tell the user later.
• [Recomputing Taxon tiles for a large database is a similar problem, takes huge resources everytime the splash screen is started, no need to recompute every session, but also maybe ways to greatly accelerate these big queries.]
• Except for a few more UI nice to have features, the big remaining issue with stats is not stats but query performance on large databases. QB, TT, Stats, are all throttled by query performance, and as we get into more analysis tools, we will need a query architecture that scales computationally.
• Currently for large data sets, it is easy to make stats that bring the server and Specify 7 FE interactivity to a stop.
• Clicking on stats TLM should not prevent people from moving on to other tabs and other work, do we need a another thread her to release the UI to continue working? At a place like RBGE or CSIRO one person making a demanding stat will stop specify use for the entire institution for 20-30 minute or more. That is not a risk we should enable and facilitate. We need to think about this scaling issue with queries on big databases from the perspective of the user an site admin, and how much people will hate us for handing them gasoline and matches. LOL!