Future Directions

I skate to where the puck is going to be, not to where it has been. — Wayne Gretsky

Agent Strategies

Priorities for builtin, custom built, third-party automaton types:

1. Environment: State Space representation options

   1. graphical models
      1. behavior trees (current)
      2. Generalized Probabilistic models
   2. RL based (Predictive State Representations (macro-actions, options), dec-pompd)
   3. Neuroevolution

2. Reasoning: Action Selection capabilities options

   1. utility ai
   2. graphical models (decision trees, bayesian, etc.)
   3. RL based (bandits, td-learning, q-learning)
   4. Evolutionary (population-based reasoning)

3. Knowledge: Working, Short-term and Long-term Memory options

   1. state spaces (Working Memory)
   2. blackboards
   3. percept memories

General

Behavior Trees

• break up the system heartbeat(update cycle) into supported abstract phases to improve designs?

• offline learning?

RL Example Tutorials

• grid world
• optimize an investment portfolio
• trade currencies