An example giving model

[arielevy8]

I'm doing research on (social) concept learning and teaching with examples (Shafto-style). As a first step, I tried to implement concept learning in a very simple environment - Tenenbaum's number game, where the hypothesis space is only intervals of up to 20 on a 100 points number line

1. This code snippet is doing great in creating accurate learner posterior probabilities after the teacher gives one example (for now, I'm not incorporating recursive mentalizing):

@memo
def learner[h: hypotheses_range, n: numbers](prior = 1):
    cast: [learner, teacher]
    learner: thinks[
        teacher: given(h in hypotheses_range, wpp = prior),
        teacher: chooses(n in numbers,wpp = 1 if is_number_in_hypothesis(h, n) else 0)
    ]
    learner: observes[teacher.n] is n
    learner: chooses(h in hypotheses_range, wpp = Pr[teacher.h == h])
    return Pr[learner.h == h]

It returns the full matrix of size hypothesis space*n – which is great! But sometimes for simulations I want to only condition on a specific value that the teacher gave and only get the posterior probability for this value (for example, condition on teacher.n == i). Now, I just take learner()[:,i] , but it seems suboptimal to compute the entire matrix when I only need one row. What’s the memo syntax for conditioning on a specific value?

2. I’d like to make the teacher’s policy for choosing example sets iterative, that is, to first randomly sample one number within the hypothesis, then sample a second number, and so on, up to K. The only implementation I’ve managed so far defines a teacher function that chooses among all possible combinations of examples up to k. The only way I managed to implement something similar is to have a teacher function that chooses among all possible combinations of examples up to k teacher[h: hypotheses_range, n: all_combinations], but this, of course, combinatorially explodes for high k (or more complex, two dimensional concepts) and returns a huge matrix. In addition, in my full model, the teacher iteratively picks examples until adding more examples is no longer cost-effective relative to the improvement in the learner’s posterior. This mechanism would also benefit from an iterative implementation, where in each iteration, the teacher can decide whether to add another example to the existing set (rather than one-shot search over the entire combination space). However, I couldn't find any examples of iterative processes of agents within a memo function in the demo folder of the repository, and I wonder how you would approach this problem?
   Thanks!
   Ariel

[kach]

Hi Ariel, thanks for writing!

For (1), you can use the observes_that statement to condition on a particular value. See Handbook.pdf for information on this construct. In your case, you can write:

learner: observes_that(teacher.n == 5)

Alternatively, you can use the observes_event statement to condition on an event with a particular probability occurring:

learner: observes_event(wpp=Pr[teacher.n == 5])

What's the difference? Well, observes_that commits to a particular observation actually happening, whereas observes_event just says that "an event with this probability occurred" but doesn't actually commit to any particular observation happening. So in the former, afterwards learner[ E[teacher.n] ] will be equal to 5, whereas in the latter case learner[ E[teacher.n] ] will continue to be the mean of the prior. Does that make sense? For this reason, you can use observes_event to repeatedly make multiple observations, because you aren't committing to a particular observation, as you are with observes_that.

learner: observes_event(wpp=Pr[teacher.n == 5])
learner: observes_event(wpp=Pr[teacher.n == 10])
learner: observes_event(wpp=Pr[teacher.n == 15])

(If you tried this with observes_that, the learner's belief would collapse because you are asserting that the learner observed mutually incompatible outcomes.)

For (2), based on my understanding of what you want to do, I would recommend that you write your teacher simulation as a Python loop that repeatedly calls a student model, iteratively updating its belief. To make this work, you would want to set up the student's model to take the prior as one of its inputs. So abstractly, your loop would look like:

prior = uniform_prior
while True:
  n = ...  # (somehow choose n)
  posterior = student(prior, n)
  if is_confident_enough(posterior):
    break
  else:
    prior = posterior

Does that make sense?

[kach]

Hi Ariel! Just checking in — did that help you solve your problem? Please feel free to ask follow-up questions if my response wasn't clear, or if more issues come up! :)

[arielevy8]

Hi Kartik! Thanks so much for your answer! and sorry for the late response, Iv'e been busy with another project and didn't get to try implementing your suggestions.

I completely understand the 'observes_that' part, and thanks! it helped me in making progress simulating a learner that learns from examples.

However, I am still struggling in modeling learning from more than one examples. For context, programming it outside of memo is very simple: likelihood is computed using size principle, multiplied by prior and being normalized:

@jax.jit
def learn_from_examples(hypotheses, priors, data):
    # priors: (num_hypotheses,)
    # data: jnp array of observed numbers (1-based)
    def likelihood(hypothesis_row):
        # hypothesis_row: (N,) array of 0s and 1s
        data_in_h = hypothesis_row[data - 1]
        all_in = jnp.all(data_in_h == 1)
        support_size = jnp.sum(hypothesis_row)
        likelihood = jnp.where(
            all_in,
            jnp.power(1.0 / support_size, data.shape[0]),
            0.0,
        )
        return likelihood
    # Vectorize likelihood over all hypotheses
    likelihoods = jax.vmap(likelihood)(hypotheses)
    unnormalized_posts = priors * likelihoods
    sum_posts = jnp.sum(unnormalized_posts)
    posteriors = unnormalized_posts / (sum_posts + 1e-40)
    predictive = jnp.dot(posteriors, hypotheses)
    return posteriors, predictive

However, when trying to model it inside memo, I struggle with the 'chooses x in y' syntax, because it require me to have a representation of all possible example sets that the teacher could have given!
A current version that I have now, and work great with great run time , looks like that:

@memo
def learner[h: hypotheses, e: possible_example_sets](example_set_id, prior = 1):
    cast: [learner, teacher]
    learner: thinks[
        teacher: given(h in hypotheses, wpp = prior),
        teacher: chooses(e in possible_example_sets, wpp = likelihood(h, e))
    ]
    learner: observes_that[teacher.e == example_set_id]
    learner: chooses(h in hypotheses_range, wpp = Pr[teacher.h == h])
    return Pr[learner.h == h]

However, it is only usable for up to 2 or 3 examples. With more than that, 'possible_examples_sets' just becomes huge and collapses my IDE.

I'm sure there is a way to model this in memo without having an explicit representation of all possible messages the teacher could have had. Maybe it's my lack of of experience with PPLs, but I feel that the 'chooses X in Y' syntax forces me to have a representation of all possible y values to be able to update my posterior given a specific y. In the non PPL-based bayesian updating function I pasted above, there is no need to know all possible example sets before updating the posterior based on a specific example set.
(another thing I tried is to have k "observes_event" statements, one for each example, stacked one after the other, then the Pr[learner.h==h] just returns the prior)

---

Regarding the while loop – thanks! that's a great idea, and indeed that's exactly the implementation I used before I tried translating my model to memo. However, I was hoping there is some way to have this while loop or other iterative message choice process within a memo function, so that I could start having a recursive teacher-learner relationship.

Thanks again for all the help and for memo!

[kach]

Hi Ariel,

Try this:

@memo
def learner[h: hypotheses, e: possible_example_sets](example_set_id, prior = 1):
    learner: thinks[
        teacher: given(h in hypotheses, wpp = prior)
    ]
    learner: observes_event(wpp=likelihood(teacher.h, example_set_id))
    learner: chooses(h in hypotheses_range, wpp = Pr[teacher.h == h])
    return Pr[learner.h == h]