log_mix on the logit scale

[syclik]

From @bob-carpenter on February 3, 2015 18:31

Now that we have log_mix defined on the natural probability scale:

log_mix(theta, lp1, lp2) 
  = log(theta * exp(lp1) + (1 - theta) * exp(lp2))
  = log_sum_exp(log(theta * exp(lp1)), log((1 - theta) * exp(lp2))
  = log_sum_exp(log(theta) + lp1, log(1-theta) + lp2)

it'd be nice to roll in the link function (inv_logit) and just work on the unconstrained log-odds scale with

log_mix_logit(alpha, lp1, lp2) 
  = log_mix(inv_logit(alpha), lp1, lp2)
  = log_sum_exp(log(inv_logit(alpha) + lp1, log(1 - inv_logit(theta)) + lp2)

Special functions log_inv_logit and log1m_inv_logit already exist.

Copied from original issue: stan-dev/stan#1260

[mike-lawrence]

Any progress on this? I was just about to create a feature-request issue asking for a inv_logit that accepts vectors, but then saw this which is the reason I was using inv_logit in the first place.

[bob-carpenter]

Nope, though adding new functions is relatively simple.
This one's low priority, given the weakness of log_mix
in the first place (only binary, no vectorization, doesn't
work like a density function).

We should have inv_logit vectorized for 2.12---Rayleigh's
working on it and the math library part is all done, we
just need to add it to the language.

• Bob

On Aug 15, 2016, at 4:27 PM, Mike Lawrence notifications@github.com wrote:

Any progress on this? I was just about to create a feature-request issue asking for a inv_logit that accepts vectors, but then saw this which is the reason I was using inv_logit in the first place.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[rtrangucci]

Just to be clear, Rayleigh's version won't be precomputing gradients for
vectorized inv_logit. The log_mix function does do precomputed gradients,
which is part of the reason it's relatively fast.

Rob

On Mon, Aug 15, 2016 at 10:30 AM, Bob Carpenter notifications@github.com
wrote:

Nope, though adding new functions is relatively simple.
This one's low priority, given the weakness of log_mix
in the first place (only binary, no vectorization, doesn't
work like a density function).

We should have inv_logit vectorized for 2.12---Rayleigh's
working on it and the math library part is all done, we
just need to add it to the language.

• Bob

On Aug 15, 2016, at 4:27 PM, Mike Lawrence notifications@github.com
wrote:

Any progress on this? I was just about to create a feature-request issue
asking for a inv_logit that accepts vectors, but then saw this which is the
reason I was using inv_logit in the first place.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
#69 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADvmhzwOM0scLXD5bxF-NKiDtiqz35C1ks5qgHf1gaJpZM4FTFYs
.

[bob-carpenter]

inv_logit does have analytic gradients.

But the elementwise functions are not more efficient
than writing the loop by hand, just more convenient. This
is very counterintuitive to those who only use interpreted
languages like R or Python, where loops are super slow.

To make the elementwise functions more efficient, we'd need
special autodiff that cuts down on the number of virtual
function calls by calculating all the adjoint updates in a single
virtual function call attached to the first variable during chaining.
I think there are some examples of that. That's non-trivial given
the way the elementwise functions are being defined now for containers.