February 11, 2005

Somewhat-improved Strawman Results

These results reflect two changes.
1) I allowed EM to train the starting probabilities, pi_0
2) I changed the way "no observation" observations were handled

The second change means that I tweaked the strawman so that it wasn't so biased by the empty observations. The insight is that the model is double biasing when it sees a "no observation". One bias is the self-transition probability which defines an exponential distribution over the length of time that the system is in the same state. When you also have different probabilities for seeing a "no observation" then switching states is controlled by the expected time in the state as well as the likelihood of seeing a "no observation". That doesn't seem quite right. So as a result, before I did inference I made all states expect to see the "no observation" with equal likelihood. Here are the results with that change:

ModelTime-Slice AccuracyEdit Distance
Primary ActivityAny Activity
"single-trained" straw man (prior)84.4% (std 6.6).90.1% (std 5.1) 16.2 std(7.4)
"full-trained" straw man (state-transitions)81.75% (std 6.2)86.1% (std 4.7)26.3 (std 7.33)
pinned straw man (partial labels)55.2% (std 7.0)56.9% (std 6.2)87.1 (std 6.4)
labelled straw man84.4% (std 6.6)90.1% (std 5.1)16.2 (std 7.4)

I'm not impressed. This morning I realized that I need to manage the EM much more closely to do what I mean to be doing. Plus this is what I have to do in order to include relational smoothing, so that's what I'm doing now.

Posted by djp3 at 11:00 AM | Comments (0) | TrackBack (0)