February 11, 2005
Somewhat-improved Strawman Results
These results reflect two changes.
1) I allowed EM to train the starting probabilities, pi_0
2) I changed the way "no observation" observations were handled
The second change means that I tweaked the strawman so that it wasn't so biased by the empty observations. The insight is that the model is double biasing when it sees a "no observation". One bias is the self-transition probability which defines an exponential distribution over the length of time that the system is in the same state. When you also have different probabilities for seeing a "no observation" then switching states is controlled by the expected time in the state as well as the likelihood of seeing a "no observation". That doesn't seem quite right. So as a result, before I did inference I made all states expect to see the "no observation" with equal likelihood. Here are the results with that change:
| Model | Time-Slice Accuracy | Edit Distance | |
| Primary Activity | Any Activity | ||
| "single-trained" straw man (prior) | 84.4% (std 6.6). | 90.1% (std 5.1) | 16.2 std(7.4) |
| "full-trained" straw man (state-transitions) | 81.75% (std 6.2) | 86.1% (std 4.7) | 26.3 (std 7.33) |
| pinned straw man (partial labels) | 55.2% (std 7.0) | 56.9% (std 6.2) | 87.1 (std 6.4) |
| labelled straw man | 84.4% (std 6.6) | 90.1% (std 5.1) | 16.2 (std 7.4) |
I'm not impressed. This morning I realized that I need to manage the EM much more closely to do what I mean to be doing. Plus this is what I have to do in order to include relational smoothing, so that's what I'm doing now.