November 17, 2004

Modeling ADLs with Untimed HMMs: updated

 There are now four models that I have tested with this data. The first model is a simple HMM that is event driven: activities are only allowed to switch when a tag is seen. This is called the "Untimed" Model. The second model is a simple HMM that is clock driven: activities are allowed to switch on every clock tick whether or not a tag is seen. I'm calling this the "Timed" Model. In fact it enforces an exponential distribution on the activities just by the nature of the self transition. The third model is an HMM that is also clock driven, but I add a countdown variable to enforce a different distribution that an exponential distribution. In this case I enforce a gamma distribution. The final model is an HMM in which each activity is a three state sequence. I train each activity separately to get an implicit gamma distribution (negative binomial). In the first three cases to model this domain with HMMs I segmented each of the traces into their 12 composite activities. I calculated P(RFID objects given activity). In the timed cases I added a null observation for each second in which there was no RFID observation. Then I constructed an HMM in which there was one node for each activity. Each node has a self-transition whose probability was calculated based on mean duration (exponential timed case) or mean number of observations per activity (untimed case). In the gamma case the "D"uration varable governs the "A"ctivity state switches. Each "A"ctivity state can transition to any of the other 13 states with a uniform probability. The HMMs start with a uniform prior over all of the activities. I then performed 17 fold cross-validation on the user traces and determined the viterbi path through the state space that corresponded to left out user trace. For the gamma distribution I used the DBN on the left to do inference. Since I'm using a plain vanilla viterbi algorithm I create the full |A|x|D| state space to represent the world. "A" is the Activity that is being performed. "D" is the number of seconds left in the activity. Normally "A" stays the same from time to time+1, and "D" counts down deterministically. When "D" reaches 0, "A" is chosen uniformly from the next possible activities. Then "D" is chosen according the gamma distribution associated with the new "A". Then I do exact inference to explain the observed tag sequence and recover the optimal state path through time. For the fourth system I enforced each activity to consist of a three state sequence which was trained using EM.

Here are some comparitive results for using flavors of HMMs to recognize ADLs. In both cases the same data set was used: The dataset that Matthai first recorded at his house.

Info about the dataset

 Activity Class # of traces in dataset Duration of Activity (sec.) Average # of objects touched 1 Personal Appearance 16 91 5.13 2 Oral Hygiene 13 108 6.38 3 Toileting 15 111 4.67 4 Hand Washing 11 75 3.82 5 Housecleaning 16 134 3.69 6 Appliance Usage 13(or 16 depending) 92 3.15 7 Prepare a snack 12 174 7.75 8 Prepare a drink 19(or 16 depending) 135 4.84 9 Control the temperature 15 67 1.13 10 Laundry 11 146 4.36 11 Leisure 15 152 3.47 12 Telephone Usage 17 149 3.00 13 Infant Care 17 160 5.94 14 Take Medication 14 74 2.5 Mean (std) 14.6 (2.4) 119(35) 4.27 (1.7)

Info about the results

 Untimed HMM Timed HMM Discrete Gamma Implicit Gamma Total Number of Activities user's performed: (17*12) = 204 Number of completely missed Activities (False Negatives) 7 18 14 9 * Average duration of missed activity (sec) 71.0 76.7 70.07 82 * Average number of objects touched in a missed activity 2.0 2.3 2.2 3.4 Number of added Activities (False Positives) 32 6 8 40 Number of confused Activities 9 14 14 13 Total errors FP+FN+Confused 49 38 36 62 Overall Recall (Correctly guessed activities)/(All true activities) 191/204=93.6% 172/204=84.3% 176/204=86.3% 182/204=89.2% Overall Precision (Correctly guessed activiites)/(All guessed activities) 191/229 =83.4% 172/192= 89.6% 176/198= 88.9% 182/235=77.4%

Confusion Matrix for HMM without timing

 Guess-> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 FN 1 14 1 1 2 13 3 14 1 4 1 8 2 5 16 6 11 2 7 12 8 1 14 1 3 9 15 10 11 11 15 12 17 13 17 14 14 FP 0 2 4 2 3 7 3 6 0 3 2 0 0 0

Confusion Matrix for HMM with exponential timing

 Guess-> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 FN 1 11 1 2 2 2 13 3 7 7 1 4 1 8 2 5 15 1 6 14 1 1 7 12 8 1 8 7 9 13 2 10 10 1 11 13 2 12 17 13 17 14 14 FP 0 2 0 2 0 2 0 0 0 0 0 0 0 0

Confusion Matrix for HMM with discretized gamma function

 Guess-> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 FN 1 14 2 0 2 13 3 8 6 1 4 1 8 2 5 15 1 6 13 1 1 1 7 12 8 1 1 8 6 9 1 14 0 10 11 0 11 12 3 12 17 13 17 14 14 FP 0 2 0 2 0 2 0 0 0 0 1 1 0 0

Confusion Matrix for HMM with implicit gamma function

 Guess-> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 FN 1 13 3 0 2 13 3 11 2 1 1 4 2 6 3 5 1 13 1 1 6 12 1 1 7 12 8 13 1 1 2 9 15 0 10 11 0 11 15 0 12 17 13 17 14 14 FP 5 1 1 3 8 4 4 2 1 6 2 0 0 3
Posted by djp3 at 10:00 AM | Comments (0) | TrackBack (0)