Propagation Networks for Recognition of Partially Ordered Sequential Actions

This paper presented a model and algorithmic technique for recognizing an activity which might have the following properties: Sequential streams: There is a natural partial ordering of components. Duration of elements: The primitives are not events but have temporal extent. Multiple, parallel streams: Many intervals may occur in parallel. Logical constraints: Some intervals can be satisfied by a disjunction of sub-intervals. Non-adjacency: Sequenced intervals may not meet but may only be ordered. Uncertainty of underlying vision component: Extracted features will always be noisy.

This was a nice paper. It described the model as a P-Net (Propagation Net) which basically requires a random variable for every possible action in the model. This, of course, is necessary if each activity can be executed concurrently. Unfortunately, though, it makes the model difficult to scale to lots of activities or even lots of steps in a single activity. As a result the treatment in this paper reflected an activity recognizer which recognized a single activity based on a threshold of the likelihood of the activity given the model.

Because of the semantics of the model, there is a lot of optimization that can be done in inference and learning by restricting the scope of the possible state transitions in light of timing and prerequisite actions. Using some of these constraints the authors introduce a technique, "D-Condensation", which is an inference method on the P-Net model.

The experiments and application were centered on trying to identify the use of a blood glucose meter using computer vision techniques from an over head view. There was also a short comparison to Stochastic Context Free Grammars (SCFG).

Finally two additional issues came up in this paper:

• The problem of particle filters tending to clump their particles into the most likely estimate was acknowledged. This is well-known among those who use them and indicates that particles filters don't represent uniform distributions well, but it was interesting to see the issue brought up formally in a paper.
• Secondly the authors made the claim that SCFG don't handle recognition in the face of deleted states well. So if an activity is performed and a state is skipped the SCFG will require exponential time (in the size of the rules maybe?) to reason that a delete has occurred.