(2)
2.2.3
Mining People’s Appearances
Our aim is to discover and exploit patterns of how people appear together in a photo collection. We do so primarily by finding repeated appearances of groups of people. While multiple people may appear together in a single photo, it is evident that people typically do not congregate for the sole purpose of taking one photo. Instead, people usually meet for a longer time period (e.g. numerous relatives attending the same family birthday party), over which multiple photos might get captured. Thus, it is also necessary to model the case of an individual person who only appears by himself or with different members of the group in all photos captured throughout such a period, which we refer to as an event. Note that for simplicity we are only interested in consecutive, non-overlapping events. Finding such patterns based solely on the information given by the training set (some labeled people) is usually not a viable option because of the often low number of training samples. Thus, we devise the following iterative approach: 1. First, we use our basic graph-based approach (as described in Section 2.2.2) to initially recognize people. 2. Then, based on these preliminary recognition results, we attempt to discover appearance patterns. 3. Lastly, we perform inference - as in Step 1 - a second time while also considering the information gained through pattern mining in Step 2 to refine the initial recognition results. for a person’s face Note that in Step 1 we also compute a measure of confidence appearance based on probabilities (corresponding to the nodes’ states) provided by the graph’s inference method. We utilize this confidence measure later.
2.2.4
Incorporating Mining Results
At this point we have numerous appearance patterns; however, we do not know to which events they apply. Thus, we show next how to match both. Like in the previous section, we first discard all uncertain appearances from confidence values
below a certain threshold
intermediate result
for each event
. These are appearances with associated . Then
. Next, we compile an
:
1. 2.
First, we form a set of individual people over that is associated with the event . We then include people in , who are part of the training set and are
3.
associated with the event , but are not contained within (because of possible recognition errors during our initial run). Next, we match the given event with any appearance pattern based on it’s set of associated people. To do so, we form a list the FIM’s result •
and including any transaction
and
is above a threshold
being the number of distinct people in
The number of distinct individuals who are either only in threshold
4.
that matches the following criteria:
The number of distinct individuals who are in with
•
by iterating through all transactions in
or
. is below a
.
Then, we transform the sets of people within into stacked vectors with a vector length equal to the number of total individuals. If an individual appears in a set, we store the transaction’s frequency
at each individual’s position in the vector.
CUbRIK First Social Network analysis, Trust & People Search Techniques
Page 8
D32 Version 1.0