We consider the problem of distributed classification of multiple observations of the same object that are collected in an ad hoc network of vision sensors. Assuming that each sensor captures a different observation of the same object, the problem is to classify this object by distributed processing in the network. We present a graph-based problem formulation whose objective function captures the smoothness of candidate labels on the data manifold formed by the observations of the object. We design a distributed average consensus algorithm for estimating the unknown object class by computing the value of the smoothness objective function for different class hypotheses. It initially estimates the objective function locally based on the observation of each sensor. As the distributed consensus algorithm progresses, all observations are gradually taken into account in the estimation of the objective function. We illustrate the performance of the distributed classification algorithm for multiview face recognition in an ad hoc network of vision sensors. When the training set is sufficiently large, the simulation results show that the consensus classification decision is equivalent to the decision of a centralized system that has access to all observations.