Identifying an appliance for interaction in commercial buildings becomes non-trivial as the number of smart appliances explodes. We present a system for users to intuitively “look up” appliances using image matching-based technique on a pre-constructed and annotated visual model of building interiors. It matched 98% images on a public robot-collected dataset and achieved 100% recall and precision. Our lab experiments with human captured videos and images also show the feasibility of real world deployments.