Student Research

Tensorflow Object Detection

Object detection is the process of locating and classifying objects, which can do such as video surveillance, robot navigation, etc. Deep learning based object detectors are trained from images with known classes of objects; these detectors can only detect a set of object classes that are known at training time. If we want to detect a new object class, we need to re-train our custom detector. However, this training process requires a large amount of images of the custom objects with annotated bounding boxes and performed manually by humans, which is costly and tedious. This causes substantial barriers to build custom detectors for arbitrary objects. To tackle this problem,our project targets a fully automatic framework of building an object detector for custom objects. In particular, we will leverage 3D model reconstruction to automate the generation of a large amount of annotated image data for target objects. Instead of annotating images and creating the bounding boxes, we only need to name the 3D object's masks once and render 3D objects to generate 2D images and its masks with different backgrounds and situations. Then, masks give the names of objects and their size and location to annotate images, which then help fine-tune deep learning models to yield an object detector with compelling identification accuracy of the custom objects. We evaluate this automatic framework, showing that it achieves satisfactory accuracy with limited user assistance while running in real-time on webcam and mobile devices.