The goal of tflite-object-detection-tracking is to help smooth the output from a *.tflite model so that the UI is more friendly. The reason is that image recongition output can be finicky at times. The tflite model outputs the results in order from highest to lowest. A slight shift of the image can change the output which will trigger changes that the user doesn't expect or intend. Additionally, hand movements on mobile devices cause the bounding boxes to constantly shift dimensions.
This library works to allow smoothing of model outputs by taking into account the location of bounding boxes - with a threshold - and keeping track of these objects.
Many thanks to @drewag for coming up with the main concepts on how this works.
npm install tflite-object-detection-tracking- Initialize the instance:
const tracker = new FrameRecognition(height, width); - Add frame data from
tfliteoutput:frameRecognition.addFrameData(boundingBoxes, labels, scores); - Get the confident bounding box output:
const confidentObj = frameRecognition.getConfidentObject(); - Use the results to render UI output as needed.
Contributors are welcome! Please open a PR.
- To test your changes in your application, before PR & merge, you will need to modify the
metro.configfile as described in the docs to allow for sym-linking