Skip to content

chiyu1203/DLC-inferencespeed-benchmark

 
 

Repository files navigation

DLC Inference Speed Benchmark DLC LIVE!

This repository stores the benchmark results for DeepLabCut-live for each standard dataset, grouped by operating system, processor, and DLC model. Each configuration is tested on a fixed set of videos.

How to contribute!

  1. Install the DeepLabCut-live! SDK
  2. git clone the DeepLabCut-live! repo: git clone https://github.com/DeepLabCut/DLC-inferencespeed-benchmark.git and run ./reinstall.sh to be sure it's properly installed.
  3. Run our benchmarking script on your system (with our data/model). Within the DeepLabCut-Live directory you will find the following structure:
DeepLabCut-Live
   -Benchmarking
   --> run_dlclive_benchmark.py

Then you can run (with python3, pythonw on MacOS):

python run_dlclive_benchmark.py

This will take some time, depending on your internet connection and hardware. Note that downloading, might take a few minutes, as the multiple models & videos comprise about 2,2 GB. Then 4 models will be run on two videos for various video sizes. To get you a sense, this takes about 90 minutes on a Titan RTX. IF you want to run the benchmark on a CPU or slow hardware, you can also change the number of frames, to 1000 in https://github.com/DeepLabCut/DeepLabCut-live/blob/master/benchmarking/run_dlclive_benchmark.py#L24.

  1. Please make a pull request here (i.e., add the resulting file to your forked repo under the _data folder--i.e., no need to hand edit the file, we will automatically convert your files into the correct yaml file format), and create a new pull request!) or email us: [email protected] if you have any trouble!

About

A database of inference speed benchmark results on various platforms and architectures

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Ruby 83.6%
  • Python 14.8%
  • SCSS 1.6%