Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimizing with multithreading #81

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

AmmarRashed
Copy link

Hello,
Loved the repo so I wanted to leave this script here in case someone needs it to run your script on a specific video. Normally, your code would take about 8fps. With the multithreading implementation below it goes up to 50-60 fps, and finally outputs a pandas dataframe.

@abdullah-2022-skipq
Copy link

@AmmarRashed Hi, Appreciate the contribution, it helped me learn threading concepts. May I ask what specs did you test your code on? I have to run this on videos ranging from 50-70 minutes, currently testing it on 16 GB RAM, and it crashes the RAM within a couple of minutes.

Not sure if it can be improved, but I'm looking for ways to optimize this for longer videos.

@AmmarRashed
Copy link
Author

@abdullah-2022-skipq Hi, I think you can reduce the batch-size, and if it still runs out of memory, you can increase the batch size and remove the parallelization part. I remember testing with different configurations. Longer videos are trickier. The fastest I got eventually was using this method> https://github.com/AmmarRashed/EmoNetTCN/blob/9bf8d5c559323980242fe98663e4f4c5888b788c/face_detection.py#L44

The trick is to skip every other frame effectively reducing the FPS of the video, and avoiding the serialization overhead of multithreading. Worked quite well.

@abdullah-2022-skipq
Copy link

abdullah-2022-skipq commented Jun 13, 2023

@AmmarRashed Hi. Thank you very much for the response. I tried reducing the batch size to 2, take only one frame per second instead of all or every other along with numerous other approaches but the RAM still crashed for longer videos (60+ minutes). I think this is due to the fact that the entire processing is being done in memory and memory is cleaned only after the execution from all threads is completed.

What I have done now is that I save the frames as jpgs and then I run the gaze detection code (not from this repo, it's based on face_recognition python package - I'll try on this one as well if the accuracy I'm currently getting is not as good as this one) on these jpgs in multiple threads.

The original code without multi-threading or preprocessing took around 65-75 minutes on an hour long video, now it takes 20 minutes without multi-threading and 7 minutes on multithreading with 4 threads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants