What is this repo?

This repo contains code pertaining to the paper “Observing Human Mobility Internationally During COVID-19” along with the actual output files for each step in our camera vetting pipeline for posterity and to serve as an example.

Image Archival

https://github.com/PurdueCAM2Project/CAM2ImageDatabase https://github.com/PurdueCAM2Project/CAM2ImageArchiver

$ crontab -l
SHELL=/bin/bash
0 0-23/5 * * 0 $HOME/cron-submit-camcheck.sh
0 1-23/5 * * 1 $HOME/cron-submit-camcheck.sh
0 2-23/5 * * 2 $HOME/cron-submit-camcheck.sh
0 3-23/5 * * 3 $HOME/cron-submit-camcheck.sh
0 4-23/5 * * 4 $HOME/cron-submit-camcheck.sh
0 3-23/5 * * 5 $HOME/cron-submit-camcheck.sh
0 2-23/5 * * 6 $HOME/cron-submit-camcheck.sh

We have our cron jobs set up to capture data from different times of the day, but on a periodicity that allows for meaningful longitudinal comparison.

Script Documentation

Each individual python script will have comments in the source code outlining its use/functionalty. camera_vetting_pipeline.py is a python script that stitches together all our filtering steps into a single script.

Pipeline Explained with our sample output files

Camera_viability.json (cameras_v1.json) is the json file containing all of the cameras ids with a number indicating how many weeks they were inactive over from april 1st 2020 to january 1st 2021 as determined by camera_filter.py

cameras_v2.json is cameras_v1 after removing all entries in the json containing 10 or more frozen weeks Script is threshold_frozen_cameras.py to output cameras_v2.json

Scene_Classification/classif_cams-gpu.py outputs ‘classification’ which is a json file classif_cams-gpu.py loads databaseiterator from Scene_Classification/database_iterator_30kcams.py Also loads scenedetection from Scene_Classification/scene_detection_gpu.py

Scene_Classification/place_country.py uses the classifications json and outputs a copy of the json but renamed and with all cameras deemed (via places output) to be irrelevant for capturing mobility removed. This is done by voting (if top two in people/vehicle locations, choose it, if not, remove it) people/vehicle locations are hard-coded in this file (107 people places, 8 vehicle places)

select_countries.py takes in ‘json with country and type specified’ (combined2021allcamerasReallyrealactual.json), outputs cameras_v3.json

At this point in the pipeline we have a json file containing all of the camera IDs that are relevant to a specific mobility study like the one in the paper. Make sure the detect_*.py files are configured correctly, are iterating over the most filtered cameras json file, and are loading the models correctly. Then, run detection and you’ll then get refined, mobility relevant data in the form of object counts over time.

Cameras_v4.json is after country vetting cameras_v3.json, ignore the locations we decided not to run detection on (for the sake of a smaller json file) - can be done with a simple python script (camera_vetting_pipeline.py optionally does this automatically depending on input parameters)

Once this is obtained the scripts detect_people_image_cams_individual.py and detect_cars_image_cams_local.py can be used to run detection on the cameras relevant to analysis.