Head-Matters--Code

Preprocessing -- We've used openface(https://github.com/TadasBaltrusaitis/OpenFace) to extract pose angles and action units for kineme and AU generation.

Model Architecture -- We've done Chunk level and video level analysis for both the datasets MIT and FICS. For more details, please refer to paper.

Types of architecture --

1. Decision fusion -- We are considering two LSTM model for AU_Only and kineme_Only. After taking teh scores from each model, we've implemented the following formula to find the DF score.
   DF_score = w*Kineme_only_score + (1-w)*AU_only_score
   w is a weight parameter between 0 and 1. We've considered it with step size 0.05
2. Feature fusion (Merged architecture) -- Kindly refer the following figure.

   https://github.com/MonikaGahalawat11/Head-Matters--Code/blob/main/Merged_architecture.png

List of parameters:

1. nKineme contains the number of kinemes clusters created (its a user defined parameter-- in our case it's 16)
2. seqLen defines the length/size of each chunk (seqLen for the datasets varies as per chunk size)
3. nAction is the size of vector for action units; as we extract 17 action units from openface

All the needed sample data to run regression code is available in the Folder "Data".

Sample data with multiple example files for Video level analysis is available in the folder "Video_level". It contains two folders: "AU_10fps" (for action units) and "Kinemes" (for kineme files) and csv file for labels.