Skip to content

Latest commit

 

History

History
114 lines (77 loc) · 4.13 KB

README.md

File metadata and controls

114 lines (77 loc) · 4.13 KB
Raw

Video to Pose3D

Predict 3d human pose from video

Prerequisite

  1. Dependencies
    • Packages
      • Python > 3.5 distribution
      • Pytorch > 1.0.0
      • torchsample
      • ffmpeg
      • tqdm
      • pillow
      • scipy
      • pandas
      • h5py
      • visdom
      • opencv-python (install with pip)
      • matplotlib
    • 2D Joint detectors

      • Alphapose (Recommended)

      • OpenPose (Not tested, PR to README.md is highly appreciated )

    • 3D Joint detectors
      • Download pretrained_h36m_detectron_coco.bin from here, place it into ./checkpoint folder

Usage

  1. place your video into ./outputs folder. (I've prepared a test video).
Single person video
  1. change the video_path in the ./videopose.py
  2. Run it! You will find the rendered output video in the ./outputs folder.
Multiple person video (Not implemented yet)

  1. For developing, check ./videopose_multi_person

    video = 'kobe.mp4'

handle_video('outputs/{video}') 
# Run AlphaPose, save the result into ./outputs/alpha_pose_kobe

track(video)					 
# Taking the result from above as the input of PoseTrack, output poseflow-results.json # into the same directory of above. 
# The visualization result is save in ./outputs/alpha_pose_kobe/poseflow-vis

# TODO: Need more action:
#  1. "Improve the accuracy of tracking algorithm" or "Doing specific post processing 
#     after getting the track result".
#  2. Choosing person(remove the other 2d points for each frame)
Tips
  1. The PyCharm is recommended since it is the IDE I'm using during development.
  2. If you're using PyCharm, mark ./Alphapose, ./data and ./pose_trackers as source root.
  3. If your're trying to run in command line, add these paths mentioned above to the sys.path at the head of ./videopose.py

Advanced

As this script is based on the VedioPose3D provided by Facebook, and automated in the following way:

args = parse_args()

args.detector_2d = 'alpha_pose'
dir_name = os.path.dirname(video_path)
basename = os.path.basename(video_path)
video_name = basename[:basename.rfind('.')]
args.viz_video = video_path
args.viz_output = f'{dir_name}/{args.detector_2d}_{video_name}.gif'

args.evaluate = 'pretrained_h36m_detectron_coco.bin'

with Timer(video_path):
    main(args)

The meaning of arguments can be found here, you can customize it conveniently by changing the args in ./videopose.py.

Acknowledgement

The 2D pose to 3D pose and visualization part is from VideoPose3D.

Some of the "In the wild" script is adapted from the other fork.

The project structure and ./videopose.py running script is adapted from this repo

Coming soon

The other feature will be added to improve accuracy in the future:

  • Human completeness check.
  • Object Tracking to the first complete human covering largest area.
  • Change 2D pose estimation method such as AlphaPose.
  • Test LightTrack as pose tracker.
  • Multi-person video(complex) support.
  • Data augmentation to solve "high-speed with low-rate" problem: SLOW-MO.