---
title: Optical Motion Capture AI
sdk: docker
app_port: 7860
app_file: app.py
---

# mocap-ai
Functionality to load FBX files, extract animation, process the animation and write it back to the file.

# Classifier
* Globals: file with hardcoded values like the marker names.
* Utilities:
  * Visualizations
* FBX Handler:
  * Load the `.fbx` file.
  * Go through each frame in the animation frame range and check if all skeleton nodes have a keyframe there.
  * If a keyframe is missing, remove that frame number from the valid frame numbers.
  * After finding all valid frames, go through all marker translation channels and store the global transform in a `pandas` DataFrame.
  * Add the actor numbers as categorical variables.
  * Save the DataFrame to a `.csv` file.
* Inference file loader
  * Same as training file loader, but this one should process all frames regardless of keyframe presence.
* Data augmentation:
  * Isolate a marker set.
  * Translate and rotate (optionally scale) with boundary check.
* Model builder:
  * Instantiate a model with various hyperparameters.
* Training loop:
  * Train given model with callbacks.
* Test loop:
  * Validate model on validation/test data.
* Development script:
  * Create new model, train it and test it.
* Deployment script:
  * Deploys the model in a Docker image on HuggingFace.


## References:
1. PointNet:
- Research paper: Qi, Charles R., et al. "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation." CVPR. 2017. [arXiv:1612.00593](https://arxiv.org/abs/1612.00593)
- Official code repository (TensorFlow): https://github.com/charlesq34/pointnet
- Official code repository (PyTorch): https://github.com/fxia22/pointnet.pytorch
2. PointNet++:
- Research paper: Qi, Charles R., et al. "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space." NeurIPS. 2017. [arXiv:1706.02413](https://arxiv.org/abs/1706.02413)
- Official code repository (TensorFlow): https://github.com/charlesq34/pointnet2
- Official code repository (PyTorch): https://github.com/erikwijmans/Pointnet2_PyTorch
3. DGCNN:
- Research paper: Wang, Yue, et al. "Dynamic Graph CNN for Learning on Point Clouds." ACM Transactions on Graphics (TOG) 38.5 (2019): 1-12. [arXiv:1801.07829](https://arxiv.org/abs/1801.07829)
- Official code repository (TensorFlow): https://github.com/WangYueFt/dgcnn
- Official code repository (PyTorch): https://github.com/muhanzhang/DGCNN