File size: 2,444 Bytes
beca475 4631d6a beca475 4631d6a beca475 4631d6a a0a49ab 4631d6a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
---
title: Optical Motion Capture AI
sdk: docker
app_port: 7860
app_file: app.py
---
# mocap-ai
Functionality to load FBX files, extract animation, process the animation and write it back to the file.
# Classifier
* Globals: file with hardcoded values like the marker names.
* Utilities:
* Visualizations
* FBX Handler:
* Load the `.fbx` file.
* Go through each frame in the animation frame range and check if all skeleton nodes have a keyframe there.
* If a keyframe is missing, remove that frame number from the valid frame numbers.
* After finding all valid frames, go through all marker translation channels and store the global transform in a `pandas` DataFrame.
* Add the actor numbers as categorical variables.
* Save the DataFrame to a `.csv` file.
* Inference file loader
* Same as training file loader, but this one should process all frames regardless of keyframe presence.
* Data augmentation:
* Isolate a marker set.
* Translate and rotate (optionally scale) with boundary check.
* Model builder:
* Instantiate a model with various hyperparameters.
* Training loop:
* Train given model with callbacks.
* Test loop:
* Validate model on validation/test data.
* Development script:
* Create new model, train it and test it.
* Deployment script:
* Deploys the model in a Docker image on HuggingFace.
## References:
1. PointNet:
- Research paper: Qi, Charles R., et al. "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation." CVPR. 2017. [arXiv:1612.00593](https://arxiv.org/abs/1612.00593)
- Official code repository (TensorFlow): https://github.com/charlesq34/pointnet
- Official code repository (PyTorch): https://github.com/fxia22/pointnet.pytorch
2. PointNet++:
- Research paper: Qi, Charles R., et al. "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space." NeurIPS. 2017. [arXiv:1706.02413](https://arxiv.org/abs/1706.02413)
- Official code repository (TensorFlow): https://github.com/charlesq34/pointnet2
- Official code repository (PyTorch): https://github.com/erikwijmans/Pointnet2_PyTorch
3. DGCNN:
- Research paper: Wang, Yue, et al. "Dynamic Graph CNN for Learning on Point Clouds." ACM Transactions on Graphics (TOG) 38.5 (2019): 1-12. [arXiv:1801.07829](https://arxiv.org/abs/1801.07829)
- Official code repository (TensorFlow): https://github.com/WangYueFt/dgcnn
- Official code repository (PyTorch): https://github.com/muhanzhang/DGCNN |