Skip to content

feat: Add Transformer-based haptic encoder with multi-task learning#2

Open
Rakshitha-Ireddi wants to merge 1 commit intoempriselab:releasefrom
Rakshitha-Ireddi:feature/transformer-multitask-learning
Open

feat: Add Transformer-based haptic encoder with multi-task learning#2
Rakshitha-Ireddi wants to merge 1 commit intoempriselab:releasefrom
Rakshitha-Ireddi:feature/transformer-multitask-learning

Conversation

@Rakshitha-Ireddi
Copy link

Authors

  • Ireddi Rakshitha
  • Yashwanth Devavarapu

Summary

  • Add TransformerHapticEncoder with temporal attention and MC Dropout uncertainty
  • Add CrossModalAttention and HierarchicalFusion for improved modality fusion
  • Add MultiTaskVisuoHapticModel for joint material/hardness/temperature prediction
  • Add comprehensive haptic time series augmentation utilities
  • Add training scripts with uncertainty-weighted multi-task loss

This contribution addresses the need for:

  1. Attention-based temporal modeling (vs. InceptionTime convolutions)
  2. Interpretable cross-modal fusion (vs. simple concatenation)
  3. Multi-task learning for richer scene understanding
  4. Uncertainty quantification for safe robotic decision-making

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant