gitwtfhub

wtf is neuralfeels?

facebookresearch/neuralfeels — explained in plain English

Analysis updated 2026-07-17 · repo last pushed 2024-11-13

★ 153PythonAudience · researcherComplexity · 5/5StaleSetup · hard

TL;DR

A research system that combines camera vision and touch-sensor data into a neural model so a robot hand can track and reconstruct the 3D shape of an object it's holding, even novel or partially hidden ones.

Mindmap

mindmap
  root((repo))
    What it does
      Combines vision and touch
      Tracks held objects
      Handles novel objects
    How it works
      Neural field optimization
      3D shape reconstruction
      Real-time updates
    Provided
      Code and trained models
      Simulation and real datasets
      Interactive visualization
    Audience
      Robotics researchers
      Manipulation engineers

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

Why would anyone build with this?

REASON 1

Train or run a neural model that fuses vision and tactile sensor data to reconstruct a held object's 3D shape.

REASON 2

Use the provided datasets and trained models to study vision-touch fusion for robot manipulation research.

REASON 3

Build dexterous manipulation tasks like assembly or sorting that need full 3D understanding of a grasped object.

What's in the stack?

PythonPyTorchCUDA

How it stacks up

facebookresearch/neuralfeelsfacebookresearch/iopathvila-lab/figmirror
Stars153153153
LanguagePythonPythonPython
Last pushed2024-11-132026-06-25—
MaintenanceStaleActive—
Setup difficultyhard—moderate
Complexity5/5—3/5
Audienceresearcherdeveloperresearcher

Figures from each repo's GitHub metadata at analysis time.

How do you spin it up?

Difficulty · hard Time to first run · 1day+

Requires a powerful GPU and a nontrivial installation process to get the neural field pipeline running.

Yoink these prompts

Prompt 1
Explain how NeuralFeels combines camera and tactile sensor data into a single neural field representation.
Prompt 2
Walk me through setting up NeuralFeels on a GPU machine and running the interactive 3D visualization.
Prompt 3
What datasets does this repo provide, and how were the simulation and real-robot datasets collected?
Prompt 4
How would I adapt NeuralFeels to track a new object shape that wasn't in the original training data?

Frequently asked questions

wtf is neuralfeels?

A research system that combines camera vision and touch-sensor data into a neural model so a robot hand can track and reconstruct the 3D shape of an object it's holding, even novel or partially hidden ones.

What language is neuralfeels written in?

Mainly Python. The stack also includes Python, PyTorch, CUDA.

Is neuralfeels actively maintained?

Stale — no commits in 1-2 years (last push 2024-11-13).

How hard is neuralfeels to set up?

Setup difficulty is rated hard, with roughly 1day+ to a first successful run.

Who is neuralfeels for?

Mainly researcher.

View the repo → Decode another repo

This repo across BitVibe Labs

Don't trust strangers blindly. Verify against the repo.

Other repos in this lane