Repository navigation
Reuse space-ros/simulation repository for IsaacSim #18
Description
Since the contents of this repository are moved to space-ros/demos based on space-ros/space-ros#178, I believe this repository can be reused exclusively for Isaac Sim which leverages photorealistic simulations.
Feature Proposal
The proposal is to repurpose the current repository to manage the simualtion assets and extensions for Space ROS simulation with Nvidia's Isaac SIM. Eventually the repository will be repurposed as the extension pack of Space ROS provided for Nvidia's Isaac SIM. The assets are made photo-realistic with a minimal sim2real gap with the information provided in the internet resources. The assets are a compilation of large set of use cases related to Space ROS applications which is summarized below. In addition to the assets, new extensions are created for space ros demos which serve as an example for building an ecosystem of simualtion extensions which could grow over time. A detailed documentation of all the assets and extensions are provided to help with the usage of the extension module.
Environment Assets
-
International Space Station Model. Useful for Canadarm2 related maintainence and docking applications.
-
Interior of International Space Station. Useful for Astrobee and Robonaut2 for assistance related applications.
-
Mars Terrain. 800 sq.m area of realistic mars terrain data acquired from NASA's Mars Exploration Data.
-
Moon Terrain. 800 sq.m area of realistic moon terrain data acquired from NASA's Moon Exploraiton Data.
-
Simple Chess Envrionment for Robonaut2. Useful in manipulation and control related tasks for Robonaut2.
-
Simple Mars Replica for Ingenuity Helicopter. Useful for Visual Odometry, Search and Mission Planning related applidations.
Robot Assets
- Canadarm2. The gazebosim equivalent of canadarm2 robot model.
- Curiosity Mars Rover. The gazebosim equivalent of curiosity Mars rover.
- Perseverance Mars Rover. The high fidelity Robot Model with advanced sensors for Mars exploration related tasks.
- Robonaut2 Upper body. Manipulation tasks and planning for robonaut2 upper body.
- Robonaut2 Full body. Useful for applications with Interior of ISS.
- Ingenuity Helicopter. Aerial robot for search applications in mars terrain.
- Canadarm2 Destro. The docking and maintainence robot for ISS.
IsaacSim Extensions
- An SpaceROS specific Omnigraph extensions for controlling rovers with rocker-bogie mechanism
- An example omnigraph extensions for building new extension specific to Space ROS
Documentation
- Detailed description and information on Environment assets
- Detailed description and usage instructions on robot assets
- Detailed description and usage instructions on extensions
The assets are widely tested for its functionality and are mostly simulation ready at the moment. The demo packages are already being built for direct integration between gazebosim and isaac sim. These assets should help cover wide range of subdomains and features exptected to be built on space ROS project.
The potential directions moving forward from this feature is to create a standalone simulator built on top of Nvidia's Omniverse for better control in packaging and distributing the simulation assets and version controls of extensions.
MetaInfo
Title: Simulation Assets and Extensions for Nvidia's Isaac Sim
Freelancer ID: @selvakumarhs
NASA Space ROS SIM Challenge
Activity
Remove Simulation assets of gazebo enviroment and urdf files
Update README.md and documentation for Isaac Sim Extension Module
Update README.md and documentation for Isaac Sim Extension Module
franklinselva commentedon Sep 8, 2024
@mkhansenbot @EzraBrooks I am trying to push the work on simulation assets for Nvidia's Isaac Sim. I am using
git-lfsfor pushing the large files to the repository but couln't succeed in pushing the changes to the fork.I am thrown with,
batch response: @franklinselva can not upload new objects to public fork franklinselva/spaceros-simulation batch response: @franklinselva can not upload new objects to public fork franklinselva/spaceros-simulation error: failed to push some refs to 'https://github.com/franklinselva/spaceros-simulation.git'Upon looking on the issue related to this, the commit on
git-lfsmentions the repository should already have lfs setup on the original repository or the contributor should have the push permissions for the original repository i.e. write access tospace-ros/simulation. Can you please help me resolve this issue?Add documentation for the repository
Add extensions for IssacSim targetting Space ROS
Minor fixes to extensions docs
Update robot documentations for required extensions
EzraBrooks commentedon Sep 9, 2024
I'm not 100% sold on the idea that Isaac Sim, a proprietary simulator, should live in the same repository as non-proprietary simulators. I don't want Space ROS to trend toward being vendor-locked by NVIDIA. I don't even have a computer with an NVIDIA GPU 😉
franklinselva commentedon Sep 9, 2024
One of the questions on the SpaceROS SIM Webinar was to choose Nvidia's Isaac Sim / Unreal engine for improved visual fidelity. The response to the question was we could go for one of the simulators in this case.
The Q&A of the webinar can be found here. Rows 3 and 27 are related to this issue.
I completely agree with the setup of requiring NVidia GPU for such simulators. For this purpose, I have already managed to write a workaround for this. More information can be found here.
Moving further, IsaacSim is not a definitive simulator but a reference application built on top of Nvidia Omniverse. This feature intends to come up with a dedicated application as I referred to in the description of this issue.
EzraBrooks commentedon Sep 9, 2024
Gotcha. I was more just saying "maybe it doesn't live in the same place as the Gz simulation", not that it doesn't have a place at all. I totally understand the desire for higher-fidelity perception simulation.
I'll take a look at setting up LFS here, since that's a good improvement anyway.
franklinselva commentedon Sep 9, 2024
Thanks a lot. I am currently setting up the LFS template for an easier entry point. I am stuck with pushing the assets to the repository at the moment. Is it something, I could overcome soon? The due date for the challenge is right around the corner.
EzraBrooks commentedon Sep 9, 2024
See #26, #27
EzraBrooks commentedon Sep 9, 2024
LFS should be enabled on the repo now. Let me know if it still gives you issues - maybe it's a permissions thing.
If it is, I'm not sure what the right route forward here is - giving upstream write access to everyone who wants to contribute doesn't seem sustainable.
Remove Simulation assets of gazebo enviroment and urdf files
Add documentation for the repository
Add extensions for IssacSim targetting Space ROS
Minor fixes to extensions docs
Update robot documentations for required extensions
Move asset images to git lfs
Add simulation assets for Nvidia's Isaac Sim
Update README.md for git-lfs instructions
franklinselva commentedon Sep 9, 2024
LFS setup works good. I have already pushed the change. Thanks for the help!
Improve suspension arm PD gains of curiosity rover for torque control
Add Perseverance Rover Robot Model