[Bug Report] Camera info does not contain all the segments label for multiple env

[exaFLOPs26]

If you are submitting a bug report, please fill in the following details and use the tag [bug].

Describe the bug

Based on the information, the info should contain the segmentation based label. But found that for more than 1 env around more than 4, some objects seems to be missing in the info label.

Currently, I ran a code and checking the camera info. And below, I only got env_0 and 1 mug but not for env_2 and 3. Please fix the bug so that we can get all the labels. Or if there a way to mitigate this bug, please let me know!

ipdb> len([p for p in info[ANNOT]["idToLabels"].values() if isinstance(p, str) and "env_1" in p])
382
ipdb> len([p for p in info[ANNOT]["idToLabels"].values() if isinstance(p, str) and "env_2" in p])
353
ipdb> len([p for p in info[ANNOT]["idToLabels"].values() if isinstance(p, str) and "env_3" in p])
364
ipdb> len([p for p in info[ANNOT]["idToLabels"].values() if isinstance(p, str) and "env_0" in p])
381

ipdb> [p for p in info[ANNOT]["idToLabels"].values() if isinstance(p, str) and "mug0" in p]
['/World/envs/env_0/Kitchen/mug0/simplified_obj/simplified_obj', '/World/envs/env_1/Kitchen/mug0/simplified_obj/simplified_obj']
ipdb>

Steps to reproduce

Any code with parallel env with a camera.

[RandomOakForest]

Thank you for posting this. What version of Isaac Lab and Sim are you using? Could you confirm this in Isaac Lab 3.0.0-beta? Thanks.