Association and annotation in non-keyframe samples

[cyh767]

I use the following code to get a non-keyframe sample

from nuscenes.nuscenes import NuScenes
nusc = NuScenes(version=version, dataroot=root_path, verbose=True)
#get keyframe sample
sample = nusc.sample[0]
cam_token = sample["data"]["CAM_FRONT"]
sd_rec_cam = nusc.get('sample_data', cam_token)
#get non-keyframe sample
cam_token = sd_rec_cam["next"]
sd_rec_cam = nusc.get('sample_data', cam_token)
sample_nonkeyframe = nusc.get("sample", sd_rec_cam["sample_token"])

It seems sample_nonkeyframe contains a data field and an anns field, like the keyframe samples. Does it mean LIDAR and 6-way cameras in non-keyframe data are also associated by the nearest timestamp? Are the annotations created by something like interpolation of keyframe ground truths?

Thanks for your help!

[whyekit-motional]

@cyh767 for non-keyframe data, there is no guarantee of closeness in timestamp across the various sensors, and neither are there annotations

[cyh767]

Thank you! I have a further question. Would top lidar calibration information and ego pose be valid for non-keyframes, if I get them using something like

lidar_token = "token of a non-keyframe top lidar frame"
sd_rec = nusc.get('sample_data', lidar_token)
cs_record = nusc.get('calibrated_sensor', sd_rec['calibrated_sensor_token'])
pose_record = nusc.get('ego_pose', sd_rec['ego_pose_token'])

[whyekit-motional]

@cyh767 you can take a look at https://github.com/nutonomy/nuscenes-devkit/blob/master/docs/schema_nuscenes.md#sample_data

In particular:

For non key-frames the sample_data points to the sample that follows closest in time.