commaai / comma2k19 Goto Github PK

The rosbag version of the dataset maybe more convenient for testing and evaluation.

In readme text, magnetic sensor value and time should be in IMU folder, but i cannot find it.
I think it is impossible to translate coordinate from ECEF to local without that value.

Please answer where the magnetic sensor value. If it is possible to estimate pose without that, can you explain about the method?

The dataset provides the decoded can data radar, steering_wheel, speed, and wheel_speed. However, it is unclear how can one access other signals that might be available within the raw can data at the path segment_number/processed_log/CAN/raw_can/.

In other words, is there a list of all the signals available in the can_raw, or how can you obtain it?

For example, how can I obtain the signals of the blinking indicator light, pedal positions, gear number, etc?

get the pixel first using the lane detected model (such as lanenet), and then map it to 3d？
use comma10k segment model to generate lane line ? thanks

Hi there - thanks so much for this dataset. I've been searching for days and this looks like the best out there for my needs. I'm currently trying to build my own E2E model from scratch using this dataset. Input: image, output: pedal, break, steering.

Hope you can help 🙏

I'm feeding my model with a DataFrame with jpeg img path (frame), steering, break and pedal. Thus I need it in this format in the DataFrame for my model to work.

If I understand correctly...

Pedal = processed_log > IMU > acceleration - correct?
Steering Angle = processed_log > CAN data > steering_angle - correct?

Question #1 Where is the break?

Also, when looking at...

Dataset_chunk_n
|
+-- route_id (dongle_id|start_time)
    |
    +-- segment_number
        |
        ...
        +-- video.hevc (video file, can be read with openpilot-tools: framereader)
        +-- processed_log/ (processed logs as numpy arrays, see format for details)

Question #2: I assume that the index in the processed_log numpy array maps to the frame in the video.hevc? e.g.

# loop through each segment, then...
fr = FrameReader(current_segment + 'video.hevc')
steering_angles = np.load(current_segment + 'processed_log/CAN/steering_angle/value')
log_index = 0

for item in steering_angles:
      steering_angle = steering_angles[log_index]
      frame_img = fr.get(frame_index, pix_fmt='rgb24')[0]
      log_index += 1

Hope that makes sense. Thanks in advance!!!

99c94dc769b5d96e--2018-08-08--11-11-16/29/ (this is a example)

Hey, First of all Thank You for this Dataset.

I read through the arxiv paper and it mentions that pose is calculated by Mesh3D and was wondering if it open source or is there a paper out there explaining the same

@BraunPhilipp @HaraldSchafer
Hi,
I have a problem in mapping the can data to frames data. I have used the following snippet of code

example_segment = '../Example_1/b0c9d2329ad1606b|2018-08-02--08-34-47/40/'`
can_time = np.load(example_segment + 'processed_log/CAN/radar/t')
fr_time = np.load(example_segment + 'global_pose/frame_times')
out = [min(range(len(can_time)), key=lambda can_idx: abs(can_time[can_idx]-ts)) for ts in fr_time]

But it gives me value error that num+count > frame_count. How to resolve this issue. Kindly help me in this regard.

Not sure if I am missing something obvious but I downloading a segment from my.comma.ai. Both the log and the video and now I am using the frame reader and log reader and successfully have the values out from my drive but I ended up with roughly ~1100 frames and ~6000 speed readings. Looks to me like the video and frames aren't necessarily lined up. Is there a way to do this? I sshed into the device and saw it was already being sharded into pieces on there. Is there any way to keep it as one continuous segment or is there already a way to splice the segments together?

Sorry for the question dump but interested in getting involved and trying some things out but these aren't completely clear to me yet.

Probably being very dumb here, but I'm trying to map the steering_angle value to a frame in the video to try to build a very basic NN to input video frame and output steering_angle.

In your dataset example, the video contains 1200 frames, but the values from processed_log/CAN/steering_angle/value contains 4974 values.

The times in processed_log/CAN/steering_angle/t do not directly map to the times in global_pose/frame_times, how does one map a frame in the video to the values?

Thanks in advance!!

Hi, I was trying to execute raw_reader.py to read the rlog files of my own, and I encountered the following error.
subprocess.CalledProcessError: Command '['make', 'PHONELIBS=/home/hao/Documents/openpilot/phonelibs']' returned non-zero exit status 2.
I found the error comes from the index_log function in logreader.py
def index_log(fn): index_log_dir = os.path.join(os.path.dirname(os.path.realpath(__file__)), "index_log") index_log = os.path.join(index_log_dir, "index_log") phonelibs_dir = os.path.join(OP_PATH, 'phonelibs') subprocess.check_call(["make", "PHONELIBS=" + phonelibs_dir], cwd=index_log_dir, stdout=subprocess.DEVNULL)

Acutally I can't understand what does the command ' make PHONELIBS= phonelibs_dir ' do? But the issues is coming from this command.

I'm using python 3.6 in Ubuntu. Can anyone provide some insights?

To whom it may concern,

When using numpy.load() to load the magnetic data,

(ex. 99c94dc769b5d96e|2018-10-12--20-13-54/7/processed_log/IMU/magnetometer/value)

there is only one axis data as shown below:

magn_val_arr.shape
(6013,)
magn_val_arr.size
6013

but accelerometer contains three axes data

accel_val_arr.shape
(6013, 3)
accel_val_arr.size
18039

Is data missing or do I load magnetic data incorrectly?

Thank you.

Hi,

After exploring the data a little, I found that sometimes the steering angles seem to take values higher than 300. I assume the units are in degrees. If so, how come these angles are higher than 60 for cars in this data?

There is a openpilot_tools.lib.exceptions.DataUnreadableError when I use openpilot_tools to read Chunk_1/b0c9d2329ad1606b|2018-07-29--16-37-17/5/video.hevc . There are two possible reasons:

• The installation of openpilot_tools is uncompleted.
• The datasets are broken, e.g. Chunk_1/b0c9d2329ad1606b|2018-07-29--16-37-17/5/video.hevc

I believe that I have installed openpilot_tools successfully. Do you have any suggestions? Thank you!

Error details:

(mapnet_release) [sun@pc]$ python
Python 2.7.15 |Anaconda, Inc.| (default, Oct 10 2018, 21:32:13) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from openpilot_tools.lib.framereader import FrameReader
>>> fr = FrameReader('video.hevc')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 869, in FrameReader
    index_data = get_video_index(fn, frame_type, cache_prefix)
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 201, in get_video_index
    index_video(fn, frame_type, cache_prefix)
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 193, in index_video
    index_stream(fn, "hevc", cache_prefix=cache_prefix)
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 119, in cache_inner
    cache_value = func(fn, *args, **kwargs)
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 136, in index_stream
    probe = ffprobe(f.name, typ)
  File "/mnt/lustre/sun/App/openpilot/openpilot_tools/lib/framereader.py", line 79, in ffprobe
    raise DataUnreadableError(fn)
openpilot_tools.lib.exceptions.DataUnreadableError: video.hevc

Hi there.

Just wondering the procedure to calculate the quaternion from any OpenPilot log. I've seen that none of the messages sent in liveLocationKalman has 4 values. All of them are 3. So, how can I do it ?

many thanks in advance.

In the README, the radar data is described as having 6 components:

|  +--radar: [forward distance (m),
|  |          left distance (m),
|  |          nan,
|  |          nan,
|  |          address,
|  |          new_track (bool)]

However, when I dumped the numpy array it had a shape of (6770, 7).

Example data:

[ 50.9 -4.4 1.55 nan nan 530. 0. ]

Is it meant to be [forward distance, left distance, down distance, nan, nan, address, new track]?

Minor Spelling change.
Sattelite to Satellite

Hello,

My question may sound trivial, but I just wanted to make sure - how can I sync the video and the sensors, e.g. steering wheel angle?

thanks for your kind to release the project! I find loc_kf.py in openpilot , I merge the Gnss to KF according to laika. but , when I merge the imu data, I find the ECEF_ORIENTATION of the KF output is abnormal, I dont find how to put imu_frame as input! I am look forward to your replay,thaks ! (I put this upload file to notebooks dir)
my.zip

Hi and thank you very much for providing this dataset!

may I ask you to provide more information about the generation of the data inside the processed_log directories? I assume it is generated from the raw_log.bz2 file also provided in each segment. I had a look at the openpilot-tools, but I couldn't locate scripts required to generate all these files.

Therefore, I would like to ask you, to add your scripts that were used to generate the processed_log directory to this repo or at least provide some more information about its generation. I especially struggle with the can/raw_can/t file, as the capnp log only provides the monotonic bus time.

Thanks a lot in advance.

In the README, it says that

+--raw CAN: This not stored as a value array but as three seperate arrays [src, address, data]

However, I did not see any example notebook that is showing information from raw CAN. How do you decode and process those arrays [src, address, data]?

Does this raw CAN have other signals apart from car_speed , steering_angle , wheel_speeds , radar?

Hello comma, I am your Chinese fan, may I ask a question？
In https://github.com/commaai/comma2k19/blob/master/assets/testmesh3d.png, how is lane line information plotted?