comma ai | Learning a Driving Simulator | Yassine Yousfi | COMMA_CON talks | Research | HQ version
YouTube Viewers YouTube Viewers
194K subscribers
9,954 views
0

 Published On Feb 1, 2024

Date of the stream 29 Jul 2023.
from $1250 - comma 3X https://comma.ai/shop/comma-3x & best ADAS system in the world - https://openpilot.comma.ai

Yassine Yousfi social media:
- https://yassineyousfi.github.io/
-   / yassineyousfi_  

Research:
- https://blog.comma.ai/
- https://github.com/commaai/commavq
- https://github.com/commaai/comma2k19
- https://github.com/commaai/comma10k
- https://github.com/commaai/research

View original video on Youtube:
-    • COMMA_CON | July 29, 2023 | San Diego  
comma.ai youtube playlist:
-    • comma.ai youtube archive (youtube.com...  

Chapters:
00:00 intro
00:20 learning a driving simulator
00:36 imagined, predicted videos via ML
01:44 why a simulator
02:50 why not a classical simulator
04:00 small offset simulator
06:25 learning a driving simulator paper
07:10 ML simulator architecture
07:25 image tokenizer
08:32 source - 5120 bits - 1280 bits
10:25 pose tokenizer (np.digitize)
11:06 dynamics transformer
14:00 it works, straight, turning right, left
14:56 accelerating, braking
15:25 commaVQ opensource dataset
15:50 2x$1000 bounties
16:50 side note 1: tokenized driving models
17:25 side note 2: smooth decoder
18:29 raw - smooth rollout
18:44 side note 3: loss function for driving in ML simulator
20:02 next steps, auto-regressive sampling
22:03 using the learned simulator for traning driving models
22:13 beautiful rollouts properties
23:40 questions
23:52 conditioning on language?
24:10 pose tokens
25:18 bottlenecks to inference for real time
26:23 aplications for human indistinguishable rollouts
27:13 model collapse from simulation data
28:35 mass auto encoders
29:40 video decoder
30:00 INB frame, general i frame and p frame
30:40 output from the simulator used for training policy models
31:00 two models, end to end, same loss function
32:08 policy model will be trained on the simulator, not the vision model
32:25 how to keep up with machine learning research
33:15 you don't want to be the first one to replicate a paper
33:48 state of the art, leveraging other work and open sourcing
34:35 where do the model names come from, Nicki Minaj, Nicholas Cage
34:55 tokenize because cross entropy is the best loss function
35:50 interpreting the tokens
36:45 conditioning on geolocation
37:35 scaling to any type of robotics problem
38:41 adding language as the first input
40:05 why does it flicker
41:30 smoothing decoder, RNN layer

We are not affiliated with comma.ai.
Official communication channels:
- https://comma.ai
-   / comma_ai  
-    / commaai  
- https://blog.comma.ai
- https://github.com/commaai
- https://discord.comma.ai

How to get a job:
- https://comma.ai/jobs
How to collaborate:
- https://comma.ai/services
Buy things to support comma.ai:
- https://comma.ai/shop

Are you interested in openpilot?
Knowledge base:
- https://wiki.comma.ai
Check out the code:
- https://openpilot.comma.ai
Is my car supported?
- https://comma.ai/vehicles
Frequently Asked Questions:
- https://comma.ai/faq
How to setup openpilot:
- https://comma.ai/setup
Comma Secure Shell:
- https://ssh.comma.ai
API Documentation:
- https://api.comma.ai
CAN analysis tool:
- https://cabana.comma.ai
Review and annotate your driving data:
- https://my.comma.ai
Leaderboard:
- https://my.comma.ai/leaderboard
Comma Connect App:
- https://apps.apple.com/us/app/comma-c...
- https://play.google.com/store/apps/de...
Research:
- https://github.com/commaai/comma2k19
- https://github.com/commaai/comma10k
- https://github.com/commaai/research

Official George Hotz communication channels:
- https://geohot.com
-   / georgehotz  
-   / georgehotz  
- https://github.com/geohot
-    / geohot  
-   / realgeorgehotz  

We archive George Hotz and comma.ai videos for fun.
Follow for notifications:
-   / geohotarchive  
Unofficial communities and resources:
-   / comma_ai  
-   / geohot  
- https://commasupport.zendesk.com

Thank you for reading and using the SHOW MORE button.
We hope you enjoy watching George's videos as much as we do.
See you at the next video.

show more

Share/Embed