Published On Feb 1, 2024
Date of the stream 29 Jul 2023.
from $1250 - comma 3X https://comma.ai/shop/comma-3x & best ADAS system in the world - https://openpilot.comma.ai
Yassine Yousfi social media:
- https://yassineyousfi.github.io/
- / yassineyousfi_
Research:
- https://blog.comma.ai/
- https://github.com/commaai/commavq
- https://github.com/commaai/comma2k19
- https://github.com/commaai/comma10k
- https://github.com/commaai/research
View original video on Youtube:
- • COMMA_CON | July 29, 2023 | San Diego
comma.ai youtube playlist:
- • comma.ai youtube archive (youtube.com...
Chapters:
00:00 intro
00:20 learning a driving simulator
00:36 imagined, predicted videos via ML
01:44 why a simulator
02:50 why not a classical simulator
04:00 small offset simulator
06:25 learning a driving simulator paper
07:10 ML simulator architecture
07:25 image tokenizer
08:32 source - 5120 bits - 1280 bits
10:25 pose tokenizer (np.digitize)
11:06 dynamics transformer
14:00 it works, straight, turning right, left
14:56 accelerating, braking
15:25 commaVQ opensource dataset
15:50 2x$1000 bounties
16:50 side note 1: tokenized driving models
17:25 side note 2: smooth decoder
18:29 raw - smooth rollout
18:44 side note 3: loss function for driving in ML simulator
20:02 next steps, auto-regressive sampling
22:03 using the learned simulator for traning driving models
22:13 beautiful rollouts properties
23:40 questions
23:52 conditioning on language?
24:10 pose tokens
25:18 bottlenecks to inference for real time
26:23 aplications for human indistinguishable rollouts
27:13 model collapse from simulation data
28:35 mass auto encoders
29:40 video decoder
30:00 INB frame, general i frame and p frame
30:40 output from the simulator used for training policy models
31:00 two models, end to end, same loss function
32:08 policy model will be trained on the simulator, not the vision model
32:25 how to keep up with machine learning research
33:15 you don't want to be the first one to replicate a paper
33:48 state of the art, leveraging other work and open sourcing
34:35 where do the model names come from, Nicki Minaj, Nicholas Cage
34:55 tokenize because cross entropy is the best loss function
35:50 interpreting the tokens
36:45 conditioning on geolocation
37:35 scaling to any type of robotics problem
38:41 adding language as the first input
40:05 why does it flicker
41:30 smoothing decoder, RNN layer
We are not affiliated with comma.ai.
Official communication channels:
- https://comma.ai
- / comma_ai
- / commaai
- https://blog.comma.ai
- https://github.com/commaai
- https://discord.comma.ai
How to get a job:
- https://comma.ai/jobs
How to collaborate:
- https://comma.ai/services
Buy things to support comma.ai:
- https://comma.ai/shop
Are you interested in openpilot?
Knowledge base:
- https://wiki.comma.ai
Check out the code:
- https://openpilot.comma.ai
Is my car supported?
- https://comma.ai/vehicles
Frequently Asked Questions:
- https://comma.ai/faq
How to setup openpilot:
- https://comma.ai/setup
Comma Secure Shell:
- https://ssh.comma.ai
API Documentation:
- https://api.comma.ai
CAN analysis tool:
- https://cabana.comma.ai
Review and annotate your driving data:
- https://my.comma.ai
Leaderboard:
- https://my.comma.ai/leaderboard
Comma Connect App:
- https://apps.apple.com/us/app/comma-c...
- https://play.google.com/store/apps/de...
Research:
- https://github.com/commaai/comma2k19
- https://github.com/commaai/comma10k
- https://github.com/commaai/research
Official George Hotz communication channels:
- https://geohot.com
- / georgehotz
- / georgehotz
- https://github.com/geohot
- / geohot
- / realgeorgehotz
We archive George Hotz and comma.ai videos for fun.
Follow for notifications:
- / geohotarchive
Unofficial communities and resources:
- / comma_ai
- / geohot
- https://commasupport.zendesk.com
Thank you for reading and using the SHOW MORE button.
We hope you enjoy watching George's videos as much as we do.
See you at the next video.