An Inside Look at Computer Vision and Radar — Video Demo

Scott Owner/Technician California Posted   Latest   Edited  

Hello everyone, I recently completed a video demonstration using some new tools I recently acquired. I purchased an NVIDIA Jetson Nano, added a 4K MIPI camera, and set up a few Github projects that allow me to run a Convolutional Neural Network (CNN) program and perform real-time Object Detection and Classification similar to how some ADAS equipped vehicles are operating today. The other tool I acquired is a TI radar demo board that allows one to create visuals in order to see what the radar sensor sees. These experiments are strictly non-scientific and really just some experimentation and as I learn more I plan to share.

I talked with my colleagues over at AESwave to help me with the radar board acquisition, so if you like what you see please let them know.

+24
Interesting
Helpful
Thanks
Agree
Brian Instructor
Ohio
Brian Default
 

Wow Scott, very cool, and crazy at the same time. AI / neural networks are fascinating. Thanks for showing us.

+1
Default Ð Awarded
Thanks
Martin Instructor
British Columbia
Martin Default
 

Nice demonstration and explanation Scott! Thanks for taking the time and expense to put this together.

+1
Default Ð Awarded
Thanks
Michael Mobile Technician
Utah
Michael Default
 

This is so awesome!

+1
Default Ð Awarded
Thanks
G Educator
New York
G Default
 

Scott very interesting....some good examples...nice job

+1
Default Ð Awarded
Thanks
Jaxon Technical Support Specialist
Australia
Jaxon Default
 

Thank you

0
Default Ð Awarded
Bill Owner/Technician
Michigan
Bill Default
 

Scott very interesting thank you for putting this together and sharing it!!!

+1
Default Ð Awarded
Thanks
Anthony Technical Support Specialist
Pennsylvania
Anthony Default
 

Hi Scott: Interesting presentation, thanks for sharing. I've got a question though. Is there a reason that you didn't make this a DN presentation (or a shared credit)? It seems like it could be a potential piece of viral guerrilla marketing. Guido

0
Default Ð Awarded
Scott Owner/Technician
California
Scott Default
 

Hi Guido, This was my first round of experimentation with these tools. Once I‘m able to master some of this tech further, my plan is to assemble some more content and distribute as DN branded material.

0
Default Ð Awarded
Brandon Diagnostician
Pennsylvania
Brandon Default
 

Wow, im very interested in having a look at this , today

0
Default Ð Awarded
Robert Diagnostician
North Carolina
Robert Default
 

This is really cool, the first thing that came to my mind was the Terminator, how they showed us in the movie what it was seeing detecting objects and not knowing back then when the first movie came out thinking it was all fiction ,that it would become a reality today for vehicles.

0
Default Ð Awarded
Hollis Technician
Colorado
Hollis Default
 

Very interesting, thanks Scott. How long did it take you to put this together ? I just spent some time reading up on, NVIDIA Jetson Nano, Convolutional Neural Network and Real-Time Object Detection and Classification then it hit me, all this because folks can't seem to pay attention and drive, almost unbelievable... Do me a favor on the next video, put the clown on the bicycle :O)

+1
Default Ð Awarded
Thanks
Scott Owner/Technician
California
Scott Default
 

Hi Hollis, Time element - many hours over months researching etc. Video shoot, 2 hrs, editing, 2-3 hrs. Can I borrow your clown to bike adapter? ;) Thanks for the feedback!

0
Default Ð Awarded
Ross Owner/Technician
Ontario
Ross Default
 

Thank you for this visual. Very interesting.

+1
Default Ð Awarded
Thanks
Jonathan Technician
Arizona
Jonathan Default
   

Hi Scott, That's pretty impressive. I can't wait until they can identify a Bozo from a mile away however. Thanks for putting that together!

+1
Default Ð Awarded
Funny
Thanks
Scott Owner/Technician
California
Scott Default
 

I’m sure that we’ve both seen our share of “Bozo’s” behind the wheel, perhaps we’ll see a level of machine learning to help these smart cars avoid those drivers!

0
Default Ð Awarded
Bill Technician
Saskatchewan
Bill Default
 

Very cool Scott, Thank you for the demo.

+1
Default Ð Awarded
Thanks
Steve Instructor
California
Steve Default
 

Hi Scott, What I find the most intriguing is the object and probability. My question to you is that video that we are seeing coming directly from the camera or is that the CNN communicating back and forth between the processor and the camera? Also, what make, model and year are we looking at? What version is the front camera? Are you going to be looking at LIDAR and compare it to Radar? If you…

0
Default Ð Awarded
Scott Owner/Technician
California
Scott Default
 

Hi Steve, The video feed is piped to the program and it is performing the inferencing and producing the data overlays AFAIK. This isn’t a specific make or model but it’s widely used technology similar to what Tesla and others do on their vehicles but is not limited to those channels. Think about the Nest/Ring camera systems are running. I would certainly love to check out other tech like LIDAR…

0
Default Ð Awarded
Anthony Engineer
New York
Anthony Default
 

Hi Scott, Great video!! Is the TI mmwave Demo Visualizer an out of the box system or a custom API? How did you get started building your radar project?

0
Default Ð Awarded
Hollis Technician
Colorado
Hollis Default
 

From Scott's post I searched these items. I'm not sure if they're exactly what he used, but it gives some insight. [NVIDIA Jetson Nano] developer​.​nvidia​.​com/embedded/jetso… [Convolutional Neural Network (CNN) and perform real-time Object Detection and Classification] ieeexplore​.​ieee​.​org/document/87233… [TI radar demo…

+3
Default Ð Awarded
Helpful
Agree
Thanks
Anthony Engineer
New York
Anthony Default
 

Hi Hollis, Thank you for the product links and the link to the iee paper on CNN. I am interested in building a project with the TI AWR1642 automotive radar evaluation module. Will also have a look at the NVIDIA AI developer kit.

0
Default Ð Awarded
Scott Owner/Technician
California
Scott Default
 

Hi Anthony, Hollis shared the right links. TI offers the code for free from their website.

0
Default Ð Awarded
Anthony Engineer
New York
Anthony Default
 

Yes, got them. Thanks. Looks like you are doing great work there at Connie and Dicks Service Center.

+1
Default Ð Awarded
Agree