Watch the noun in the title: not "vehicle," but "autonomous machine." NVIDIA's grant US11688181B2 ("Sensor fusion for autonomous machine applications using machine learning," issued June 27, 2023; inventors Minwoo Park, Junghyun Kwon, Mehmet K. Kocamaz, Hae-Jong Seo, Berta Rodriguez Hervas, Tae Eun Choe) is drafted to span vehicles and robots alike. That generality is the strategic core for a company that wants to be the perception layer of every autonomous thing.
Claim 1 is short and deliberately abstract, and reading it explains the breadth. A processor's circuits "receive first data representative of a plurality of outputs of a plurality of deep neural networks (DNNs)," where at least one output corresponds to "a respective sensor having a respective field of view different from" the others; "compute, using a fusion DNN…second data representative of a fusion of the plurality of outputs"; and "perform one or more operations using the autonomous machine" on that fused result. The key move is that the inputs to the fusion network are themselves already-DNN outputs — fusion happens late, on per-sensor detections, and the fusion itself is learned. There is no hand-tuned Kalman filter or hand-weighted modality blend in the claim; a network arbitrates. That is the camp marker: data, not hand-coded sensor models, decides how camera, LiDAR, and radar combine.
The field-of-view language is doing real work, and the dependent claims show why. Claim 7 recites a first output for "a first field of view," a second for "a second field of view different from the first," and a fusion that "corresponds to both," with claim 8 adding that the two FOVs are "at least partially overlapping." Claim 10 then states the actual problem being solved: a first output has "a first representation of an object," a second output has "a second representation of the object," and the fusion produces "a fused representation of the object." The invention is reconciling the same object seen by two sensors with overlapping but offset views — the duplicate-detection problem the abstract calls out, where the fusion network is "trained to account for multiple instances of a same object appearing in different input representations." A naive union of detections double-counts; a learned fusion that knows about boundary/overlap regions does not.
“In various examples, a multi-sensor fusion machine learning model—such as a deep neural network (DNN)—may be deployed to fuse data from a plurality of individual machine learning models.”— U.S. Patent No. 11,688,181 source
The data representation is concrete in the second independent claim. Claim 12 has the fusion DNN consume "a first rasterized image" from one DNN and a "second rasterized image" from another, and output "a fused rasterized image including both the first object and the second object." Rasterizing each sensor's detections into a common image grid before fusion is the engineering trick that lets a convolutional fusion network operate — it puts heterogeneous sensor outputs into one spatial frame the network can convolve over. Claim 15 enriches each cell with "at least one probability distribution function…corresponding to one or more potential locations of an detected object" (so the fusion sees per-detection uncertainty), and claim 16 adds "velocity representations" with "a velocity in an x-direction or a velocity in a y-direction" encoded into the grid. The network fuses not just positions but uncertainties and motion.
The modality and platform breadth are then layered on. Claim 9 lets the inputs include outputs from "a LiDAR sensor, a RADAR sensor, or an ultrasonic sensor," generated "using another DNN or without using another DNN" — so a sensor whose output never went through a network can still feed the fuser. Claim 20 makes the heterogeneity explicit: a camera image processed by a DNN fused with a LiDAR/radar/ultrasonic image "generated without using a DNN." And claim 11/17's boilerplate list — "a control system…a perception system…a robot…a system implemented at least partially using cloud computing resources" — is the deliberate platform-agnostic framing. CPC-wise, G06T 7/292 marks multi-sensor tracking and G06V 20/588/58 the learned recognition of road features and objects, with B60W 60/00272 marking autonomous control on object behavior.
For the control-and-perception beat, learned late fusion of per-sensor DNN outputs — rasterized into a shared grid, carrying uncertainty and velocity — is a meaningful bet. It aligns with NVIDIA's Isaac/Drive strategy: own the learned perception primitives that every robot and car builder will license rather than build. The broad platform language maximizes the number of downstream systems that read on the claim.
From a portfolio angle, this is arms-dealer IP. The "autonomous machine" framing keeps the claims platform-agnostic, so they read on a humanoid, a robotaxi, and a warehouse AMR alike. Read alongside NVIDIA's other perception grants, this is a deliberate horizontal fence around learned multi-sensor fusion.
Caveats. "Autonomous machine" in a title doesn't guarantee claim 1 is platform-agnostic — the B60W code signals a vehicle embodiment, and an accused product likely has to fuse DNN outputs with a fusion DNN (not just any sensor blend) to read on it. Learned fusion is also heavily published; the grant turns on the specifics — late fusion of per-sensor network outputs, the rasterized common frame, the duplicate-object reconciliation. Read claim 1 with claims 10 and 12 for whether an accused system matches that architecture.
For the file: a horizontally-framed learned-late-fusion grant central to NVIDIA's autonomy-platform play, distinguished by fusing per-sensor DNN outputs in a shared rasterized frame and reconciling duplicate objects across overlapping FOVs. Pull US11688181B2 in the patent record, read claims 1, 10, and 12 for the fusion architecture and the platform language — that determines how wide the fence really is.
Comments
Loading comments…