Landmark location reconstruction in autonomous machine applications
Granted: April 27, 2021
Patent Number:
10991155
In various examples, locations of directional landmarks, such as vertical landmarks, may be identified using 3D reconstruction. A set of observations of directional landmarks (e.g., images captured from a moving vehicle) may be reduced to 1D lookups by rectifying the observations to align directional landmarks along a particular direction of the observations. Object detection may be applied, and corresponding 1D lookups may be generated to represent the presence of a detected vertical…
Adaptive shading in a graphics processing pipeline
Granted: April 27, 2021
Patent Number:
10991152
One embodiment of the present invention includes a parallel processing unit (PPU) that performs pixel shading at variable granularities. For effects that vary at a low frequency across a pixel block, a coarse shading unit performs the associated shading operations on a subset of the pixels in the pixel block. By contrast, for effects that vary at a high frequency across the pixel block, fine shading units perform the associated shading operations on each pixel in the pixel block. Because…
Using previously rendered scene frames to reduce pixel noise
Granted: April 27, 2021
Patent Number:
10991079
This disclosure presents a method to denoise a ray traced scene where the ray tracing uses a minimal number of rays. The method can use temporal reprojections to compute a weighted average to the scene data. A spatial filter can be run on the scene data, using the temporal reprojection count to reduce the size of the utilized spatial filter radius. In some aspects, additional temporal filters can be applied to the scene data. In some aspects, global illumination temporal reprojection…
System frequency margin recovery via distributed critical path monitors (CPM)
Granted: April 27, 2021
Patent Number:
10990732
Introduced herein is an improved technique of recovering system frequency margin via distributed CPMs. The introduced technique creates and distributes multiple sets of always sensitized critical path replicas across a chip and monitors them for timing failure. The introduced technique takes feedback from these critical path replicas and dynamically boosts the clock frequency of the chip to remove the margin. The introduced technique provides more accurate and more comprehensive coverage…
Performing traversal stack compression
Granted: April 20, 2021
Patent Number:
10984049
A method, computer readable medium, and system are disclosed for performing traversal stack compression. The method includes traversing a hierarchical data structure having more than two children per node, and during the traversing, creating at least one stack entry, utilizing a processor, where each stack entry contains a plurality of intersected nodes, and adding the at least one stack entry to a compressed traversal stack stored in a memory, utilizing the processor.
Scene flow estimation using shared features
Granted: April 20, 2021
Patent Number:
10986325
Scene flow represents the three-dimensional (3D) structure and movement of objects in a video sequence in three dimensions from frame-to-frame and is used to track objects and estimate speeds for autonomous driving applications. Scene flow is recovered by a neural network system from a video sequence captured from at least two viewpoints (e.g., cameras), such as a left-eye and right-eye of a viewer. An encoder portion of the system extracts features from frames of the video sequence. The…
Haptic control interface for detecting content features using machine learning to induce haptic effects
Granted: April 20, 2021
Patent Number:
10984637
Haptic effects have long been provided to enhance content, such as by providing vibrations, rumbles, etc. in a remote controller or other device being used by a user while watching or listening to the content. To date, haptic effects have either been provided by programming controls for the haptic effects within the content itself, or by providing an interface to audio that simply maps certain haptic effects with certain audio frequencies. The present disclosure provides a haptic control…
Virtual photogrammetry
Granted: April 20, 2021
Patent Number:
10984587
Multiple snapshots of a scene are captured within an executing application (e.g., a video game). When each snapshot is captured, associated color values per pixel and a distance or depth value z per pixel are stored. The depth information from the snapshots is accessed, and a point cloud representing the depth information is constructed. A mesh structure is constructed from the point cloud. The light field(s) on the surface(s) of the mesh structure are calculated. A surface light field…
Estimating depth for a video stream captured with a monocular rgb camera
Granted: April 20, 2021
Patent Number:
10984545
Techniques for estimating depth for a video stream captured by a monocular image sensor are disclosed. A sequence of image frames are captured by the monocular image sensor. A first neural network is configured to process at least a portion of the sequence of image frames to generate a depth probability volume. The depth probability volume includes a plurality of probability maps corresponding to a number of discrete depth candidate locations over a range of depths defined for the scene.…
Domain stylization using a neural network model
Granted: April 20, 2021
Patent Number:
10984286
A style transfer neural network may be used to generate stylized synthetic images, where real images provide the style (e.g., seasons, weather, lighting) for transfer to synthetic images. The stylized synthetic images may then be used to train a recognition neural network. In turn, the trained neural network may be used to predict semantic labels for the real images, providing recognition data for the real images. Finally, the real training dataset (real images and predicted recognition…
Addressing cache slices in a last level cache
Granted: April 20, 2021
Patent Number:
10983919
An addressing scheme in systems utilizing a number of operative memory slices in a last level cache that is not evenly divisible by a number of memory channels utilizes the operative slices exposes the full last level cache bandwidth and capacity to data processing logic in a high-performance graphics system.
Queue manager for streaming multiprocessor systems
Granted: April 20, 2021
Patent Number:
10983699
A queue manager apparatus converts inbound commands of a first width into scalar format commands to be queued in a command queue. Furthermore, the queue manager converts the scalar format commands residing in the command queue into outbound commands of a second width for transmission. Converting inbound commands to scalar format commands and then converting the scalar format commands to a target width for transmission allows the queue manager to advantageously provide efficient and…
Method and system for low latency high frame rate streaming
Granted: April 13, 2021
Patent Number:
10979744
Embodiments of the present invention provide a low-latency approach for local or remote application streaming that reaches high FPS targets without overloading the available streaming bandwidth, for example, by limiting the bit rate to the same value that is used by traditional 60 FPS streaming solutions. A client device and server device cooperate to actively monitor and control a video stream to maintain an acceptable balance between latency and video quality by adjusting the frequency…
Codebook to reduce error growth arising from channel errors
Granted: April 13, 2021
Patent Number:
10979176
Techniques for limiting the growth of errors in decoded data words that arise from bit errors incurred during transmission. The growth of 3+ bit errors in the decoded data word is limited at the expense of a higher number of two bit errors, which are correctable using practical error correcting codes.
Techniques for comprehensively synchronizing execution threads
Granted: April 13, 2021
Patent Number:
10977037
In one embodiment, a synchronization instruction causes a processor to ensure that specified threads included within a warp concurrently execute a single subsequent instruction. The specified threads include at least a first thread and a second thread. In operation, the first thread arrives at the synchronization instruction. The processor determines that the second thread has not yet arrived at the synchronization instruction and configures the first thread to stop executing…
Head-mounted display having an image sensor array
Granted: April 6, 2021
Patent Number:
10972719
Head-mounted Displays (HMDs) are commonly used for virtual reality, mixed reality, and augmented reality. HMDs are, by definition, worn on the head of a user to provide a display in the line of sight of the user. By viewing the display, the user is able to experience one of the aforementioned types of reality. Oftentimes, HMDs are configured to integrate live video captured from the user's perspective, especially in the case of the HMD providing augmented reality where a virtual…
Motion blur and depth of field reconstruction through temporally stable neural networks
Granted: April 6, 2021
Patent Number:
10970816
A neural network structure, namely a warped external recurrent neural network, is disclosed for reconstructing images with synthesized effects. The effects can include motion blur, depth of field reconstruction (e.g., simulating lens effects), and/or anti-aliasing (e.g., removing artifacts caused by sampling frequency). The warped external recurrent neural network is not recurrent at each layer inside the neural network. Instead, the external state output by the final layer of the neural…
Reconciler for a distributed storage system
Granted: April 6, 2021
Patent Number:
10970256
A technique is applied to eventually converge on a single data storage strategy for any set of object data which had an inconsistent data storage strategy applied during storage while there was a network partition. This state could occur in instances of a highly available distributed object storage system which can store objects according to multiple data storage strategies. Upon the healing of a network partition, the technique discovers if multiple data storage strategies were applied…
System and method for near-eye light field rendering for wide field of view interactive three-dimensional computer graphics
Granted: April 6, 2021
Patent Number:
10969740
A method for rendering a light field comprises projecting rays from a viewpoint positioned at a first side of a spatial light modulator (SLM) to a clipping plane positioned at an opposing side of the SLM to form an elemental view frustum within a three-dimensional scene and rendering objects within the elemental view frustum to generate components of a first elemental image for the first elemental region. The SLM may include a tiled array of non-overlapping elemental regions and a top…
Techniques for reducing noise in video
Granted: March 30, 2021
Patent Number:
10964000
Systems and techniques for noise reduction in video are described. Example implementations provide improved motion-adaptive temporal or spatio-temporal noise reduction that use an improved blending of the current frame with previous frames. The improved blending may be particularly effective for processing video captured in noisy environments such as low-light and/or mobile environments. In some example implementations, the improved blending is based on more accurately distinguishing…
