Rendering images for non-standard display devices
Granted: October 29, 2024
Patent Number:
12131419
A method of rendering an image includes receiving information of a virtual camera, including a camera position and a camera orientation defining a virtual screen; receiving information of a target screen, including a target screen position and a target screen orientation defining a plurality of pixels, each respective pixel corresponding to a respective UV coordinate on the target screen; for each respective pixel of the target screen: determining a respective XY coordinate of a…
Content presentation and layering across multiple devices
Granted: June 25, 2024
Patent Number:
12022357
A method includes causing background content to be displayed on a display device with a first virtual object and a second virtual object; causing augmented reality (AR) content to be rendered based on a location of an AR device relative to the display device; determining that the AR content is in front of the first virtual object in the scene when viewed through the AR device and rendering the background content with a cutout in the first virtual object when the first virtual object…
System and techniques for lighting adjustment for an immersive content production system
Granted: May 7, 2024
Patent Number:
11978154
In at least one embodiment, an immersive content generation system may receive a first input from a user indicating a lighting value. The computing device may receive a second input indicating a region of an immersive virtual environment to which the lighting value is to be applied. The computing device may apply the lighting value to the region of the immersive virtual environment. The computing device may output one or more images of the immersive virtual environment, the one or more…
Streaming mixed-reality environments between multiple devices
Granted: April 2, 2024
Patent Number:
11948260
An immersive content presentation system can capture the motion or position of a performer in a real-world environment. A game engine can be modified to receive the position or motion of the performer and identify predetermined gestures or positions that can be used to trigger actions in a 3-D virtual environment, such as generating a digital effect, transitioning virtual assets through an animation graph, adding new objects, and so forth. The use of the 3-D environment can be rendered…
System and techniques for patch color correction for an immersive content production system
Granted: January 30, 2024
Patent Number:
11887251
A computing device in communication with an immersive content generation system can generate and present images of a virtual environment on one or more light-emitting diode (LED) displays at least partially surrounding a performance area. The device may capture a plurality of images of a performer or a physical object in the performance area along with at least some portion of the images of the virtual environment by a taking camera. The device may identify a color mismatch between a…
Obtaining high resolution and dense reconstruction of face from sparse facial markers
Granted: October 10, 2023
Patent Number:
11783493
Some implementations of the disclosure are directed to techniques for facial reconstruction from a sparse set of facial markers. In one implementation, a method comprises: obtaining data comprising a captured facial performance of a subject with a plurality of facial markers; determining a three-dimensional (3D) bundle corresponding to each of the plurality of facial markers of the captured facial performance; using at least the determined 3D bundles to retrieve, from a facial dataset…
Light capture device
Granted: September 19, 2023
Patent Number:
11762481
In some implementations, an apparatus may include a housing enclosing a circuitry may include a processor and a memory, the housing forming a handgrip. In addition, the apparatus may include a plurality of light sensors arranged in a particular configuration, each of the plurality of light sensors coupled to an exterior the housing via a sensor arm. Also, the apparatus may include one or more controls mounted on the exterior of the housing and electrically coupled to the circuitry. The…
Camera systems for motion capture
Granted: June 6, 2023
Patent Number:
11671717
Embodiments of the disclosure provide systems and methods for motion capture to generate content (e.g., motion pictures, television programming, videos, etc.). An actor or other performing being can have multiple markers on his or her face that are essentially invisible to the human eye, but that can be clearly captured by camera systems of the present disclosure. Embodiments can capture the performance using two different camera systems, each of which can observe the same performance…
Photogrammetric alignment for immersive content production
Granted: February 14, 2023
Patent Number:
11580616
A method of content production includes generating a survey of a performance area that includes a point cloud representing a first physical object, in a survey graph hierarchy, constraining the point cloud and a taking camera coordinate system as child nodes of an origin of a survey coordinate system, obtaining virtual content including a first virtual object that corresponds to the first physical object, applying a transformation to the origin of the survey coordinate system so that at…
System for deliverables versioning in audio mastering
Granted: February 14, 2023
Patent Number:
11581970
Some implementations of the disclosure relate to using a model trained on mixing console data of sound mixes to automate the process of sound mix creation. In one implementation, a non-transitory computer-readable medium has executable instructions stored thereon that, when executed by a processor, causes the processor to perform operations comprising: obtaining a first version of a sound mix; extracting first audio features from the first version of the sound mix obtaining mixing…
Camera calibration tool
Granted: January 31, 2023
Patent Number:
D976992
Scene interactions in a previsualization environment
Granted: December 20, 2022
Patent Number:
11532102
Views of a virtual environment can be displayed on mobile devices in a real-world environment simultaneously for multiple users. The users can operate selections devices in the real-world environment that interact with objects in the virtual environment. Virtual characters and objects can be moved and manipulated using selection shapes. A graphical interface can be instantiated and rendered as part of the virtual environment. Virtual cameras and screens can also be instantiated to…
Streaming mixed-reality environments between multiple devices
Granted: November 29, 2022
Patent Number:
11514653
An immersive content presentation system can capture the motion or position of a performer in a real-world environment. A game engine can be modified to receive the position or motion of the performer and identify predetermined gestures or positions that can be used to trigger actions in a 3-D virtual environment, such as generating a digital effect, transitioning virtual assets through an animation graph, adding new objects, and so forth. The use of the 3-D environment can be rendered…
Navigating a virtual environment of a media content item
Granted: November 22, 2022
Patent Number:
11508125
Systems and techniques are provided for switching between different modes of a media content item. A media content item may include a movie that has different modes, such as a cinematic mode and an interactive mode. For example, a movie may be presented in a cinematic mode that does not allow certain user interactions with the movie. The movie may be switched to an interactive mode during any point of the movie, allowing a viewer to interact with various aspects of the movie. The movie…
Smart edge loop selection
Granted: April 19, 2022
Patent Number:
11308693
A method of edge loop selection includes accessing a polygon mesh; receiving a selection of a first edge connected to a first non-four-way intersection vertex; receiving, after receiving the selection of the first edge, a selection of a second edge connected to the first non-four-way intersection vertex; in response to receiving a command invoking an edge loop selection process: evaluating a topological relationship between the first edge and the second edge; determining a rule for…
Multi-device interaction with an immersive environment
Granted: February 1, 2022
Patent Number:
11238619
Views of a virtual environment can be displayed on mobile devices in a real-world environment simultaneously for multiple users. The users can operate selections devices in the real-world environment that interact with objects in the virtual environment. Virtual characters and objects can be moved and manipulated using selection shapes. A graphical interface can be instantiated and rendered as part of the virtual environment. Virtual cameras and screens can also be instantiated to…
Color correction for immersive content production systems
Granted: December 14, 2021
Patent Number:
11200752
In at least one embodiment, an immersive content generation system may receive a first user input that defines a three-dimensional (3D) volume within a performance area. In at least one embodiment, the system may capture a plurality of images of an object in the performance area using a camera, wherein the object is at least partially surrounded by one or more displays presenting images of a virtual environment. In at least one embodiment, the system may receive a second user input to…
Obtaining high resolution and dense reconstruction of face from sparse facial markers
Granted: November 9, 2021
Patent Number:
11170571
Some implementations of the disclosure are directed to techniques for facial reconstruction from a sparse set of facial markers. In one implementation, a method comprises: obtaining data comprising a captured facial performance of a subject with a plurality of facial markers; determining a three-dimensional (3D) bundle corresponding to each of the plurality of facial markers of the captured facial performance; using at least the determined 3D bundles to retrieve, from a facial dataset…
Communication protocol for streaming mixed-reality environments between multiple devices
Granted: October 12, 2021
Patent Number:
11145125
An immersive content presentation system can capture the motion or position of a performer in a real-world environment. A game engine can be modified to receive the position or motion of the performer and identify predetermined gestures or positions that can be used to trigger actions in a 3-D virtual environment, such as generating a digital effect, transitioning virtual assets through an animation graph, adding new objects, and so forth. The use of the 3-D environment can be rendered…
Immersive content production system with multiple targets
Granted: September 28, 2021
Patent Number:
11132837
An immersive content presentation system and techniques that can detect and correct lighting artifacts caused by movements of one or more taking camera in a performance area consisting of multiple displays (e.g., LED or LCD displays). The techniques include capturing, with a camera, a plurality of images of a performer performing in a performance area at least partially surrounded by one or more displays presenting images of a virtual environment. Where the images of the virtual…
