LATENCY REDUCTION FOR IMMERSIVE CONTENT PRODUCTION SYSTEMS
Granted: March 21, 2024
Application Number:
20240096035
A method of content production may include receiving tracking information for a camera with a frustum configured to capture images of a subject in an immersive environment. a first image of a virtual environment corresponding to the frustum may be rendered using a first rendering process based on the tracking information to be perspective-correct when displayed on the displays and viewed through the camera. A second image of the virtual environment may also be rendered using a second…
OPTIMIZATION FOR TECHNICAL TARGETS IN AUDIO CONTENT
Granted: February 1, 2024
Application Number:
20240038256
Some implementations of the disclosure relate to a non-transitory computer-readable medium having executable instructions stored thereon that, when executed by a processor, cause a system to perform operations comprising: obtaining a first energy-based target for audio; obtaining a first version of a sound mix including one or more audio components; computing, for each audio frame of multiple audio frames of each of the one or more audio components, a first audio feature measurement…
OPTIMIZATION FOR TECHNICAL TARGETS IN AUDIO CONTENT
Granted: February 1, 2024
Application Number:
20240038256
Some implementations of the disclosure relate to a non-transitory computer-readable medium having executable instructions stored thereon that, when executed by a processor, cause a system to perform operations comprising: obtaining a first energy-based target for audio; obtaining a first version of a sound mix including one or more audio components; computing, for each audio frame of multiple audio frames of each of the one or more audio components, a first audio feature measurement…
MOTION CAPTURE USING SYNCHRONIZED AND ALIGNED DEVICES
Granted: October 19, 2023
Application Number:
20230336679
A motion capture system comprising: a master clock configured to repeatedly generate and output, at a frame rate, a primary clock signal that conveys when a video frame starts; a first camera configured to capture light within a first set of wavelengths and operably coupled to receive the master clock signal and initiate an image capture sequence on a frame-by-frame basis in fixed phase relationship with the primary clock signal to generate a first set of images at the frame rate from…
IMMERSIVE CONTENT PRODUCTION SYSTEM WITH MULTIPLE TARGETS
Granted: October 12, 2023
Application Number:
20230326142
An immersive content presentation system and techniques that can detect and correct lighting artifacts caused by movements of one or more taking camera in a performance area consisting of multiple displays (e.g., LED or LCD displays). The techniques include capturing, with a camera, a plurality of images of a performer performing in a performance area at least partially surrounded by one or more displays presenting images of a virtual environment. Where the images of the virtual…
METHOD AND SYSTEM FOR LATENT-SPACE FACIAL FEATURE EDITING IN DEEP LEARNING BASED FACE SWAPPING
Granted: October 5, 2023
Application Number:
20230316587
A computer-implemented method of changing a face within an output image or video frame that includes: receiving an input image that includes a face presenting a facial expression in a pose; processing the image with a neural network encoder to generate a latent space point that is an encoded representation of the image; decoding the latent space point to generate an initial output image in accordance with a desired facial identity but with the facial expression and pose of the face in…
AI-ASSISTED SOUND EFFECT EDITORIAL
Granted: May 4, 2023
Application Number:
20230136632
Some implementations of the disclosure relate to a method, comprising: obtaining, at a computing device, first video clip data including multiple sequential video frames, the multiple sequential video frames including at least a first video frame and a second video frame that occurs after the first video frame; inputting, at the computing device, the first video clip data into at least one trained model that automatically predicts, based on at least features of the first video frame and…
AI-ASSISTED SOUND EFFECT EDITORIAL
Granted: May 4, 2023
Application Number:
20230136632
Some implementations of the disclosure relate to a method, comprising: obtaining, at a computing device, first video clip data including multiple sequential video frames, the multiple sequential video frames including at least a first video frame and a second video frame that occurs after the first video frame; inputting, at the computing device, the first video clip data into at least one trained model that automatically predicts, based on at least features of the first video frame and…
COLOR AND LIGHTING ADJUSTMENT FOR IMMERSIVE CONTENT PRODUCTION SYSTEM
Granted: October 27, 2022
Application Number:
20220343591
A computing device in communication with an immersive content generation system can generate and present images of a virtual environment on one or more light-emitting diode (LED) displays at least partially surrounding a performance area. The device may capture a plurality of images of a performer or a physical object in the performance area along with at least some portion of the images of the virtual environment by a taking camera. The device may identify a color mismatch between a…
SYSTEM FOR DELIVERABLES VERSIONING IN AUDIO MASTERING
Granted: October 27, 2022
Application Number:
20220345234
Some implementations of the disclosure relate to using a model trained on mixing console data of sound mixes to automate the process of sound mix creation. In one implementation, a non-transitory computer-readable medium has executable instructions stored thereon that, when executed by a processor, causes the processor to perform operations comprising: obtaining a first version of a sound mix; extracting first audio features from the first version of the sound mix obtaining mixing…
SYSTEM AND TECHNIQUES FOR LIGHTING ADJUSTMENT FOR AN IMMERSIVE CONTENT PRODUCTION SYSTEM
Granted: October 27, 2022
Application Number:
20220343590
In at least one embodiment, an immersive content generation system may receive a first input from a user indicating a lighting value. The computing device may receive a second input indicating a region of an immersive virtual environment to which the lighting value is to be applied. The computing device may apply the lighting value to the region of the immersive virtual environment. The computing device may output one or more images of the immersive virtual environment, the one or more…
COLOR AND LIGHTING ADJUSTMENT FOR IMMERSIVE CONTENT PRODUCTION SYSTEM
Granted: October 27, 2022
Application Number:
20220343562
In some implementations, a computing device in communication with an immersive content generation system may generate a first set of user interface elements configured to receive a first selection of a shape of a virtual stage light. In addition, the device may generate a second set of user interface elements configured to receive a second selection of an image for the virtual stage light. Also, the device may generate a third set of user interface elements configured to receive a third…
LIGHT CAPTURE DEVICE
Granted: October 27, 2022
Application Number:
20220342488
In some implementations, an apparatus may include a housing enclosing a circuitry may include a processor and a memory, the housing forming a handgrip. In addition, the apparatus may include a plurality of light sensors arranged in a particular configuration, each of the plurality of light sensors coupled to an exterior the housing via a sensor arm. Also, the apparatus may include one or more controls mounted on the exterior of the housing and electrically coupled to the circuitry. The…
BREAK AWAY MOCAP TRACKER
Granted: March 3, 2022
Application Number:
20220067948
A motion capture tracking device comprising a base portion including a first alignment feature, a first magnetic element and an attachment mechanism operative to mechanically couple the base portion to a rod, a detachable end cap configured to be removably mated with the base portion, and a plurality of motion capture markers coupled to the end cap. The detachable end cap can include a second alignment feature and a second magnetic element, such that, during a mating event in which the…
OBTAINING HIGH RESOLUTION AND DENSE RECONSTRUCTION OF FACE FROM SPARSE FACIAL MARKERS
Granted: February 24, 2022
Application Number:
20220058870
Some implementations of the disclosure are directed to techniques for facial reconstruction from a sparse set of facial markers. In one implementation, a method comprises: obtaining data comprising a captured facial performance of a subject with a plurality of facial markers; determining a three-dimensional (3D) bundle corresponding to each of the plurality of facial markers of the captured facial performance; using at least the determined 3D bundles to retrieve, from a facial dataset…
IMMERSIVE CONTENT PRODUCTION SYSTEM WITH MULTIPLE TARGETS
Granted: January 6, 2022
Application Number:
20220005279
An immersive content presentation system and techniques that can detect and correct lighting artifacts caused by movements of one or more taking camera in a performance area consisting of multiple displays (e.g., LED or LCD displays). The techniques include capturing, with a camera, a plurality of images of a performer performing in a performance area at least partially surrounded by one or more displays presenting images of a virtual environment. Where the images of the virtual…
SMART EDGE LOOP SELECTION
Granted: December 30, 2021
Application Number:
20210407199
A method of edge loop selection includes accessing a polygon mesh; receiving a selection of a first edge connected to a first non-four-way intersection vertex; receiving, after receiving the selection of the first edge, a selection of a second edge connected to the first non-four-way intersection vertex; in response to receiving a command invoking an edge loop selection process: evaluating a topological relationship between the first edge and the second edge; determining a rule for…
RENDERING IMAGES FOR NON-STANDARD DISPLAY DEVICES
Granted: December 30, 2021
Application Number:
20210407174
A method of rendering an image includes receiving information of a virtual camera, including a camera position and a camera orientation defining a virtual screen; receiving information of a target screen, including a target screen position and a target screen orientation defining a plurality of pixels, each respective pixel corresponding to a respective UV coordinate on the target screen; for each respective pixel of the target screen: determining a respective XY coordinate of a…
PHOTOGRAMMETRIC ALIGNMENT FOR IMMERSIVE CONTENT PRODUCTION
Granted: November 4, 2021
Application Number:
20210342971
A method of content production includes generating a survey of a performance area that includes a point cloud representing a first physical object, in a survey graph hierarchy, constraining the point cloud and a taking camera coordinate system as child nodes of an origin of a survey coordinate system, obtaining virtual content including a first virtual object that corresponds to the first physical object, applying a transformation to the origin of the survey coordinate system so that at…
OBTAINING HIGH RESOLUTION AND DENSE RECONSTRUCTION OF FACE FROM SPARSE FACIAL MARKERS
Granted: May 20, 2021
Application Number:
20210150810
Some implementations of the disclosure are directed to techniques for facial reconstruction from a sparse set of facial markers. In one implementation, a method comprises: obtaining data comprising a captured facial performance of a subject with a plurality of facial markers; determining a three-dimensional (3D) bundle corresponding to each of the plurality of facial markers of the captured facial performance; using at least the determined 3D bundles to retrieve, from a facial dataset…
