1) Maintain a common database(1) of vectorized objects, images, skins, textures and persons and load onto both the media management server/transmitter(3) and media management server/receiver device(4) prior to retail purchase.
2) Create universal internet update service(5) that downloads all newly identified vector objects to all receiver servers as they become available or as the are needed.
Vector-based Video Pre-production:
3) Thoroughly scout(6) and videotape all scenes to be shot for all objects in future production. Identify and log(7) all objects in scene.
4) The AVIE(8) creates a rough draft 3D interpretation of all scenes for the upcoming shoot.
5) The authorized art direct modifies the 3D scene to remove all traces of production equipment(9) and then adds an artificial illusion of an adjacent scene(10) that complements the main set.
5) All talent(11) and props(12) are imaged for inclusion in the production.
6) The more objects identified and vectorized prior to a production the less processing will be needed for broadcast.
7) For live events, vector interpreting production assistants (13) identify unknown AVIE flagged objects(14) in real time as the technical director(15) previews(16) or takes camera shots.
8) As stereoscopic video(17) is shot for a production the AVIE creates a real-time 3D map(11) of all animated objects and persons(18) within each scene using pre-vectorized object data(19).
9) As the footage is shot it is sent to the AVIE database(20) to be further interpreted. Any AVIE unidentified objects are flagged for future human identification(21).
10) After all scenes and safeties(22) are shot the footage data(23) is sent to a post production object identifier(24). This person reviews all AVIE flagged objects and identifies and logs each previously unknown object.
11) The editor(24) arranges all scenes and shots for the best presentation possible. From this point on all views, angles and camera shots(25) are infinitely variable.
Vector-based A/V uses an object recognition processor or interpreting engine and isolates an object in a scene over a any given period that it exists in the video. The interpreting engine then quickly generates random 3D objects fitting the fuzzy criteria of the object given dimensions and relations with reference to identified objects surrounding it. The engine also looks for reference shadows, reflections and other trace inter-courses with identified objects in the unidentified object’s proximity. Similar 3D vector objects are randomly generated on a matching linear video timeline until a shape most closely resembles the unknown shape. The shape is then compared with all existing object indexes and objects until a match is found. If no match is found the the object is identified as unknown and flagged for identification by a Audio Vector Interpreting Operative (AVIE).
Challenges of a vector-based production
a) Gigantic Polygon Meshes
1 – Scouting for vector-based recordings (video and audio) is done by a vector scout. The scout creates close-up video of all objects and scenes that are expected to be in the production. The scout then processes that video with the vector A/V interpreting engine and identifies and logs all XYZ objects in the scout video.
2 – The vector A/V interpreting engine initially identifies the objects it can interpret using character recognition routines, the pre-identified objects entered by the vector scout and an extensive database of pre-installed reference objects.
Any object that cannot be identified is flagged within running frames as an unknown entity and placed on the vector matrix at an XYZ location best represented by AI deduction objects suitably recreated using the production video, raw footage, safety shots and the original scouting video.
4 – Posting vector-based productions is done by a vector interpreter.
a) After a production is recorded it initially takes an operative to decipher and identify unknown objects and sounds which were misidentified or missed by the initial vector scout.
5 – Correspondence problem: “Given two or more images of the same 3D scene, taken from different points of view, the correspondence problem is to find a set of points in one image which can be identified as the same points in another image. A human can normally solve this problem quickly and easily, even when the images contain significant amount of noise. In computer vision the correspondence problem is studied for the case when a computer should solve it automatically with only the images as input. Once the correspondence problem has been solved, resulting in a set of image points which are in correspondence, other methods can be applied to this set to reconstruct the position of the corresponding 3D points in the scene.”
“The correspondence problem typically occurs when two images of the same scene are used, the stereo correspondence problem. This concept can be generalized to the three-view correspondence problem or, in general, the N-view correspondence problem. In the general case, the images can either come from N different cameras which depict (more or less) the same scene or from one and the same camera which is moving relative to the scene. An even more difficult version of the correspondence problem occurs when the objects in the scene can be in general motion relative to the camera(s).” From Wikipedia
“A typical application of the correspondence problem occurs in image mosaicing — when two or more images which only have a small overlap are to be stitched into a larger composite image. In this case it is necessary to be able to identify a set of corresponding points in a pair of images in order to calculate the transformation of one image to stitch it onto the other image.” From Wikipedia
(25) Establishing Shot, EWS (Extreme Wide Shot), VWS (Very Wide Shot), WS (Wide Shot), MS (Mid Shot), MCU (Medium Close Up), CU (Close Up), ECU (Extreme Close Up), CA (Cutaway), (OSS) Over-the-Shoulder Shot, Noddy (Reaction Shot) Shot, POV (Point-of-View Shot) and Weather Shots