TEKsystems - Apple

TEKsystems - Apple

Synthetic Data Software Engineer - our client makes the newest technologies accessible to create amazing user experiences. You will have the opportunity to work in a multi-functional team to provide synthetic character data to develop innovative computer vision and machine learning algorithms. We are looking for a synthetic digital human software engineer who has exceptional knowledge and experience in computer graphics and visual effects for character development. You will need a keen understanding and deep experience in all aspects of character development - photogrammetry, shading/rendering, rigging, simulation, and pipelines for offline and real-time environments. You will be responsible for solving character problems, managing and troubleshooting synthetic data generation.

Skills:

Python, maya, Houdini, 3D, 3d graphics, 3d graphics software, character rigging, 3d geometry, Linear Algebra, C++, Synthetic Data, Shading, Rendering

Top Skills Details:

Python,maya,Houdini,3D,3d graphics,3d graphics software,character rigging,3d geometry,Linear Algebra

Additional Skills & Qualifications:

- Programming skill in C/C++ is a bonus

- Experience in generating synthetic data for computer vision problems is a big bonus

- Good understanding of modern shading/rendering methodologies such as PBS/PBR, IBL

- Hair/cloth simulation setup/pipeline experience is a plus

Experience Level:

Expert Level

FIRST ROUND 

• someone that fulfills every one of those requirements are either working for 20+ years or lacks the fidelity in those departments. 
• Digital Humans 
  • Rendering using post production VRAY, then deferred look, then forward look, then headset   
  • Getting rid of FACS blendshapes for performance
  • at ML focused on scanning pipeline 
  • At Samsung focused on QC pipeline and MAM 
  • give high level overview of Metahumans pipeline and how it compares to traditional
• 3D Geometry 
  • basic 3d character pipeline, UVS and matching topology 
• Character Rigging 
  • IK FK, skin weights.
  • machine learning methods
    • meshcapade 
    • quick rigging tool in maya  
    • A Stance, T Stance helper joints 
    • smooth select when weight painting
  • mocap, traditional vs cv based 
• Python 
  • basic ds and a and how it applied to python
  • pep8
  • Some design patterns
• Prototyping (Pipelines)
  • What makes a MVP 
  • Validating it works ASAP 
    • testing with thirdparty before writing own implmentation 
    • architecting the solution in python, then handing it to SWE to code in C++ 
      • examples. keying, carrying multiple layers for real time 
• Synthetic Data 
  • How we use ML in pipleine 
    • concept art gen ai
    • rigging meshcapade
    • building the new pipeline for modern era, generalists over specialists 
  • Talk about building the hand scanning pipeline for ML headset for CV 
    • built using reality capture license bound
  • Talk about building 3d head generator for ML that is used for CV 
    • built using metashape not license bound 
    • @SRA , talk about purchasing heads over building solid scanning system 
    • working with ML engineers
  • Benefits of synthetic data over real data
    • privacy!
    • generate whatever you want and narrow down requirements 
    • Get real data (scan or purchase from RenderPeople), ML model that generates different heads... then use that model to test computer vision. 
      • look at microsoft stuff 
      • Talk about siggraph papers
  • how it applies to training CV https://www.youtube.com/watch?v=6Bts8WeZ6nA

from chatgpt

The dot product is a mathematical operation that has various applications in computer graphics and technical art. Technical artists often use the dot product for tasks related to shading, lighting, and geometry. Here are some specific applications of the dot product in the context of technical art:

1. Lighting and Shading: The dot product is commonly used in lighting calculations, such as the Phong reflection model. In this model, the dot product of the surface normal and the light direction vector is used to determine the amount of diffuse and specular reflection at a given point on a surface. This helps create realistic lighting effects in computer-generated images.

2. Normal Mapping: Normal mapping is a technique used to add fine surface details to 3D models without increasing the model's geometry. The dot product is often employed to transform per-pixel normal maps into the space of the rendered surface, allowing for accurate lighting calculations based on the details provided by the normal map.

3. Culling and Visibility: The dot product is used in techniques like backface culling, where the dot product of the surface normal and the view direction is checked to determine whether a polygon is facing toward or away from the camera. This information is crucial for optimizing rendering by skipping the rendering of surfaces facing away from the viewer.

4. Reflection and Refraction: The dot product is involved in calculations related to reflection and refraction of light rays. When simulating reflective or refractive materials, technical artists use the dot product to compute the reflection and refraction vectors, allowing for realistic rendering of materials like glass or water.

5. Orientation and Alignment: Technical artists may use the dot product to determine the alignment or orientation of objects in a scene. This information can be useful for various purposes, including procedural generation, animation, and rigging.

6. Geometry Operations: In certain cases, the dot product is used for geometric operations, such as checking if two vectors are orthogonal (have a dot product of zero) or finding the angle between two vectors.

In summary, the dot product is a versatile mathematical operation that technical artists use in various aspects of computer graphics and rendering to achieve realistic and visually appealing results.

Here are some applications of the cross product in the context of technical art:

1. Normal Vector Calculation:

   • One of the primary uses of the cross product is in computing normal vectors for surfaces. Given two vectors lying in the plane of a surface, the cross product provides a vector that is perpendicular to that surface. Normal vectors are essential for lighting calculations and shading in 3D graphics.

2. Tangent and Bitangent Calculation:

   • In texture mapping, the cross product can be used to calculate tangent and bitangent vectors, which are crucial for mapping textures onto 3D surfaces, especially in normal mapping techniques.

3. Orientation and Rotation:

   • Technical artists use the cross product to determine the orientation of objects or to calculate the rotation axis when performing rotations in 3D space. This is valuable for rigging and animation tasks.

4. Culling:

   • In addition to the dot product, the cross product is used for backface culling. By comparing the result of the cross product of two vectors (edges of a polygon), the orientation of the polygon in relation to the camera can be determined.

5. Collision Detection:

   • The cross product is involved in collision detection algorithms, particularly in determining the direction of the collision response force when two objects collide.

6. Angular Momentum and Torque:

   • In physics simulations and game development, the cross product is used to calculate angular momentum and torque, providing realistic rotational behavior for objects.

7. Quaternion Rotation:

   • Quaternions, which are often used to represent rotations in 3D space, involve the cross product. The cross product of two vectors can be used to construct a quaternion that represents the rotation between them.

8. Volumetric Effects:

   • In certain rendering techniques, such as volume rendering or fluid simulations, the cross product is used to calculate vorticity or rotational aspects of the fluid flow.

9. Procedural Generation:

   • The cross product is employed in procedural generation algorithms, especially when creating diverse and natural landscapes. It can be used to generate variations in terrain features or patterns.

10. Particle Systems:

    • When simulating particle systems, the cross product is used to determine the rotational forces acting on particles, allowing for realistic spinning or rotation effects.

Terminology water tight