Research Scientist - Efficient Audio Visual Machine Learning
Description For Research Scientist - Efficient Audio Visual Machine Learning
At Meta's Reality Labs Research, our goal is to make world-class consumer virtual, augmented, and mixed reality experiences. We are developing all the technologies needed to enable breakthrough Smartglasses, AR glasses and VR headsets, including optics and displays, computer vision, audio, graphics, brain-computer interfaces, haptic interaction, eye/hand/face/body tracking, perception science, and true telepresence.
The Audio team within RL Research is looking for an experienced and innovative Research Scientist with a specialty in real-time and efficient audio-visual learning and machine learning to join our growing team. You will be doing core and applied research in technologies that improve listener's hearing abilities under challenging listening conditions using wearable computing, and alongside a team of dedicated researchers, developers, and engineers. You will operate at the intersection of egocentric perception, acoustics, computer vision, and signal processing algorithms with hardware and software co-design.
Responsibilities include:
- Develop novel AI algorithms and associated real-time systems for source tracking, localization, diarization, and semantic scene understanding for egocentric wearable computing in AR and VR.
- Design and develop efficient AI frameworks and real-time technical systems with constraints on low-compute, low-power and overall system latency.
- Lead the development of systems and methods for quick prototyping, proof of concept, and demonstrations.
- Contribute to datasets designs and large-scale data processing for real-time evaluations.
- Contribute to technical strategy and establish new execution methods for efficient compute driven AI systems in Audio AR and VR applications.
- Summarize technical findings and influence system design and integration decisions of multi-modal AI systems supporting hearing technologies in AR and VR.
Join us in creating the future of augmented and virtual reality, changing everything about how we work, play, and connect.
Responsibilities For Research Scientist - Efficient Audio Visual Machine Learning
- Develop novel AI algorithms and associated real-time systems for source tracking, source localization, source diarization, and relevant semantic scene understanding with application into egocentric wearable computing in AR and VR
- Design and develop efficient AI frameworks and real-time technical systems with constraints on low-compute, low-power and overall system latency
- Lead the development of systems and methods to enable quick prototyping, proof of concept, or proof-of-experience and demonstrations
- Contribute to datasets designs and large-scale data processing for real-time evaluations of efficient audio-visual machine learning methods
- Contribute to the technical strategy and establish new execution methods where relevant for efficient compute driven AI systems in Audio AR and VR applications
- Summarize technical findings to cross-org collaborators, and influence system design and integration decisions of multi-modal AI systems supporting hearing technologies in AR and VR
Requirements For Research Scientist - Efficient Audio Visual Machine Learning
- PhD degree or equivalent experience in Deep Learning, AI, Machine Learning, Computer Science, Robotics, Computer Vision, Computational Neuroscience, Signal Processing, Speech and Language technologies, or related field
- 4+ years of experience working on applied computer vision methods for wearable computing
- 2+ years of experience working on efficient multimodal machine learning algorithms for low-compute and low-power devices
- Research-oriented software engineering skills, including fluency with machine learning (e.g., PyTorch, TensorFlow, Scikit-learn, Pandas) and libraries for scientific computing (e.g. SciPy ecosystem)
- Experience with cross-group and cross-cultural collaboration
- Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
Benefits For Research Scientist - Efficient Audio Visual Machine Learning
- Medical Insurance
- Dental Insurance
- Vision Insurance
- 401k
