Data Pipeline Engineer
Description For Data Pipeline Engineer
Near Space Labs is revolutionizing earth imaging through their fleet of stratospheric robots operating at altitudes of 40,000-60,000 feet. They're seeking a Data Pipeline Engineer to lead the evolution of their petabyte-scale geospatial imagery pipeline. This role combines cutting-edge technology with practical data engineering challenges, working with technologies like Python, TypeScript, Kubernetes, and various cloud services.
The position offers a unique opportunity to work on large-scale geospatial data processing, handling petabytes of imagery data while building and maintaining sophisticated data pipelines. The ideal candidate will have 5+ years of experience in data pipeline development, strong expertise in Python, and significant experience with distributed systems and cloud technologies.
Working from Barcelona (remote options available), you'll join an international team focused on building unique technologies and solving novel problems. The role demands both technical expertise and creative problem-solving skills, as you'll be responsible for everything from pipeline maintenance to developing new data processing tasks and optimizing workflows.
The company offers an exciting startup culture where you'll have significant impact on core technology development. They emphasize diversity and inclusion, providing an environment where different backgrounds and perspectives are valued. The position combines technical challenges with the opportunity to work on meaningful projects that have real-world applications in earth imaging technology.
This role is perfect for someone who wants to work at the intersection of space technology and data engineering, offering both technical growth and the chance to contribute to innovative earth observation solutions.
Responsibilities For Data Pipeline Engineer
- Build and maintain proprietary data pipeline for batch geospatial data processing
- Monitor and support data pipeline performance and reliability
- Develop new data processing tasks to support evolving data requirements
- Design and implement data storage solutions using cloud-based storage services
- Develop and optimize data workflows using distributed systems
- Rapid generation of functional prototypes
Requirements For Data Pipeline Engineer
- 5+ years of experience developing robust data pipelines and applications
- Strong understanding of data structures, algorithms and geospatial data
- Experience designing, building, and maintaining scalable data pipelines
- Ability to interact and manage containerized data applications using Docker and Kubernetes
- Experience working with cloud-based storage solutions and data warehousing technologies
- Knowledge of distributed systems
- Strong ability to diagnose and resolve complex data infrastructure issues
- Ability to work independently and collaboratively in a fast-paced environment
Benefits For Data Pipeline Engineer
- Exciting startup culture
- International and motivated team environment
- Diverse and inclusive workplace
- Opportunity to work on unique technologies
Jobs Related To Near Space Labs Data Pipeline Engineer
