Data Engineer
Description For Data Engineer
Acelab, a B2B marketplace connecting architects and designers with building material manufacturers, is seeking an experienced Data Engineer to design, develop, and maintain robust data pipelines and infrastructure. This crucial role involves managing the data ecosystem, implementing advanced analytics, and supporting data science and engineering teams in delivering high-quality insights and products.
Key Responsibilities:
- Design, develop, and maintain pipelines for importing and cleaning semi-structured data from various sources
- Specify, deploy, and manage scalable and reliable data pipeline infrastructure
- Design and implement database schemas for efficient storage and retrieval of complex and sparse data
- Collaborate with data science and engineering teams to ensure data consistency across systems
- Develop and maintain data quality checks and monitoring systems
- Optimize data storage and retrieval processes for improved performance and cost-effectiveness
- Participate in code reviews and contribute to best practices for data engineering
- Stay up-to-date with the latest trends and technologies in data engineering and big data processing
The ideal candidate would have experience in data-intensive industries such as e-commerce (e.g., Wayfair, Amazon, Shopify) or similar high-volume data environments. This role offers the opportunity to work with some of the brightest minds in AEC, including the former CEO of Revit, and provides a range of benefits including unlimited PTO, comprehensive health plans, and a flexible, remote work schedule.
Join Acelab to contribute to cutting-edge data engineering practices in the architecture and design industry, while enjoying opportunities for growth, creativity, and ownership in a collaborative and innovative environment.
Responsibilities For Data Engineer
- Design, develop, and maintain pipelines for importing and cleaning semi-structured data from various sources
- Specify, deploy, and manage scalable and reliable data pipeline infrastructure
- Design and implement database schemas to efficiently store and retrieve complex and sparse data
- Collaborate closely with data science and engineering teams to reconcile datasets and ensure data consistency across systems
- Develop and maintain data quality checks and monitoring systems
- Optimize data storage and retrieval processes for improved performance and cost-effectiveness
- Participate in code reviews and contribute to best practices for data engineering
- Stay up-to-date with the latest trends and technologies in data engineering and big data processing
Requirements For Data Engineer
- Bachelor's or Master's degree in Computer Science, Data Science, or a related field
- 3+ years of experience in data engineering or a similar role
- Strong proficiency in SQL, and at least one big data processing framework (e.g., Apache Spark, Hadoop)
- Some experience with .NET Web API
- Experience with cloud platforms (AWS, Azure, or GCP) and their data services
- Solid understanding of data modeling, ETL processes, and data warehousing concepts
- Familiarity with version control systems (e.g., Git) and CI/CD practices
- Knowledge of data governance and security best practices
- Strong problem-solving skills and attention to detail
- Excellent communication skills and ability to work in a collaborative environment
- Experience with Agile development methodologies
Benefits For Data Engineer
- Work with some of the brightest minds in AEC, including former CEO of Revit
- Unlimited PTO, Sick Days, Paid National Holidays, and more
- Medical, dental, vision, short-term/long-term disability plans, and 401(k)
- Opportunities for growth, creativity, and ownership
- Flexible and remote work schedules
