Data Engineer
Description For Data Engineer
Microsoft's Azure Data engineering team is at the forefront of analytics transformation, developing products like Microsoft Fabric, Azure SQL DB, Azure Cosmos DB, Azure PostgreSQL, and more. The team's mission is to build the data platform for the AI age, enabling data-first applications and fostering a data culture.
The Fabric Telemetry Data Platform team is specifically focused on building a next-generation data platform that revolutionizes how data engineers and partner teams work. This platform emphasizes scalability, resilience, and automation, supporting high-performance querying and self-service capabilities.
As a Data Engineer, you'll be instrumental in:
- Designing and building scalable data pipelines
- Developing tools for metadata management
- Creating job monitoring and notification services
- Optimizing data processing and storage
- Implementing automated anomaly detection
- Streamlining operations through automation
The role requires expertise in:
- Big data technologies (Hadoop, Hive, Spark)
- Cloud computing platforms (Azure Data Lake, Azure Data Factory, Azure Synapse)
- Programming languages (C#, Java, Python)
- ETL processes and data pipeline development
- Data governance and security practices
You'll work in a collaborative environment that values diverse perspectives and innovative thinking, contributing to Microsoft's mission of empowering every person and organization on the planet to achieve more. The position offers comprehensive benefits, professional development opportunities, and the chance to work with cutting-edge technologies in a global team setting.
Responsibilities For Data Engineer
- Design, develop, and maintain data pipelines for data collection, cleansing, transformation, and usage
- Build frameworks to validate data quality and completeness, detect anomalies
- Design and implement end-to-end software life cycles, including CI/CD and service reliability
- Implement performance monitoring protocols across data pipelines
- Design data modeling and handling procedures for compliance
- Implement security and access control measures
- Perform database administration tasks
Requirements For Data Engineer
- Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 2+ years experience
- 2+ years of experience in software or data engineering, with proven proficiency in C#, Java, or equivalent
- 2+ years in one scripting language for data retrieval and manipulation (e.g., SQL or KQL)
- 2+ years of experience in big data scripting languages like Python, SCOPE, SQL or equivalent
- 2+ years of experience with ETL and data cloud computing technologies
- Must pass Microsoft Cloud background check
Benefits For Data Engineer
- Industry leading healthcare
- Educational resources
- Discounts on products and services
- Savings and investments
- Maternity and paternity leave
- Generous time away
- Giving programs
- Opportunities to network and connect
Jobs Related To Microsoft Data Engineer
