Software Engineer, Data Acquisition (Crawler)

Anthropic is seeking a Software Engineer to lead the "Tokens: Data Acquisition" team, responsible for acquiring internet data through large-scale web crawling and data partnerships. This role is critical for producing the best pretrained AI models.

Key responsibilities include:

• Developing and maintaining an internet-scale web crawler
• Building pipelines for data ingestion, analysis, and quality improvement
• Creating specialized crawlers for high-value data sources
• Improving observability and debuggability of the crawler system
• Collaborating on improving data acquisition processes

The ideal candidate should:

• Believe in the transformative potential of advanced AI systems
• Have extensive experience with large distributed systems
• Be familiar with cloud-based compute and storage solutions
• Have expertise in Python
• Understand the non-technical aspects of internet-scale crawling (e.g., data privacy, robots.txt adherence)

Anthropic offers a competitive compensation package, including:

• Salary range: $280,000 - $625,000 USD
• Equity options with higher-than-average compensation
• Comprehensive health, dental, and vision insurance
• 401(k) with 4% matching
• 22 weeks of paid parental leave
• Unlimited PTO (most take 4-6 weeks annually)
• Various stipends and benefits

The company values collaboration, impact-driven work, and communication skills. They encourage applications from diverse candidates and offer visa sponsorship for eligible roles.

Join Anthropic in their mission to create safe and beneficial AI systems with enormous social and ethical implications.