Job Title: Software Engineer III
Contract Duration: 1 year, possible extension
Location: Burlingame, CA
Work Arrangement: Remote
Summary
We are seeking a skilled and experienced engineer to join our team as a Distributed Systems Specialist. As a generalist with expertise in distributed systems, you will play a key role in designing, developing, and optimizing our data generation pipelines for maximum efficiency. Your experience with DAGs/Graphs, large fan-out loads, and distributed workers/jobs will be instrumental in helping us scale our systems to meet the growing demands of our users.
Responsibilities
- Design, develop, and optimize data processing pipelines using DAGs/Graphs and distributed workers/jobs
- Collaborate with cross-functional teams to identify and prioritize system improvements
- Develop and implement solutions to handle large fan-out loads and ensure system scalability
- Work closely with data scientists and engineers to integrate and scale solutions
- Identify opportunities for optimization and implement changes to improve system performance and reduce resource utilization
Requirements
- 5+ years of experience in software development, with a focus on distributed systems
- Experience with large fan-out loads and distributed workers/jobs
- Proficiency in at least one of the following programming languages:
- Python (primary language)
- Rust
- C++
- Hack (PHP)
- Experience with cloud-based infrastructure, such as AWS or GCP
- Strong problem-solving skills and ability to work independently
- Excellent communication and collaboration skills
- Bachelors with tech field related
Must Have Skills
- Experience running compute clusters or building data infrastructure
- Proficiency in at least one of the following programming languages: Python (ideally), Rust, C++, Hack (PHP)
- Experience with cloud-based infrastructure such as AWS or GC
Nice to Have Skills
- Experience with companies using large data scales
- Experience with Digital Content Creation (DCC) tools such as blender
- Experience with machine learning frameworks, such as TensorFlow or PyTorch
| | |