⚡ Data Infrastructure Engineer

📍 Remote within the US or San Diego

💲 130k - 170k + equity

Working with a clinical-stage biotechnology company that leverages human genetics to transform therapeutic discovery. They utilize its platform to analyze extensive biological datasets, aiming to identify and develop novel medicines that mimic naturally occurring genetic variants with beneficial health effects.

You will work closely with a team of engineers and scientists to build and extend the internal data platform, which include cloud-based systems, services, and web applications that help our scientists discover drug targets from some of the largest biological datasets in the world. An ideal candidate is curious, detail-oriented, and enjoys working in collaborative environments with teammates coming from diverse backgrounds and skill sets.

👨‍💻 Responsibilities:

1. Building tools for internal use that improve the ability of data engineers and analysts to build pipelines and integrate with cloud compute platforms
2. Collaborate with a team of scientists and engineers to build reusable components that solve common problems during pipeline development
3. Be a champion of data engineering best practices, enabling others through the creation of tools and frameworks that enforce and simplify their implementation
4. Implement observability and instrumentation for real-time tracking and alerting of application and data metrics
5. Build tools to integrate and abstract integration with cloud vendors and external services
6. Create documentation and trainings to enable pipeline developers to use internal and external tools
7. Take ownership of the code and subsystems that you work on

👩‍🎓Requirements:

1. Strong experience with Python (4 years professional experience minimum)
2. Comfortable with SQL
3. Significant experience building data pipelines (4 years professional experience minimum)
4. Strong experience with distributed transformation frameworks such as Spark
5. Strong understanding of distributed systems
6. Experience building internal tools and automation, and improving developer workflows
7. Experience implementing data observability tooling
8. Docker / containerization experience
9. Proficiency with git / GitHub
10. Experience with a CI / CD solution such as Github Actions, CircleCI, Travis, Code*, etc.
11. Experience building documentation for internal APIs and tools
12. Experience administering and building on AWS, including exposure to most of the following: EC2, ECS, VPC, Lambda, RDS, IAM, Athena
13. Proven ability to self-manage all aspects of SDLC for a given project, including requirements gathering, specifications, implementation, and ongoing maintenance

⭐ Bonus:

1. Experience working in a complex scientific domain
2. Experience with a workflow orchestration tool (Dagster, Airflow, Prefect, etc.)
3. Experience using or implementing a data catalog
4. Knowledgeable about data governance and organizational best practices
5. Exposure to data warehousing techniques such as star / snowflake schemas, medallion architecture
6. Experience building backend applications and REST / GraphQL / GRPC services
7. Apache Spark tuning and administration experience
8. Experience deploying software with complex dependencies
9. Experience with next-gen ETL frameworks such as Polars, Daft, Duckdb
10. Experience with Ray as a user or administrator
11. Experience building tools or services that handle and manage large (>100TB) datasets
12. Experience building tools on top of Databricks
13. Significant experience with next-gen table formats such as Delta, Iceberg, Hudi, etc.
14. Working knowledge of relational databases such as Postgres, MySQL, MariaDB, etc.
15. Working knowledge of log aggregation / monitoring tools (we use Datadog)
16. Exposure to Scala / Java

📧 Interested in applying? Please click on the ‘Easy Apply’ button

⚡ Storm3 is a HealthTech recruitment firm with clients across London, Europe and North America. To discuss open opportunities or career options, please visit our website www.storm3.com and follow the Storm3 Linked In page for the latest jobs and intel.

Are you looking for remote jobs near your area? At Yulys, thousands of employers are looking for exceptional talent like yours. Find a perfect job now.