Data Engineer
We are looking for a data engineer to join our growing technical team. The ideal candidate for this position is passionate about building high-performance data pipelines, automating labor-intensive processes, and ensuring data integrity. In this role, you will work closely with BlastPoint product, engineering and data science staff to ensure smooth and timely delivery of actionable insights to customers by taking a leadership role on tasks related to data onboarding, cleansing, enrichment, processing, curation, and validation.
This position is based in Pittsburgh, PA, but will be fully remote at least while the pandemic remains, and possibly longer.
Core duties include:
- Design, build, deploy, automate, document, test and maintain production-quality data pipelines
- Design and automate our customer dataset analysis and insights delivery process to smoothly handle greater data variety and velocity, and applications from custom scoring metrics to anomaly detection to monitoring pipeline health to ML solutions
- Integrate client data with a wide variety of public and private sources, making them ready to sing harmony to a customer dataset’s melody
- Improving data storage, indexing and API-based service of data to user interfaces
- Protect the security and integrity of our data throughout the project lifecycle
- Working in an agile team, collaborating with engineers, data scientists and others
The balance of work day-to-day will likely be ~60% data engineering on our core platform, ~30% client project-specific work to integrate with client data sources and systems, and ~10% data-science modeling, support, or ad-hoc needs.
Professional Requirements
- Bachelor’s degree or equivalent experience in computer science or a related field
- 2+ years of experience in data engineering; likely other software engineering experience as well
- Ability to write robust, maintainable, and performant Python code
- Comfort with relational database systems and complex SQL queries
- Deep knowledge of how to diagnose and remediate performance issues in a UNIX environment
- Authorized to work in the United States (H-1B transfers and OPT are OK, but no new H-1B applicants please)
Preferred Skills and Experience
- Documented record of building robust and performant end-to-end data pipelines
- Comprehensive knowledge of cloud computing technologies, particularly those of the Amazon Web Services platform
- Strong UNIX command line and shell scripting skills
- Familiarity with prominent data wrangling utilities including csvkit, jq, sed, awk, and grep
- Experience with platforms and libraries related to data transformation, processing, and analysis, including Jupyter, Pandas, Apache Airflow, and Pony ORM
- Hands-on experience with relational databases like PostgreSQL, with the ability to profile and optimize queries to improve the scalability of the platform
- Exposure to modern DevOps, Continuous Integration & Delivery, and Infrastructure as Code technologies and best practices, including but not limited to Docker, Terraform, Ansible, and Jenkins
- Expertise building secure services and systems in a multi-tenant environment
- Expertise evaluating and integrating with third-party platforms and libraries in a way that enhances the functionality of the product without adding unnecessary complexity
- Highly attentive to detail, with a keen eye for unclean data
- Strong communications skills, with an ability to convey ideas to audiences regardless of their technical knowledge
- Ability to self-motivate, self-organize, work independently, and assume ownership of your deliverables in a challenging, fast-paced environment with several ongoing concurrent projects
- Recognition that there are always multiple answers to a problem, with the ability to engage in a constructive dialogue to find the best path forward
- Curious and eager problem solver, able to self-teach new skills when needed, with a hunger for building well-designed, high-quality solutions
- Experience in, exposure to, or a high level of comfort with the constraints of a startup environment
- Ability to commute to the Pittsburgh, PA office regularly (once current pandemic constraints are lifted); note that this is a preference, but we are open to permanent-remote candidates located within the United States
Email us your cover letter and resume to apply: careers@blastpoint.io