Duration: 6 Month Contract - w/extensions or conversion
Location: Remote ; East Coast!!!
Must Haves:
2-3 years exp. as a data engineer
3 years of RECENT experience with Data Bricks
Must be their most relevant experience!!
Experience with Cloud ( Azure, GCP, or AWS)
Experience with SQL - they will be writing queries
Lead experience - Okay with not just being heads down
Python/pyspark experience
Plusses:
AWS Certifications
Data Governance experience
Bach Degree
Day to Day:
Insight Global is seeking a Lead Data Engineer to sit remote (EAST COAST) for a large retail client headquartered in the greater Pittsburgh area! Our client recently kicked off a major project to build their own home-grown eCommerce platform and applications that will be used internally and then deployed out into their 600+ retail locations. In this role, the Data Engineer will be a member of the RISC team and will be a design, develop, and maintain scalable and efficient data pipelines for extracting, transforming, and loading (ETL) data from various sources into our data warehouse leveraging AWS, Databricks, Python and SQL technologies.
Additional responsibilities will include:
Collaborate with cross-functional teams to understand data requirements and implement solutions that meet business needs.
Optimize and troubleshoot existing data pipelines for performance and reliability.
Implement data quality and validation processes to ensure accuracy and consistency of data.
Stay updated on industry trends and best practices in data engineering to continuously improve our data infrastructure.
Work closely with data scientists, analysts, and other stakeholders to understand data use cases and provide the necessary infrastructure and support.
Assemble large, complex sets of data that meet non-functional and functional business requirements.
Identify, design and implement internal process improvements including re-design infrastructure for greater scalability, optimize data delivery, and automating manual processes.