Senior DataBricks Developer at CAI in Harrisburg, Pennsylvania

Job Description:

Senior DataBricks Developer

Req number:

R2321

Employment type:

Full time

Worksite flexibility:

Remote

Job Summary

As a Senior DataBricks Engineer, you will be responsible for coding and modifying websites and web applications, from layout to function and according to a client's specifications.

Job Description



We are looking for a Senior DataBricks Engineer to develop robust code, understand and implement data pipelines and perform data transformations within the DataBricks platform. This role requires a combination of technical expertise, strong analytical skills, leadership skills, and a strong understanding of data engineering principles. This is a full-time, remote, contract opportunity.



What You'll Do

• Work closely with Software Designers to ensure best practices are being followed and offer suggestions for improvements to code proficiency and maintainability
  
  


• Occasionally interface with customers to assist with analyzing user needs to determine technical requirements
  
  


• Design, build, and maintain scalable and reliable data pipelines using DataBricks
  
  


• Develop high-quality code with an emphasis on performance, scalability, and security
  
  


• Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs
  
  


• Implement data transformations and complex algorithms within the DataBricks environment
  
  


• Optimize data processing and refine data architecture to improve system efficiency and data quality
  
  


• Mentor junior engineers and contribute to the development of best practices within the team
  
  


• Stay up-to-date with emerging trends and technologies in data engineering and cloud computing
  
  


• 
  
  
  

What You'll Need



Required:

• 7+ years of experience in data engineering or a related field
  
  


• Strong hands-on experience with DataBricks, including development of code, pipelines, and data transformations
  
  


• Proficient in at least one programming language (e.g., Python, Scala, Java).
  
  


• In-depth knowledge of Apache Spark and its integration within DataBricks
  
  


• Experience with cloud services (AWS, Azure, or GCP) and their data-related products
  
  


• Familiarity with CI/CD practices, version control (Git), and automated testing.
  
  


• Excellent problem-solving skills and the ability to work independently or as part of a team
  
  


• Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical field
  
  

Physical Demands

• Ability to safely and successfully perform the essential job functions consistent with the ADA and other federal, state and local standards
  
  


• Sedentary work that involves sitting or remaining stationary most of the time with occasional need to move around the office to attend meetings, etc.
  
  


• Ability to conduct repetitive tasks on a computer, utilizing a mouse, keyboard, and monitor
  
  

Reasonable Accommodation Statement



If you require a reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employment selection process, please direct your inquiries to application.accommodations@cai.io or (888) 824 - 8111.



Equal Employment Opportunity Policy Statement



It is the policy of CAI not to discriminate against any employee or applicant due to race, color, religion, sex, sexual orientation, gender identity, national origin, age, marital status, genetic information, disability or being a protected veteran. It is also the policy of CAI to take affirmative action to employ and to advance in employment, all persons regardless of race, color, religion, sex, sexual orientation, gender identity, national origin, age, marital status, genetic information, disability or being a protected veteran, and to base all employment decisions only on valid job requirements. This policy shall apply to all employment actions, including but not limited to recruitment, hiring, upgrading, promotion, transfer, demotion, layoff, recall, termination, rates of pay or other forms of compensation and selection for training, including apprenticeship, at all levels of employment. Employees and applicants of CAI will not be subject to harassment due to race, color, religion, sex, sexual orientation, gender identity, national origin, age, marital status, genetic information, disability or being a protected veteran. Additionally, retaliation, including intimidation, threats, or coercion, because an employee or applicant has objected to discrimination, engaged or may engage in filing a complaint, assisted in a review, investigation, or hearing or have otherwise sought to obtain their legal rights under any Federal, State, or local EEO law is prohibited. Apply Now