JOB LOCATION: 2900 W. Plano Parkway, Plano, TX 75075
DUTIES: Design, implement, and maintain big data engineering modules and solutions as part of AI, and as a service using different cloud provider services. Develop Python-Spark (PySpark) ETL, and ELT Pipelines to perform distributed big data processing in both batch and real time streaming from different data sources: Snowflake, MongoDB, Redis cache, Microsoft Event hubs and Kafka topics. Integrate multiple cloud data SAAS technologies with cloud provider services: Azure function applications. Create serverless framework asynchronous APIs for distributed data processing and computing tasks. Develop and maintain different high performance Kubernetes spark application microservices and restful API's for distributed big data processing and computation tasks. Work with Hive data warehouse creating tables and data distribution by implementing Partitioning and Bucketing strategies, writing, and optimizing SQL queries. Perform analytics on structured and semi-structured data using PySpark, Spark SQL queries, operations, Joins, tuning queries and UDF. Integrate cloud data lakes with downstream cloud provider visualization and analytics tools: Power-BI. Implement several design patterns, make code robust and highly scalable, and use high performance data structures to make applications to perform quickly and efficiently. Consume Web Services through CLI to interact with other external interfaces to exchange data in different forms by using RESTful serverless framework and Handling Different File Formats: CSV, JSON, Multi-line JSON, Nested JSON, Text, Avro, Parquet file formats, Snappy, BZ2, and Gzip compression. Develop different Spark pipelines in Databricks software, schedule notebook jobs for daily sync jobs, and monitor feature engineering pipelines. Develop scripts to use modern Big-Data tools: PySpark, Spark SQL. Develop scripts to use modern Big-Data tools to convert schema-less data into structured files for further analysis. Modify SQL queries to Spark using Spark RDDs, Scala, and Python. Work on creating combiners, partitioning, and distributed cache to improve the performance of Apache Spark jobs. Utilize big data programming languages and technology. Writing code and complete programming and documentation. Perform the testing and debugging of applications. Analyze, design, program, debug and modify software enhancements and/or new products used in distributed, large scale analytics and visualization solutions.
REQUIREMENTS: Requires a Master's degree, or foreign equivalent degree, in Information Systems and Technology or Computer Science and three (3) years of experience in the job offered or three (3) years of experience in a related occupation developing Python-Spark (PySpark) ETL, and ELT Pipelines to perform distributed big data processing in both batch and real time streaming from different data sources: Snowflake, MongoDB, Redis cache, Microsoft Event hubs and Kafka topics; utilizing big data programming languages and technology; writing code and completing programming and documentation; performing the testing and debugging of applications; and analyzing, designing, programming, debugging and modifying software enhancements and/or new products used in distributed, large scale analytics and visualization solutions.
Our Senior-Big Data Software Engineers earn between $157,286 - $215,800 yearly. Not to mention all the other amazing rewards that working at AT&T offers.
Joining our team comes with amazing perks and benefits:
Medical/Dental/Vision coverage
401(k) plan
Tuition reimbursement program
Paid Time Off and Holidays (based on date of hire, at least 23 days of vacation each year and 9 company-designated holidays)
Paid Parental Leave
Paid Caregiver Leave
Additional sick leave beyond what state and local law require may be available but is unprotected
Employee discounts up to 50% off on eligible AT&T mobility plans and accessories, AT&T internet (and fiber where available) and AT&T phone
Weekly Hours: 40
Time Type: Regular
Location: Plano, Texas
It is the policy of AT&T to provide equal employment opportunity (EEO) to all persons regardless of age, color, national origin, citizenship status, physical or mental disability, race, religion, creed, gender, sex, sexual orientation, gender identity and/or expression, genetic information, marital status, status with regard to public assistance, veteran status, or any other characteristic protected by federal, state or local law. In addition, AT&T will provide reasonable accommodations for qualified individuals with disabilities.