Lead Data Engineer Job at Argyllinfotech, San Jose, CA

aHhCNzMySTlaRkN5M0pVVXBhRXhHWno2RVE9PQ==
  • Argyllinfotech
  • San Jose, CA

Job Description

Hello Everyone,

No GC,

Role: Senior Lead Data Engineer

Location: San Jose, CA 95125 Hybrid(Locals)

Key: Hadoop, Spark SQL strong

Key Responsibilities:

Architect, construct , test and maintain ELT pipelines using Hadoop and Spark SQL

- Manage and optimize the infrastructure ensuring reliability and performance

-Design and implement data models and schemas for enterprise data warehouse and data marts

-Write and maintain scripts in Python for various applications

-Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases

- Work closely with analysts and product managers to understand data requirements and deliver high-quality data

-Provide technical guidance and mentorship to other team members fostering best practices in data engineering

-Monitor system performance, identify bottlenecks, implement solutions to ensure high data availability

Required Skills:

-High proficiency in the Hadoop ecosystem (HDFS, MapReduce, Yarn) and Apache Spark, Scala for large scale data processing

-Good knowledge in Python for data manipulation and scripting

-Mastery of advanced SQL and deep experience working with relational databases and data warehouses

-Hands-on experience with workflow management tools like Apache Airflow

-Familiarity with Unix shell scripting for automation and system management

Job Tags

Local area,

Similar Jobs

Bethel Church of Redding

Marketing Copywriter & Editor Job at Bethel Church of Redding

The Marketing Copywriter and Editor is a key member of the Creative Team, blending strong writing skills with strategic insight and editorial excellence. The role works closely with Bethels Marketing Manager to execute marketing materials for campaigns to support the... 

Mayo Clinic

Registered Nurse - Hematology/Oncology/Cellular Therapy - RN Job at Mayo Clinic

 ...*Location:** Eisenberg 72 (EI72), Hematology/Oncology/Cellular Therapy (General Care)**Patient Care Type:** This unit specializes in...  ...:** Patient care services include chemotherapy, immunotherapy, radiation therapy, chimeric antigen receptor T-cell (CAR T) therapy, pain... 

Amazon Data Services, Inc.

Data Center Technician, DCC Communities Job at Amazon Data Services, Inc.

 ...gangways safely and without limitation - Work in a noisy environment - Work shifts longer than 8 hours to support 24/7 operations (covering both night and day shifts). NOTE: Amazon does not sponsor for immigration, including for H-1B, TN, and other non-immigrant... 

Vision Truck Lines

Class A Truck Driver 1625 per week Felony n Accident Friendly OTR EEO MAPFAOTR Job at Vision Truck Lines

 ...accidents are subject to Safety Review. Termination - Needs 6 months of safe Driving after a termination for safety Felony amp; Misdemeanor Convictions - :No drug/possession charges in the last 5 years and no more than two in a life time :No... 

Enterprise Medical Recruiting

Sports, Total Joint & General Orthopedics with No Call - Southwest Minnesota Job at Enterprise Medical Recruiting

 ...A recognized orthopedic and sports medicine practice located in southwest Minnesota on the shores of beautiful Lake Okabena would like to add an orthopedic surgeon to its team-focused group. Enjoy a diverse mix of sports medicine, total joint and general orthopedic cases...