Lead Data Engineer Job at WorkHQ, Los Angeles, CA

aEJONTBtazVaRm16M1pVU3BxODVFNVQvRnc9PQ==
  • WorkHQ
  • Los Angeles, CA

Job Description

Company Context

Series A, well-funded US startup in HRTech developing WorkHQ.com and an AI Recruiter product.

This is a US-only, Remote role (Mainland).

Role Overview

Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.

Hire data engineers to aid you in that journey.

Core Responsibilities

  • Design scalable data pipelines processing massive record volumes

  • Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)

  • Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch

  • Integrate new data sources into the main pipeline

  • Implement advanced data matching using Splink

Technical Requirements

  • 5-8 years professional data engineering experience

  • Good proficiency in:

    • PySpark and distributed computing

    • AWS data services (EMR, Glue, Athena)

    • Docker

    • Pandas and DataFrame manipulation

    • Complex data format handling (JSONL, Parquet)

  • Strong background in:

    • Big data processing architectures

    • Data warehouse design

    • Performance optimization

  • Advanced Python, SQL skills

Nice to Have

  • Probabilistic record linking expertise

  • OpenSearch/elasticsearch technologies

  • Machine learning data pipeline design

  • Recruitment tech ecosystem knowledge

Technical Stack

  • Big Data: PySpark, EMR

  • Databases: Postgres, OpenSearch

  • Cloud: AWS

  • Containerization: Docker

  • Data Formats: JSONL, Parquet

  • Analytics: Metabase, Athena, Glue

  • Data Processing: Pandas, Splink

Other Considerations

While this role has specific requirements - if you lack a few technical skills, but motivated to learn and lead the platform, please apply for consideration.

If you are coming from Director/Head of/VP levels that is relevant to this job, you can apply as well.

You will need to apply directly on our platform.

Thank you for your time.

Job Tags

Permanent employment, Remote work, Shift work,

Similar Jobs

Hawthorne Lane

Appellate Paralegal Job at Hawthorne Lane

 ...encouraged to apply. This firm offers substantive work, excellent benefits, and a great work...  ...staff as needed.~Utilize MS Office and legal technology to manage document databases...  ...~This position is approved to work from home three days a week after training and onboarding... 

Reality Defender

Applied AI Engineer - (Audio) Job at Reality Defender

 ...platform offering comprehensive deepfake detection. A Y Combinator graduate, Comcast NBCUniversal LIFT Labs alumni, and backed by DCVC,...  ...deepfake and AI-generated content detection technology is developed by a leadership team with over 20 years of experience in applied... 

EFitz Logistics

Dry Van CDL-A Truck Driver | Home Weekly Job at EFitz Logistics

 ...Qualifications: ~ Valid CDL A license. ~ At least12 months of verifiabletractor...  ...You must live inCalifornia. ~ No SAP drivers. Job Details: Average Weekly Pay...  ...Running area: Southeast region. Type of Trucks: Kenworth, Freightliner & International... 

Keller Williams Rockwall

Newly Licensed Real Estate Agent Job at Keller Williams Rockwall

 ...Summary Real Estate Career Launchpad | Keller Williams West Cobb Youre not looking for another job Youre looking for a career...  ..., we built a proven mentorship pathway designed to take new agents from overwhelmed confident producing, and profitable.... 

Home Care Providers of Texas

Home Health-RN-PRN All Days Job at Home Care Providers of Texas

 ...Seeking for a PRN Home Health Registered Nurse for Weekdays and/or Weekends in the North Dallas, TX area! Are you seeking relief from the constant pressure and time constraints of your current nursing job? Look no further than Home Care Providers of Texas! At HCPT, our...