Contract position for Apache Iceberg Engineer Location: Austin, TX Job at Veah Consulting Services, Austin, TX

bGx3T3hKRTRsbEZoQUpnZE9vUU1Bb09DWUE9PQ==
  • Veah Consulting Services
  • Austin, TX

Job Description

Position: Apache Iceberg Engineer
Location: Austin, TX
Duration: 6+ Months

Job Details:

Exp: 8 +Years

  • Create and maintain scalable data pipelines and workflows to ingest, transform, and load data into Apache Iceberg tables using Apache Spark, Python, PySpark
  • Write SQL queries using Spark and Trino, optimize and remove performance bottlenecks for query response and resource utilization
  • Develop and execute migration strategies and scripts to move data from Snowflake to Iceberg tables, ensuring data integrity and consistency
  • Write reliable and maintainable code, debug complex issues in large-scale distributed systems, document, participate in code reviews, and mentor other engineers

Minimum Qualifications

  • 3+ years of experience in Data Lake Architectures, data warehousing technologies and data engineering.
  • Strong understanding of Apache Iceberg, including schema evolution, partition management, and data versioning
  • Proficient in Apache Spark, PySpark and other big data processing frameworks like Presto or Trino, Flink
  • Proficient in Snowflake Database and Architecture, SQL development and data modeling
  • Experience with cloud platforms and technologies like EMR, AWS IAM, AWS S3, including object storage services and compute services
  • Hands-on experience of working with Git, Jupyter notebooks, Splunk and Sandbox environments
  • Exceptional communication and collaboration skills, with the ability to work effectively across multiple teams and functions.

Job Tags

Contract work,

Similar Jobs

test ad

IT Recruiter Job at test ad

A Senior IT Recruiter is a specialized role focused on attracting, recruiting, and hiring qualified IT professionals. They are responsible for the full recruitment lifecycle, from sourcing and screening candidates to conducting interviews and negotiating offers, while also... 

JPMorgan Chase Bank, N.A.

Investment Banking - Corporate Finance Advisory - Associate Job at JPMorgan Chase Bank, N.A.

 ...Job Summary Join the Investment Banking Division, as an Associate in the Corporate Finance Advisory team, where you will be a key player in the execution of the firm's business. You will work with expert professionals at the heart of a leading global investment bank... 

Avint

SOC Analyst Job at Avint

 ...Avint LLC is seeking a SOC Analyst who has experience providing support in a dynamic, fast-paced environment within the public sector. This is a unique opportunity to shape the growth, development, and culture of an exciting and fast-growing company in the cybersecurity... 

HomeLight

Junior Paralegal Job at HomeLight

 ...administrative processes, and are motivated by the opportunity to grow within the legal field. Whether youre a recent graduate, a law student, or someone with prior administrative experience, you're ready to make an impact and build your legal career from the ground up.... 

Amentum

Records Administrator Job at Amentum

 ...or an Associate's Degree. Must have excellent communication skills and be able to communicate between vendors, engineers, and the Amentum Business Office. Most time in this position is spent communicating on the phone, in person, and through email. Candidate must have...