Job Description

Position: Apache Iceberg Engineer
Location: Austin, TX
Duration: 6+ Months

Job Details:

Exp: 8 +Years

• Create and maintain scalable data pipelines and workflows to ingest, transform, and load data into Apache Iceberg tables using Apache Spark, Python, PySpark
• Write SQL queries using Spark and Trino, optimize and remove performance bottlenecks for query response and resource utilization
• Develop and execute migration strategies and scripts to move data from Snowflake to Iceberg tables, ensuring data integrity and consistency
• Write reliable and maintainable code, debug complex issues in large-scale distributed systems, document, participate in code reviews, and mentor other engineers

Minimum Qualifications

• 3+ years of experience in Data Lake Architectures, data warehousing technologies and data engineering.
• Strong understanding of Apache Iceberg, including schema evolution, partition management, and data versioning
• Proficient in Apache Spark, PySpark and other big data processing frameworks like Presto or Trino, Flink
• Proficient in Snowflake Database and Architecture, SQL development and data modeling
• Experience with cloud platforms and technologies like EMR, AWS IAM, AWS S3, including object storage services and compute services
• Hands-on experience of working with Git, Jupyter notebooks, Splunk and Sandbox environments
• Exceptional communication and collaboration skills, with the ability to work effectively across multiple teams and functions.

Job Tags

Similar Jobs