Information Technology_USA - USA_Engineer

**Please strictly adhere to the following resume naming convention:

ALL CAPS, NO SPACES BETWEEN UNDERSCORES

PTN_US_GBAMSREQID_CandidateBeelineID
Example: PTN_US_9999999_SKIPJOHNSON0413

: -

MSP Owner: Michelle Lee
Location: Seattle, WA
Duration: 6 months
skill id: 10729592

Responsibilities:
• Design, develop, and maintain scalable data pipelines and ETL workflows using Azure Databricks and PySpark.
• Build and optimize data models, data transformations, and distributed data processing solutions.
• Work closely with data architects, analysts, and business stakeholders to understand requirements and deliver robust data solutions.
• Develop and maintain data ingestion frameworks for structured and unstructured data sources.
• Implement data quality checks, validation rules, and automated monitoring for pipeline reliability.
• Optimize Spark jobs for performance and cost efficiency in Azure cloud environments.
• Develop reusable components, frameworks, and best practices for data engineering processes.
• Collaborate with DevOps teams for CI/CD deployment of data pipelines.
• Ensure security, compliance, and governance across all data processes and environments.
• Maintain detailed technical documentation and support production systems.

Skills
• 6-8 years of experience in building scalable data pipelines and data solutions.
• Hands-on experience with:
o Azure
o Databricks
o PySpark
• Strong experience designing, developing, and optimizing big data processing systems on cloud platforms.
• Expertise in Azure data services and advanced Spark-based data engineering.
• Experience with ETL workflows and distributed data processing.
• Knowledge of CI/CD practices for data pipelines.
• Strong understanding of data quality, monitoring, and governance practices.
• Ability to work with structured and unstructured data sources.

Skills: Digital : Microsoft Azure ~ Digital : Databricks ~ Digital : PySpark

Experience Required: 6-8, Project Code :