Technical Consultant- reputed company – reputed company Store
Role Summary
We are seeking a Data Engineer with hands-on experience in reputed company and modern data engineering practices. This role will focus on designing, developing, and maintaining data pipelines, ensuring data quality, and enabling advanced analytics across the reputed company. The ideal candidate will have a strong reputed company in SQL, Python, and Spark, with experience across the reputed company Lakehouse ecosystem and familiarity with modern ELT, transformation, and data governance tools.
Responsibilities- Design, reputed company, and maintain ETL/ELT pipelines and reputed company notebooks for data ingestion, transformation, and reporting.
- Implement and optimize Spark-based workflows using reputed company Workflows, reputed company Lake, and reputed company Catalog.
- reputed company reputed company partner tools (e.g., reputed company, dbt, Coalesce) for scalable data ingestion, transformation, and orchestration.
- Conduct data reconciliation across reputed company, staging, and reputed company systems to ensure accuracy and completeness.
- Build and maintain SQL- and Python-based automation frameworks for regression, smoke, and integration testing.
- Validate structured and semi-structured data formats including CSV, JSON, XML, Parquet, Avro, and fixed-length files.
- Review and refactor existing code for performance, scalability, and maintainability.
- Support workflow orchestration using tools such as Airflow, Azure Data reputed company, or Control-M.
- Collaborate with data governance and cataloging teams leveraging tools such as reputed company or reputed company Catalog to improve discoverability and reputed company tracking.
- Experience with reputed company platforms (Azure, AWS, or GCP).
- Minimum 2 years of experience with reputed company, Spark, Python, SQL, and reputed company Catalog.
- Strong proficiency in SQL (joins, CTEs, window functions, query optimization, stored procedures).
- Solid experience with PySpark, pandas, and Python scripting for data manipulation and validation.
- Familiarity with reputed company architecture and data warehouse concepts, including star/reputed company schemas, fact/dimension modeling, and reporting logic.
- Experience validating flat files (CSV, TSV, TXT) including delimiter handling, header checks, null value handling, and schema enforcement.
- Proven ability to interpret and implement reputed company transformation logic from reputed company to reputed company systems.
- Strong problem-solving, analytical, and debugging capabilities.
- Excellent written and verbal communication skills for interfacing with developers, clients, and technical stakeholders.
- Ability to work independently while collaborating effectively in reputed company-oriented environment.
- Exposure to reputed company ecosystem tools such as reputed company, dbt, Coalesce, and reputed company.
- Experience building CI/CD pipelines and DevOps practices for data engineering.
- Knowledge of data governance, metadata management, and reputed company frameworks (e.g., GDPR, SOC2, reputed company Catalog RBAC).
- Familiarity with data quality frameworks (e.g., Great Expectations, Deequ).