Sr. Data Engineer

Long Description

Job Summary

Oversee and manage various aspects of data integration and data warehouse processes. This includes implementing data quality measures, designing and deploying integration solutions, collaborating with team members, troubleshooting production issues, refining best practices, educating end users, participating in code reviews, exploring, and adopting new technologies, leading prototype development, and ensuring adherence to industry standards. The role involves working within Agile teams, reporting on project status, and partnering with IT Solution Architects to optimize data integration designs. The goal is to contribute to efficient and effective data analytics integration solutions while staying informed about technological trends and mentoring team members.

Job Responsibilities

Operational Work – 75% of time 

• Implement Data Warehouse data quality processes. Provide solutions where issues are identified
• Design, develop, test, and deploy data integration solutions based on business requirements
• Collaborate with team members on data integration designs and alternatives
• Troubleshoot and resolve production issues
• Continually refine and document data integration best practices
• Educate end users (where applicable) in the use of metadata and understanding where data originates
• Participate in code reviews
• Identify, evaluate, and refine leading-edge data integration tools and methods 
• Build prototypes and POCs
• Bring a passion to stay on top of tech trends, experiment with and learn new technologies, participate in internal & external technology communities, and mentor other members of the engineering community
• Own the development of cross-functional, multi-platform prototypes
• Work within and across Agile teams to design, develop, test, implement, and support data analytics integration solutions
• Document and support deployment activities
• Report on status of IT Data Operations products and projects on a regular and timely basis
• Ensure the timely resolution of issues
• Follow standards and guidelines according to industry best practices 
• Escalate anticipated risks to management promptly and properly
• Partner with IT Solution Architects regarding data integration design options

Strategic Work – 25%

• Educate and inform yourself and team members about data integration technology solutions and emerging trends 
• Continually participate in visioning exercises and contribute to the data strategy

Candidate Requirements

Required years of relevant experience:

• 4+ years of experience designing, developing, and maintaining data integration solutions

Desired Qualifications:

• Education - B.S. in Computer Science or related discipline.
• Advanced degree a plus

Specialized Skills:

Required:

• Data integration development experience with the following skill sets:
• Extract, Transformation, and Load (ETL) of disparate data sources to targets including databases, files, and API’s. Talend, Matillion, Glue Data Studio preferred
• Ability to detect, identify and resolve source to target performance / load issues
• Expert level proficiency in structured query language (SQL)

               o    Data Warehousing
                     -Thorough understanding of relational data models including third normal form and fact / dimension designs
                     -Change data capture (CDC) design options and application
                     -Data marts
             o    Oracle database platform features including indexing, partitioning and PL/SQL
             o    AWS Data Migration System (DMS), Amazon Redshift, Aurora Postgres Expertise including but not limited to performance tuning
             o    Linux O/S commands and scripting
             o    Secure File Transfer Protocol (SFTP)
             o    Desire to work in the data integration space (data warehouse, data lake, AWS, data API’s) while continuing to grow skill sets

Desired:

• Modern development practices – e.g. DevOps
• Experience with open source database platforms:

                    - PostgreSQL
                     - MySQL
                     -MariaDB

• Data Lake integration design and development (ETL & ELT Methodologies)
• Experience with data quality tools
• Experience with AWS platform:

               o    DMS
               o    Glue
               o    Redshift
               o    S3, etc

• AWS Certifications
• Experience in API development
• Master data management integration and concepts
• Experience with job scheduling solutions such as Airflow
   

Education