Sr. Data Engineer

The Sr. data engineer will play a pivotal role in designing, building and operationalizing data pipelines that feed the enterprise data ecosystem for a variety of multi-faceted analytics created by data consumers. The data engineer needs to guarantee compliance with data governance and data security requirements while creating, improving and operationalizing integrated and reusable data pipelines. The data engineer will be measured on their ability to create repeatable, efficient, high-quality code, automation-friendly processes, and delivering work products quickly.



Responsibilities:

Lead construction, maintenance, & optimization of data pipelines.

Drive Automation through effective metadata management, using innovative and modern tools, techniques, and architectures to improve productivity.

Lead renovation of the data management infrastructure to drive automation in data integration and management.

Collaborate with multiple Analytics Center of Excellence teams and EIM team in refining their data requirements for various DnA initiatives and data consumption requirements.

Identify gaps in new data initiatives and how to address new data requirements.

Transfer knowledge of data and/or domain understanding in addressing new data requirements.

Propose appropriate (and innovative) data ingestion, preparation, integration and operationalization techniques to optimize data requirements.

Work with data governance teams to ensure data scientists and consumers use corresponding data responsibly through data governance and compliance initiatives.

Design and implement data quality monitoring systems, which includes source-to-target data validations as well as anomaly detection



Requirements:

Minimum 7 years of work experience in data management disciplines, including data integration, modeling, optimization and data quality, and/or other areas directly relevant to data engineering responsibilities and tasks.

Minimum 7 years of hands-on experience designing and implementing production grade data warehousing solutions on large scale databases such as SQL Server, Oracle or DB2

5+ years data pipeline development experience with Informatica PowerCenter

3+ years of experience with advanced analytics tools for Object-oriented/object function scripting using languages such as R, Python, Java, C++, Scala, others

3+ years of experience with Client/Perl Scripting

5+ years of experience with popular database programming languages including SQL, PL/SQL (required), others for relational databases and certifications on upcoming NoSQL/Hadoop oriented databases (preferred) like MongoDB, Cassandra, others for nonrelational databases.

Minimum 2 years designing and implementing a fully operational production grade large scale data solution on Snowflake Data Warehouse.



Preferred skills:

Hands on experience with Palantir Foundry + AiP

Preferred experience in working with data science teams in refining and optimizing data science and machine learning models and algorithms

Full Lifecycle Data Warehouse, Data Mart Development experience is a big plus.

Agile Software development methodologies

Preferred experience with any automation tool such as AutoSys, Cntl-M