Data Engineer II

About the job

About the Role

Our dev/ops team builds and supports the data engineering solutions that power analytics, reporting, and operational decision-making across the business and Field. We develop and maintain scalable ELT pipelines using tools like Python, Spark, Databricks, and orchestration frameworks to deliver reliable, high-quality data for downstream consumers, including Power BI and SSRS reporting solutions. We focus on production reliability, data quality, observability, and continuous improvement, and we partner closely with business and technology teams to turn complex data needs into usable data products.

What You'll do

• Design, implement, and maintain ELT pipelines and transformations using Python/PySpark/SQL.
• Own production reliability: monitor SLAs, respond to incidents, notebooks, and reduce MTTR.
• Implement and maintain CI/CD pipelines to automate addition and modification of custom stages in ELT workflows.
• Orchestrate workflows with Airflow, Control M (or equivalent); handle retries, backfills, and schema changes.
• Build and maintain streaming integrations (Kafka/Kinesis or equivalent) for near‑real‑time use cases.
• Implement and maintain data quality, observability and auditing (e.g., DQX, data expectations, pydat, custom checks).
• Optimize pipelines for cost and performance on cloud platforms (Databricks, Snowflake, or similar).
• Participate in code reviews, design discussions, and documentation; mentor junior engineers.
• Identify root causes of data issues and implement robust fixes and preventative measures.
• Collaborate with stakeholders to translate business requirements into reliable data solutions.

What You'll Bring to the Role

• 2–5 years professional experience in data engineering or software engineering with production systems.
• Strong Python programming skills and experience with distributed processing (PySpark or Scala + Spark).
• Solid SQL skills and experience with at least one cloud data platform (Snowflake, Databricks, or equivalent).
• Working knowledge of Power BI, including building datasets and reports, data modeling, and troubleshooting dataset refreshes and performance issues.
• Experience with orchestration tools (Airflow, Control-M, or equivalent) and CI/CD pipelines.
• Familiarity with event streaming systems (Kafka, SQS, or similar) and batch/stream integration patterns.
• Experience implementing data quality checks, testing data pipelines, and handling schema changes.
• Proficient with Git and modern DevOps practices; comfortable reading and troubleshooting logs.
• Has or develops understanding of 1-3 subject areas/domains of data.
• Good communicator; able to explain technical solutions to peers and stakeholders; self-directed.

Preferred Qualifications

• Familiarity with Databricks ELT architecture (Declarative Automation Bundles, Delta Lake, Jobs/Workflows, and Delta Live Tables).
• Experience with Terraform, Kubernetes, Docker, or other IaC/container tooling.
• Familiarity with observability/monitoring tools (Grafana, Dynatrace, etc.) and alerting best practices.
• Experience with data quality frameworks (DQX, Deequ) and data governance/catalog tools.
• Domain knowledge in one or more business areas outside of the Field domain (e.g., Client, Product, etc).

Skills You'll Have

Consulting (NM): Partners with stakeholders to gather requirements, solve problems, and develop effective technical solutions.

Data Architecture (NM): Designs and maintains scalable data structures, integration frameworks, and data management solutions.

Data ETL (NM): Develops and supports data extraction, transformation, and loading processes across multiple systems and platforms.

Data Security (NM): Protects sensitive data and ensures compliance with governance, privacy, and regulatory requirements.

Databases & Data Platforms (NM): Utilizes databases and modern data platforms to store, manage, and access enterprise data.

DevOps (NM): Applies CI/CD practices and automation to improve delivery speed, reliability, and quality.

Engineering Expertise & Practices (NM): Applies engineering principles and technical best practices to deliver reliable and scalable data solutions.

Compensation Range:

Pay Range - Start:
$89,360.00
Pay Range - End:
$134,040.00
Geographic Specific Pay Structure:

Structure 110:
$98,320.00 USD - $147,480.00 USD
Structure 115:

$102,800.00 USD - $154,200.00 USD
We believe in fairness and transparency. It’s why we share the salary range for most of our roles. However, final salaries are based on a number of factors, including the skills and experience of the candidate; the current market; location of the candidate; and other factors uncovered in the hiring process. The standard pay structure is listed but if you’re living in California, New York City or other eligible location, geographic specific pay structures, compensation and benefits could be applicable, click here to learn more.

Grow your career with a best-in-class company that puts our clients' interests at the center of all we do. Get started now!

Northwestern Mutual is an equal opportunity employer that welcomes talented individuals of all backgrounds. We are committed to creating and maintaining an environment in which each employee can contribute creative ideas, seek challenges, assume leadership and continue to focus on meeting and exceeding business and personal objectives.