> Markdown version of [/jobs/ext/3968163-senior-data-engineer-cybersecurity-big-data](https://www.wearedevelopers.com/jobs/ext/3968163-senior-data-engineer-cybersecurity-big-data). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Engineer - Cybersecurity & Big Data

- **Company:** Northern Base
- **Location:** Charlotte, NC, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Amazon Web Services, Microsoft Azure, Big Data, Unix, Cloud Computing, Cyber Security, Continuous Integration, Information Engineering, Data Integration, Data Transformation, Data Structures, Data Stores, Data Systems, Elasticsearch, Apache Hadoop, Apache HBase, Apache Hive, Python (Programming Language), Meta-Data Management, Scrum Methodology, Systems Development Life Cycle, Release Management, Standard Sql, Cloudera, Software Engineering, SQL Databases, Software Vulnerability Management, Software Organization, Data Processing, Performance Testing, Apache Spark, Pyspark, Information Technology, Data Analytics, Bitbucket, Data Management, Software Coding, Data Pipelines
- **Published:** October 27, 2026
- **Apply:** https://www.careerjet.com/jobad/usa6f278fa442f76bcc72f28f06cc09b31

## About the Role

Must-Have Skills:
8+ Years of Hands-On Data Engineering / Software Development Experience
Strong SQL and UNIX Experience
Hadoop Ecosystem Hive, Spark and PySpark
Big Data Processing Platforms Hadoop / Cloudera
Python Programming and Modular Python Development
Data Pipeline Development and Data Transformation
Data Integration, Cleaning, Validation and Quality Controls
Data Stores Hive, HBase and Elasticsearch
Cloud Technologies AWS or Azure
Version Control Bitbucket or Equivalent
CI/CD Pipelines and Release Management
Data Engineering Principles, SDLC and Software Development Best Practices
Cybersecurity Domain Knowledge, Preferably Vulnerability Data Management
Technical Leadership, Hands-On Coding and Mentoring
Preferred Experience:
Cybersecurity Data Analytics and Vulnerability Management
Complex Data Processing Across Multiple Systems
Metadata Management, Dependency Management and Workload Optimization
Data Quality Controls and Internal Control Frameworks
Integration, Regression and Performance Testing
Agile / Scrum Delivery Environments
Guiding Junior Engineers and Coordinating CI/CD Releases
Strong Stakeholder Communication and Requirements Analysis, Education: Bachelor's degree in Computer Science required.

## Description

Design, develop and deliver complex data solutions supporting cybersecurity technology and business objectives.
Build and maintain data pipelines for integration, cleaning, transformation and processing across operational and analytics systems.
Develop scalable data solutions using Hadoop, Hive, Spark, PySpark, Python and SQL.
Assemble large, complex datasets that meet functional and non-functional requirements.
Build data structures, metadata frameworks, data quality controls and workload management processes.
Collaborate with business stakeholders, product owners, architects, data scientists and development teams to translate requirements into technical solutions.
Evaluate new technologies, conduct prototypes and validate architectural approaches.
Identify data management gaps, technical risks and requirement challenges, and communicate them to stakeholders.
Contribute to integration, regression and performance testing; analyze results and troubleshoot issues.
Maintain technical documentation for deployment, maintenance, support and business functionality.
Follow established release processes, CI/CD practices and code deployment standards.
Mentor junior data engineers and support pull requests, issue triage, merge conflict resolution and release documentation.
Define key performance indicators and internal controls to improve data quality and operational efficiency.