Data Platform Engineer
RIVAGO INFOTECH INC.
Charlotte, NC, United States
3 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Job source
Tech stack
Big Data
BigQuery
Code Review
Cyber Security
Data as a Services
Data Cleansing
Data Deduplication
Data Infrastructure
Github
Apache Hadoop
Apache Hive
Information Systems Security Architecture Professional
+28 more
Python (Programming Language)
Enterprise Messaging Systems
SonarQube
Toolchain
Software Vulnerability Management
Data Processing
Google Cloud
Large Language Models
Sonatype
Software Security
Veracode
Gitlab
Data Lakes
Pyspark
Data Analytics
Apache Kafka
Checkmarx
Video Streaming
Cloud Migration
Restful APIs
Data Pipelines
Devsecops
Jenkins
Servicenow
Static Application Security Testing
Golang
Programming Languages
Dynamic Application Security Testing
Job description
- Pipeline Development & Optimization: Design and implement robust batch and real-time streaming pipelines using PySpark, Hadoop, Hive, and Apache Kafka to process large volumes of security vulnerability findings.
- Data Cleansing & Deduplication: Implement advanced data quality, enrichment (severity, criticality scoring), and deduplication logic comparing multiple scanning feeds (e.g., LLM scanners vs. Checkmarx/Snow AR) to eliminate duplicate findings.
- Cloud Migration & Data Lake Integration: Ingest and process data within Google Cloud Platform (Google Cloud Platform) leveraging Google Cloud Platform Data Lake, BigQuery, and modern cloud storage/compute frameworks, facilitating the transition from on-premise processing to Google Cloud Platform.
- Extensible Scanning Framework: Build a modular, pluggable framework that enables the security team to integrate, evaluate, and compare output results from newly introduced LLM/security scanner models (e.g., Mythos, Codex, Astra) within days.
- ServiceNow & Remediation Workflows: Format, validate, and deliver clean, actionable data to downstream systems.
Requirements
- Programming Languages: Proficiency in Python and Go (Golang).
- Big Data & Streaming Technologies: Hands-on experience with PySpark, Apache Hadoop, Apache Hive, and Apache Kafka (both batch and real-time/event-driven data pipelines).
- Google Cloud Platform (Google Cloud Platform): Proven experience with BigQuery, Google Cloud Platform Data Lake architectures, and Google Cloud Platform data services.
- Data Processing & Architecture: Demonstrated experience building data quality validation, cross-source deduplication, data enrichment, and schema standardization pipelines.
- Tools & Integration: Experience integrating data pipelines with enterprise REST APIs, messaging systems, and security/ITSM platforms., * Prior domain experience in Vulnerability Management, Application Security, or Cyber Security Data Analytics.
- Familiarity with Code Scanning / SAST / DAST tools (e.g., Checkmarx) and ServiceNow modules (Vulnerability Response / SecOps / Snow AR).
- Experience handling enterprise-scale code repository metadata (GitHub/GitLab pipelines).
- Financial services or highly regulated enterprise environment experience.
Preferred Qualifications
-
Certified Information Systems Security Professional (CISSP), Certified Application Security Engineer (CASE), or CSSLP.
-
Experience working in a large enterprise banking environment with multiple business units.
-
Familiarity with DevSecOps toolchains (Jenkins, GitHub Actions, SonarQube, Veracode, Checkmarx, Snyk).
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦