Engineer Software Design
Tata Consultancy Services Limited
Herndon, VA, United States
8 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.careerbuilder.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Compensation
$100,000.0 - $120,000.0
Working hours
Regular working hours
Job source
Tech stack
Artificial Intelligence
Airflow
Cloud Computing
Databases
Extract Transform Load (ETL)
Apache Hive
Python (Programming Language)
Performance Tuning
Systems Development Life Cycle
Software Tools
Software Engineering
Data Streaming
+10 more
Data Processing
Scripting
Google Cloud
Data Ingestion
Apache Spark
Pyspark
Integration Frameworks
Apache Kafka
Data Management
Data Pipelines
Job description
- Google Cloud Platform (GCP)
- PySpark
- Python
- Airflow
- Hive
- Kafka
Data Pipeline Development, Enhancement & Maintenance
- Design, build, and maintain scalable ETL/ELT pipelines.
- Automate data ingestion from multiple data sources.
- Ensure reliable, high-performance, and scalable data movement.
- Develop and optimize PySpark-based data processing frameworks.
- Build and manage Airflow workflows and orchestration pipelines.
- Work with Hive and Kafka-based batch and streaming data solutions.
- Implement data quality, monitoring, governance, and operational controls.
- Support production incidents, performance tuning, and continuous improvement initiatives
Roles & Responsibilities
- The Data Engineer will be responsible for designing, building, governing, and optimizing data platforms, pipelines, and integrations to ensure secure, reliable, and scalable data availability for analytics, reporting, AI/ML, and business operations.
Requirements
| Engineering Software, Software Design, GCP (Good Clinical Practices), Cloud Computing, Python Programming/Scripting Language, Apiary/Beekeeping, Data Management, Database Extract Transform and Load (ETL), Scalable System Development, Data Processing, Required Skills: Kafka | Python | ETL Testing | Hive |
Experience: 8 - 12 Years, * Strong Functional Knowledge in Retail (Search & Personalization) Added advantage.
- Strong experience with large-scale GCP Data Engineering solutions.
- Hands-on experience with Airflow DAG development and support
- Experience building and optimizing Spark/PySpark data pipelines.
- Exposure to Hive-based data processing and Kafka-driven ingestion frameworks.
- Experience supporting analytics, personalization, recommendation, or search platforms is highly desirable
Benefits & conditions
- Discretionary Annual Incentive.
- Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
- Family Support: Maternal & Parental Leaves.
- Insurance Options: Aut& Home Insurance, Identity Theft Protection.
- Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
- Time Off: Vacation, Time Off, Sick Leave & Holidays.
- Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
LI-KR3
Salary Range-$100,000-$120,000 a year
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.careerbuilder.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦