Data Engineer
MarineTraffic
yesterday
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
JuniorJob location
Tech stack
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Big Data
Databases
Data Architecture
Data Governance
Data Integration
ETL
Data Transformation
Data Warehousing
Relational Databases
Database Design
Identity and Access Management
Key Management
Metadata Repositories
Cloud Services
DataOps
Spatial Data Infrastructures
Cloud Platform System
Spark
AWS Lambda
Data Lake
Amazon Web Services (AWS)
Data Management
Lidar
Redshift
Job description
The role of the Data Architect will be to manage the development of data base projects. Plan and budget staff and data base resources. When necessary, reallocate resources to maximize benefits. Prepare and deliver presentations on DBMS concepts. Provide daily supervision and direction to support staff. Extensive knowledge of relational database environment., * Based on priorities identified by the FEMA RMD Data Governance Board, work with data stewards to identify and document authoritative data sources
- Create data governance standards, processes and documentation based on direction from the RMD Data Governance Board or Data Stewards.
- Integrate data governance into CDS operations and application development process.
Data Quality:
- Reduce Redundancy by automating manual business processes and streamline the workflow to prevent replication of content.
- Work with the FEMA RMD Data Governance Board and stakeholders to recommend high-priority quality objectives and measures.
- Monitor data quality and escalate issues as appropriate.
Data Architecture:
- Develop strategy to improve support for large datasets, such as LiDAR and Automated Engineering, particularly uploading and downloading these datasets.
- Review and develop object and data models and the metadata repository to structure the data for better management and quicker access.
- Translate business needs into long-term architecture solutions.
- Lead team in executing data modernization to cloud-native data lake according to agency modernization plan
- Develop and maintain data catalog and monitor standards and procedures and integration of systems through database design
Requirements
- 8+ years' experience in designing and optimizing data lakes and data warehouses using Amazon S3, AWS Glue, and Amazon Redshift
- 8+ years' experience architecting and deploying modernized data architectures into cloud environments
- 8+ years' experience developing and executing data governance and data management practices within large organizations including experience using AWS Lake Formation, AWS IAM, and AWS KMS.
- 5+ years' experience with data integration platforms and ETL tools. Proficient using AWS Glue, Apache Spark on EMR, AWS Step Function and AWS Lambda
- 5+ years' experience performing data management and data operations in AWS
- 3+ years' experience working with GIS spatial data
- 1+ years' experience delivering on a large Agile program
- Strong problem solving and analytical skills
- Ability to work effectively in a fast-paced and dynamic environment