ETL Developer
Role details
Job location
Tech stack
Job description
We are seeking an experienced ETL Developer to provide onsite support for a geospatial visualization platform. This role focuses on developing, maintaining, and optimizing data pipelines to ensure reliable data flow into a PostgreSQL database. The ideal candidate will have strong experience in ETL processes, data integration, and working with large, complex datasets, including geospatial data.
Responsibilities:
-
Design, develop and manage ETL workflows using ApacheNiFito support a geospatial visualization platform
-
Learn the nuances of theexistingNiFisystem and the related AWS and PostgreSQLinfrastructurein order tomonitorand troubleshoot real time issues
-
Obtain an understandingof existingNiFiflows with minimal support from others
-
Develop newNiFiflows that adopt best practicesof existingNiFiflows
-
Take feedback onNiFiflows and develop requested changes
-
Take direction and communicate status via a ticketing system
-
Interface with analysts via email, chat, and in person meetings tocomprehendfuture data flow needs and refine general requirements intospecificactions
-
Interface with a development cycle using a local development environmentofcontainerized infrastructureandversion control viaGitandNiFiRegistry
-
Gain competency ingeospatialdatabase design and employ best practicesfor table creation, indexing, and unique constraints
-
Take feedback from analysts on nuances to support application data layers
-
Programmatically manipulate and transform data from multiple sources
-
Query and analyze large datasets using SQL
-
Retrieve and process data from external systems via APIs
-
Support data quality, integrity, and performance optimization efforts
-
Understand Python and data science frameworks whiledifferentiatingbetweenexperimental prototypesand enterpriselevelsoftwarerequirements
-
Develop Python in support of data science related efforts or ETL prototypes
Requirements
-
Experience:Bachelor's degree with 8+ years of relevant experience, ORMaster's degree with 7+ years of experience, ORDoctorate with 6+ years of experience
-
Degree in Computer Science, Systems Engineering, Geography, or a related technical field
-
Active TS/SCI clearance with required polygraph
-
U.S. Citizenship required
-
Willingness to work onsite full-time
-
Willingness todevelopNiFiflows using existing best practices
-
Strong understanding of ETL concepts and best practices
-
Strong experience withdatabasesfor data management
-
Proficiency in SQL for querying large datasets
-
Experience with API integration and data retrieval
-
Demonstrated experience with programmatic data manipulation
-
Competency to rapidly adoptcontainerizeddevelopmentproficiency
-
Competency to rapidly adopt AWS architectureproficiency
-
Competency to rapidly adoptNiFiregistry and Git for code management
Preferred Qualifications:
-
Creativityto experimentand recommendbest practicecodingimprovements
-
Drive to adopt and recommendAIintodevelopment processes
-
Ability to design and develop custom NiFi processors using Java
-
Experience with AWS cloud services for data ingestion and processing
-
Strong experience with PostgreSQL for data management
-
Hands-on experience with Apache Airflow
-
Hands-on experience with ApacheNiFi
-
Hands-on experience withNiFiregistry and Git
-
Hands-on experience with Git and repository actions and pull requests
-
Hands-on experienceinPython for data manipulationand APIinterfacing
-
Experience working with geospatial data and related tool
-
Understanding of basic data science concepts and tools
Pay Transparency: In support of pay transparency at Vantor, we disclose salary ranges on all U.S. job postings. The successful candidate's starting pay will fall within the salary range provided below and is determined based on job-related factors, including, but not limited to, the experience, qualifications, knowledge, skills, geographic work location, and market conditions. Candidates with the minimum necessary experience, qualifications, knowledge, and skillsets for the position should not expect to receive the upper end of the pay range.
Benefits & conditions
- The base pay for this position within the Washington, DC metropolitan area is: $113,000.00 - $188,000.00 annually.
For all other states, we use geographic cost of labor as an input to develop market-driven ranges for our roles, and as such, each location where we hire may have a different range.
Benefits: Vantor offers a competitive total rewards package that goes beyond the standard, including a robust 401(k) with company match, mental health resources, and unique perks like student loan repayment assistance, adoption reimbursement and pet insurance to support all aspects of your life. You can find more information on our benefits at:https://www.Vantor.com/careers
The application window is three days from the date the job is posted and will remain posted until a qualified candidate has been identified for hire. If the job is reposted regardless of reason, it will remain posted three days from the date the job is reposted and will remain reposted until a qualified candidate has been identified for hire.