Software Engineer
Role details
Job location
Tech stack
Job description
Reports to the Director of Democracy's Library and will work as a member of the Democracy's Library U.S. team. Responsibilities include: designing, building, and maintaining technical infrastructure; ensuring pathways to government documents and public records can be reliably acquired, processed, and made accessible at scale to the public; and ensuring integrity, quality, and completeness of collection data throughout the ingestion and storage lifecycle., * Design, develop, and maintain automated data ingestion pipelines that integrate seamlessly with Internet Archive's platform, and advance the acquisition of government documents and public records from web sources, application programming interfaces (APIs), and partner repositories.
- Integrate backend infrastructure with Internet Archive's systems - including Petabox storage and the Archive.org platform - to ensure collections are accurately deposited, cataloged, and then made accessible.
- Develop and maintain Python-based scripts, tools, and services that support the library's collection workflow from acquisition to public access.
- Collaborate closely with the Collections Manager and other team members to understand content acquisition priorities and translate them into technical workflows.
- Support bridge-building efforts between collections and engineering to ensure government documents and public records are reliably acquired, processed, and made accessible.
- Build and maintain APIs and backend services that support internal collection management and enable easy to discover and access library collections.
- Monitor and maintain pipeline integrity and completeness of collection data throughout the ingestion and storage lifecycle.
- Perform diagnostics, and implement improvements to ensure reliability and completeness of collection data.
- Participate in code reviews and contribute to engineering best practices within the team.
- Draft and maintain easy to understand technical documentation and reference guides for systems, pipelines, APIs, and data models that team members and key stakeholders can refer to and follow.
- Other duties may be assigned as needed.
Requirements
- A commitment to the mission: to provide, "Universal Access to All Knowledge."
- Ability to stay focused, organized, and meet deadlines in a fast-paced technical environment.
Communication Skills
- Able to verbally communicate technical concepts in an easy to understand manner to both technical and non-technical colleagues.
- Strong technical writing skills, able to produce clear documentation and reference guides for systems, APIs, and workflows.
Computer Knowledge
- Proficient in Python; comfortable building and maintaining data pipelines and backend services.
- Proficient in Google Workspace, Slack, Jira and Confluence.
Able to Reason & Think Critically
- Capable of solving complex technical problems with sound judgment and practical approaches - solution-oriented.
- Able to assess trade-offs in system design and make informed decisions about pipeline architecture and data quality., * Required: Bachelor's Degree in Computer Science, Information Science, Software Engineering, or a related technical field, or equivalent practical experience.
- Required: Two (2) or more years of professional-level experience in a software development, data engineering, or systems development role.
- Required: Experience writing Python for data pipelines, automation, or backend systems.
- Required: Experience working with application programming interfaces (APIs); consumption, build-out, and/or maintaining them in a production environment.
- Required: Experience working with Google Workspace, Slack, Jira, and Zoom.
- Preferred: Familiarity with Internet Archive systems, including Petabox, Archive.org platform, or the Internet Archive Python library.
- Preferred: Experience with web scraping, bulk data acquisition, or automated harvesting of government or open-data sources.
- Preferred: Experience working in a digital library, archive, open-access, or civic technology context.
Benefits & conditions
Health insurance, Paid time off, Vision insurance, Dental insurance, Flexible spending account, Disability insurance, Commuter assistance, Paid holidays, IMPORTANT: We like cover letters. It helps us see how you communicate and express yourself in writing. Candidates who provide a cover letter will be prioritized. Benefits & Perks Internet Archive works with Open Staffing Foundation to recruit, hire and retain staff. Open Staffing Foundation offers a comprehensive benefits package, including PTO, paid holidays, medical, dental, and vision benefits, health savings and flexible spending accounts, commuter benefits, short-term and long-term disability coverage, and retirement programs.
