Big Data Engineer, Profit Intelligence

Profit Intelligence team in Amazon Retail is seeking a seasoned and talented Senior Data Engineer to join the Historical Contribution Profit (HCP) team. HCP is a fast growing team with a mandate to build tools to automate profit-and-loss forecasting and planning for the Physical Consumer business. We are building the next generation data and Business Intelligence solutions using big data technologies and native AWS (NAWS). As a Data Engineer in Amazon, you will be working in a large, extremely complex and dynamic data environment. You should be passionate about working with big data and are able to learn new technologies rapidly and evaluate them critically. You should have excellent communication skills and be able to work with business owners to translate business requirements into system solutions. You are a self-starter, comfortable with ambiguity, and working in a fast-paced and ever-changing environment. Ideally, you are also experienced with at least one of the programming languages such as Java, Spark/Scala, Python, etc.

Major Responsibilities:

• Work with a team of product and program managers, engineering leaders, and business leaders to build data architectures and platforms to support business
• Design, develop, and operate high-scalable, high-performance, low-cost, and accurate data pipelines in distributed data processing platforms
• Recognize and adopt best practices in data processing, reporting, and analysis: data integrity, test design, analysis, validation, and documentation
• Keep up to date with big data technologies, evaluate and make decisions around the use of new or existing software products to design the data architecture
• Design, build and own all the components of a high-volume data warehouse end to end.
• Provide end-to-end data engineering support for project lifecycle execution (design, execution and risk assessment)
• Continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers
• Interface with other technology teams to extract, transform, and load (ETL) data from a wide variety of data sources
• Own the functional and nonfunctional scaling of software systems in your ownership area.
• Implement big data solutions for distributed computing., As a Data Engineer on our team, you will be responsible for leading the data modelling, database design, and launch of some of the core data pipelines. You will have significant influence on our overall strategy by helping define the data model, drive the database design, and spearhead the best practices to delivery high quality products.

About the team

Profit intelligence systems measures, predicts true profit(/loss) for each item as a result of a specific shipment to an Amazon customer. Profit Intelligence is all about providing intelligent ways for Amazon to understand profitability across retail business. What are the hidden factors driving the growth or profitability across millions of shipments each day?

We compute the profitability of each and every shipment that gets shipped out of Amazon. Guess what, we predict the profitability of future possible shipments too. We are a team of agile, can-do engineers, who believe that not only are moon shots possible but that they can be done before lunch. All it takes is finding new ideas that challenge our preconceived notions of how things should be done. Process and procedure matter less than ideas and the practical work of getting stuff done. This is a place for exploring the new and taking risks.

We push the envelope in using cloud services in AWS as well as the latest in distributed systems, forecasting algorithms, and data mining.

• 1+ years of data engineering experience
• Experience with SQL
• Experience with data modeling, warehousing and building ETL pipelines
• Experience with one or more query language (e.g., SQL, PL/SQL, DDL, MDX, HiveQL, SparkSQL, Scala)
• Experience with one or more scripting language (e.g., Python, KornShell)

Preferred Qualifications

• Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
• Experience with any ETL tool like, Informatica, ODI, SSIS, BODI, Datastage, etc.