Job Details

 

Principal Data Engineer

  • 12 - 18 years
  • Bengaluru/Bangalore
Industry :
IT-Software / Software Services
Functional Area :
IT Software - Application Programme, Maintenance
submit your CV

Quote job ref: 214040

Job Description

Job Title

Principal Data Engineer

Job Description

We help people save money, so they can live better. This mission serves as the foundation for every decision we make and drives us to create the future of retail. We can’t do that without the best talent – talent that is innovative, curious, and driven to create exceptional experiences for our customers.

 

Do you have boundless energy and passion for engineering data used to solve dynamic problems that will shape the future of retail? With the sheer scale of environment comes the biggest of big data sets. As a Data Engineer, you will dig into our mammoth scale of data to help unleash the power of retail data science by imagining, developing, and maintaining data pipelines that our Data Scientists and Analysts can rely on. You will be responsible for contributing to an orchestration layer of complex data transformations, refining raw data from source into targeted, valuable data assets for consumption in a governed way.?You will partner with Data Scientists, Analysts, other engineers and business stakeholders to solve complex and exciting challenges so that we can build out capabilities that evolve the retail business model while making a positive impact on our customers’ lives.

 

About the Data Strategy and Insights Organization:

Our organization focuses on managing and delivering world-class data assets, including creating and maintaining data standards, driving policy compliance, creating partnerships, and developing pipelines and self-service tools. We empower our business to leverage data to fuel growth, driving revenue in our core and building new business model opportunities.

 

About Data Ventures:

Data Ventures exists to unlock the full value of data by developing and productizing B2B data initiatives that empower merchants and suppliers to make better, faster decisions for the business. As part of this transformation, we’re seeking entrepreneurial individuals to help drive data productization from concept to deployment.

 

 

You’ll make an impact by:

  • Data Modeling: Influences the overarching data strategy and vision for data pipelines and data products. Oversees and governs the expansion of existing data architecture and the optimization of data query performance via best practices. Presents and socializes data models to business and information technology stakeholders.
  • Data Architecture: Designs, implements and improves processes in data management.
  • Culture champion: Standardize and build a fact based metric driven engineering culture across the engineering organization. Help hire, upskill and mentor a strong team of data engineers.
  • Leads and participates in medium- to large-scale, complex, cross-functional projects by reviewing project requirements. Translating requirements into technical solutions. Gathering requested information, writing and developing the code. Communicating status and issues to team members and stakeholders, collaborating with project team and cross functional teams. Troubleshooting open issues and bug-fixes. Enhancing design to prevent reoccurrences of defects, ensuring on-time delivery and hand-offs. Daily interacting with project manager to provide input on project plan and providing leadership to the project team.
  • Leads the work of other small groups of six to ten engineers, including offshore associates. Leads assigned Engineering projects by providing pertinent documents, direction, and examples: identifying short- and long- term solutions and timeline. Reviewing and approving proposed solutions, implementing new architectural patterns and performing design and code reviews of changes.
  • Troubleshoots business and production issues by reviewing and analyzing information (for example, issue, impact, criticality, possible root cause), engaging support teams to assist in the resolution of issues. Formulating an action plan, directing actions as designated in the plan. Interpreting the results to determine further action, performs root cause analysis to prevent future occurrence of issues and completing online documentation.
  • Supports business objectives by collaborating with business partners to identify opportunities, addressing high-priority initiatives (for example, business strategy, technical feasibility, implementation alternatives), identifying short- and long-term solutions and leading cross-functional partnership.
  • Leads the discovery phase of medium to large projects to come up with high level design by partnering with the product management, Web Operations, project management, business and user experience teams. Obtaining cross-function approvals, driving proof-of-concept and implementing prototypes to validate ideas.
  • Develops Innovation strategies, processes and best practices by leading internal technical teams. Partnering with cross-functional teams across the business. Developing assessments of key opportunities, documenting project scopes, developing long-range plans and project timelines. Communicating with and influencing decision-makers and executives within the organization and resolving technology differences across teams through informed discussions.
  • Utilizes industry research to improve technology environment by analyzing industry best practices, benchmarking industry against internal processes and solutions. Researching or influencing future industry solutions for fit with internal needs and defining software development guidelines, standards and processes.
  • Drives the execution of multiple business plans and projects by identifying customer and operational needs. Developing and communicating business plans and priorities, removing barriers and obstacles that impact performance. Developing contingency plans and demonstrating adaptability and supporting continuous learning.
  • Ensures business needs are being met by evaluating the ongoing effectiveness of current plans, programs, and initiatives. Applying suggestions for improving efficiency and cost effectiveness; and participating in and supporting community outreach events.

 

 

You’ll sweep us off our feet if:

  • Well versed with Hadoop, Hive, Spark using Scala.

· You evangelize an extremely high standard of code quality, system reliability, and performance

· You have a proven track record coding with at least one programming language (e.g., Java, Python)

  • You’re experienced in computing platforms (e.g., GCP, Azure)
  • You’re skilled in data modeling & data migration protocols
  • Experience with ThoughtSpot, Druid, Big Query and ClickHouse is added advantage.
  • Experience with the integration tools like Automic, Airflow

· Ability to write requirements for ETL and BI developers

· Ability to write designs for data architecture of data warehouse or data lake solutions or end to end pipelines

· Expert in data architecture principles, distributed computing

· Intake prioritization, cost/benefit analysis, decision making of what to pursue across a wide base of users/stakeholders and across products, databases and services,

 

Mandatory Skills

Application Development, Platform Building, Java/J2EE, Hadoop, Spark, Hive, SQL

Desirable Skills

Scala, Airflow, GCP, CI CD pipeline

Years Of Experience

13 to 20 Years

Education/Qualification

BE

Key Skills
Application Development
Platform Building
Java/J2EE
Hadoop
Spark
Hive
SQL
Education
  • B.Tech/B.E., M.Tech, MCA
submit your CV