Job description
Develop, program, and maintain applications using the Apache Spark open-source framework
Work with different aspects of the Spark ecosystem, including Spark SQL, DataFrames, Datasets, and streaming Spark Developer must have strong programming skills in Java, Python and Unix Shell scripting
Familiar with big data processing tools and techniquesCreate Spark jobs for data transformation and aggregation
Produce unit tests for Spark transformations and helper methods

Design data processing pipelinesCreate and maintain optimal data pipeline architecture,
Assemble large, complex data sets that meet functional / non-functional business requirements.
Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.

Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.

Process Skills:

Strong analytic skills related to working with unstructured datasets.
Build processes supporting data transformation, data structures, metadata, dependency and workload management.
A successful history of manipulating, processing and extracting value from large disconnected datasets.
Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data store
Agile/Scrum methodology experience is required.

Behavioral Skills:

Experience working with offshore / onshore model

Strong communication skills and able to work effectively in a team environment.

Must have strong knowledge in Object oriented development, version control, release management, change management

Skills
PRIMARY COMPETENCY : Big Data Technologies PRIMARY SKILL : Apache Spark PRIMARY SKILL PERCENTAGE : 51 SECONDARY COMPETENCY : Cloud Apps SECONDARY SKILL : Amazon Web Services-Cloud platform SECONDARY SKILL PERCENTAGE : 49

APPLY
Role: Technical Lead
Industry Type: IT Services & Consulting
Department: Engineering - Software & QA
Employment Type: Full Time, Permanent
Role Category: Software Development
Education
UG: Any Graduate
PG: Any Postgraduate
Key Skills
Skills highlighted with ‘‘ are preferred keyskills
Big Data
JavaAmazon Web ServicesScrum methodologydata structuresAgile methodologySparkdata transformationUnix Shell scriptingSQLApache SparkPython