Principal Software Engineer - Big Data Technologies

Hiring for one of the leading based MNC into Ad-Tech Company : MNC (in to Services Product) Job Location : Bangalore Profile : Principal Software Engineer Scala Experience : 10 to 18 years Required Skills : - Experience in Scala - Experience in AWS - Experience in Spark - Strong experience (12 years) in Scala programming language and extensive experience with Apache Spark for Big Data processing for design, developing and maintaining scalable on-prem and cloud environments, especially on AWS EMR. - Proficiency in performance tuning of Spark jobs, optimizing resource usage, shuffling, partitioning, and caching for maximum efficiency in Big Data environments. - In-depth understanding of the Hadoop ecosystem, including HDFS, YARN, and MapReduce. - Expertise in designing and implementing scalable, fault-tolerant data pipelines with end-to-end monitoring and alerting. - Using Python to develop infrastructure modules. - Hence, hands-on experience with Python. - Solid grasp of database systems and SQLs for writing efficient SQL's (RDBMS/Warehouse) to handle TBS of data. - Familiarity with design patterns and best practices for efficient data modelling, partitioning strategies, and sharding for distributed systems and experience in building, scheduling and maintaining DAG workflows. - End-to-end ownership with definition, development, and documentation of software's objectives, business requirements, deliverables, and specifications in collaboration with stakeholders. - Experience in working on GIT (or equivalent source control) and solid understanding of Unit and integration test frameworks. - Must have the ability to collaborate with stakeholders/teams to understand requirements and develop a working solution and the ability to work within tight deadlines and effectively prioritize and execute tasks in a high-pressure environment. - Must be able to mentor junior staff. Advantageous to have experience on below : - Hands-on with Databricks for unified data analytics, including Databricks Notebooks, Delta Lake, and Catalogues. - Proficiency in using the ELK (Elasticsearch, Logstash, Kibana) stack for real-time search, log analysis, and visualization. - Strong background in analytics, including the ability to derive actionable insights from large datasets and support data-driven decision-making. - Experience with data visualization tools like Tableau, Power BI, or Grafana. - Familiarity with Docker for containerization and Kubernetes for orchestration. - Good to have AWS certification. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 5/5/2025