comScore, Inc

Lead Data Engineer (Core Java)

2 months ago
# of Openings
Software Engineering & Technology

Job Description

comScore is a data company. We collect and process 60+ billion events each day, have 10s of petabytes online and each month our processes read nearly an exabyte. We use this capability to provide our clients with deep insights that no other company can match. Data Engineering is the team responsible for managing this vast dataset. We are looking for a Staff Data Engineer to join our team.


What our team does:

We build data processing pipelines that handle 100+ terabyte datasets. We automate as much as we can so that we can stay focused on writing code. We troubleshoot and quickly resolve issues. We work with Analysts and Data Scientists to design and implement new methodology.


As a staff engineer, you're a senior individual contributor, who is well-versed in design, proof of concept development, methodology and architecture. Your previous big data experience will have been on production products delivered on systems such as Hadoop. 


What you bring:

  • 10+ years of professional programming experience
  • You have a solid understanding of Computer Science fundamentals
  • You write good code and take pride in that fact regardless of which language you are currently using
  • You have a strong affinity towards working with data
  • You enjoy working as a member of team and consider feedback a learning opportunity
  • You are comfortable in an environment that values quickly providing our customers with solutions
  • You have an innate drive to grow and develop

The following are considered a plus:

  • You treat performance as a feature not an afterthought
  • You have experience writing analytical queries that run on MPP databases
  • You are comfortable reading query execution plans
  • You can describe multiple MapReduce join strategies and their tradeoffs
  • You have experience scaling Machine Learning algorithms in a distributed environment

Technologies we use:

(Previous experience in these technologies is not required. This is a great opportunity for someone looking for a new challenge.)

  • Hadoop - MapR, Apache Pig, Spark, HBase,Java, SQL (CTEs, window functions, UDFs, etc.)
  • An internal framework for job scheduling and execution (similar to AirFlow)
  • Scala and Spark for data processing
  • Java and JVM, for running code on Spark


About comScore: comScore, Inc. (OTC: SCOR) is a leading cross-platform measurement company that precisely measures audiences, brands and consumer behavior everywhere. 


comScore completed its merger with Rentrak Corporation in January 2016, to create the new model for a dynamic, cross-platform world. Built on precision and innovation, our unmatched data footprint combines proprietary digital, TV and movie intelligence with vast demographic details to quantify consumers’ multiscreen behavior at massive scale. This approach helps media companies monetize their complete audiences and allows marketers to reach these audiences more effectively. With more than 3,200 clients and global footprint in more than 75 countries, comScore is delivering the future of measurement. For more information on comScore, please visit


EEO Statement: We are an equal employment opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, disability status, sexual orientation, gender identity, age, protected veteran status or any other characteristic protected by law.




Sorry the Share function is not working properly at this moment. Please refresh the page and try again later.
Share on your newsfeed

Connect With Us!

Don't see your ideal role at comScore listed here? That's OK, we still want to stay in touch, because new roles become available frequently. Click below to join our Talent Network, and we'll keep you updated on all the happenings at comScore and alert you when there is a role that might be exactly what you are looking for!

Connect with us to stay informed!