Crunchbase - Data Engineer

Data Engineer

Crunchbase San Francisco, CA, USA


Our Mission

It’s nearly impossible to find an authoritative source of truth about companies that is both always up-to-date and freely available. Our mission is to continue building the largest high-quality, live, openly editable and accessible dataset of company and people information in the world -- and to build a delightful product with slick, easy-to-use APIs around that dataset.

We love building enjoyable user experiences, complex data schemata, APIs, and scalable systems that support tens of millions of interactions per day. Our team is a group of highly skilled engineers working on formidable problems throughout the stack.

How We Operate

We have a flexible model for team organization at Crunchbase, centered around the principles of agile software development, allowing us to fluidly adapt to new requirements and situations as they arise. We tend towards two main types of teams:

• Feature-oriented teams:operate independently and own their projects from cradle to grave, building everything they need to execute. While they don't necessarily have to do all the work themselves, it's their responsibility to make sure the business needs are met all the way into production.

• Function-oriented teams: provide support to feature teams and have deeper expertise on particular parts of the stack. It's their responsibility to build platforms that are so good the other teams want to use them, rather than being forced into it.

We also ship code as early as we possibly can; getting betas into customer hands is a top priority. Our goal is to tune our process to customer needs as efficiently and quickly as possible. Crunchbase is built with a variety of tools - a core part of our engineering philosophy is to use the right tool for the job:

• Ruby, Scala, Java

• Node.js, Angular.js, Typescript

• Redis, PostgreSQL, ElasticSearch

• Git, Nginx, Docker, NewRelic, Jenkins

• AWS - EC2, ELB, VPC, S3, R53, RDS, SQS, etc.

Data Engineering at Crunchbase

Our data team is responsible for building and maintaining the infrastructure for our data needs. As Crunchbase grows, so too does the amount of data we process, the number of sources it comes from, and the number of ways that people want to slice and dice it. We are currently building out a robust pipeline for our core data and we’re continually expanding use cases for it, so it must be both scalable and flexible.

At its core, Crunchbase is a data company, and data engineering is at the heart of our platform and will propel us into the future. The responsibilities of data engineers at Crunchbase include:

• Architect and build new dimensional data models and schema designs to improve accessibility, efficiency, consistency and quality of both internal and production data.

• Build, monitor, and maintain analytics and production data ETL pipelines.

• Provide the foundation for a data-driven culture by empowering other engineers and the Product team to ask questions of the dataset in an easy, reliable way.

• Enable data scientists to implement NLP and ML algorithms at scale, in fault-tolerant, highly available systems.


• Solid understanding of computer science and software engineering fundamentals.

• Motivated to participate in ongoing learning and growth through pair programming, code reviews, application of new technologies and best practices.

• Excellent verbal and written communication skills.

• Familiarity with tools we presently use is a plus, but not required -- if you know something better, we may use that instead!

What Crunchbase offers:

•  Competitive salary and equity

•  A team of creative, transparent entrepreneurs driven to accomplish our mission

•  Daily catered lunches

•  Fitness reimbursement (to work off the catered lunches)

•  Unlimited Paid Time Off (PTO) -- we don't track vacation days

•  Incredible medical, vision, and dental benefits for employees and their families

•  401(k) and Roth plans, and free annual financial adviser check-in

•  Free One Medical Group membership for employees and their families

•  Commuting stipend

•  Free Lyft and Uber rides anywhere in the Bay Area after late nights at the office

•  Prime location in the Financial District of SF, near Bart Muni stops

•  Company and team offsites, retreats, events, and happy hours

Every day our team is honored to work with entrepreneurs and innovators from every corner of the globe, and we aim to build a team that reflects the diversity of our customers. Each individual at Crunchbase brings their own perspectives, work experiences, lifestyles, and cultures with them, and we believe that a more diverse team creates more innovative products, provides a better service to its customers, and helps us all grow and learn as individuals. Crunchbase does not discriminate on the basis of race, creed, color, ethnicity, national origin, religion, sex, sexual orientation, gender expression, age, height, weight, veteran status, military obligations, or marital status.


Social links: