What do I look for when hiring a Data Engineer?
Over a period of time, I had an opportunity to talk to many people who wanted to apply for Data Engineering positions. The count is as big as few hundreds plus. Every single candidate I met/spoke with had something different in it.
First filtering criteria?
Some of them were extremely enthusiasts about the work they have done while some were looking for an opportunity to do something different. Some of them only wanted to work on latest technologies while some of them wanted to continue working on technologies they were comfortable with. For some, this was a stepping stone into a Data Science world while for some this was an escape from traditional software engineering work!
Unfortunately a very few wanted to create a value for the organisation! And I my first filter criteria was exactly the same.
What about Technology?
Yes, technology was my second criteria! I looked for basics! For instance, Hadoop is the most used technology in Data Engineering space, the first question was always to explains — “How does the HDFS copyFromLocal/put command works internally?”
Interestingly a lot candidates failed to answer this basic question but surprisingly they could explain some tough questions very easily which was a big put off for me.
Too many technologies in the CV?
Sometimes I was surprised to see too many technologies in the CV. Some clearly mentioned that they learned those but have not used in practice but some clearly said they have used those in production. I guess all were right but then I asked to explain the details and then the picture was quite clear!
Strict NO to Jumping Jacks!
Sometimes I came across the CVs which had too many jumps in a very short time! There could be reasons for that but not always! Hence I was definitely looking for someone with a stable career path!
If your previous companies could not retain you for longer then I am sure I will fail in it as well !
Its all about the right team combination!
Interestingly I have seen people hiring for a candidate without looking for the right team composition. If you have an all-star team then you are going to have a tough time managing them.
My management training taught me, there 4 types of people, I should look for in a team
- Drivers — Self-confident, Aggressive, Blunt, Sometimes impatient
- Amiables — Relaxed, Team players, prefer to follow and not lead, good listeners
- Exprerssors — Extroverted, Enthusiastic, Optimistic, Friendly
- Analyticals — Organised, collects data, avoid mistakes, get into details
I must say, you need people who can lead and as well follow. You need people who like change and who does not like change!
Those who like the change will innovate and those who don’t will sustain your business and projects! So both and important!
I never expected people to answer 100% questions
Nobody can and will ever do! Not even me! There was never the expectation that the person will know everything! And I did try to put people in a not-so-comfortable situation and tried to see how they would get out of it.
The intention was not to disrespect anyone but just to see their ability to handle the situation.
Any questions?
During the interview, I tried to give the opportunity to candidates to ask the questions. And I always tried to look beyond “How is the company culture?”, “What projects I will be working on?”! The reason was quite clear, I find some of these questions quite vague as the first one is how you see it and the later one is situational.
The way the market trend is, technologies will change and so does the business focus as well.
Unfortunately, most of the candidates were only preferred to say — “No questions, I am good”
Results?
At last, I know I may not be 100% right in the selection but I know what I was looking for and I am glad, I could find what I was looking for! And I am glad I am evolving and learning in the process!
Hey, if you enjoyed this story, check out Medium Membership! Just $5/month!
Your membership fee directly supports me and other writers you read. You’ll also get full access to every story on Medium.