Summary

Data Engineering is a gradual process that requires consistent learning, community involvement, and a mindset geared towards long-term growth and mastery of fundamentals.

Abstract

The article emphasizes that success in Data Engineering is not an overnight achievement but a long-term endeavor akin to running a marathon. It advises junior Data Engineers to approach their career with patience and persistence, focusing on daily practice and continuous learning to build a strong foundation. The author suggests adhering to the 80/20 rule by prioritizing the basics, networking with supportive communities, maintaining a positive and self-compassionate mindset, and sharing knowledge through writing. The journey to becoming a proficient Data Engineer is portrayed as one that demands time, dedication, and the understanding that progress is made through incremental improvements.

Opinions

The author believes that the journey from rookie to veteran in Data Engineering is a process that requires training every day and a commitment to lifelong learning.
There is a strong emphasis on mastering the fundamentals, as they are the cornerstone for more advanced knowledge and skills.
The article suggests that new Data Engineers often burn out quickly by trying to learn everything at once, rather than pacing themselves and focusing on the most impactful areas.
The author advocates for joining a community of like-minded individuals for support, encouragement, and shared learning experiences.
A change in mindset is recommended, where self-compassion and recognition of the learning curve are key to overcoming frustration and self-doubt.
Writing is highlighted as a tool for solidifying one's understanding and contributing to the collective knowledge of the Data Engineering community.

Rookie to Veteran

5 Tips to Make Data Engineering a Marathon, Not a Sprint

Tips to Help Junior Data Engineers Succeed

Data Engineering is a marathon — read that part twice so it sinks in.

Marathon runners don’t wake up one-morning lace up their shoes and bang out a full-blown marathon. No, No.

They build up their mileage and consistency by training every day. Getting good at anything, not just Data Engineering isn’t so different. You need training and you need experience. The world of Data Engineering is full of surprises and challenges, requiring constant learning and adaptation to stay up-to-date with the latest technologies and best practices.

If you can resist the urge to run out there and try to learn all the things you need to know (or think you need to know), you’ll stick around long enough to get great. The Greats have the fundamentals down. They keep learning and pushing themselves daily. They work hard, collaborate and share their knowledge. That can be you too.

Too often I’ve seen the newbie Data Engineer come into the fold filled with drive and determination only to fizzle out by the sheer amount of things there is to learn.

It plays out something like this…

They see Data Engineering (or insert a similar title) as a new and exciting career path. It has lots to learn and seemingly pays well how hard could it be? I can do that.

They start learning. Sign up for all the courses. Buy all the books. They are motivated to learn. They jump from thing to thing dizzying themselves.

And, within 4 weeks, they completely burn out. Overwhelmed.

Their frustration spills over, the success doesn’t come quickly enough, they expect it to be easy and their motivation fades. It’s over in the blink of an eye like summer rain — Over before it even began.

Don’t get me wrong. I get it, it’s what I did when I was getting started.

If you want to be a successful Data Engineer you need to pace yourself. Train daily, get involved with the community, and share your struggles. I went through all the emotions on my journey: excitement, frustration, and self-doubt. I still feel this way from time to time.

Here is my advice.

Train every day — Work on your skills and knowledge by practising and learning something new every day. Putting in a little daily effort over time adds up. Read up on topics that are unfamiliar to you, sign up for courses, or expose yourself to new concepts by getting hands-on and building and testing solutions. Remember, small continuous improvements make big things happen.
The 80/20 rule — Focus on the 20% that will give you 80% of the results. Focus on getting the basics right if you can do that the rest falls into place. Too often I see junior Data Engineers fall short with the fundamentals focus on that and you’re on solid ground.
Find a community — Surround yourself with great people, network with kind, empathic people the ones who will celebrate your wins and spur you on.
Change your mindset — Be kind to yourself all the greats out there started at zero just like you. If they can get to where they are so can you. Determination, hard work, and perseverance are the secret ingredients.
Write — Writing helps you learn and to think clearly. Share what you learn, documents your journey, and help others succeed where you struggled. Someone out here is on the same path as you and needs your help.

Remember

It’s a marathon, not a sprint.

Training to get good at something takes time. Data Engineering is vast, too vast to conquer in a day. What you need is time, patience, dedication, and daily improvement.

You have it in you to be great, you just have to play the game long enough to get there.

Thanks for reading! If you liked it, don’t forget to subscribe to my latest articles. Want to get in touch? feel free to find me on LinkedIn.