Python dataflow coordination enhancements you don’t want to miss! — In the five months since Prefect v2 was released to general availability, the Prefect team has published over 30 releases! While the release velocity is impressive, it can be hard to keep up with all this awesomeness, so I’m going to distill my ten favorite enhancements in this article. Let’s…

Why and how to attempt certification — Considering taking the AWS Machine Learning Specialty certification? I took the test in October 2022 and wasn’t able to find much updated information about it. This post is designed to help you decide whether to take the exam and how to invest your time if you decide to go for…

Store your flow code in your Docker Image — In the first post in this series we saw how to use Prefect with Kubernetes without explicitly making a Prefect infrastructure block. In the second post we saw three ways to make a custom Kubernetes Job block. In all of our examples to this point we’ve stored our code in…

Creating custom Kubernetes Job blocks — In the first post in this series, we used Prefect with Kubernetes to run our Python flow code without creating a custom Kubernetes Job block. In this post, we’ll show three more ways to use Prefect with K8s. Create a Kubernetes Job block Recall that our deployment in the previous article automatically created an anonymous…

Part 1: Up and running — Kubernetes is a workhorse! 🐴 It’s a great way to declaratively scale containerized infrastructure up and down for long running processes. Kubernetes can also be a bear. 🐻 It has a lot of moving parts that can be cumbersome to run in harmony.

A straightforward guide to getting involved — Getting started with open source software can feel intimidating, but it doesn’t need to. In this guide you’ll learn how to contribute to open source, no matter your experience level. First let’s look at why you might want to get involved. Why contribute to open source? There are many reasons to contribute to projects that…

Scheduling, with or without cron, is only a small part of the data coordination puzzle — I was showing off Prefect, the open source dataflow coordination platform, the other day and someone watching asked, “Why should I use Prefect instead of Cron?” That’s such a great question that I decided to write this blog post to answer it! 🚀 Cron is a great option to take…

Introducing Blocks: open source, modular, reusable, and sharable configuration plus code — Everyone who codes can save time by reusing configuration — whether for logging in to cloud providers or databases, configuring Docker containers, or sending notifications. Install the Prefect library and add a few lines of code and you’ve got all these super powers and more! 🦸 Have you ever wanted…

How to quickly make a Prefect Python deployment file — Prefect is an open-source dataflow coordination solution. It’s like air traffic control for your dataflows, providing you with observation and orchestration. In this post, you’ll see how to orchestrate your Prefect decorated functions using a new feature — a deployment file written in Python. 🎉 Prefect makes it easy to…