The best way to use App Engine pull duties (Module 18)

Posted by Wesley Chun (@wescpy), Developer Advocate, Google Cloud

The Serverless Migration Station mini-series helps App Engine builders modernize their apps to the most recent language runtimes, comparable to from Python 2 to three or Java 8 to 17, or to sister serverless platforms Cloud Features and Cloud Run. One other aim of this sequence is to exhibit the right way to transfer away from App Engine’s authentic APIs (now known as legacy bundled providers) to Cloud standalone alternative providers. As soon as not depending on these proprietary providers, apps develop into way more moveable, making them versatile sufficient to:

App Engine’s Activity Queue service offers infrastructure for executing duties outdoors of the usual request-response workflow. Duties might encompass workloads exceeding request timeouts or periodic tangential work. The Activity Queue service offers two completely different queue varieties, push and pull, for builders to carry out auxiliary work.

Push queues are coated in Migration Modules 7-9, demonstrating the right way to add use of push duties to an present baseline app adopted by steps emigrate that performance to Cloud Duties, the standalone successor to the Activity Queues push service. We flip to pull queues in in the present day’s video the place Module 18 demonstrates the right way to add use of pull duties to the identical baseline pattern app. Module 19 follows, displaying the right way to migrate that utilization to Cloud Pub/Sub.

Including use of pull queues

Along with registering web page visits, the pattern app must be modified to trace guests. Visits are comprised of a timestamp and customer data such because the IP deal with and person agent. We’ll modify the app to make use of the IP deal with and monitor what number of visits come from every deal with seen. The house web page is modified to point out the highest guests along with the latest visits:

The pattern app’s up to date residence web page monitoring visits and guests

When visits are registered, pull duties are created to trace the guests. The pull duties sit patiently within the queue till they’re processed in mixture periodically. Till that occurs, the highest guests desk stays static. These duties might be processed in quite a few methods: periodically by a cron or Cloud Scheduler job, a separate App Engine backend service, explicitly by a person (by way of browser or command-line HTTP request), event-triggered Cloud Operate, and so forth. Within the tutorial, we subject a curl request to the app’s endpoint to course of the enqueued duties. When all duties have accomplished, the desk then displays any adjustments to the present prime guests and their go to counts:

Processed pull duties replace the highest guests desk

Under is a few pseudocode representing the core a part of the app that was altered so as to add Activity Queue pull process utilization, specifically a brand new information mannequin class, VisitorCount, to trace customer counts, enqueuing a (pull) process to replace customer counts when registering particular person visits in store_visit(), and most significantly, a brand new perform fetch_counts(), accessible by way of /log, to course of enqueued duties and replace general customer counts. The bolded traces signify the brand new or altered code.

Including App Engine Activity Queue pull process utilization to pattern app

Wrap-up

This “migration” is comprised of including Activity Queue pull process utilization to assist monitoring customer counts to the Module 1 baseline app and arrives on the end line with the Module 18 app. To get hands-on expertise doing it your self, do the codelab by hand and observe together with the video. Then you definitely’ll be able to improve to Cloud Pub/Sub must you select to take action.

In Fall 2021, the App Engine group prolonged assist of most of the bundled providers to 2nd era runtimes (which have a 1st era runtime), which means you might be not required emigrate pull duties to Pub/Sub when porting your app to Python 3. You’ll be able to proceed utilizing Activity Queue in your Python 3 app as long as you retrofit the code to entry bundled providers from next-generation runtimes.

For those who do wish to transfer to Pub/Sub, see Module 19, together with its codelab. All Serverless Migration Station content material (codelabs, movies, and supply code) can be found at its open supply repo. Whereas we’re initially specializing in Python customers, the Cloud group is masking different runtimes quickly, so keep tuned. Additionally take a look at different movies within the broader Serverless Expeditions sequence.