AWS re:Invent 2022 — My Seasonal Highlights
The world is asynchronous. The world is event-driven. — Dr. Werner Vogels, CTO, Amazon.com
It required a world stage to propagate asynchrony and event-driven architecture in our modern era of cloud computing. And that’s precisely what Dr. Werner Vogels delivered in his keynote at re:Invent 2022.
As a cloud-scale conference, re:Invent serves a different purpose to different people. It is a stage to bring and sell ideas, show technology directions, showcase innovations, incubate business visions, listen to the community, learn from the experts, create business opportunities, and of course, criticize when things fall short of expectations!
In this brief article, I focus on the things that I liked and will help with the future of serverless and its adoption.
Intellectual re:Invent
As Werner mentioned in his keynote, many thought-leaders of software architecture have been advocating for event-driven computing and building loosely coupled applications for years. Thanks to cloud and serverless, what many of us pushed aside as hard to achieve with previous generation technologies is now available as a commodity for everyone. Hence the urge to shift our minds to change our thinking!
As I was making room to accommodate new books, the following two-decade-old book on Java Message Service (JMS) caught my attention. JMS might have a new name, but it had the basic concepts for building event-driven applications.
Messages, queues for point-to-point communication, events, and topics for publishing and subscribing (pub-sub) were all there. Cloud and serverless rejuvenated these fundamentals with readymade building blocks, i.e., managed services. Instead of hand-coding with JMS APIs, we compose our applications using managed services!
Though it may sound like we have come a full circle, the scale of applications we build, and the speed of development are beyond comparison. Hence the theme of asynchrony and event-driven architecture struck a chord with me immediately.
Technical re:Invent
For cloud and serverless builders, there have been several new announcements. Here are the ones that I found interesting in the serverless space.
Amazon EventBridge Scheduler
EventBridge Scheduler is one of the most asked-for and awaited services. It eliminates the reliance on DynamoDB TTLs and Step Function wait states with a clean built-for-purpose solution!
As I understand, several improvements and features are in the pipeline. Currently, there is no option to automatically clean up the already completed one-time schedules. Another missing feature is to dynamically create a one-time schedule as an EventBridge target without writing a lambda function.
Amazon EventBridge Pipes
When I first heard from the product team that Pipes was in the pipeline, I was a bit confused! Because here is yet another choice available for event-driven computing on AWS in the already overloaded space. It’s always good to have more options than no options.
EventBridge Pipes has its purpose and use cases where it fits well. My word of caution is to take time, assess your need, and assert Pipes is the best-fit solution. Failing to do so may get you into a Ball of Serverless Mud (BoSM) situation discussed in the article on granularity in serverless.
AWS Step Functions Distributed Map
While increasing the parallel iterations limit from 40 to 10,000, the Distributed Map feature is mainly geared to work with S3 to process thousands of objects at one time.
This feature opens up new high-volume data processing use cases. One thing to be careful about is the concurrency limits of lambda functions in the account and the service limits of other services used in the execution flow.
AWS Lambda SnapStart
Similar to how the VPC lambda cold-start performance improvements happened a couple of years ago, the Lambda SnapStart is massive for Java based lambda functions.
It’s a great boost for Java applications, protecting the engineering investment in many enterprises and encouragement to move to serverless.
AWS Application Composer (Preview)
From a serverless engineer’s point of view, there are two main reasons why I like Application Composer.
- For a new serverless engineer — it helps in their learning process.
- For an experienced serverless engineer — it helps in their service design process.
I favor engineers learning architecture and designing solutions in collaboration with others (rather than being fed with the architecture and seeing them as programmers). Application Composer has a role in that quickly realizing the solution as the design goes through reviews and changes.
There has been an array of new features and ideas poured in at one of the AWS Heroes meetings with the product team, including the option to export the diagram with just the service icons without the boxes and many more!
Amazon SNS Payload-based Message Filtering
In the past, with attribute-based filtering, teams created different topics to differentiate between event types and subscriptions. With payload-based filtering, it becomes easier to identify and route different types of events via a single topic to respective subscribers.
It is important to recognize the use cases specific to SNS and not to get confused with the similar event filtering capabilities offered by Amazon EventBridge.
Other announcements
Amazon Inspector for Lambda — Amazon Inspector can now scan the Lambda functions to identify software vulnerabilities in application package dependencies used in the function code. It is critical to building secure services, but the cost is my only concern here.
Amazon VPC Lattice (Preview) — VPC Lattice the creation of a logical application layer network to connect clients and services across different VPCs and accounts, abstracting network complexity.
Community re:Invent
Among the dozens of conversations I had, product ideas listened to, and bold initiatives I had a chance to look at, the two newer product teams I closely interacted with were-
momento
The main attraction of momento is its simplicity and efficiency. As a consumer, it only takes just a few lines of code to get going!
The team led by Khawaja Shams is very knowledgeable with plenty of experience. The meeting I had with the momento team, along with a bunch of AWS Serverless Heroes, where we went through their architecture, was a testament to their confidence in their product. Rarely do you find a start-up sharing its trade secrets with a bunch of folks!
sedai
sedai has bold ambitions in the DevOps space to take cloud automation to a deep level. While we cannot fully eliminate DevOps (just yet), sedai makes its strides in reducing the load by taking care of a lambda function’s resource needs to have a seamless operation.
While sedai may not be everyone’s cloud automation ally, its growing customer base is demonstrating its early success. With a well-established team of engineers, sedai is constantly pushing features for both serverless and non-serverless cloud operations.
The two areas I will keep a keen interest in the coming days are the cloud automation space and Infrastructure from Code (IfC).
Thoughtful re:Invent
AWS re:Invent is not just a conference with a distributed-map of parallel sessions! As I recently tweeted, it’s about the celebration of cloud and technology where everyone becomes an actor to play our part.
It’s massive, crowded, noisy, busy, and expensive, but everything works like clockwork. The unsung heroes are the thousands of support staff who make our participation memorable.
So, let’s re:Member and spare a thought to-
- the woman who patiently guided us when we got stranded in the middle of a casino,
- the person who cleared the table at the dining hall for the next batch of diners,
- the ones who held the placard high and helped us board the right shuttle to get to the right hotel for the next session,
- the man who refilled the drinking water to rehydrate from the desert dryness,
- those who carefully pushed the coffee trolleys to keep us awake,
- the lady who spotted us a seat in an overflowing breakout room, and
- everyone who made it a great success.
That’s my re:Invent 2022!