Google Cloud Pub/Sub is an asynchronous messaging service that helps in decoupling services that produce events from services that process events.
This blog covers an outline of cloud pub/sub including its key features, working, Pub/Sub Lite, pricing, etc.
What Is Cloud Pub/Sub?
Cloud Pub-Sub is a real-time messaging service offering by Google for applications to publish and subscribe to events.
Pub/Sub= Publisher-Subscriber
One can publish the message manually or watch the changes in Gmail using Gmail API and notify the user application on a real-time basis.
Core Components of Google Cloud Pub/Sub
- Topic: A named resource (which acts as an alias) to which the publishers send messages.
- Publisher: It is an application that creates and sends a message to a Pub/Sub topic. It is additionally referred to as Producer.
- Subscriber: It is an application that registers itself to a specified topic of interest in order to get appropriate messages.
- Subscription: It represents a stream of messages from a single, specific topic, which is to be delivered to the subscribing application.
- Message: It is a combination of data and other attributes which is sent by a publisher to a topic and is delivered to the subscribers at the end.
- Message attribute: A key-value pair used by a publisher to define a message.
Also, Check Our blog post on the Google Cloud Account Free Trial. Click here
How Does Pub/Sub Works?
The diagram below explains the key components in the Google Cloud Pub/Sub messaging system and the flow of messages between publishers and subscribers.
- A publisher application creates a topic in the Publisher/Subscriber service and sends messages to the topic. The message comprises a payload and optional attributes describing the payload content.
- The service makes sure that published messages are retained on behalf of the subscriptions. The message published is retained for a subscription till it is acknowledged by a subscriber consuming messages from that particular subscription.
- Pub/Sub forwards messages individually from a topic to all of its subscriptions.
- A subscriber receives messages either by Pub/Sub pushing them to the subscriber’s chosen endpoint or by the subscriber pulling them from the service.
- An acknowledgment is sent by the subscriber to the Pub/Sub service for each received message.
- The service then removes acknowledged messages from the subscription’s message queue.
Also, Read Our previous blog post on Google Cloud Run. Click here
Key Features
1.) In-order delivery at scale: Optional per-key ordering simplifies stateful application logic without sacrificing horizontal scale (i.e., no partitions required).
2.) Stream analytics and connectors: The native Dataflow integration allows reliable, expressive, processing and integration of event streams in languages such as Java, Python, and SQL.
3.) Cost-optimized ingestion with Pub/Sub Lite: Pub/Sub Lite aims to be the lowest cost option for high-volume event ingestion. It offers zonal storage and users have control of capacity management.
4.) Google Cloud–native integrations: Take advantage of integrations with multiple services, such as Cloud Storage and Gmail update events and Cloud Functions for serverless event-driven computing.
5.) Dead letter topics: It allows for messages unable to be processed by subscriber applications to be put aside for offline examination and debugging so that other messages can be processed without any delay.
Check Out: Cloud Spanner vs Cosmos DB, to know the major differences between them.
Publisher-Subscriber Patterns
The publisher application creates and sends messages to a topic and the subscriber applications create a subscription to that topic to receive messages from it.
There are three pub/sub patterns:
1.) ONE-TO-MANY/Fan-Out
2.) MANY-TO-ONE/Fan-In
3.) MANY-TO-MANY
Common Use-Cases
Here are some common use cases of Pub/Sub service:
- Balancing workloads in network clusters
- Implementing asynchronous workflows
- Distributing event notifications
- Refreshing distributed caches
- Data streaming from various processes or devices
- For reliability improvement
Read More: About Google Cloud Services. Click here
Pub/Sub vs Pub/Sub Lite
In a Pub/Sub system, publisher applications send messages to topics, and subscriber applications receive messages from subscriptions. Topics and subscriptions can be in different projects as they are global resources.
Pub/Sub Lite is a high-volume message service built for a low cost of operation by offering zonal storage and pre-provisioned capacity. Here, publishers send messages to Lite topics and subscribers receive messages from Lite subscriptions. Lite topics and Lite subscriptions must be in the same Cloud project and zone as they are zonal resources.
Pricing
Pub/Sub pricing is based on:
- Message ingestion and delivery
- Seek-related message storage i.e., snapshots and retained acknowledged messages
Pub/Sub pricing is based upon the monthly data volumes consumed. The first 10 GB of data per month is offered at no charge and beyond 10GB price is $40 per TB (tebibyte, or 240 bytes).
Read More: About GKE Cluster. Click here
Frequently Asked Questions
What is a pub/sub subscription?
The subscription connects the topic to a subscriber application that receives and processes messages published to the topic. A topic can have multiple subscriptions, but a given subscription belongs to a single topic.
Are messages delivered in order?
The Pub/Sub service can deliver messages in the order if the messages received have the same ordering key and publishers send the messages to the same region,
How to detect duplicate messages?
Pub/Sub assigns a unique message_id to each message, which is used to detect duplicate messages received by the subscriber.
Can Pub/Sub be used for cross-region or cross-project communication?
Yes, Pub/Sub supports cross-region communication within the same project and even across projects. You can publish messages to topics in one region and subscribe to those topics from subscribers in different regions or projects.
How does Pub/Sub guarantee message delivery?
Pub/Sub guarantees at-least-once delivery, which means that messages are delivered to subscribers at least once. If a subscriber acknowledges a message, Pub/Sub removes it from the system. If a message acknowledgment is not received within a certain time, Pub/Sub retries delivery.
How does Pub/Sub work?
Pub/Sub uses a publish-subscribe model, where publishers send messages to specific topics, and subscribers receive and process messages from those topics. Topics act as communication channels, and subscribers can receive messages in real-time or at their own pace.
Related References
- GCP Associate Cloud Engineer: All You Need To Know About
- GCP Professional Cloud Architect: Everything You Need To Know
- Introduction To Google Cloud Platform
- Google Cloud Services & Tools
- Introduction To Google Compute Engine
- Overview of Google App Engine
Next Task For You
If you are also interested and want to know more about the Google Professional Cloud Architect certification then register for our Free Class.
yash says
This is very helpful for beginners. Thanks a lot for sharing it Atisha !
Content is well presented
Rahul Dangayach says
Hi Yash,
Glad you liked our blog.
Please stay tuned for more informative blog like these.
Thanks and Regards
Rahul Dangayach
Team K21Academy