Google Cloud Pub/Sub

Overview

Google Cloud Pub/Sub is a fully managed, asynchronous global messaging service offered by Google Cloud. It facilitates real-time communication between independent applications and microservices, acting as middleware for event ingestion and distribution. The service operates on a publish-subscribe model, where publishers send messages to a topic, and subscribers receive messages from that topic via a subscription. This decoupling allows components to interact without direct knowledge of each other, enhancing system flexibility and resilience.

Pub/Sub is engineered for high throughput and low latency, capable of processing millions of messages per second. It supports both standard and Lite variants. Pub/Sub Standard provides a global service with automatic scaling and high availability, making it suitable for general-purpose messaging, event streams, and background task queues across regional boundaries. Pub/Sub Lite offers a lower-cost option with regional storage and explicit capacity provisioning, designed for scenarios where predictable throughput and cost are prioritized, such as data pipelines with consistent loads within a specific region Google Cloud Pub/Sub Lite overview.

Developers use Pub/Sub for building event-driven architectures where services react to changes or events in real-time. Common use cases include ingesting data from IoT devices, processing user activity streams, distributing tasks to worker services, and replicating data between databases. The service guarantees at-least-once message delivery, ensuring that a message is delivered to a subscriber at least one time, with mechanisms for acknowledgment to prevent reprocessing or loss Google Cloud Pub/Sub subscriber guide. Client libraries are available for multiple programming languages, simplifying integration into existing applications.

In comparison to traditional message queues, Pub/Sub offers automatic scaling and global reach, abstracting away infrastructure management. For instance, while Apache Kafka provides robust message streaming capabilities, it typically requires self-management of clusters. Google Cloud Pub/Sub's managed nature reduces operational overhead, allowing developers to focus on application logic rather than message broker administration Apache Kafka vs. Cloud Pub/Sub comparison.

Key features

• Global and Regional Messaging: Supports a global message bus for standard Pub/Sub and regional topics for Pub/Sub Lite, catering to different latency and data locality requirements Google Cloud Pub/Sub overview.
• Automatic Scaling: Automatically scales to handle fluctuating message volumes without manual intervention for Pub/Sub Standard.
• At-Least-Once Delivery: Guarantees that each message is delivered to a subscribing application at least once, with duplicate messages possible in rare cases.
• Message Durability: Messages are stored redundantly across multiple zones until acknowledged by subscribers, ensuring data persistence.
• Pull and Push Subscriptions: Offers both pull-based subscriptions, where subscribers fetch messages, and push-based subscriptions, which deliver messages to a webhook endpoint Google Cloud Pub/Sub push subscriptions.
• Dead-Letter Topics: Allows configuring dead-letter topics to store messages that cannot be successfully processed, aiding in debugging and error handling.
• Message Filtering: Subscribers can filter messages based on attributes, receiving only messages relevant to their processing logic Google Cloud Pub/Sub message filtering.
• Ordering Guarantees: Provides ordered delivery within a message key for Pub/Sub Standard and strict ordering for Pub/Sub Lite within a partition.
• Client Libraries: Official client libraries are available for Python, Java, Node.js, Go, C#, Ruby, PHP, and C++.

Pricing

Google Cloud Pub/Sub employs a pay-as-you-go pricing model based primarily on the volume of message data processed. There are separate pricing structures for Pub/Sub Standard and Pub/Sub Lite, reflecting their different operational models and use cases.

As of May 2026, the general pricing for Pub/Sub is:

Detailed and up-to-date pricing information, including regional variations and specific terms for Pub/Sub Lite, is available on the official Google Cloud Pub/Sub pricing page.

Common integrations

• Google Cloud Dataflow: Used for real-time stream processing of messages from Pub/Sub topics, enabling transformations and analysis Dataflow with Pub/Sub.
• Google Cloud Functions: Triggers serverless functions in response to messages published to a Pub/Sub topic, supporting event-driven microservices Cloud Functions triggered by Pub/Sub.
• Google Cloud Storage: Messages can be streamed to Cloud Storage for archival, batch processing, or integration with data warehousing solutions Pub/Sub notifications for Cloud Storage.
• BigQuery: Pub/Sub messages can be directly ingested into BigQuery tables for real-time analytics and reporting Loading data from Pub/Sub to BigQuery.
• Google Kubernetes Engine (GKE): Microservices deployed on GKE can use Pub/Sub for inter-service communication and event coordination.
• Cloud Logging and Monitoring: Integrates with Cloud Logging for message delivery logs and Cloud Monitoring for operational metrics and alerts Monitoring Pub/Sub.

Alternatives

• Amazon SQS: A fully managed message queuing service by AWS, offering standard and FIFO queues for decoupling and scaling microservices, distributed systems, and serverless applications.
• Azure Service Bus: A reliable cloud messaging service by Microsoft Azure for connecting applications and services, providing advanced capabilities like message sessions and topic subscriptions.
• Apache Kafka: An open-source distributed streaming platform for building real-time data pipelines and streaming applications, often deployed and managed by users or through managed services like Confluent Cloud.
• RabbitMQ: An open-source message broker that implements the Advanced Message Queuing Protocol (AMQP), providing robust and flexible messaging for various distributed application architectures.
• Redis Streams: A data structure in Redis that implements a log-like data type, allowing for append-only data storage and consumption, suitable for simple event sourcing and messaging patterns.

Getting started

This Python example demonstrates how to publish a message to a Google Cloud Pub/Sub topic and subscribe to receive messages. First, ensure you have the Google Cloud SDK installed and authenticated, and the google-cloud-pubsub library installed (pip install google-cloud-pubsub).

from google.cloud import pubsub_v1
import time

# --- Publisher Setup ---
project_id = "YOUR_PROJECT_ID"
topic_id = "YOUR_TOPIC_ID"

publisher = pubsub_v1.PublisherClient()
topic_path = publisher.topic_path(project_id, topic_id)

# Create the topic if it doesn't exist (optional)
# try:
#     publisher.get_topic(request={"topic": topic_path})
#     print(f"Topic {topic_id} already exists.")
# except Exception:
#     publisher.create_topic(request={"name": topic_path})
#     print(f"Topic {topic_id} created.")

# --- Subscriber Setup ---
subscription_id = "YOUR_SUBSCRIPTION_ID"
subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path(project_id, subscription_id)

# Create the subscription if it doesn't exist (optional)
# try:
#     subscriber.get_subscription(request={"subscription": subscription_path})
#     print(f"Subscription {subscription_id} already exists.")
# except Exception:
#     subscriber.create_subscription(request={"name": subscription_path, "topic": topic_path})
#     print(f"Subscription {subscription_id} created for topic {topic_id}.")

def callback(message):
    print(f"Received message: {message.data.decode('utf-8')}")
    message.ack() # Acknowledge the message

# Start an asynchronous subscription pull
streaming_pull_future = subscriber.subscribe(subscription_path, callback=callback)
print(f"Listening for messages on {subscription_path}. Press Ctrl+C to exit...")

# Publish a message
data = "Hello, Pub/Sub from Python!"
data = data.encode("utf-8")

future = publisher.publish(topic_path, data)
print(f"Published message with ID: {future.result()}")

# Keep the main thread alive to allow the subscriber to receive messages
try:
    streaming_pull_future.result() # Blocks until exception or cancellation
except KeyboardInterrupt:
    streaming_pull_future.cancel() # Triggers the callback to exit
    subscriber.close()
    publisher.close()

Replace YOUR_PROJECT_ID, YOUR_TOPIC_ID, and YOUR_SUBSCRIPTION_ID with your actual Google Cloud project and resource identifiers. This script sets up a publisher to send a message and a subscriber to receive and acknowledge it. The comments indicate how to optionally create topics and subscriptions if they don't already exist. For detailed setup and more advanced features, refer to the official Google Cloud Pub/Sub Python client library quickstart.