> ## Documentation Index > Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt > Use this file to discover all available pages before exploring further. # Connecting to Cloud PubSub > Overview of connecting applications to Google Cloud Pub/Sub, covering client libraries, authentication with Application Default Credentials, publish and subscribe patterns, code examples, and operational best practices Welcome back. In this lesson we’ll examine how applications and services connect to Google Cloud Pub/Sub, the managed messaging service for asynchronous communication. You’ll learn where connections occur, how client libraries communicate with the service, recommended authentication patterns, commonly used publish/subscribe designs, code examples in popular languages, and operational best practices. ## Architecture overview Connections to Cloud Pub/Sub happen in two primary places: * Publisher side — applications that send messages (for example, microservices, data collection APIs, or logging agents like Fluent Bit). * Subscriber side — applications that receive messages from a subscription (for example, other microservices, data pipeline workers, or serverless functions). Both publishers and subscribers use Google Cloud client libraries. Think of the client library as a language-specific connector that handles authentication, retries, batching, flow control, and network details so your application can focus on message semantics. ## How client libraries communicate * Protocols: gRPC is used by default for performance and low latency. REST/JSON endpoints are also available for certain use cases and tooling. * Endpoint: Client libraries communicate with the Pub/Sub endpoint at `pubsub.googleapis.com` unless you explicitly configure a different endpoint. * Responsibilities handled by libraries: authentication (Application Default Credentials or explicit service account keys), automatic retries, publish batching, and subscription flow control/backpressure. ## Authentication and IAM Application Default Credentials (ADC) are the recommended authentication method: * In Google Cloud runtimes (Compute Engine, GKE, Cloud Run, Cloud Functions), ADC is usually available automatically. * When running outside Google Cloud (for example, local development), set `GOOGLE_APPLICATION_CREDENTIALS` to a service-account key file: ```bash theme={null} export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json" ``` Use least-privilege IAM roles. For publishing and subscribing, grant the `roles/pubsub.publisher` or `roles/pubsub.subscriber` role respectively to the service account. ## Publish vs. subscribe patterns * Publishing: Clients publish messages to a topic. Libraries support synchronous and asynchronous (future/promise-based) publish APIs and provide batching options for higher throughput. * Pull subscribers: Clients pull messages from a subscription. Modern libraries support streaming pull with callback handlers and automatic lease (ack deadline) management. * Push subscribers: Pub/Sub sends HTTP POST requests to a configured HTTPS endpoint when new messages arrive. Push endpoints must acknowledge the request (e.g., return HTTP 200) to avoid redelivery. ## Client library packages (common languages) | Language | Official package | | -------- | -------------------------------------- | | Python | `google-cloud-pubsub` | | Java | `com.google.cloud:google-cloud-pubsub` | | Node.js | `@google-cloud/pubsub` | | Go | `cloud.google.com/go/pubsub` | | C# | `Google.Cloud.PubSub.V1` | Installation examples: ```bash theme={null} # Python pip install google-cloud-pubsub # Node.js npm install @google-cloud/pubsub ``` ## Minimal code examples Python — publish (synchronous) ```python theme={null} from google.cloud import pubsub_v1 project_id = "my-project" topic_id = "my-topic" publisher = pubsub_v1.PublisherClient() topic_path = publisher.topic_path(project_id, topic_id) data = "Hello, Pub/Sub!".encode("utf-8") future = publisher.publish(topic_path, data, origin="python-sample") message_id = future.result() print(f"Published message ID: {message_id}") ``` Python — subscribe (streaming pull with callback) ```python theme={null} from google.cloud import pubsub_v1 project_id = "my-project" subscription_id = "my-subscription" subscriber = pubsub_v1.SubscriberClient() subscription_path = subscriber.subscription_path(project_id, subscription_id) def callback(message): print(f"Received message: {message.data.decode('utf-8')}") message.ack() streaming_pull_future = subscriber.subscribe(subscription_path, callback=callback) print(f"Listening for messages on {subscription_path}...") try: streaming_pull_future.result() except KeyboardInterrupt: streaming_pull_future.cancel() ``` Node.js — publish (async) ```javascript theme={null} const {PubSub} = require('@google-cloud/pubsub'); const pubsub = new PubSub({projectId: 'my-project'}); async function publishMessage() { const topicName = 'my-topic'; const dataBuffer = Buffer.from('Hello, Pub/Sub!'); const messageId = await pubsub.topic(topicName).publish(dataBuffer); console.log(`Published message ID: ${messageId}`); } publishMessage().catch(console.error); ``` ## Operational considerations * Batching and latency: Publishers batch messages to improve throughput. Tune batch size and delay to balance latency and throughput. * Flow control: Configure max outstanding messages/bytes on subscribers to avoid resource exhaustion and to maintain predictable memory/CPU usage. * Retries and dead-letter topics: Configure retry policies and consider a dead-letter topic for messages that fail processing repeatedly so they can be inspected and reprocessed. * Push endpoints: Ensure push endpoints use HTTPS, validate requests as needed (e.g., authentication or token validation), and respond with the correct status codes to control acknowledgment behavior. * Monitoring and alerting: Track metrics such as publish/ack latency, undelivered message backlog, and subscriber errors to detect and respond to issues quickly. ## Security warning Never commit service account keys to source control. Use secret management, environment-level credentials (ADC), or workload identity to securely provision credentials. ## Quick reference: patterns and choices | Use case | Recommended pattern | | --------------------------------- | ---------------------------------------------------------------------- | | High-throughput ingestion | Asynchronous publishing with batching | | Real-time processing with scaling | Pull subscribers using streaming pull or serverless subscribers | | Webhook-style delivery | Push subscriptions to HTTPS endpoints (must handle retries/validation) | | Error isolation | Use dead-letter topics and retry policies | ## Links and references * Cloud Pub/Sub documentation: [https://cloud.google.com/pubsub](https://cloud.google.com/pubsub) * Pub/Sub client libraries: [https://cloud.google.com/pubsub/docs/reference/libraries](https://cloud.google.com/pubsub/docs/reference/libraries) * Authentication overview (ADC): [https://cloud.google.com/docs/authentication/production](https://cloud.google.com/docs/authentication/production) Summary * Publishers and subscribers connect to Cloud Pub/Sub via Google Cloud client libraries. * Libraries abstract network, auth, retries, batching, and flow control so you can focus on message handling. * Choose publish/pull/push based on latency, throughput, and runtime environment. * Use Application Default Credentials or secure credential provisioning and follow least-privilege IAM. That’s it for this lesson—see you in the next one.