Spring Cloud Using Google Cloud Pub-Sub

By amikblogger30 07/17/202310 Comments

Spring Cloud GCP provides various cloud computing service that helps to run a Spring Boot application. It provides a variety of features or components to use. In this blog, I have discussed the Publisher-Subscriber concept.

Initial Setup

The first step is to register for Google Cloud Console using a Google account.

Now after registering you can scroll to the left navigation panel, scroll down click the Pub-Sub option
Once clicked, it will open a page where you can create the Topic.
After you create the Topic, you have to create a Subscription.
Once the Subscription is created, now you can push the data and even pull the data.
You need to enable one API to access Pub/Sub named Pub/Sub API
To access the Pub/sub from your Spring Boot Application, you need to create IAM roles as follows:
- Pub/Sub Admin
- Pub/Sub Editor

Steps to install Google Cloud SDK

Download the Google Cloud SDK.
Install the SDK following the instructions on that same page.
After successful installation, it will run the gcloud init command
It will authenticate with your Google account that you are using in the Google Cloud Platform
Once authenticated, it will ask to select the Project, provide the no of the project
Then it will ask for the Zone and Region. I have chosen the asia-south1-c region, which is available at option no 35. It can vary. But based on your location, you choose your region.

Steps to use the same Google account to connect/configure to various GCP Services from local

Open the Google Cloud SDK from your machine and run a command

gcloud auth application-default login

The above command will store the credentials of your Google account that you use in GCP in your local machine in a json file format. By default, it stores in the location as C:/Users/<UserName>/AppData/Roaming/gcloud/application_default_credentials.json.

The advantage is with the help of this json file you can connect to various GCP Services for this blog I have connected to Google Cloud Pub-Sub.

Discussion on spring-cloud-gcp-pub-sub masterclass application

Spring gradually making a strong bonding with Cloud Frameworks like GCP, AWS, Azure, etc. In this article, I have highlighted one of the Cloud Components of Google, which is called Pub-Sub. This is a cloud messaging service. Similar, to any messaging service, Google’s messaging service requires a Topic to publish the messages and from that Topic, a list of Subscribers will consume those messages. In this article, I just use the Google Pub-Sub to publish and consume messages. While you, people can persist the data in a DB after consuming it as well. In support of this context, I have created a Demo application named spring-cloud-google-pub-sub-masterclass and uploaded it to my personal GitHub account. One can clone the project and test it locally. All the steps that are required to clone the project from GitHub and run it locally by installing the software are mentioned in the README.md file of the application.

Brief Description – In this application, I have used Java, Spring Boot, Maven, Spring-REST, and Google Cloud Pub-Sub.
Software Used – Software required to develop this project.
- Spring Tool Suite-4.7.0-RELEASE – If the latest version is available then download that one
- Apache Maven 3.6.3 – If the latest version is available then download that one
- Java 8 – Not less than Java8
- Git 2.27.0 – Latest version as available
- Google Cloud SDK – To configure the Google account as used in Google Cloud Console to configure the required services
- Postman v8.3.0 – To test the REST Service
- The list of the software used in this application and their required download links with the installation steps are briefly described in the README.md file of the spring-cloud-gcp-pub-sub-masterclass repository.
Project Components – Maven is used as the building tool and to provide the dependencies for this demo application. This application connects with one of the Cloud frameworks named Google Cloud Platform. For this, I define BOM (Bills of Materials) in pom.xml file. Let’s discuss this in brief.

<dependencyManagement>
	<dependencies>
		<dependency>
			<groupId>com.google.cloud</groupId>
			<artifactId>spring-cloud-gcp-dependencies</artifactId>
			<version>3.3.0</version>
			<type>pom</type>
			<scope>import</scope>
		</dependency>
        </dependencies>
</dependencyManagement>

The above dependency spring-cloud-gcp-dependencies of com.google.cloud groupId is used to load all the required google-cloud dependencies in support to develop the application.

You also need a specific dependency by which you will be able to successfully connect with Google Pub-Sub. The name of the dependency is spring-cloud-gcp-starter-pubsub.

<dependency>
	<groupId>com.google.cloud</groupId>
	<artifactId>spring-cloud-gcp-starter-pubsub</artifactId>
</dependency>

The above dependency will let you connect to Google Pub-Sub from your Spring Boot application.

Apart from this, I have defined 2 properties in application.properties file. One can also use application.yml in replace of application.properties. The properties are provided below:

spring.cloud.gcp.project-id=spring-gcp-sql-storage

spring.cloud.gcp.credentials.location=file:<Folder location>/application_default_credentials.json

spring.cloud.gcp.project-id=spring-gcp-sql-storage ⇾ This is the name of the project as created in Google Cloud Console using your own Google Account or Company Account.
spring.cloud.gcp.credentials.location=file:<Folder location>/application_default_credentials.json ⇾ If you want to connect your application from a local machine with Google Cloud Services, you need to provide the credentials of that account that you have used in Google Cloud Console to configure various services. This is a JSON file, and it contains the credentials in token format. Here, the red highlighted <Folder location>is the place in your hard drive where you keep this file. A detailed description is provided in the README.md file of the spring-cloud-gcp-pub-sub-masterclass repository.

Structure of the Project – If you are using Maven as the building tool, then the project structure is always the same. PFB the screenshot.

Spring Cloud integration with Google Cloud Pub-Sub – The application is a Spring Boot application and I have used Google Cloud dependencies to connect with the Google Cloud platform. Here, using the Google Cloud dependencies I am able to successfully connect to Google Pub-Sub by using some classes like PubSubAdmin and PubSubTemplate.

A code snippet of the use of the above classes:

// Use of PubSubAdmin class
// To create the Topic
Topic topic = this.pubSubAdmin.createTopic(topicName);
// To create the Subscription
Subscription subscription = this.pubSubAdmin.createSubscription(subscriptionName, topicName);
// To get the list of Topics
List<Topic> topics = this.pubSubAdmin.listTopics();
// To get the lsit of Subscriptions
List<Subscription> subscriptions = this.pubSubAdmin.listSubscriptions();

// Use of PubSubTemplate class
// To publish a message in a Subscription of a Topic
ListenableFuture<String> future = this.pubSubTemplate.publish(topicName, message);
// To pull message from Subscription of a Topic
Collection<AcknowledgeablePubsubMessage> messages = this.pubSubTemplate.pull(subscriptionName, 10, true);
// Acknowlege the message once pulled
ListenableFuture<Void> ackFuture = this.pubSubTemplate.ack(messages);

A brief description

PubSubAdmin → com.google.cloud.spring.pubsub.PubSubAdmin, is a very special class as using this class we can create a Topic, Subscription, List all the Topics available, List all the Subscriptions available, delete a Topic, and Delete a Subscription.
PubSubTemlplate → com.google.cloud.spring.pubsub.core.PubSubTemplate is another important class by which we can publish and pull messages from the Subscription of a Topic, and after pulling the message we can acknowledge it.

The most important point to remember

PubSubAdmin and PubSubTemplate classes need to initialize in the Constructor, as they should be declared private and final. Please refer to the below code snippet regarding this.

private final PubSubAdmin pubSubAdmin;
private final PubSubTemplate pubSubTemplate;

@Autowired
public SpringGCPPubSubService(PubSubAdmin pubSubAdmin, PubSubTemplate pubSubTemplate) {
	this.pubSubAdmin = pubSubAdmin;
	this.pubSubTemplate = pubSubTemplate;
}

Testing the application – In this application there is no UI component. So, I have used Postman to test the REST endpoint as defined. But you can integrate UI components to result in the output. Even though I have not used any DB to persist the data, you can use that as well.

You clone the application from GitHub and set up the application locally in any one of the IDEs like Spring Tool Suite (STS) or Eclipse.
Right-click on the application
Click the Option Run As
Select the option Spring Boot App.
It will start the application in port no 7121.

Below, I have provided the list of endpoints as available in this application.

Create Topic and Subscription in Google Cloud Console – localhost:7121/createTopic/{topicName}/{subscriptionName}
Publish a message in Google Cloud Pub-Sub Topic – localhost:7121/publishMessage
Pull Message from Google Cloud Pub-Sub Topic – localhost:7121/pullMessage
Delete Subscription from Google Cloud Pub-Sub – localhost:7121/deleteSubscription
Delete Topic from Google Cloud Pub-Sub – localhost:7121/deleteTopic

Sample Post RequestBody – Body for the POST method to publish a message in Google Cloud Pub-sub Topic. For this method, the body type is raw then selected Text from Postman. Provide the message in text format. Like, "Hi! Welcome to Google Cloud Pub-Sub"

Pictorial Presentation of Pub-Sub in Google Cloud Console

Once the application runs locally following the above steps, you will hit your 1st endpoint to create the Topic and Subscription in Google Cloud Pub-Sub. Once it runs successfully, it will create the desired Topic in Google Cloud Console. A screenshot for the same is provided below.

Similarly, a Subscription is also created in Google Cloud Console. A screenshot is provided below.

Now once the above Topic and Subscription creation are successful then you will be able to push the message in the Subscription of a Topic. For that, I have another REST Endpoint by which I am publishing the messages in the Subscription of a Topic in Google Cloud Console. A screenshot is provided below.

After this, once you call another REST Endpoint to pull the message, then that API will consume the message from the topic, and then it will acknowledge that it has consumed the message from the Topic. And after consuming, the message will be not available in the Topic. A screenshot is provided below.

See the yellow highlighted color segment in the above screenshot, it clearly states that “No messages found yet“.

Last updated on 07/14/2024

amikblogger30

View All Posts

Initial Setup

Steps to install Google Cloud SDK

Steps to use the same Google account to connect/configure to various GCP Services from local

Discussion on spring-cloud-gcp-pub-sub masterclass application

Pictorial Presentation of Pub-Sub in Google Cloud Console

GitHub Code Link

Common Faced Problems

Interview FAQs

Other Useful Links of Spring Cloud using Google

Other Useful Links

10 Comments