As your Pulsar deployment grows, keeping track of all the topics and understanding what data they carry, can become a challenge. The Streamvisor data catalog helps you document, organize and search your topics. This way you will always understand what is flowing through your pipelines and who owns it.
In this quick guide, we’ll show you how to use the data catalog to register metadata, filter by owners and applications, and quickly navigate to your detailed topic information.
To get started, open the Explorer page. This is your main starting point for interacting with your Pulsar environment.
Once you are on the Explorer page, you'll see a list of tenants, the logical groups that help organize your messaging infrastructure. Each tenant contains one or more namespaces, which in turn contain the actual topics where your messages live.
Here’s a quick path through the structure:
This clear hierarchy ensures you always know exactly where you are in the system.
Once you’ve selected a topic, you’ll land on its overview page. This page gives you general information about the topic, such as its configuration and metrics.
If you have not previously added documentation, you will notice that no catalog information is registered yet.
Click the Edit button to update the documentation for this topic. In the dialog that opens, you can define:
This metadata forms the entry in your data catalog. You can also leave fields empty if you do not need them.
Once you have added documentation, head to the Catalog section in the sidebar. Here, you'll see a different visualization compared to the Explorer. The catalog shoes all documented topics in one place, along with their descriptions, owners and labels.
From here, you can:
This makes it easy to find the topics you care about.
The Data Catalog is not just a static list. When you find an interesting entry, click on it to open an extended dialog. Here the full metadata will be displayed and clicking on the context menu at the top will give you the option to jump right into the topic details by clicking on the Explore button.
The Data Catalog makes Pulsar more transparent and easier to manage. By documenting topics with owners, descriptions and labels, you can create a shared understanding across your different teams and roles. Try it out next time you want to bring clarity to your topics, improve collaboration, or just make your Pulsar streams more discoverable!
