🦄 Making great presentations more accessible.

This project aims to enhances multilingual accessibility and discoverability while maintaining the integrity of original content. Detailed transcriptions and keyframes preserve the nuances and technical insights that make each session compelling.

Overview

📖 AWS re:Invent 2025 - Build, govern, and share Amazon Quick Suite dashboards with Amazon SageMaker

In this video, Luis Campos and Utkarsh demonstrate the integration between Amazon SageMaker Unified Studio and Amazon QuickSight (Quick Suite) to solve two critical problems: disconnected dashboard creation from data pipelines and lack of integrated governance for dashboards. The session shows how data analysts can discover data through SageMaker's catalog, request access via subscription workflows, analyze data using built-in query editors, and seamlessly transition to QuickSight with one click to create visualizations. Dashboards are automatically cataloged back in SageMaker for discovery and sharing. Mirko Buholzer from Natera shares how his company leverages this integration in a data mesh architecture, enabling distributed teams to independently manage data domains while maintaining centralized governance, resulting in 80% automated billing eligibility and improved lab operations insights.

; This article is entirely auto-generated while preserving the original presentation content as much as possible. Please note that there may be typos or inaccuracies.

Main Part

The agenda that we have for you today is going to be a packed agenda. I'm first going to show you a bit of the map of the cave and lay out the problem and what is that problem, how do we intend to solve it. Later, Utkarsh is going to show you what happens behind the scenes and also an end-to-end demo of this integration. Mirko is going to come and tell his story and how his team is innovating using these technologies. So let's get started.

Now, this problem is not exclusive to technology. In many day-to-day situations, you will find decision fatigue because of this plethora of options, too many options. Well, retail consumers go through this either in in-store experiences or online with too many products, sometimes from the same category, and being very hard to distinguish what's the difference between them.

In companies, in enterprises, the organizations that have to manage data, you have a similar issue with data coming from different sources, separate systems, different technologies as well. And so, once you have all these vast amounts of data that is being sourced, being a consumer of that data becomes a much harder task, let alone if you need to consume that data, transform it, and then serve it to someone else.

From Filters to Recommendations: The Evolution of Data Discovery

Navigating through all these options has always the challenge to bridge these two worlds, the options that we have and the consumer. Now let's go back to the retail example or entertainment example, if you wish. Once you have curated these options and extracted the metadata from the products, let's say from an e-commerce website or let's say a travel agency that has an online website, that metadata is being extracted to create what we call the filters.

This is not something new, yet it's something that a lot of organizations are still struggling with. And this is not a technology example, this is just from a consumer perspective. Now, let's get a little bit more, let's dive a little bit deeper. Let's transport to the enterprise data world, where not only you have several different sources from several different technologies, sometimes the same data in several different forms, and on the other side, you have humans, you have machines, you have agents, you have software, you have different sizes and different shapes of consumers.

The first step is to curate that data, extract that data, or simply Zero ETL that data over to a lakehouse. At the lakehouse, that's exactly where you're going to start doing your pre-processing. See if your data is clean, if it provides context, but most importantly, to correlate the data between all the sources, because now you have everything in one place. So far, so good, okay?

This happens a lot. These tools getting fatter and fatter in terms of capability. They do more and more, but they are not supposed to. So, where is the data that I need to build my dashboard, and I need to move my data. Turns out that this hinders the time it takes to create a dashboard. The second issue is the opposite side of the same coin.

That plethora of options I was talking about happens for two reasons. First of all, the dashboards that are created, leaving the private workspaces of the BI analyst, or sometimes the business user. And there are a lot of duplications, we know that. And so what tends to happen is that when someone needs a dashboard, they don't know where to find the exact, maybe someone has built something similar, so what do they do?

Amazon SageMaker Unified Studio: A Single Platform for All Data Personas

But the key topic for this talk today is the integration between both, okay. Let's just overview what Amazon SageMaker is. SageMaker is, SageMaker and SageMaker Unified Studio were launched in this same place last year, one year ago, as an integrated platform for all your data, AI and analytics needs, with three main layers. The first one is the face of the tool, which we call SageMaker Unified Studio, a single environment.

Every persona will use a different tool once they are in SageMaker Unified Studio, because for instance, a data scientist may use SageMaker AI to train and deploy AI models, while at the same time, a Gen AI developer can use the Bedrock IDE to create their custom Gen AI applications. A data engineer may prepare and integrate their data with EMR or Athena, or business analysts or data analysts may run their SQL queries using Redshift. So they all will enter through the same door, but at the end they will use different capabilities of the tool. SageMaker will integrate natively with Redshift, with EMR, with SageMaker AI, and with Bedrock.

Amazon Quick Suite Integration: Seamless Dashboard Creation and Cataloging

I'm not sure if you were attentive two months ago, but we did a big launch, and that was introducing Amazon Quick Suite. It's a tool that supersedes our previous tool called QuickSight, our BI tool. And I'm going to expand a bit more on what Amazon Quick Suite does. It has three main pillars, and all of these pillars are made consistent by something called spaces. This means that you can create a space, provide data assets inside that space, provide files, dashboards, and something called actions. An action is an integration with a third-party tool. And once you have all that, you can create something new, a new colleague, and that would be your custom chat agent.

You can specialize a chat agent inside that space that will be an expert on all the things you've put inside. So now you can not only talk with your dashboard, you can also have a specialist that knows about that. And most of the times, this represents a big advancement in people that are already working together. Now, Amazon Quick Suite is comprised of several different tools, and I'm going to very quickly skim through them, and then we're going to see how the integration works.

First of them is Quick Index that will allow, through an extensive list of connectors, to connect Amazon Quick Suite to a plethora of external sources from SharePoint to Salesforce and others. The other interesting part would be QuickSight and Quick Research. QuickSight is the tool that you use to create dashboards, you use to create storyboards, you use to create pixel-perfect reports. And Quick Research, we have been using internally at AWS and it's a game changer. You will provide an input in terms of natural language, you will provide the spaces that you have created, and you will toggle something: go to the web or don't. And it will create the topics of a research report you would like to create. And after that, of course, you will be able to customize your own research paper.

The automation inside Amazon Quick Suite is done through the use of Quick Flows, so you will create several different flows. And in order for those flows to connect with the tools, you will have Quick Automate. Automate will be where you create those actions, where you connect to those external sources like Salesforce and SharePoint and whatever tool you use, for instance, for CRM. Now, this is where all of these users will work together inside Amazon Quick Suite,

Demo Part 1: Data Discovery and Self-Service Access in SageMaker

Well, to show you how all this magic works in the background, let me call on stage my colleague Utkarsh. Thank you. Well, thank you so much, Luis. So, Luis talked about how Amazon SageMaker and Amazon Quick Suite together help you simplify the end-to-end analytics journey. Now, let's explore a real-life use case to see how it works in practice.

A data analyst has been asked to explore whether customer support interactions have an impact on customer churn. So it sounds like a simple problem at first, but where do you even begin? Your churn data might live in a Redshift data warehouse managed by a different team. Your customer support data might be in the AWS Glue Data Catalog in a completely different account with no business context at all. So first of all, how do you find this data? And once you find this data, how do you get access to this data? How do you understand this data with business context? And then how do you analyze and visualize this data using the tools of your choice?

Amazon SageMaker Unified Studio and Amazon Quick Suite help you do all of that in a single streamlined governed workflow that Luis talked about, and that is exactly what we'll explore in this demo. We'll start with data discovery. We'll show you how you can use the catalog in Amazon SageMaker Unified Studio to find data that you may not even have access to. We will then show you how you can access that data in a self-service manner. We will show you how you can analyze that data using some of the tools built in within Amazon SageMaker Unified Studio. And then once you have analyzed your data, how you can transition into Amazon Quick Suite so that you can visualize that data and build a dashboard. And once you have built that dashboard, how you can catalog and share that data so that the insights that you have generated do not just simply exist in a silo within your own personal workspace.

Now, before we start, let's take a look at the three personas who will be part of this demo. First, we have Daniel, who is a data analyst, and his responsibility is to turn raw data into meaningful insights. Then we have John, who is a business user. He heads the customer support team, and he's looking to understand how the customer support interactions of his team impact customer churn. And finally we have Priya, who is a data owner. She owns the data in the catalog, and she is responsible for managing access to the data that she owns. All right, so with that, let's jump into the demo and see how the data analyst can start with data discovery.

Now at this point, this request goes to the owner of the data, who is Priya in this case. So let's take a look at how Priya's experience would look like. We are again in SageMaker Unified Studio, logged in as Priya this time, and if we go to the subscription request page, we can see that there are a couple of pending requests. Priya can open one of the requests. She can see who has requested access for what data, for what purpose. And she can choose whether she wants to approve with full access, which means Daniel gets access to the full table, and she can simply approve the request.

So let's go back to Daniel's project. And if we go to the assets page, you will see that under subscribed data, there's an asset. The same asset that Daniel subscribed to, and it shows up as accessible. Just to confirm that Daniel can actually access it, let's go to the data page, and you can see that the same table also shows up under the Data Explorer. Daniel can access it.

Demo Part 2: From Data Analysis to Dashboard Visualization in QuickSight

So let's take a look at that. He goes to the query editor and he starts exploring the data. Here he's writing a SQL query, he executes it, and he's basically seeing how the churn rate varies with CSAT score, how does the average sentiment look like, how does the average response time look like. Once he has done this exploratory analysis, he decides to create a table with all the dimensions and metrics that he needs to be able to visualize this data. So here, he again runs a query to create that table, and you can see in the Data Explorer on the left-hand side that the table is created.

Great. So with that, Daniel is now ready to visualize this data in Amazon QuickSight. Now, before we jump into QuickSight, let's first understand how this integration actually works between SageMaker and QuickSight. So, with this integration, like Luis was saying, we provide a one-click transition from SageMaker to QuickSight, so that as a data analyst who is working in SageMaker, you can go into QuickSight and visualize the same data that you're able to access in SageMaker.

Now to make that happen, we do a lot of orchestration behind the scenes. First of all, any project that you have in SageMaker, if that project has QuickSight enabled in it, we create behind the scenes a folder in QuickSight with the same name as the project, and any datasets or dashboards that you create in QuickSight will exist in this folder itself.

What that means is from that point you can add more business metadata to that dashboard and make it discoverable in the catalog so that other users who do not have access to your dashboard can find the dashboard. If they want to use the insights that you have generated, they can do that. Finally, we make sure that the membership, members in the project are always in sync with the members in the folder in QuickSight. So anytime you add a member to the project or remove a member from the project, we make sure that the same members are also added and removed from the folder as well.

Great. So with that understanding, let's see how Daniel can go from SageMaker to QuickSight. So he simply chooses this option to open in QuickSight, and we open QuickSight in a new browser tab, and he lands on this dataset in QuickSight, which is automatically created for him. It's the same with the same name as the table that he needs to visualize. Now all he needs to do is simply use this dataset in an analysis. He adds this, he creates an analysis using this dataset.

Now he also wants to see how the sentiment in the chat impacts the churn rate. And this time instead of creating a chart manually, he describes it in natural language, and QuickSight is able to create the right chart for him. All he has to do is add this chart to his analysis. So you can see how easy it is to go from SageMaker to come to QuickSight and start building your analysis and charts.

Now, after doing all of this ad hoc analysis, Daniel realizes that customer support interactions are really impacting the customer churn and it's really important to monitor these metrics on a regular basis. So he goes and creates some more beautiful charts like these. Now I won't go into showing you how exactly he built those charts, but I guess you get the point. Once he has built this nice analysis with all these beautiful charts, he wants to make these charts available to other users, and for that, he can simply publish this analysis as a dashboard.

Another thing that Daniel can do here is that he can also start sharing this dashboard with the other users and groups in the organization from within the catalog itself. He does not have to necessarily go into QuickSight to do that. So here you will see that he's sharing this dashboard with a business user. He can choose to share it with users or groups, but in this case, he's sharing it with a user. And once he has shared this dashboard, again, SageMaker goes into the underlying service, which is QuickSight in this case, and manages the permissions on the dashboard so that this business user can access the dashboard.

So, Natera was founded in 2004, pioneering the cell-free DNA technology, and we're basically building non-invasive tests which can give us critical information about health conditions early on in the journey of many, many patients. We have labs in the San Francisco Bay Area as well as in Austin, Texas. And it's really key for us on a mission to help as many patients as we can.

So with that, we can aim at transforming the management of disease worldwide, and as you can see from the statement, this is kind of a very moonshot type activity that we would like to achieve. And with that we know this is not something which is done within a month or two, right? We have quite a bit of tests that we're actually already providing from women's health all the way to oncology and organ health, and we have a deep portfolio of tests that we're providing to our patients. The key part here is we want to expand and enhance those tests as quick as possible.

So going a little bit back to the key aspects in regards to combining data with visualization, the journey of many companies basically starts probably with applications having siloed data and analytics tools within each of the applications. Once that stage is past, we are moving into an area where a lot of companies are basically building centralized analytics teams, and with that we're introducing a bottleneck in the organization where we're trying to have one team solve all the answers for everyone else. So what we moved into is really a data mesh model, which allows everyone to collaborate in a very distributed, very self-serve model where we use SageMaker Unified Studio and have the ability to really contribute all the data products and assets to that platform.

For example, here we have a data domain on the lab side which is combining all the DNA data, the clinical data, as well as some of the instrument and execution and operations data onto the platform and moving that and allowing everyone else to use that data across the organization. So this includes obviously interpretation, but then all the way to R&D to develop new tests.

What we were doing initially when we introduced this was really making sure we're moving the data responsibility to the different applications. A key aspect that you see many times with analytics organizations is that they start to just copy databases over into the analytics environment, and then you have the central team trying to make sense out of it. What we changed here is really making sure that data is a first-class citizen within the application teams where they define the schema, where they define the actual shape of the data that we want to publish and are able to own that similar to an API or something else within the application where you really have a defined way of how you push that into the system.

The application generates events that are clearly formatted and then moves them into the data mesh. We have an option for serverless transformation as well if needed, but the goal is really to have the application shape the data so we don't have to do a lot of transformation anymore. Everything is basically done at the application level, and we just ingest it into the data mesh and have it available in the catalog.

Similar to that, there's the question of how do we organize our teams. Usually you have a technology team which owns the application. As we mentioned, we are basically pushing and shaping the data already into the data mesh, so that's called the application owner. We have product owners who really take that data already, create visualizations, create metrics, create insights where we know what type of functionality or features we need to develop based on the insights we get, and have the ability to really run the application data-driven so that we can develop applications which meet the user's requirements and are valuable.

At the same time, obviously you have a domain owner which makes sure that the different data is quite coherent across the different applications. With that, you generate high quality data because you're basically drinking your own champagne and are making sure that the data that is coming in meets the metrics that you want to be able to develop your application. This model creates high quality data which then can be used by the business users.

On the business side, obviously they might have similar or different metrics. They probably look across different applications as well and combine some of this data. You have analysts doing this, combining some of the data sources, maybe creating new data assets and visualizations from it, and they are able to really work independently on that level. Then obviously sometimes you have the business users come in and say, oh, by the way, I have this idea, can you give me some data around it? That's when the data scientist can just run on their own, create a Jupyter notebook, run the analysis, and come back with a validation.

So the business side really is independent as well. They usually have very small analytics teams within the business organization and are able to just leverage the infrastructure that was already built from the technology organization. There might be some requests when some data is missing, but in general, the goal is really to have those two teams being able to run independently.

Another good example that we have here is around billing eligibility. This is a big topic for our patients. They want to know if their insurance company is actually paying for the test or what they need to pay out of pocket. We worked closely together with the business team, the billing team, the RCM team in this case, together with technology, kind of having three in a box where they're able to take the data that is coming in from the different eligibility responses and then optimizing the rules and the executions to get highly automated billing eligibility responses back from our vendors.

We can see we're close to 80% fully automated where we're getting eligibility responses directly from our system, and believe me, that wasn't at 80% when we started. So it was really great to see the collaboration between the different teams to get up to that level.

Key insights really for us is creating reusable components within the organization where not everyone has to recreate everything from scratch. So starting with data assets or data products that we have in the organization, being able to share that in the catalog, being able to reuse that across the organization is a big advantage. And now with the integration into Amazon QuickSight and Amazon Quick Suite, we have the ability now to also have dashboards with it. A lot of times some of our business users just created their whole catalog of own dashboards which weren't shared across the organization.

And then the other part is really the distributed data ownership. So having the ability to have small teams run on their own pace, being able to innovate, being able to have quick answers, and having the ability to get back to the business owners in a good way is a key aspect that we now have and allows us to really walk and run fast. And obviously all of this is the platform that we need for our Gen AI journey that we're going through. I mean, it's all great to have agents and everything else running, but if you don't really know and have a good data foundation, you're not going to be able to get the results that you want. So with this we're able to combine all of that.

All right, that's it from my end, handing it over to Luis. Just wrapping it up. So I would like to thank your presence here today. Wish you the rest of a good week. If you enjoyed this session, please give us five stars in the app.

; This article is entirely auto-generated using Amazon Bedrock.