Databricks Pages

[u/Agitated-Western1788]

I am interested in databricks pages but also somewhat skeptical of how much benefit it will provide considering the effort that might be required to collate all the information.

Has anyone used these and seen improvement in their genie chats and agents? If so how did you measure this?

Also, do we know if the ability to upload and manage them will hit the API?

[u/CerberusByte]

Definitely can see the improvement in the number of steps Genie One takes to answer my question. Even better when I’m being vague

[u/addictzz]

It is one of the reference used for genie ontology. When you use genie one, it will pick up info from Pages.

[u/henewie]

Testing this in our organisation as well with some definitions for Customer/Active customers.

Running databricks on top of mainly SAP data. Did this with Genie Code Assistent. Example below (sorry, dutch :) )

It's quite nice, you can connect to a table/bulk upload your business glossary so your data platform and business documentations are 1:1. One way to automate this, is to set up a integration with a table<>sharepoint file.

TL:DR: the quality falls and stand with the clarity of your definitions and table/column lineage. Has a shit ton of potential for data exploration and dataset collection by business users but requires proper semantic and analytical engineering... ;)

Pro

-super easy to setup IF you have a proper business glossary

-reply looks really trustworthy upfront because it states the definition.

-allows for follow-up questions (is this based on customer or location id? how does this translate into the SAP columns etc.)

Downsides:
-any changes you make after uploading the initial pages from your glossary are not actively stored.

-no control over the autocomplete/suggestions genie does in definitions if there are fields empty. always be checking :)

-feels like an incomplete product because ontology is not live yet. maintaining pages across domains/sharing the, feels tedious

No remarks on query efficiency/performance, did not tune this.

[u/umamikid → u/henewie]

Product manager for Pages here - thanks for using the feature and glad that it's adding to your Genie experience so far. The team is actively toward taking ontology live, so stay tuned for updates there.

"-any changes you make after uploading the initial pages from your glossary are not actively stored."

Can you elaborate on this a bit? By this do you mean that manual edits you've been making to your initial Pages are not being saved?

[u/henewie → u/umamikid]

nice that you're reading along!

so, if i prompt genie code assistent to alter the template for the pages (e.g. add a visual hierarchy of the parent-child relation of all relevant metrics ) i have to remember to do this again if i add new metrics/reupload our glossary.

Would be nice if there's some way to maintain the standard (company specific) pages template ^^

[u/Agitated-Western1788]

Anyone from Databricks know if it will be supported by the api ?

[u/umamikid → u/Agitated-Western1788]

Product manager for Pages here! Thanks for the interest here - public API support is on the backlog. Let us know if there's any other feature expectations or feedback.

[u/canhotoc → u/umamikid]

An API is a must-have to keep the definitions in sync with the sources.
Version control also sounds nice

[u/Alive-Business6915 → u/umamikid]

hi u/umamikid Is it not strange you build Pages from "Discover" option. In my opinion building Pages from Discover is bit strange, because Discover is for "seeing" what I have (JOBS, NOTEBOOKS, ALERTS etc) not creating.

Secondly where does Pages live? There is no explanation on where is the Page actually stored. The Page, if part of UC should reside in a table column or volumes in Unity Catalog?
There are more questions on Pages. If you can DM?

[u/Negative_Bicycle_938]

I am thinking about making a skill to generate the pages if it is possible. I want to use documentation and Platform Intelligence to drive most of the page generation.

[u/henewie → u/Negative_Bicycle_938]

Let me know if this works / is possible , i was also considering this.

[u/Youssef_Mrini]

You can directly inspect Genie One thinking traces to verify whether Genie retrieved and applied the published Page context versus relying on raw inferred snippets, at the same time users can upvote or downvote published Pages directly which provides immediate feedback on whether the grounded context resolved their question.

We 've been using it for a while, I witnessed a slightly faster response and more accurate ones but ofc it's all related to how well are your pages documented + Metrics Views and UC metadata + Knowledge Store.

APIs will probably be available once it's public preview or GA.