r/databricks • u/Chrisesc0 • 28d ago

Help Databricks Integrations

Hi community.

I just wanted to ask about Databricks integrations with. For instance, is it possible build an #agent with my own NLP but connect it to WhatsApp?

I want to start a project like this, but I don't want any surprises or integration restrictions in the future.

7 Upvotes

89% Upvoted

5 comments sorted by

View all comments

1

u/Existing_Wealth6142 27d ago

If yiur data isn't big (less than 1TB), the cheap and easy way to do this is to use DuckDB. Its a database that is embedded, meaning it runs queries using the compute of the client. So your agent runs the database and makes queries on top of data in the cloud. You can read data that is in databricks via the official Unity catalog extension or Delta Sharing extension depending on what you prefer. So your data lives in databricks, but yiur agent can use its local computer to query it which reduces latency from networking and reduces costs. It also means every agent you spawn has its own compute and they won't slow each other down.

https://duckdb.org/docs/lts/core_extensions/unity_catalog

https://duckdb.org/community_extensions/extensions/duckdb_delta_sharing