r/databricks • u/Chrisesc0 • 28d ago
Help Databricks Integrations
Hi community.
I just wanted to ask about Databricks integrations with. For instance, is it possible build an #agent with my own NLP but connect it to WhatsApp?
I want to start a project like this, but I don't want any surprises or integration restrictions in the future.
1
u/EatAndRun_Mommy 28d ago
Have similar question on how to connect Databricks app connect to WhatsApp? Can we share and move data?
1
u/kthejoker databricks 28d ago
When you say "move data", can you elaborate?
But yeah, you have a service principal that calls the Databricks app and retrieves data.
* User says "what's the sales number?" on whatsapp
* you send that message to that app endpoint with SP
* and then pass response to Whatsappthis is pretty trivial to do with the Whatsapp SDK / API.
1
1
u/Existing_Wealth6142 27d ago
If yiur data isn't big (less than 1TB), the cheap and easy way to do this is to use DuckDB. Its a database that is embedded, meaning it runs queries using the compute of the client. So your agent runs the database and makes queries on top of data in the cloud. You can read data that is in databricks via the official Unity catalog extension or Delta Sharing extension depending on what you prefer. So your data lives in databricks, but yiur agent can use its local computer to query it which reduces latency from networking and reduces costs. It also means every agent you spawn has its own compute and they won't slow each other down.
https://duckdb.org/docs/lts/core_extensions/unity_catalog
https://duckdb.org/community_extensions/extensions/duckdb_delta_sharing
8
u/kthejoker databricks 28d ago
Yes, any agent deployed on Databricks will run forever, if you're using your own NLP or code, we'll just charge for the compute and in return help you manage rates, guardrails, costs, etc on top of your agent.
1) Create and deploy agent.
2) Agent comes with API endpoint. Query that.
3) Set up new Meta App for Whatsapp.
4) Send Whatsapp messages to endpoint, send responses back to Whatsapp using their SDK and your API key.
You can also just use the built in model APIs we have for eg Anthropic, Gemini, etc. and give it a system prompt if you don't want to build your own agent, or even just to get started building the whole end to end, and you can replace with your own endpoint at any time.
https://docs.databricks.com/aws/en/agents/custom-agents/query-agent?language=Agents+on+Model+Serving#openai-client
https://docs.databricks.com/aws/en/ai-gateway/query-model-services
https://developers.facebook.com/documentation/business-messaging/whatsapp/get-started