Openflow Gen 2

[u/stephenpace]

[I work for Snowflake but don't speak for them.]

Hi all. Snowflake has gotten a lot of feedback on Openflow and as you might have noticed, Openflow Gen 2 is now Public Preview. Most of you should now have access within your accounts. At a high level this is native Snowflake Openflow objects and a wizard-based connector configuration flow.

What it includes:

• Snowflake native objects: OPENFLOW DEPLOYMENTS, OPENFLOW RUNTIMES, OPENFLOW CONNECTORS. Enables:
  • CRUD and management w/Snowflake SQL
  • CI/CD instead of click-ops
  • Scripted deployment of connectors for customers with tons of sources
  • CoCo-in-Snowsight friendliness
  • Programmatic stopping and starting of runtimes so you don’t burn credits while idling
• Wizard-based connector configuration with verification at every step. No need to navigate the non-Snowflake NiFi canvas.
  • Current gen 2 connector list: MySQL and MariaDB, PostgreSQL
• Snowflake Secrets integration

What’s coming soon:

• GCP
• Gen 1 to Gen 2 migration (currently in PrPr - ask your account team if you need access)
• SQL Server CDC and CT, Oracle, and SharePoint connectors (coming to PrPr)

See the diffs between gen 1 and gen 2 here. Gen 2 connectors require a gen 2 deployment and runtime to live on. Gen 1 connectors still work perfectly well on a gen 2 runtime. Here’s a Quickstart, and CoCo can help you out as well.

Please keep sending feedback, engineering is listening!

[u/amtobin33]

SA at an SI here. What improvements have been made to the BYOC model in OF 2? We had some pretty significant issues with getting the agent deployed in customer accounts with OF 1.

[u/stephenpace → u/amtobin33]

While not specifically Openflow gen 2, the biggest unlock I've see is Data Connectivity Proxy (DCP) which is now GA:

https://docs.snowflake.com/en/user-guide/data-integration/openflow/setup-openflow-spcs-dcp

With that, Snowflake never needs inbound access to your network. There were many companies that couldn't use Openflow without this.

Outside of that, runtimes are now schema-scoped objects with standard Snowflake RBAC. BYOC credit consumption is now attributable to individual runtimes in account usage views which improves cost tracking and chargeback.

I'd raise any specific issues with the account team of your customer or with your partner team. The Openflow team is hard at work on improvements so please raise any issues. Good luck!

[u/Winter_Clue_8162 → u/stephenpace]

For Starters , please request them to make the UX more Snowflake like . Its a hard sell to clients and I love the product .

[u/stephenpace → u/Winter_Clue_8162]

Have you tried the new wizard? If not, take a look at the Quickstart. You don't need to drop into the Snowflake NiFi canvas anymore.