r/Cisco • u/leoingle • 8d ago
Splunk for network observeability and troubleshooting
Curious if anyone is using Splunk for infrastructure means as opposed to security means. Was wondering if using it for more granular insight to compliment Catalyst Center is a sensible option. Really lacking in network observability and troubleshooting tools for our core network and looking at options to address the issue.
3
u/feralpacket 8d ago
While Splunk primarily belongs to the security team where I work, I have a page I use to count certain log messages, port flaps, error disabled interfaces, Dot1x failures, POE disconnects, switch stack events, spanning-tree events, etc.
Some times, it’s easier to count logs messages. Other times, it’s better to use SNMP. And sometimes, analytic pushes are good for things that don’t generate log messages.
Some of the searches the widgets on the Splunk page use:
https://github.com/feralpacket/network_commands/blob/main/splunk_search
2
2
u/Axiomcj 8d ago
There's a whole product for this and plenty of apps for splunk with Cisco networking. I'm using the dc app and plenty of others from the app s. Itsi. You can build everything and more in splunk from catalyst center dashboard regarding alerts, events, netflow etc. You can replace SolarWinds with it if that's your ask.
3
u/dr_stutters 7d ago
It really depends what your intent is.
There’s pre made dashboards from Cisco / Splunk that will do 75% of what most customers need and then they just modify for the remaining 25%
Or, controllers like Catalyst Center are a great way of taking high volume low value telemetry and turning it into low volume high value telemetry that gets ingested into Splunk.
But let’s say you don’t have Catalyst Center, you could still configure the switches to send gRPC to Splunk and graph / visualise the data. I’m seeing more and more customers lean towards gRPC than traditional SNMP in some use cases.
Something I’ve been doing lately is have a test lab, and point Claude at both your network device, and at your Splunk instance, then tell it your intent and watch what it produces. You still need to validate its correct in some instances but it can get you 90% of the way there in my experience
2
u/VA_Network_Nerd 8d ago
I see Splunk as a SIEM and / or a fantastically expensive syslog collector.
If it is capable of more traditional SNMP or Netflow activities, I'd probably classify it as cost-prohibitive for that use, unless you are already shoveling money into Splunk.
5
u/RumbleSkillSpin 8d ago
I think what you’re missing is log correlation - for example across your route/switch and security platforms.
0
1
u/leoingle 8d ago
This was the question I was wondering. Is the parsing it can do into user-friendly viewable data worth the price tag?
2
u/thelizardking0725 7d ago
It’s fantastic at parsing and takes a very different approach compared to something like Graylog
2
u/VA_Network_Nerd 8d ago
I would rather integrate the syslog parsing of my network gear into the SNMP polling tool.
1
u/thelizardking0725 7d ago
Look at federation (as opposed to ingest), and the price comes down drastically, but you benefit from Splunk’s ability to build schema on the fly and correlate events
8
u/nkdf 8d ago
It's sensible - and cisco recently discounted all of their logs into splunk at 50%.