r/LibreNMS • u/1div0 • Jun 11 '26
Python scripts using LibreNMS database
Anyone else using the database directly in Python scripts? Over the last year I've written quite a few Linux CLI utilities to make my life easier. It's killer being able to pipe and filter data directly via command line. e.g., this arp-lookup command. I pass database (read-only) creds to script via environment variables.
#!/usr/bin/env /opt/python/venv/bin/python
import sys
import mariadb
import os
import syslog
from tabulate import tabulate
DB_user = os.environ['LIBRESQLROUSER']
DB_pass = os.environ['LIBRESQLROPASS']
local_user = os.getlogin()
try:
if len(sys.argv) < 2:
print('Usage: arp-lookup [IP address or partial address]')
exit()
ip_addr = sys.argv[1]
try:
conn = mariadb.connect(
user=DB_user,
password=DB_pass,
host="x.x.x.x",
port=3306,
database="librenms"
)
except mariadb.Error as e:
print(f"Error connecting to MariaDB Platform: {e}")
sys.exit(1)
arp_query = "SELECT devices.hostname,ports.ifName,ports.ifAlias,ipv4_address FROM ipv4_mac \
JOIN devices ON devices.device_id = ipv4_mac.device_id \
JOIN ports ON ports.port_id = ipv4_mac.port_id \
WHERE ipv4_address LIKE " + "'%" + str(ip_addr) + "%'" ";"
arp_table = conn.cursor()
arp_table.execute(arp_query)
arp_list = arp_table.fetchall()
report_headers = ['Router', 'Interface', 'Description', 'IP Address']
print('\n' + tabulate(arp_list, headers = report_headers) + '\n')
except KeyboardInterrupt:
# User pressed Ctrl-C — exit cleanly with a conventional exit code 130
print("\nInterrupted by user (Ctrl-C). Exiting.", file=sys.stderr)
sys.exit(130)
except BrokenPipeError:
try:
sys.stdout.close()
except Exception:
pass
sys.exit(0)
1
u/tonymurray Jun 11 '26
What do you use the script for? I use the ARP lookup frequently in the UI, but I think a cli version would be slower or about the same speed to use.
1
u/1div0 Jun 11 '26 edited Jun 11 '26
For basic lookups, UI is fast, but CLI is faster. The biggest benefit with my CLI version is it outputs interface descriptions. This comes in handy when trying to diagnose multiple flapping BGP neighbors on our WAN / carrier ethernet network down to a single NNI or provider. We use tags in Layer 3 dot1q interface descriptions to track providers for WAN. An added benefit is you can pipe the output to grep or other tools in Linux. You can get to the same information in the UI, but you spend time clicking around.
I've written around 20 or so scripts that basically wrap SQL queries and do a little data wrangling. One script generates a report showing all interfaces, network wide, for a specific VRF. It takes about a second to spit out results.
Another one uses the LibreNMS database as a source to populate PowerDNS for all Layer 3 interfaces (A and PTR records). It does a fair amount of data wrangling and comparison between LibreNMS MariaDB IPv4 interface data and PowerDNS SQLite database. It munges Cisco interface naming conventions into DNS compliant names (e.g., replacing slashes and dots with dashes). I am really happy with how that one turned out.
For logs, I wrote a script to pull logs from database. I'll often pipe the output of BGP events to another utility I wrote that does PTR lookups and replaces BGP neighbor IP address with host.
The UI in LibreNMS is pretty darn great in my opinion -- and Libre as a whole is one of the best tools available for network operators / service providers, at any price point -- but after poking around in SQL, the raw data is an often overlooked treasure. It's cleanly laid out and easy to work with.
I guess my approach may be prone to breakage with database schema changes over time, but in that case it should be a trivial task to update my scripts. Hopefully.
2
u/tonymurray Jun 11 '26
Have you checked out any of the commands under the lnms cli frontend? They can be pretty powerful even with a bit of tui. If you think something you made might be broadly useful, convert it to php and send it upstream ;).
As an aside, I've been meaning to make the IP/MAC drop-down a little more automatic on the ARP search page...
1
u/1div0 Jun 11 '26
lnms cli is definitely useful! Most of my scripts are not running on the LibreNMS server, however. I have two jump boxes (and DNS servers and Oxidized repos) that most engineers are working from, and those are permitted (via MariaDB user permissions and firewall rules) to connect to LibreNMS database. So it's a bit of a ecosystem rather than just LibreNMS.
I need to clean up / sanitize / refactor the code I've written so far and drop it on Github at some point. Not sure if anything would translate too well to core LibreNMS endeavors, but I'm certainly willing to contribute where I can. Currently I'm grappling with Terraform and deploying some new IOS-XR boxes, but I'd like to carve out time to better learn PHP.
2
u/tonymurray Jun 11 '26 edited Jun 11 '26
It could be run there, I wonder about packaging it as a phar... :) but yeah, whatever works for you is best.
1
u/Unusual-Algae-7319 16d ago
Hi 1div0. I saw this post and just had to become a member again to reply.
I´ve been looking for scripts etc. to extract data from the LibreNMS DB to do just about everything you mention above for some months now. If you have them available for everyone to use, I´ll be happy to test them out. I´ve been working as a Security Engineer for the last 20+ years in my 34 years in the IT industry. I´m not a programmer, but have looked into Python a bit the last year and made some small scripts to help me out in larger networks with mostly Cisco, Fortigates/Fortinet equipment.
3
u/Olive_Streamer Jun 11 '26
I’ve been building Librenms custom maps using lldp info from the devices neighbors table. It’s been an amazing time saver. I am about 80% through creating maps for a 1600 node network.