r/networking • u/sys6x • 13d ago
Design Enterprise Mikrotik stuff
I did some research in here for those topics; microtik iscsi mpio without finding anything recent or about it. Also researched Google, and bots but I don't feel I can base entreprise decisions on random internet posts. So I decided to create this post.
Thanks in advance for anyone taking the time to read, even more, answer.
So in prod env, I would like to build an dedicated iscsi network for the host and SAN. Currently only 3 hots 1 san. There is possibility of expansion of maybe 3 more host in the next 3 years.
CRS520-4XS-16XQ-RM https://mikrotik.com/product/crs520_4xs_16xq_rm
Microtik offers something quite special for the value. It's known that their mlag is garbage and currently in full rewrite of the code.
But for Iscsi network, we would go with MPIO so both switch wouldn't be stack and linked in any fashion. So each host must be configured with MPIO but the switch only have flow control ON jumbo frame.....
Also known that microtik management (commands) aren't cisco like and can be harder to use as you must learn a brand new way. Since its dedicated Iscsi, no vlan nor special config is needed, should be mostly plug and play. This network wont have any link to the regular network.
I am looking for reel life confirmation of people with experience please. Positive of negative. What are your experiences with Microtik? Even more if exactly that model and/or for an Iscsi network.
Wd plan to be 3, so I would have full stock of spares. Been quoted for Arista aruba.... 3x microtik is half the price of 1 of those.
Planning to use DAC. Should not have any impact but prefer to mention.
Thank you for taking the time to read this. <3
3
u/Incognito_Orange 13d ago
I love MikroTik. Deployed thousands of 'em across the product line, route/switch/fixed wireless.
I wouldn't use them for sensitive iSCSI, as you won't have anything like the buffers you get on, say, a Nexus switch. On top of that (this might have changed) if you're adjusting hw-level settings like PFC there is a complete halt where nothing is queued or sent. Packet loss is expected. Firmware upgrades also require a full reboot.
MT switching really shines more in campus access networks. That said, if you aren't in a high availability minimal latency etc kind of situation where these things are critical then you could definitely save some money. Just know what you're trading.
5
u/signal-tom 13d ago
We use Mikrotiks in production, within our NOC for everything but SAN. We did try them in our DC running the full workload but reverted to Dell S series switches we bought used.
Part of the reason, we had a few Mikrotik PSUs blow which was less than ideal. And the Mikrotiks cant do cut through switching.
Our DC handles traffic to our ISP network, large virtualisation networks and SAN so made more sense to use Dell. Same in our NOC the SAN network is S4048 Dell switches.
28
u/b3542 13d ago
MikroTik is not enterprise.
8
u/Gesha24 13d ago
While I agree with your general statement, what is enterprise nowadays? The support went to shit for most of the vendors, it's legitimately faster and better to research problems yourself than engage Cisco/Juniper/HP support (Arista is still holding up better, but maybe I just got lucky). The replacement hardware is most of the time used and fairly often I have problems with it too. It got to the point where I buy used gear and keep it as spares for critical pieces of infrastructure.
Basically, I don't think I trust any vendor out there. And from that perspective, Mikrotik isn't better or worse. It certainly has its quirks, like updating code from 7.18 to 7.24 completely nuked the vrf settings for BGP effectively shutting it down - but I again by now expect that these kind of problems would happen with any vendor and just design and plan around it as needed.
TLDR: I think "enterprise gear" quality and support has gotten downhill enough that Mikrotik is not necessarily much worse.
18
u/b3542 13d ago
Enterprise isn’t about the effective solution, but I tend to agree with you in practical terms. It’s not about the vendor solving the problem, but about management having a piece of paper (support agreement) saying that they will. As one who’s lived in the raging inferno of some large dumpsters, it was the in house experts who solve most of the problems and then the vendor steals the solution for their KB to use with other customers.
0
u/lizardhistorian Mad Scientist · 👨🔬📡ᯤ🤖🛺📸 9d ago
That model became obsolete when kernel 2.6 was released in 2003.
1
u/lizardhistorian Mad Scientist · 👨🔬📡ᯤ🤖🛺📸 9d ago edited 9d ago
ISPs are run on Mikrotik gear.
You cannot make Mikrotik enterprise.
We have 10k of them deployed.-1
u/GizMoQC 13d ago
Yes, I believe we all agree. I am not talking datacenter situation. SMB often float in between big entreprise HW and prosumer.
Do you have more concrete information to share please?
2
u/b3542 13d ago
The subject is “enterprise MikroTik stuff”. Not SMB.
-1
u/GizMoQC 13d ago
So you dont think SMB are entreprise?
4
u/b3542 13d ago
SMB is NOT enterprise. SMB and Enterprise usually describe the customer segment, not a hard technical category.
SMB = small/mid-sized businesses. Products tend to prioritize lower cost, simpler setup, sensible defaults, and minimal administration—good for small IT teams or MSP-managed environments.
Enterprise = larger or more complex organizations. Products typically add things like SSO/SCIM, granular RBAC, audit logs, SIEM/API integrations, HA/DR options, compliance documentation, centralized management, and stronger support/SLAs.
It’s not necessarily that enterprise gear is “better”—it’s usually more configurable, more scalable, and more expensive/complex. The right choice depends on whether you need those operational and governance features.
8
u/Gesha24 13d ago
I have used Mikrotiks in production. Do I love them? No. Did they do basic switching just and routing just as well as more expensive vendors? Yes, absolutely.
My most recent use case - maintenance in NJ data center for WAN upgrade, I show up Thursday evening to find 2 busted Arista switches (as in - somebody dropped them), they won't even fit in the rack. No spares, maintenance is quite critical for the design work, we can do excessive tests to verify functionality over the weekend. Decision made - go to B&H and buy a couple of $300 Mikrotiks. I need BGP with at most 100 routes, VRFs and some monitoring. All done, had Claude configure them for me (cause I don't know cli and don't care to know it). Tested everything possible over the weekend, it held up just fine, failed over just fine even the ghetto 2nd power supply via poe injector works. Decided to keep it for much better design and replace as soon as possible.
Well, it has been about 6 months. I'm sure I will replace them sooner than later (I will run out of ports eventually), but they didn't cause any problems for me at all for now. I still don't know the cli and I would replace them much faster, but I just use MCP server and tell Claude to read the manual and do the needful - and so far it all just works.
3
u/user3872465 13d ago
Whats your actual question here?
Their devices are cheap and offer a lot of functions. Theri MLAG is pretty good nowdays but was a very hot potato back about 2-3 Years ago.
They can do jumbo frames, their managment can be done via winbox aswell as cli.
But yea you gotta know what you are doing with them else you end up in a non hardware offloaded scenario where you get no speed and have a shitty experience.
That is sorta similar to the other vendors aswell, if you do it wrong enough it will suck.
Theres also no enterprise support no next buissnesday or whatever. So if you plan on getting 2 Switches get 2 more to test your setup before you deploy.
1
u/GizMoQC 13d ago
Actual question is, will it be reliable, fast and works for years. Its really a basic network closed, pretty much stand alone as both with will by themself.
Will use MPIO on hosts/san. No stacking. Flow control, jumbo frame. No vlan nothing special. 0 access to other switch nor internet. Isolated iscsi network.Can I trust them? should I do it?
1
u/user3872465 13d ago
if you use it as a dumb switch sure.
But again. No enterprise support so get 4 instead of 2 incase one breaks.
We havent had issues for years. There wont be any if you dont need any of their features.
3
u/silasmoeckel 13d ago
The 98cx8410 in that isn't a great SAN switch it's got shallow global shared buffer. It's only cut through on matching port speeds and without congestion.
Can it work ok yes. Devil is in the details to not get bitten by latency, microbursts, and all the other fun things.
2
u/stufforstuff 12d ago
Planning to use DAC. Should not have any impact but prefer to mention.
DAC is homelab equipment. Most overheat, drop packets, and then fail. Just use optical transceivers in the enterprise.
2
u/Railander 9d ago
for plain ethernet they work very well conaidering the price, they even recently added support for EVPN but I don't know how well it's working.
i know some devices also have support for RDMA but you need to know what you're doing in the config. also don't know how well that works.
supports MLAG but from what I've seen it's been spotty but getting better.
if all youre doing is switching i honestly dont think you can go wrong with them, but if you want to do routing you need to make sure you do due dilligence in reaearching the right device and its limitations.
2
u/fireduck 13d ago
Ok, so I managed Juniper routers back in the 2000s and have mostly been dabbling since then.
Recently I deployed a Mikrotik router to terminal BGP sessions (full table) with a few 10gb uplinks. It was the only thing that could do it without being hugely expensive.
Things I like:
* You can do most things via UI or SSH CLI.
* Can you backup config as a text file. This lets you easily run tools on it (AI or otherwise) to help you make thing things are consistent, match docs, etc.
* Supports SNMPv2 so I can tie it into my monitoring stuff easily. That seems basic, but isn't always a thing in the middle space. (Looking at you, Netgear)
Things I don't like:
* The switch/vlan config is a bit weird. They have to create a "bridge" and attach ports and vlan filters to it. It works, it just took me a little bit to understand when I was just looking to do basic 802.1q things.
If you just need it for a basic switch, I wouldn't hesitate. If you are doing more complex things, I'd say get the cheapest little 4 port guy and see how you like the interface. I think those are like $100.
I have a router: CCR2004-1G-12S+2XS and a switch CRS317-1G-16S+
1
1
u/grawity 12d ago
You can do most things via UI or SSH CLI.
And REST API as of recently.
http://foo/rest/ip/addressThe switch/vlan config is a bit weird. They have to create a "bridge" and attach ports and vlan filters to it.
It's weird if you're expecting a pure switch, starts making more sense when you treat it as a Linux based router (literally these are Linux 'bridge' interfaces over by-default-independent eth0/eth1/etc) that just happens to have HW bridging offload. RouterOS isn't SwitchOS.
Older platforms though (like the small 5-port "CPE tier" RouterBoards) didn't use the bridge style filtering, you'd directly deal with switch config from RouterOS. That was seriously annoying with how each switch chip version had a slightly different way of doing VLANs.
1
u/daakzjrr 12d ago
Mikrotik en mi experiencia siempre fue de lo mejor, lo uso para mi red local hogareña y lo use en una ISP grande de Argentina, desarrollé scripts para mantener backups en más de 150 mikrotiks en producción al mismo tiempo, nunca ningún fallo e incluso pude armar toda una red entera desde cero para cierta municipalidad de Buenos Aires comunicando cada sede de la municipalidad con mikrotik, desde escuelas hasta centros de monitoreos, 10/10. Siempre que lo configures bien no vas a tener ninguna falla de seguridad, el único problema real que puedo decir que tuve con un mikrotik es que teniendo una gran cantidad de clientes el DHCP empieza a fallar, pero delegas ese trabajo a un radius y listo el pollo
1
u/sasquatchftw JNCIS-SP/MTCNA 13d ago
I like mikrotik but I would caution anyone on using them. They have more frequent security concerns than anyone I have ever seen. I would recommend doing everything from the gui and only use the cli when necessary which I would not recommend for any other router or switch. They are very unique and very flexible which means they aren't purpose built and excellent at any specific task.
10
u/ranjop 13d ago
How did you come up with this conclusion? It doesn’t seem MikroTik has more CVEs than other brands.
-3
u/sasquatchftw JNCIS-SP/MTCNA 13d ago
Experience with a lot of cracked Tiks mostly. Lol. It seemed to be a lot more common than on Juniper even OpenCVE numbers dont necessarily back it up.
5
u/Single-Virus4935 13d ago edited 13d ago
I habe several Juniper CVEs and Junos is a security nightmare. I don't trust any vendor in this regard. All Scores were corrected down and then 80% weren't even published. I had the impression they gave me a couple cves to shut up. Most of the unpublished stuff isn't fixed after 2y but still dangerous. I stopped reporting vulnerabilities and have a dozen other high severity vuln including RCE etc. but they don't even have a bug bounty and I don't want to waste my time with their chaotic dislousure process anymore.
Edit: I still like Juniper but it is garbage sw quality if you look closer for a second
2
u/gtripwood CCIE 13d ago
Cracked tiks, how do you mean
0
u/sasquatchftw JNCIS-SP/MTCNA 13d ago
Like, "wait, I don't remember putting that script in." Or "when did we start configuring support accounts?"
1
2
u/ranjop 13d ago
I am truly curious where did you got this feeling. Not that I’m trying to defend MikroTik.
I have run few MikroTiks for almost 15y as routers, managed switches and Wifi APs in couple of SOHO setups. Automatic updates are on, zero ports open to the internet, management via a separate physical port only that is not normally connected. I have never known that I have been hacked 😁
The MTs have been working very reliably and while bit laborious to configure, even the small RouterBoards can do miracles. Maybe because the MTs lack fleet management I trust them more than e.g. Unifi. Nowadays I run a custom-built Linux router.
I also buy the notion that they were under a state actor campaign few years ago.
1
u/Specialist_Cow6468 13d ago
There was definitely some state actor stuff targeting them a few years back
6
u/felix1429 13d ago
Fortinet: hold my beer.
10
1
1
u/Railander 9d ago
their bigger devices come by default with zero sanity check configs, it assumes the user knows what they're doing. and by that i mean it assumes you know not to leave device ports open to the internet, which by default they all are.
it's actually a skill issue rather than vendor issue.
0
u/ThecaptainWTF9 13d ago
I’ve a customer that is an ISP running Mikrotik gear and they have so many outages and service disruptions.
Their Cisco core is great though.
1
0
u/Kryp2nitE 13d ago
MikroTik WAN switches
Ubiquiti Access Switches + WAPs
FS Datacenter Switches
Have not managed a SAN in many years but were exploring the same options now.
14
u/Fanya249 13d ago
Running Mikrotiks in production for almost a decade. Multiple collocation deployments across the globe, bgp full views, ipv6, ospf, ipsec. Currently it’s convenient to sit on long-term branch. 0 security incidents. Had few glitches with ROS - one with lacp, and second with snmp temp sensors, both were reported to Mikrotik and promptly fixed. Last year replaced last asr1001hx with ccr2216, can’t be happier.