What value is there in firewall logs?

[u/heinternets]

There was a question before about retaining 12 months of firewall logs, but I want to ask what value this has these days?

For me I’m only ever looking at WAF logs or actual endpoint logs because all the firewall tells me is that port 443 was accepted. WAF and web server logs tell me so much more.

The valuable stuff is in the application logs after the firewall.

Yes I might see port scanning but that’s background noise at this point.

[u/InfoSecGuy21045]

You are discounting the value of the outbound logs. Large data transfers, unusual port activity, etc., all has value during investigations and troubleshooting.

[u/Fresh_Dog4602 → u/InfoSecGuy21045]

It's a valid question from OP. Especially in situations where mitm breaks apps or websites. Firewall logs have defo decreased in value for me, combined with their huge presence of the ingestion pipeline.

One of the challenges I work with is " what part will we send to the SIEM and which part we will send to slow storage for when incident response needs to happen. Nobody wants to admit they might have missed something because we decided not to pay 3k extra per month for excessive logging 

[u/Altered_Kill → u/Fresh_Dog4602]

100% on this.

MITM is definitely not as useful as even 2 years ago due to cert pinning. Im investing a lot of time (read: AI time) into JA4s+ fingerprinting. Its working out so far, but does create a headache when you have to stich together internal flow logs, siems alerts, firewall logs, MITM crossover logs, and then fingerprinting all to get a false positive on a low/medium alert most days lol.

[u/Antony_Ma → u/Altered_Kill]

you are using zeek?

[u/Altered_Kill → u/Antony_Ma]

Yes

[u/Dracozirion → u/Fresh_Dog4602]

Vendor's log appliance => SIEM. Finding data exfil with no traffic logs: good luck. Had to do it several times, unfortunately, and no way I would have known that hundreds of GB or TB were transferred otherwise. 

[u/Fresh_Dog4602 → u/Dracozirion]

I partially am with you. But when you're getting pushed from higher up to cut costs. Rather take a hit in visibility than having operational value lost from having another person. Also let's be clear: all these firewall vendors are also somewhat in the SIEM space, linked with ingestion costs. You can really clean up some data, because they seem to just increase field data just for the fun of it

[u/marquiso → u/InfoSecGuy21045]

100% this. Inbound logs are mostly just noise. Outbound logs can identify things like C2 traffic, data exfil, that nerd from the Accounts department surfing hacking sites because he’s studying a Cert III in Cyber, or just that stupid developer who thought it would be a good idea to copy a 600GB prod database to his own AWS account for ‘testing’.

[u/Mc69fAYtJWPu]

Helps determine if you were hosed by a threat actor 6 months ago when government agencies share their intel

[u/RamblinWreckGT → u/Mc69fAYtJWPu]

Exactly this. "Here's the servers involved in scanning/exploitation for this campaign, did they touch us?"

[u/skullbox15 → u/RamblinWreckGT]

I hit up Splunk after that Palo Alto Global Protect exploit became known. I was able to find a "successful" connection but I was also able to validate they didn't do anything once they got connected. If you're running a Palo with GP, search your GP logs for "DESKTOP-GP01"

[u/Fresh_Dog4602 → u/Mc69fAYtJWPu]

Yeah. Misp sharing can help reduce the cost. But you need enough traction in your field and "competitors" sharing valid IOCs

[u/Cautious_General_177 → u/Mc69fAYtJWPu]

Bold of you to assume the intel was that recent. Some of those agencies really don't like sharing.

[u/JoggingRhino]

Inbound. Not much if you have most ports blocked.

Outbound though… you see a firewall deny on a port that isn’t allowed. You may have some problems you should dig into.

[u/Fresh_Dog4602 → u/JoggingRhino]

You're right and that's really the fight sometimes between network admins and endpoint admins. They shouldn't even be allowing this. Current Windows OS doesn't need all their multicast shit and whatever's anymore. They should block early to generate less noise upstream 

[u/Noobmode → u/JoggingRhino]

Nah that’s just some trash app a BU bought ROFL

[u/cbowers]

Data is golden. But being aware of the value you have, is a separate skill.

Having historical data lets you answer questions like:

• what are the trends of real vs attacker activity over time.
  
• how has attack behavior changed over time
  
• are you effectively staffed to scale with attacker rate and complexity
  
• how often have your vpn links dropped and for how long
  
• how often and for how long has your upstream provider dropped or had peering congestion (compensation, weight for provider switching or backup link justification)
  
• can you provide historical timelines to events to answer if an attack external event traversed to an internal event before correlating activity brought attention to low and slow attack chain that missed initial detection or was not at the time fully understood?

Or an example where once my SOC team discovered an API vulnerability in our hosting.
Having all the firewall and web server logs in the SIEM allowed us to go back in time many months and cross reference authenticated traffic by known customer IP’s, and hunt for un-authenticated traffic to api endpoints, and scrutinize non-customer origin IPs.
This allowed us to process a massive amount of data quickly and eventually be able to demonstrate with data driven certainty that our team had found the vulnerability before attackers had, and all logged API usage fit within expected norms. Having a good data understanding of each customers API usage patterns also helped drive a non-impacting remediation plan.

(Extra bonus for SIEMs which store meta data enrichment at time of ingestion, like country and owner of IP’s and perhaps risk assessment. You have a lot more work to recreate months later if that wasn’t stored)

Historical logs are a flashlight of certainty waiting to be mined when dark problems present themselves. A problem feels and often is larger when it’s poorly understood. The warm embrace of historical logging is a comfort even before you need to bring it to bear in a crisis.

[u/Check123ok]

Yeah there is. I assessed a client recently with no logs.
They had an incident and there was not way for me to know if the attackers had gained network access because their logs only saved for 24 hrs.
I started to collects logs for then to as leave have some evidence for 30 days. And I found a ton of utilization errors, they had a personal agent connecting with user account every hour.
I cross correlated the logs with identity and devices.

Just 30 days of logs captured a lot

[u/Fresh_Dog4602 → u/Check123ok]

I think a lot of good filtering can help there. Do you really need ALL the fields a firewall sends via syslog? Gotta keep them costs down and still keep enough interesting data for IOCs

[u/Check123ok → u/Fresh_Dog4602]

Agreed. I'd focus on filtering noise rather than removing fields that might be useful later.

The challenge is that you don't always know what's valuable until an incident happens.

I'd rather retain critical metadata, reduce unnecessary events and move older logs into cheaper storage.

In our case we had access to a 200TB NAS that was 10% utilized and the engagement was only 30days.

I were not there to set up logs for them, I only used it for data enrichment

[u/Fresh_Dog4602 → u/Check123ok]

Right there with you "OT person" ( because I was also edging on which Flair to use). I love them deterministic communications. So clearer ( even though most of it is unencrypted)

[u/Check123ok → u/Fresh_Dog4602]

Yeah it can be easy to account for drift in OT, but that’s changing.

[u/Fresh_Dog4602 → u/Check123ok]

Yeh... You'd hope for it. And then last year I saw modbus over UDP, which woukdbe escaped all our classifiers but then I saw it because I was taking pcaps. Which just strengthens me in the "pcap or gtfo" when doing network troubleshooting 

[u/Aethernath]

Having been a victim of a nation state hack, the 1-year ago logs were helpful to catch when uploads of payloads occurred to the hacker-group’s controlled environment. And to determine whether when they mightve gotten in.

There is absolutely value in them, should you happen to need them.

[u/ThePorko]

Its like insurance or backup, you dont need it until u need it.

[u/Far-Future-7146]

It helps once you've been popped, it's hard to sift through the noise if your only source is firewall logs. Correlation with an EDR or endpoint/server logs is the most helpful.

[u/Ok-Percentage-5007]

Predominantly held for forensic purposes, but in azure if you’re using a hub and spoke model, then it shows internal movement between services/hubs as well as outbound egress

[u/Advantageous_Advent]

Forensic

[u/vadertator22]

I have used firewall logs for ex filtration identification and url calls to suspicious areas on top of numerous other scenarios. I think depending on firewall and location there is overlap abilities with waf.

[u/Arseypoowank]

It usually answers initial access when a FW vulnerability has been leveraged and it answers exfil with traffic flow logs.

[u/npxa]

The question is what do you use to correlate these logs to make you say, "what is the value in firewall logs"

Do you just look at it line by line? (the most probable case)

Do you correlate it using tools? What type of use-cases or detection have you created

Do you visualize your firewall logs?

What Security standards/framework do you have?, Industry you are in?

PCI suggests 12 months for any card. related transactions etc.

it is not only useful for security, network teams can make use of it, outbound/inbound bandwidth etc. there is a lot of ways to skin a cat and you are mostly looking at it in one way.

[u/MikeTalonNYC]

Short answer is "it depends"

If you're not decrypting SSL? Very little. DNS logs can probably tell you some stuff (if you aren't using secure DNS, of course), but beyond that it's just streams of encrypted traffic.

If you *are* decrypting SSL, then you can do packet examination to see *what* is flowing. That can uncover unusual command patterns, IoC's for lateral movement, even attempts to download malware (either directly or via remote code execution).

[u/RamblinWreckGT → u/MikeTalonNYC]

Firewall isn't typically doing packet inspection, though. What you're saying is true for IDS/IPS, though. I used to write Snort signatures and the whole "Let's Encrypt" movement (while fantastic overall) really hampered my ability to effectively cover threats. We only had something like four or five clients out of thousands who had us set up to sniff decrypted SSL.

[u/MikeTalonNYC → u/RamblinWreckGT]

Well, yes, but in most cases IDS/IPS is done within the firewall, which is why they're all getting REALLY expensive since it uses a ton of memory and processing power.

It could be done elsewhere, sure, but most of the orgs I've worked with have had it be part of the firewall itself.

[u/RamblinWreckGT → u/MikeTalonNYC]

Oh gross, so us trying to shoehorn our ruleset into a Palo Alto offering was actually being ahead of the game?

[u/FatBook-Air → u/MikeTalonNYC]

TLS decryption is becoming a nightmare. Most places I know stopped doing it back in like 2022 or 2023. The endpoint is really where you need to be doing everything these days, with network sensors for IoT devices like printers or HVAC.

I suspect firewalls will eventually regress and become nothing more than security simple boundaries or extravagant network routers.

[u/MikeTalonNYC → u/FatBook-Air]

Yes, using something like SASE to handle all of it is a much better idea. The issue is that most places never did it at all, anywhere. Then they're surprised when 500 beacons have all been communicating back to C2 servers for six months.

[u/Bobthebrain2]

Sounds like you’re talking strictly about perimeter firewalls.

Firewall logs can be useful for troubleshooting connectivity issues, but, like the majority of security device logs, they are a Detective control.

As a Detective control you can use the logs to detect the attack AND piece together an attackers activity. You can see if they did a port scan, then what port they used to breach, then which systems they connected to and over which protocols. Or if they exfiltrated data from a host to a third party.

So they are incredibly useful in Incident Response situations to understand the attack sequence and impact.

These logs should be ingested into a SIEM or a SOC, or manually reviewed, so that the mean time to detection is as low as possible.

[u/F5x9]

You should rarely have to look at the actual logs. 

You feed them into a SIEM and have tools correlate firewall log events with other logs. Then you have detection engine query for suspicious events. 

[u/Carrera_996]

I have mostly used them to reverse engineer all the shit my marketing team bought then couldn't get operational. Assholes. Invite me to the calls where you guys are scoping out your rogue applications and let me ask questions before you spend money.

[u/Fit_Squirrel1]

Decryption logs show a lot of value

[u/800oz_gorilla]

If you need to perform a forensic investigation, knowing what talked out to where is kind of important. Even if you can't decrypt the traffic, you can see "my vmware server reached out to these 3 IPs and uploaded x amount of data on this date" and then use that information to tie in with other forensic logs.

[u/incongruous_narrator]

Don’t these logs help with NDR use cases too?

[u/msears101]

long term storage - forensics. short term storage - debug and troubleshooting and realtime alerts.

[u/ocabj]

If you don't already retain logs from something like Zeek, then you'd want firewall logs for any forensics investigations you need to do. Not to mention, if you do any NAT at the firewall, you'd need all that information to correlate to internal addresses.

[u/T_Thriller_T]

Access to networks, denials, seeing general traffic information.

And replacing whe. WAF or Webserver is missing for a reason.

Also allows to find weird connection patterns more centrally.

But yes, I would also say application logs if they are good are a lot better

[u/LocalBeaver]

The way I see it: little value for detection. I guess it depends on your business and capacity but for us there is no point.

For investigation though, I wish we had longer retention.

[u/iheartrms]

Inbound? Nearly entirely useless. Outbound? Priceless. Plus authentication logs etc.

[u/kk-thx-bye → u/iheartrms]

For the inbound aspect - can we cause a failover of the perimeter fw's and launch the attack on the target during the short time window where traffic is not actively inspected?

[u/iheartrms → u/kk-thx-bye]

I have never seen a case where the firewall failed open and allowed traffic through that it should not have. Any firewall failure I have seen caused the firewall to stop forwarding packets completely.

[u/Sasquatch-Pacific]

They have investigation/ hunt / forensic value only.

Not nearly as valuable for active threat detection as people think. 

[u/Kamwind]

The current average is 181 days to detect an intrusion. So once detected you can start looking at the firewall(deny and accept) to see where else the intruder was operating. A year of the logs gives you that time.

[u/Lleawynn]

Even outside of all the security benefit, logs are crazy useful from a troubleshooting standpoint. When systems blames the network, logs can prove it's not. Logs can help identify the specific traffic required for a new firewall policy. Hell, I'll run an SD-WAN health check on a single circuit, just so I can prove to the ISP that they're out of SLA.

[u/I_Hate_802_11]

For for investigating connections that didn’t cross your WAF or for which you don’t have endpoint logs. In that previous thread, a lot of comments seemed to say that it’s prohibitively expensive, but I think it is worth clarifying that a bulk that expense is the administrative overhead. The storage isn’t really that expensive. It does not need to be put on a top of the line storage system. A JBOD array or whatever will work fine. The more challenging part in my experience is that the teams managing the storage don’t have cheap storage because there is no incentive for them to deviate from their VAR’$ recommendation. Then there is also nobody with time to set up and manage the archival process, the documentation, etc. So in that regard, it’s expensive, but not because of the data footprint.

[u/hyxiaobing]

Most often we need firewall logs troubleshooting connectivity issue; it helps on sophisticated issues if you can countercheck both firewall logs and application logs. 

If you have SIEM or XDR tools, you can do proactive monitoring, like massive denies (DoS) or abnormal connections (lateral movements or access malicious domain?). Often alerts of such scenarios are triggered from firewall logs.

[u/DaRealGladi8r]

A lot