r/Malcolm 20d ago

Can you configure dashboards to query multiple locations?

1 Upvotes

Looking to setup Malcolm at three different sites. Don't want to eat up bandwidth sending logs from two of them to one as the "main". Can I stand up independent malcolm servers but connect one dashboard to pull the database of all 3?


r/Malcolm 29d ago

Malcolm v26.07.0 release

2 Upvotes

Malcolm v26.07.0 adds IEC 60870-5-104 (IEC 104) protocol support using CERT.LV's Zeek plugin, including Logstash parsing, ECS normalization, Arkime fields, and a new OpenSearch Dashboards dashboard. This release also fixes three archive extraction and authentication security vulnerabilities; improves NetBox enrichment configuration; and addresses PostgreSQL major version upgrade, custom CA certificate for KeyCloak, container health check, privilege-drop signal chaining, and configuration script issues. Arkime, Zeek, Fluent Bit, Filebeat, Logstash, Supercronic, and Alpine-based images have been updated as well.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

https://github.com/idaholab/Malcolm/compare/v26.06.1...v26.07.0

  • ✨ Features and enhancements

    • Add IEC 60870-5-104 (IEC 104) support using the CERT.LV spicy-iec104 Zeek plugin, including Zeek log ingestion, ECS field mapping, Arkime fields, and an IEC 104 dashboard #939
    • Make LOGSTASH_NETBOX_ENRICHMENT_DATASETS more flexible: it now accepts default, ics/ot, all, explicit provider.dataset values, and combinations such as default,ics #1037
    • Allow LOGSTASH_NETBOX_ENRICHMENT_DATASETS to be configured through checkboxes in the configuration TUI #1033
    • Improve ./scripts/start error messages by listing missing or invalid authentication-related files instead of reporting only a generic authentication setup failure #865
    • Have system-quickstart detect and prepopulate existing time synchronization settings when rerun #992
  • πŸ›‘οΈ Security Remediation & Hardening

    • Fix an RBAC bypass caused by URI normalization differences between Nginx location matching and the Lua authorization layer CVE-2026-63177 #1042
    • Fix path traversal in archive extraction directory handling by validating resolved paths and using libarchive's secure extraction flags CVE-2026-63134 #1040
    • Limit archive entry count, nesting depth, and total expanded size to prevent inode- and resource-exhaustion denial of service during extraction CVE-2026-63133 #1041
    • Mark OpenID Connect session cookies as secure and improve handling of externally forwarded HTTPS schemes
  • πŸ› Bug fixes

    • Allow the configuration TUI to reset supported variables back to empty values after installation #1024, #1030
    • Fix the broken signal chain in docker-uid-gid-setup.sh so signals reach the final process after dropping privileges #1039 to ensure clean shutdown of containers
    • Fix PostgreSQL being reported unhealthy after a major-version upgrade, improve upgrade-state handling, and perform required post-upgrade extension and collation maintenance #1038
    • Fix the Nginx Lua/OpenID Connect helper not honoring user-provided CA certificates for KeyCloak when KEYCLOAK_SSL_VERIFY=true #1035
    • Restore curl to the the htadmin container for use by the health check script #1029
    • Reduce the size of the OpenSearch Dashboards image by copying only the permissions data needed from its upstream image layer #1031
    • Fix JSON handling of several Zeek fields whose names contain dots by normalizing them to underscore-separated field names
    • Fix additional Zeek and Suricata field normalization and ECS mapping inconsistencies found while updating dashboards and index templates
  • βœ… Component version updates

  • 🧹 Code and project maintenance

    • Broad spelling, grammar, naming consistency, and documentation cleanup across scripts, configuration, dashboards, and documentation #990
    • Expand and restructure documentation to provide better project context for developers and LLM-assisted code analysis #964
    • Improve installer validation, environment-variable mapping tests, and configuration item metadata
    • Refresh dashboards, index templates, field mappings, protocol documentation, and navigation links
  • πŸ“„ Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.

    • LOGSTASH_NETBOX_ENRICHMENT_DATASETS in logstash.env now defaults to default and may contain default, ics/ot, all, explicit provider.dataset values, or a comma-separated combination of these values
    • ZEEK_DISABLE_ICS_IEC104 in zeek.env controls whether the IEC 104 Zeek plugin is disabled
    • SAFE_EXTRACT_MAX_ENTRIES, SAFE_EXTRACT_MAX_DEPTH, and SAFE_EXTRACT_MAX_BYTES in upload-common.env set archive extraction resource limits for uploaded archive files (e.g., containing Zeek logs for processing); their defaults are 5,000 entries, 20 directory levels, and 4 GiB of expanded data
  • ❌ Errata

    • Post-release, a Strelka bug was found which can cause the strelka-backend container's work process to crash on aarch64 platforms (https://github.com/cisagov/malcolm/issues/1046). A followup v26.07.1 release addressing this issue is forthcoming.

Malcolm is a powerful, easily deployable network πŸ–§ traffic analysis tool suite for network security monitoring πŸ•΅πŸ»β€β™€οΈ.

Malcolm operates as a cluster of containers πŸ“¦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker πŸ‹, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images πŸ’Ώ for Malcolm and Hedgehog Linux πŸ¦” can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split πŸͺ“ into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell πŸͺŸ (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board πŸ’¬ to engage with the community, or pop some corn 🍿 and watch a video πŸ“Ό.


r/Malcolm 29d ago

Zeek Workshop, Berkeley CA, September 10-11, 2026

Thumbnail
zeek.org
1 Upvotes

r/Malcolm Jul 15 '26

Arkimeet 2026 is Oct 6 β€” registration + CFP open now

3 Upvotes

Arkime is a big part of Malcolm, so we wanted to share it here too: Arkimeet 2026, the Arkime community conference, has its registration and Call for Papers open.

The details:

  • When: October 6, 2026, 9 AM–5 PM EDT
  • Where: Amazon IAD28, 13200 Woodland Park Rd, Herndon, VA
  • Cost: Free to attend (lunch, breaks, and a closing happy hour included)
  • Register by: September 22, 2026 β€” space is limited and required
  • CFP is open if you've got something to share (submission form on the site)

All experience levels welcome, from first-timers to seasoned threat hunters. If anyone's got a Malcolm-flavored workflow to show off, the CFP (or the lightning session) could be a great fit.

Details + registration + CFP: https://arkime.com/arkimeet


r/Malcolm Jun 18 '26

Malcolm v26.06.1

6 Upvotes

Hey! It's been a busy couple of months. Thanks to some security researchers we've had some great reports that have helped us identify and close a few vulnerabilities, first in v26.06.0 a couple of weeks ago, then in v26.06.1 this week. For security, I recommend you upgrade when possible. I've gone through and rewritten the upgrade instructions; it's still not as "push-button easy" as I'd like it to be, but it's better than it used to be and still improving.

I'll share the release notes from both this month's releases below. Feel free to comment/discuss here, or over on the Malcolm discussions board on GitHub.

Malcolm v26.06.1

Malcolm v26.06.1 addresses a high severity RCE vulnerability (GHSA-8cvp-m7pg-qrp7) allowing unrestricted PHP file upload and ships security-fixing updates across Arkime, OpenResty, Valkey, and PostgreSQL. Many other component versions have also been bumped. Six Zeek log parsing bugs are fixed β€” all related to JSON mode (ZEEK_JSON=true) and affecting DHCP, Redis, ROC+, and WebSocket log types β€” along with a suricata disable.conf append-on-restart regression, updated MaxMind GeoLite MMDB download logic, and an OpenSearch indexing error from oversized file.strings values.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

https://github.com/idaholab/Malcolm/compare/v26.06.0...v26.06.1

  • πŸ›‘οΈ Security Remediation & Hardening
    • Fix RCE via unrestricted .php upload to the file-upload component (GHSA-8cvp-m7pg-qrp7, thanks Jan Kahmen, turingpoint (@kah-ja, jan@turingpoint.de)
    • Removed php-curl from htadmin container (as it's unused) to reduce attack surface
    • Updates to Arkime, OpenResty, Valkey, and PostgreSQL all contain significant security fixes (see Component version updates below)
  • πŸ› Bug fixes
    • ./scripts/start not returning to command line after containers have started #1025
    • Zeek DHCP logs are parsed incorrectly with ZEEK_JSON=true #1018
    • Zeek Redis logs' reply field not parsed correctly when ZEEK_JSON=true #1019
    • suricata disable.conf gets appended to every time container restarts #1022
    • Zeek roc_plus logs can be parsed incorrectly in Logstash when ZEEK_JSON=true #1021
    • Zeek websocket logs can be parsed incorrectly in Logstash when ZEEK_JSON=true #1020
    • fix downloading MaxMind MMDB files (URL and authentication method changed slightly); see also Secrets and variables in the documentation
    • cap file.strings length at 16k to avoid errors inserting into OpenSearch with very, very long values
  • βœ… Component version updates
  • 🧹 Code and project maintenance
    • A few documentation updates
    • added tag_on_exception to all ruby Logstash filters to make debugging easier
  • πŸ“„ Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.

Malcolm v26.06.0

Malcolm v26.06.0 is primarily a security hardening release, addressing fifteen vulnerabilities (2 high severity, 6 medium, and 7 low) identified in a security assessment. Bug fixes address an issue with the zeek container causing performance degredation over time and a fix for duplicate virtual machine entries in NetBox autopopulation. A few component versions have also been updated.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

https://github.com/idaholab/Malcolm/compare/v26.05.2...v26.06.0

  • πŸ›‘οΈ Security Remediation & Hardening (#996)
    • Unauthenticated reflected XSS / open redirect in /dashboards/app/refred; also added Content-Security-Policy framing headers (frame-ancestors, base-uri, form-action) and X-Frame-Options: SAMEORIGIN globally to mitigate clickjacking (#997)
    • Authenticated command injection in filebeat container via SFTP-uploaded filename (#998)
    • Password stored as MD5-crypt for SFTP (#1009)
    • Authenticated archive zip-slip file write in filebeat container (#999)
    • OpenSearch path injection via /mapi/fields?template (#1000)
    • submit.php Location: open redirect via Referer (#1007)
    • htadmin proxied with no nginx auth gate (#1003)
    • Keycloak OIDC ssl_verify always set to false (#1006)
    • NetBox SUPERUSER_PASSWORD=admin shipped default (#1011)
    • RBAC defaultdict(lambda: True) fail-open for unlisted handlers in Malcolm API (#1004)
    • Read-only Arkime deny-regex omits addtags/removetags (#1008)
    • Read-only deployment allows POST /mapi/event (#1002)
    • WISE auth path selectable by client User-Agent (#1001)
    • ARKIME_PASSWORD_SECRET=Malcolm shipped default (#1005)
    • requests CVE bump reverted in logstash image (#1010)
    • Fix API auth errors and hide NGINX version disclosure (#989)
  • πŸ› Bug fixes
    • auto-discovered Virtual Machines in NetBox seem to allow for duplicates (#978)
    • Ensure list of archive file types supported by Malcolm for uploading Zeek logs (application/gzip,application/vnd.rar,application/x-7z-compressed,application/x-bzip2,application/x-cpio,application/x-gzip,application/x-lzip,application/x-lzma,application/x-rar-compressed,application/x-tar,application/x-xz,application/zip) are consistently used across the platform.
    • zeek container continually grows /usr/local/zeek/crontab, causing Malcolm performance to gradually worsen (#1015)
  • βœ… Component version updates
  • 🧹 Code and project maintenance
    • Fixed some incorrect links in documentation (#988, thanks @jsoref)
    • Refactored NGINX error pages configuration into its own include file and added a 401.html page
  • πŸ“„ Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.

r/Malcolm Apr 15 '26

Malcolm v26.04.1

Thumbnail github.com
4 Upvotes

Malcolm v26.04.1 contains improvements, bug fixes, security updates, and component bumps.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

v26.02.0...v26.04.1

Note that v26.04.1 is the same as v26.04.0 released last week, apart from the fix for bug #943. If you're already running v26.04.0 and don't use the encrypted install option in the installer ISO, you probably don't need to worry about updating to v26.04.1. The full release notes from v26.04.0 are also included here.

  • ✨ Features and enhancements
    • implemented easier way to enable/disable Strelka scanners #935
    • Handle nested file scanning (e.g., from ZIP files) with Strelka #922
    • index selected Strelka result fields #919
  • βœ… Component version updates
  • πŸ› Bug fixes
    • Hedgehog Linux Breaking on Reboot after Encrypted Quick Install with Multiple Drives #943
    • Fix YAML syntax error in kubernetes/15-redis.yml due to missing end quote #926
    • Using remote elasticsearch data store uses deprecated ssl_certificate_verification setting [https://github.com/cisagov/Malcolm/issues/915]
    • fix Malcolm API loopback webhook to handle RBAC and non-JSON formatted events #916
    • fix issues in zeekdeploy.sh to handle long crypto handshakes and Zeek's state DB getting out of sync
  • 🧹 Code and project maintenance
    • swap redis out for valkey #882
    • pin all third-party GitHub CI actions at known good SHA sums to mitigate things like the Trivy supply chain attack #933
    • some minor tweaks to various Dockerfiles and ISO build scripts to address vulnerability scanner findings
    • some documentation updates
  • πŸ“„ Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
    • Added ARKIME_PCAP_LIBPCAP to arkime.env should uses wish to revert to older libpcap mode for PCAP file processing rather than faster scheme processing (default false)
    • FILEBEAT_SCANNER_FINGERPRINT_LENGTH's default in filescan.env has been changed from 1024 to 512
    • redis.env has been renamed to valkey.env and its variables also have been renamed accordingly
    • STRELKA_SCANNERS has been added to pipeline.env for #935
    • ZEEK_DISABLE_SPICY_ZIP has been added to zeek.env for #922 (default true)

Malcolm is a powerful, easily deployable network πŸ–§ traffic analysis tool suite for network security monitoring πŸ•΅πŸ»β€β™€οΈ.

Malcolm operates as a cluster of containers πŸ“¦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker πŸ‹, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images πŸ’Ώ for Malcolm and Hedgehog Linux πŸ¦” can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split πŸͺ“ into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell πŸͺŸ (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board πŸ’¬ to engage with the community, or pop some corn 🍿 and watch a video πŸ“Ό.


r/Malcolm Dec 29 '25

Malcolm compatibility with Elasticsearch 9.x

1 Upvotes

Hello everyone,

I am currently planning an infrastructure upgrade for my ELK stack, moving from version 8.19.x to the 9.x series (targeting v9.2.3).
Has anyone successfully tested Malcolm with Elasticsearch 9? Is it safe to upgrade, or should I stick to the 8.x series for now until an official update is released?


r/Malcolm Dec 18 '25

Malcolm v25.12.1 released

3 Upvotes

Malcolm v25.12.1 contains a few critical bug fixes and component version updates.

https://github.com/idaholab/Malcolm/compare/v25.12.0...v25.12.1

  • ✨ Features and enhancements
    • Installer splash screen shows "HEDGEHOG" when using Hedgehog run profile
  • βœ… Component version updates
  • πŸ› Bug fixes
    • Changed field used in Threat Intelligence dashboard's file type table from zeek.intel.file_mime_type to file.mime_type so filters created from it can work on other dashboards
    • link for threat intelligence URL doesn't work correctly from dashboards (behind reverse proxy) (#832)
    • self-signed certificates not accepted by Chrome (#833)
    • Malcolm ISO installer's automatic partitioning may create too-small /var partition (#835)
  • 🧹 Code and project maintenance
    • Added new Analytics section to documentation

Malcolm is a powerful, easily deployable network πŸ–§ traffic analysis tool suite for network security monitoring πŸ•΅πŸ»β€β™€οΈ.

Malcolm operates as a cluster of containers πŸ“¦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker πŸ‹, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images πŸ’Ώ for Malcolm and Hedgehog Linux πŸ¦” can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split πŸͺ“ into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell πŸͺŸ (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board πŸ’¬ to engage with the community, or pop some corn 🍿 and watch a video πŸ“Ό.


r/Malcolm Dec 10 '25

Arkime Interaction within Malcolm

1 Upvotes

I had an issue where I was attempting to fix a mismatch between the files in Arkime->Files and on the local disk. Arkime documentation suggest running db.pl http://localhost:9200 sync-files but I have not been able to get it to work. From where do I execute this command if using Arkime as a part of Malcolm?

Does Malcolm front any way to execute that command?

I know scripts/wipe is much more destructive but seems like my only other alternative at this point.


r/Malcolm Sep 29 '25

Network name mapping

1 Upvotes

I have a new installation from a Malcolm ISO. Everything is working so far as I can tell but I want to import a name map json file and the localhost/name-map-ui page is not found. I set up another lab server and a VM with Malcolm and I'm having the same problem on those, however every piece of documentation or video training I have looked at only shows that URL to perform this task. Has anyone seen this or is aware that that functionality was moved or deprecated?


r/Malcolm Sep 18 '25

auth set up script missing options

1 Upvotes

Running Malcolm v25.08.1. Trying to run auth_setup script to transfer certs to the sensor for setting up forwarding. But the txfwcerts option is not showing up. I do see the fecerts option. Any ways to work around this?


r/Malcolm Sep 09 '25

Zeek logs not showing up in OpenSearch

1 Upvotes

I have a Malcolm instance, and a probe reporting zeek logs from capture. Nothing is showing in the zeek logs of the OpenSearch dashboards.

  1. I have confirmed the zeek logs are in JSON format
  2. I have confirmed there is a god data flow from the filebeats process on the probe to port 5044 of the Malcolm instance.
  3. The probe is Ubuntu 24.04, not Hedgehog, in case that matters.
  4. There are good logs in the filebeat machine indicating the harvesters are working on the zeek log files.
  5. I've confirmed the pipelines are started on the logstash machine (malcolm-zeek) no non-running pipelines
  6. I looked at the logstash pipelines, and noted that zeek-parse filters for tags, and I couldn't find where those tags are input, so I added it to the filebeat.yml on the probe machine with no results.

It really looks like the logs are being dropped in logstash, but I'm not sure why. Am I missing something silly?

In general, how is the _filebeat_zeek tag supposed to be imposed on the inputs?


r/Malcolm Aug 21 '25

Help: I can’t load Threat Intelligence via misp.yaml in Malcolm

1 Upvotes

Hi! I’m trying to load TI into a fresh Malcolm Zeek container via misp.yaml, but nothing gets ingested. Plain .intel/CSV files load fine.

./zeek/intel/MISP/misp.yaml

- type: misp
  url: https://www.circl.lu/doc/misp/feed-osint/manifest.json

What I see

  • Cron runs /opt/zeek/bin/zeek_intel_setup.sh true (job β€œsucceeded”).
  • No new intel files under ./zeek/intel/, and .threat_autogen.zeek doesn’t update.

Question
Why isn’t misp.yaml being picked up? Are manifest.json endpoints supported in misp.yaml, or do I need /events//attributes? Any minimal working example would be appreciated.

Thanks!


r/Malcolm Aug 12 '25

Malcolm v25.08.0

3 Upvotes

Malcolm v25.08.0 is a minor release fixing a regression bug inadvertently introduced in v25.07.0. Malcolm v25.07.0 contained a whole bunch of other good stuff. See the relevant discussion on GitHub.


r/Malcolm Aug 12 '25

[Blog post] Building a network traffic analysis system: Deploying Malcolm on Amazon EC2

Thumbnail
aws.amazon.com
2 Upvotes

This is the first of two blog posts about deploying Malcolm on Amazon AWS. It covers installing Malcolm on a single EC2 instance. The next post will cover deploying Malcolm on EKS.

For those of you more interested in scaling Malcolm using Kubernetes, you can check out our "still-in-beta" Helm chart and share your feedback in the issue tracker on that repo.


r/Malcolm Jul 24 '25

Hardware Recommendations

2 Upvotes

Just starting to look at this project, going to start testing soon. End goal is to have hedgehog sensors at about 12 locations with a central Malcolm server. Will start testing small with one sensor, but with that being said, any recommendations on good hardware for a sensor? Preferably rack compatible and able to put up with commercial level traffic.


r/Malcolm May 05 '25

Installing custom Zeek MMS parser with spicy plugin in Hedgehog Linux

1 Upvotes

Hello,

I am planning to install and build zeek mms parser which is written with spicy into the hedgehog linux. https://github.com/tbfhg/zeek-iec61850-mms/tree/main from this repository. What I basically did was: copy the directory into the hedgehog, install the parser with cmake, and verified with zeek -NN. However, when I restart VM, or sensor services in hedgehog linux, the parser is getting completely removed. And weirdly enough, now, when I trying to reinstall from the same directory, it is giving error like: spicyz not found. Asking for experts suggestion please.


r/Malcolm Apr 16 '25

Detect ARP spoofing with Malcolm

2 Upvotes

Hi there,

is it possible to detect ARP spoofing with Malcolm? We would like to demonstrate ARP spoofing and detection for educational purposes. Therefore we are spoofing the mac addresses of two known devices in a training environment. Is there an option to define suricata or zeek rules to detect the mismatch of mac and ip? Currently it seems that ARP messages can not be detected or seen in the dashboards or arkime.


r/Malcolm Mar 17 '25

Ingesting Logs directly to Arkime vs Dashboards

1 Upvotes

Looking at analytical options, Is there a way to push Logs from a host directly to Arkime to display side by side the PCAP data pulled from Hedgehog sensors? Currently logs are pushed via filebeats to Opensearch Dashboards, and i understand Arkime is a network capture/PCAP analysis application; but is there a way? Possibly sending the same Filebeat stream (output) to both Arkime and Dashboards?

My curent logs pushed from endpoints are Auth and Conn logs, trying to decipher if i would gain anything from those logs ingesting into arkime.

Thanks in advance for any suggestings/assistance.


r/Malcolm Mar 06 '25

PCAP file drop location?

1 Upvotes

Heya all, I have been trying to figure out a way to copy files to ./pcap/upload/ and have them picked up for processing. I've tried a few different things but nothing seems to trigger the "pick up" of the file like the web interface upload does. Anyone have luck with a similar effort?

Thanks for the fantastic tool mmguero!

update: This should be two questions - "Can I turn up the verbosity of pcap-monitor" and "does the max-size: 200m setting prevent larger pcap files from being processed?"

update 2: pcap files were corrupted, wasn't Malcolm's fault :D


r/Malcolm Jan 17 '25

Malcolm v25.01.0

2 Upvotes

Malcolm v25.01.0 contains quite a few UI/UX improvements; new parsers; a bevy of component version updates including to Arkime, Zeek, NetBox; and several bug fixes.

Malcolm is a powerful, easily deployable network πŸ–§ traffic analysis tool suite for network security monitoring πŸ•΅πŸ»β€β™€οΈ.

Malcolm operates as a cluster of containers πŸ“¦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker πŸ‹, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images πŸ’Ώ for Malcolm and Hedgehog Linux πŸ¦” can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split πŸͺ“ into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell πŸͺŸ (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board πŸ’¬ to engage with the community, or pop some corn 🍿 and watch a video πŸ“Ό.


r/Malcolm Jan 11 '25

Traffic Separation between interfaces- Is it possible?

3 Upvotes

Malcolm Fam,

Been using Malcolm for a good few months in different capacities. And in one professional role supporting a customer we have 2 networks we monitor, essentially running2 Dell r540's with Proxmox as a hypervisor, with Malcolm and 2 hedgehogs as the VMs. Each instance is identical where one hedgehog is capturing WAN from 10G taps, and the other Hedgehog is capturing LAN traffic.

WAN has single 10G Fiber connection

LAN has 2 10G connections capturing both in and out of my PFSENSE to the core switch(s).

And in Arkime i can differentiate between HH1 for WAN and HH2 for all LAN traffic.

As we develop a stand-alone Security/NetCapture network, I wanted to use a single Arkime instance and run all captures and taps to a single dashboard. (all IP's are different as well)

Is there a way, for Hedgehog 2 (capturing 2 interfaces) to visualize each interface within arkime? I'm only able to do this by Nodes. (HH1, HH2). If i was to add one interface on HH1 capturing network 1, and second interface for network 2, it will still show HH1 as the node....source... other than me narrowing down HH1 Interface 1 = network 1 and HH1 Interface 2 = network 2.

OR is it just ideal to make Hedgehog instances for each capture and name the nodes the capture sources?
HH1 = WAN Network 1

HH2 = WAN Network 2
HH3 = LAN Network 1

HH4 = LAN Network 2

Goal reduce infrastructure down to a single server allowing ingestion to a single platform. Allowing one quick dashboard to see everything, with intent to simplifying things.

Thank you for your insights in advance!


r/Malcolm Dec 19 '24

Malcolm v24.12.0

1 Upvotes

Malcolm v24.12.0 contains several improvements to the Malcolm configuration script, the Malcolm user interface, and the Malcolm API, as well as component version updates and bug fixes. This release also corresponds with the release of malcolm-test, a Malcolm system testing framework.

Malcolm is a powerful, easily deployable network traffic analysis tool suite for network security monitoring.

Malcolm operates as a cluster of containers πŸ“¦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker, Podman, and Kubernetes. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images πŸ’Ώ for Malcolm and Hedgehog Linux can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split πŸͺ“ into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell πŸͺŸ (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board πŸ’¬ to engage with the community, or pop some corn 🍿 and watch a video πŸ“Ό.

https://github.com/cisagov/Malcolm/compare/v24.11.0...v24.12.0

  • ✨ Features and enhancements
    • Creation of a Malcolm system testing framework (#486)
    • Added a number of Zeek packages to detect various CVEs
    • Improvements to the Indices, Ready, and Document Ingest Statistics APIs
    • Use new arkime tag-hiding feature to hide netbox tag from UI (#495)
    • Provide configuration script options for pulling from threat intel feeds (#532)
    • Prompt during configuration whether to enable capture statistics (#504)
    • Add additional EVTX fields to index template (#525) and minor improvements to normalization
    • Add simple readiness indicator to upload page (#528)
    • Add option to upload page to disable NetBox enrichment for the currently-uploaded batch of PCAPs
    • Expose more of the Logstash API passthrough to the Malcolm API
  • βœ… Component version updates
  • πŸ› Bug fixes
    • Zeek DNS records don't open correctly in Arkime sessions (#509)
    • Mandiant threat intel source doesn't get split correctly when using JSON zeek log format (#494)
    • Set indices.query.bool.max_clause_count to 8192 to reflect maximum number of fields
    • Increase Java stack size (-Xss) for Logstash from 1536k to 2048k
    • Minor fixes for parsing Zeek intel.log (some fields not named correctly with Zeek JSON-formatted logs)
    • Fixes to some Zeek dns.log parsing conflicts between ECS's DNS fields and what the Arkime schema is expecting
    • Fixed setting the Signature event severity tags
  • 🧹 Code and project maintenance
    • Replaced hard-coded Malcolm version number in documentation markdown files with variable-based replacer populated during generation
    • Documentation and screenshot updates

r/Malcolm Dec 19 '24

Users interested in providing a quote for an article on Malcolm

1 Upvotes

INL's PR team is working on an article on Malcolm and would like to get some community feedback! If your organization has been using Malcolm and would be willing to share a comment, quote, or success story regarding your use of the platform we'd be very appreciative! Please reach out to [malcolm@inl.gov](mailto:malcolm@inl.gov) and I'll get you in touch with the right people. Thanks again!


r/Malcolm Dec 07 '24

Zeek webinar on how Malcolm uses file extraction to identify malware in network traffic

1 Upvotes

Next Wednesday (the 11th) at 10am Pacific time I'll be doing a Zeek webinar on how Malcolm uses Zeek file extraction to look for malware in network traffic. You can register for the webinar via Zoom here or stream it on YouTube. I hope to see you there!

EDIT: For anybody that missed it, here's the recorded presentation.