Watch the cluster at runtime with Falco
Admission and scanners look at objects before they run. Falco looks at what the process actually does: a shell in a container that should not have one, a write under /bin, a read of /etc/shadow.
It is a userspace engine plus a kernel probe (eBPF, or the older kernel module). Rules match syscall fields. Output goes to stdout, a file, syslog, or an HTTP webhook.
Install
DaemonSet via Helm is the usual cluster install:
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update
helm install falco falcosecurity/falco \
--namespace falco-system \
--create-namespace
kubectl get pods -n falco-system
kubectl get daemonset -n falco-system
On a node with the package:
systemctl status falco
journalctl -fu falco
Files that matter
| Path | Role |
|---|---|
/etc/falco/falco.yaml |
Engine config, outputs, which rule files to load |
/etc/falco/falco_rules.yaml |
Default rules. Do not edit these. |
/etc/falco/falco_rules.local.yaml |
Your rules and overrides |
/etc/falco/k8s_audit_rules.yaml |
Kubernetes audit-driven rules |
In-cluster, that config is usually a ConfigMap. After you change it:
kubectl rollout restart daemonset/falco -n falco-system
kubectl logs -f -l app.kubernetes.io/name=falco -n falco-system
Turn on a file you can tail
# falco.yaml
file_output:
enabled: true
keep_alive: false
filename: /var/log/falco_events.txt
stdout_output:
enabled: true
json_output: true
Restart, then tail -f /var/log/falco_events.txt.
A local rule
# /etc/falco/falco_rules.local.yaml
- rule: Package Management in Container
desc: Detect package managers running in containers
condition: >
spawned_process and container and
proc.name in (apt, apt-get, yum, rpm, apk, pip)
output: >
Package manager in container (proc=%proc.name container=%container.name)
priority: WARNING
tags: [process, container]
Disable a noisy default without touching the shipped file:
- rule: Some Default Rule Name
enabled: false
Validate before restart:
falco --validate /etc/falco/falco.yaml
falco --list-rules
Prove it
kubectl exec -it <pod> -- /bin/bash
kubectl exec -it <pod> -- touch /bin/test-file
kubectl exec -it <pod> -- cat /etc/shadow
You should see the matching rule in the Falco log. If you see nothing, the DaemonSet is not on that node, the rule did not load, or you edited falco_rules.yaml and the next upgrade wiped it.
Keep conditions short. Restart after every config change. Watch journalctl -fu falco while you test.