Container Runtime Security: Beyond Image Scanning

November 30, 2020

Image scanning catches known vulnerabilities before deployment. But containers can behave unexpectedly in production—executing binaries, making network connections, accessing files they shouldn’t. Runtime security monitors and controls container behavior as it happens.

Here’s how to implement container runtime security effectively.

Why Runtime Security

What Static Analysis Misses

Static scanning catches:
✓ Known CVEs in packages
✓ Hardcoded secrets in images
✓ Misconfigurations in Dockerfiles

Static scanning misses:
✗ Zero-day exploits
✗ Runtime behavior anomalies
✗ Compromised container actions
✗ Lateral movement attempts
✗ Data exfiltration

The Runtime Threat Model

Attacks that bypass static analysis:
1. Supply chain compromise (malicious code, not vulnerability)
2. Memory corruption exploits (no signature yet)
3. Credential theft and misuse
4. Container escape attempts
5. Cryptomining injection
6. Reverse shells

Runtime Security Layers

Kernel-Level Security

Seccomp (Secure Computing Mode): Restrict syscalls containers can make:

{
  "defaultAction": "SCMP_ACT_ERRNO",
  "architectures": ["SCMP_ARCH_X86_64"],
  "syscalls": [
    {
      "names": ["read", "write", "open", "close", "stat", "fstat", "mmap", "mprotect", "munmap", "brk", "ioctl", "access", "pipe", "poll", "select", "sched_yield", "mremap", "msync", "mincore", "madvise", "dup", "dup2", "pause", "nanosleep", "getpid", "socket", "connect", "accept", "sendto", "recvfrom", "shutdown", "bind", "listen", "getsockname", "getpeername", "socketpair", "setsockopt", "getsockopt", "clone", "fork", "vfork", "execve", "exit", "wait4", "kill", "uname", "fcntl", "flock", "fsync", "fdatasync", "truncate", "ftruncate", "getdents", "getcwd", "chdir", "fchdir", "mkdir", "rmdir", "creat", "link", "unlink", "symlink", "readlink", "chmod", "fchmod", "chown", "fchown", "lchown", "umask", "gettimeofday", "getrlimit", "getrusage", "sysinfo", "times", "getuid", "getgid", "setuid", "setgid", "geteuid", "getegid", "setpgid", "getppid", "getpgrp", "setsid", "setreuid", "setregid", "getgroups", "setgroups", "setresuid", "setresgid", "sigaltstack", "rt_sigaction", "rt_sigprocmask", "rt_sigreturn", "epoll_create", "epoll_ctl", "epoll_wait", "exit_group", "set_tid_address", "futex", "set_robust_list", "get_robust_list", "epoll_create1", "eventfd2", "epoll_pwait", "timerfd_create", "timerfd_settime", "timerfd_gettime", "accept4", "signalfd4", "eventfd", "arch_prctl", "prctl", "prlimit64", "clock_gettime", "clock_getres", "clock_nanosleep", "openat", "mkdirat", "fchownat", "newfstatat", "unlinkat", "renameat", "linkat", "symlinkat", "readlinkat", "fchmodat", "faccessat", "pselect6", "ppoll", "utimensat", "splice", "tee", "vmsplice", "sendmsg", "recvmsg", "pipe2", "inotify_init1", "preadv", "pwritev", "setns", "getrandom"],
      "action": "SCMP_ACT_ALLOW"
    }
  ]
}

AppArmor: Mandatory access control:

# /etc/apparmor.d/container-profile
profile container-default flags=(attach_disconnected,mediate_deleted) {
  # Network access
  network inet tcp,
  network inet udp,

  # File access
  /bin/** rx,
  /lib/** rm,
  /usr/** rm,
  /app/** r,

  # Deny sensitive paths
  deny /etc/shadow r,
  deny /etc/passwd w,
  deny /proc/*/mem rw,
}

SELinux: Security-Enhanced Linux policies:

# Kubernetes with SELinux
securityContext:
  seLinuxOptions:
    level: "s0:c123,c456"
    type: "container_runtime_t"

Runtime Monitoring

Falco (CNCF): Behavioral monitoring using eBPF/kernel modules:

# falco_rules.yaml
- rule: Unauthorized Process
  desc: Detect unauthorized process execution
  condition: >
    spawned_process and
    container and
    not proc.name in (allowed_processes)
  output: >
    Unauthorized process started
    (user=%user.name command=%proc.cmdline container=%container.name)
  priority: WARNING
  tags: [process]

- rule: Outbound Connection to Unusual Port
  desc: Detect outbound connections to unusual ports
  condition: >
    outbound and
    fd.sport not in (80, 443, 8080, 8443, 5432, 6379, 3306)
  output: >
    Unusual outbound connection
    (user=%user.name connection=%fd.name container=%container.name)
  priority: NOTICE

- rule: Write to /etc
  desc: Detect writes to /etc directory
  condition: >
    open_write and
    fd.directory = /etc and
    container
  output: >
    File written to /etc in container
    (user=%user.name file=%fd.name container=%container.name)
  priority: WARNING

Network Policies

Kubernetes network segmentation:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-policy
spec:
  podSelector:
    matchLabels:
      app: api
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 8080
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: database
      ports:
        - protocol: TCP
          port: 5432
    - to:
        - namespaceSelector: {}
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53

Detection Strategies

Baseline Behavior

Learn normal, alert on abnormal:

# Conceptual baseline learning
class ContainerBaseline:
    def __init__(self, container_id):
        self.processes = set()
        self.network_connections = set()
        self.file_accesses = set()

    def observe(self, event):
        if event.type == 'process':
            self.processes.add(event.process_name)
        elif event.type == 'network':
            self.network_connections.add((event.dest_ip, event.dest_port))
        elif event.type == 'file':
            self.file_accesses.add(event.path)

    def check_anomaly(self, event):
        if event.type == 'process':
            if event.process_name not in self.processes:
                return Anomaly('Unexpected process', event)
        # ... similar for network and file

Known-Bad Detection

Signatures for known attack patterns:

# Detect reverse shell
- rule: Reverse Shell
  condition: >
    spawned_process and
    (proc.name in (bash, sh, zsh) and
     proc.args contains "&" and
     proc.args contains "/dev/tcp")
  priority: CRITICAL

# Detect crypto mining
- rule: Crypto Mining
  condition: >
    spawned_process and
    (proc.name in (xmrig, minerd, cpuminer) or
     proc.args contains "stratum+" or
     proc.args contains "pool.minergate")
  priority: CRITICAL

# Detect container escape attempt
- rule: Container Escape
  condition: >
    open_write and
    (fd.name startswith /proc/sys or
     fd.name startswith /sys/kernel)
  priority: CRITICAL

Drift Detection

Alert when containers change from their image:

# Detect files added to container
- rule: Binary Added to Container
  condition: >
    (open_write or create) and
    container and
    (fd.name endswith .sh or
     fd.name endswith .py or
     fd.directory = /usr/bin or
     fd.directory = /bin)
  output: >
    Binary or script added to container
    (file=%fd.name container=%container.name image=%container.image.repository)
  priority: WARNING

Response Actions

Automated Response

# Integration with response system
actions:
  - alert:
      condition: priority >= WARNING
      destination: siem

  - kill_container:
      condition: priority == CRITICAL
      enabled: true

  - network_isolate:
      condition: rule == "Reverse Shell" or rule == "Data Exfiltration"
      enabled: true

  - snapshot:
      condition: priority >= WARNING
      capture: [memory, filesystem, network]

Integration with Security Stack

integrations:
  siem:
    type: splunk
    endpoint: https://splunk.example.com
    events: all

  pagerduty:
    severity_map:
      CRITICAL: P1
      WARNING: P3
    routing_key: xxx

  kubernetes:
    actions:
      - label_pod_suspicious
      - cordon_node
      - delete_pod

Implementation

Falco Deployment

# Helm installation
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm install falco falcosecurity/falco \
  --set falcosidekick.enabled=true \
  --set falcosidekick.config.slack.webhookurl=https://hooks.slack.com/xxx

Custom Rules

# Organization-specific rules
- list: allowed_processes_api
  items: [node, npm, sh]

- rule: Unexpected Process in API Container
  desc: Detect unexpected process in API containers
  condition: >
    spawned_process and
    container.image.repository = "mycompany/api" and
    not proc.name in (allowed_processes_api)
  output: >
    Unexpected process in API container
    (process=%proc.name command=%proc.cmdline container=%container.name)
  priority: WARNING

- rule: API Container Database Access
  desc: API container should only access authorized databases
  condition: >
    outbound and
    container.image.repository = "mycompany/api" and
    fd.sport = 5432 and
    not fd.rip = "10.0.1.100"  # authorized DB IP
  output: >
    API container connecting to unauthorized database
    (dest=%fd.rip container=%container.name)
  priority: CRITICAL

Key Takeaways

Container security is defense in depth. Static analysis, runtime monitoring, and network controls each catch different threat types. Layer them for comprehensive protection.