How-to Debug a Running Docker Container from a Separate Container

Alternate title: How-to debug freaking go binary containers

Containers are great for shipping software, but sometimes you can go too far when stripping down your container to make it as small as possible. There’s a fine balance between a “no-frills” image and something impossible to debug (I’m looking at you, single binary go containers).

$: curl https://getcaddy.com | bash -s personal && mv /usr/local/bin/caddy .
FROM scratch
ADD caddy /
$: docker build -t caddy .
<output trimmed>
$: docker run -d --name caddy -p 2015:2015 caddy /caddy
$: docker run -it --rm --link caddy:caddy alpine sh/ # ping caddy -c 1
PING caddy (172.30.238.2): 56 data bytes
64 bytes from 172.30.238.2: seq=0 ttl=64 time=0.075 ms
/ # ps aux
PID USER TIME COMMAND
1 root 0:00 sh
8 root 0:00 ps aux
FROM alpine
RUN apk update && apk add strace
CMD ["strace", "-p", "1"]
$: docker build -t strace .
<output trimmed>
$: docker run -t --pid=container:caddy \
--net=container:caddy \
--cap-add sys_admin \
--cap-add sys_ptrace \
strace
strace: Process 1 attached
futex(0xd72e90, FUTEX_WAIT, 0, NULL
$: docker run -it --pid=container:caddy \
--net=container:caddy \
--cap-add sys_admin \
alpine sh
/ # ps aux
PID USER TIME COMMAND
1 root 0:00 /caddy
13 root 0:00 strace -p 1
34 root 0:00 sh
40 root 0:00 ps aux
/ # ls -l /proc/1/root/caddy 
-rwxr-xr-x 1 root root 16099400 Jan 24 15:30 /proc/1/root/caddy
/ # apk update && apk add curl lsof
/ # curl localhost:2015
404 Not Found
/ # lsof -i TCP
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
caddy 1 root 4u IPv6 330044347 0t0 TCP *:2015 (LISTEN)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store