# What a wicked NAT we weave: detangling the cloud

- Link: https://www.agilicus.com/what-a-wicked-nat-we-weave-detangling-the-cloud/
- Published: 2019-04-11T07:17:58+00:00
- Author: Don Bowman

Cloud has 3 levels of address translation. Shocking I know. But… Believe it or 
not, this is the chain of events for a stream that arrives at your service in the
cloud.

 The sequence ends up being:

 Client->LoadBalancer->Ingress->Sidecar->Service

 and, LoadBalancer does a NAT, Ingress and Sidecar are proxies, so, well, Service
never sees the IP of client.

 Other people have been working at this problem (e.g. [RFC7974](https://tools.ietf.org/html/rfc7974)),
HAProxy ‘[Proxy Protocol](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt)‘,
others.

 Today lets look at a practical example, using the HAProxy Proxy Protocol. Specifically,
lets look at a tool [CloudFlare](https://github.com/cloudflare/mmproxy) did that
allows adding transparency on the far side. They talk about it more [here](https://blog.cloudflare.com/mmproxy-creative-way-of-preserving-client-ips-in-spectrum/).

 Here’s a recipe for you to try it out at home:

 Start a new container (as per first line) and run the following lines

    ```
    docker run --name mmp --privileged --rm -it -v $PWD:$PWD ubuntu:18.04
    apt update && apt install -y iproute2 curl iptables python3 netcat

    iptables -t mangle -I PREROUTING -m mark --mark 123 -j CONNMARK --save-mark 
    iptables -t mangle -I OUTPUT -m connmark --mark 123 -j CONNMARK --restore-mark
    ip6tables -t mangle -I PREROUTING -m mark --mark 123 -j CONNMARK --save-mark 
    ip6tables -t mangle -I OUTPUT -m connmark --mark 123 -j CONNMARK --restore-mark 
    ip rule add fwmark 123 lookup 100 
    ip route add local 0.0.0.0/0 dev lo table 100 
    ip -6 rule add fwmark 123 lookup 100 
    ip -6 route add local ::/0 dev lo table 100 
    echo 1 | tee /proc/sys/net/ipv4/conf/eth0/route_localnet

    python3 -m http.server -b 127.0.0.1 8000
    ```

 Now run this from host:

    ```
    docker exec -it mmp $PWD/mmproxy -a $PWD/networks.txt \
      -l 0.0.0.0:80 -4 127.0.0.1:8000 -6 '[::1]:8000'
    ```

 Now run this from host:

    ```
    echo -en "PROXY TCP4 1.2.3.4 1.2.3.4 11 11\r\nGET / HTTP/1.1\r\n\r\n" | \
     docker exec -i mmp nc -v 127.0.0.1 80
    ```

 On the first window, you will see something like:

    ```
    1.2.3.4 - - [18/Jun/2018 14:32:04] "GET / HTTP/1.1" 200 
    ```

 The 1.2.3.4 indicates the source IP.

 What sourcery is this? Is this a tool to undo the magic NAT stuff of the cloud?
Or a security nightmare?