How (Not) To Containerise Securely Lessons Learned the Hard Way

Right, hello everybody. Uh, this is a bit of a monster of a talk because it has a lot of live demos in. So I will ask you to uh to bear with me as I flip around all the time between. Whoops. There we go. Okie dokie. Hello, I am Andy. I like to break stuff and put it back together once I know how it's been working. I work at ControlPlane and uh I'm very proud to say that I am a trainer for various organizations. I'm this here eventually. Um, watch out for SANS SEC 584, which is uh Cloud Native Security course, five days coming out next year that I'm uh pleased to be working with on, that's out of battery, marvelous. I'm a founder of ControlPlane, which is continuous security engineering practices with a focus on cloud native and regulated industries. And I want to talk about container breakouts, Kubernetes break-ins, cluster drive-bys, API exploits, powning everything that we can find, fingers crossed, and how we fix the security skills gap in the cloud native ecosystem. Spoiler alert. We do it by training the next generation of cloud native security engineers and architects with production-like systems that they can hack, they can play CTFs until they can pop shells, and they really understand what they're doing. They test, remediate, and harden so we can hack back in. We do this with models that we build the system. These are threat models, attack trees. This is all open source under the Financial Services User Group in the CNCF. And attack simulators, which are production-like infrastructure in safe testing environments, more on this later. So, what are we going to do? I've got some local VMs, I've got GKE, got some droplets, got some Docker versions and some kernel versions, and I will demo a hacking simulator right at the end if 30 minutes is enough. Uh, also, I am drinking through this. I'm sure, perhaps, ladies and gentlemen.

The video opens with a title slide for a presentation. A terminal window is shown, then a slide with the presentation title. A slide introduces the speaker, Andy, and his roles. A slide displays the 'controlplane' logo. A humorous image of a car with multiple spoilers is shown. A complex flowchart diagram titled 'Kubernetes Attack Trees - Compromised Container Scenario' is displayed. A slide introduces 'kubesim.io - K8s Hacking and Hardening Simulator'. A slide with a cartoon image of Charmander trying to put out a fire on its tail.

Okay. So, what are we doing? Docker and Kubernetes. Do we love Kubernetes? Well, broadly, does it love us? Absolutely not. Why is it so difficult? It is a distributed system. We're all trained on monoliths, and all of a sudden, we have to deal with all sorts of different network-based problems. What is the problem with Kubernetes? Well, it's a layered security onion, but if I can get onto a single node, if I can root one of your workers, I can probably root your whole cluster. So, containers. Containers are awesome. We have to thank the uh hosts of this dev room for all the work that they put in, not only on LXD but also upstream into the Linux kernel. But they emerged from the primordial kernel soup. A child of evolution rather than intelligent design. That has morphed, refined, and been coerced into something usable. But don't forget, containers do not exist. They are merely the resultant bundling and isolation left once we've set up our name spaces and C groups and Linux security modules and started a precious little process inside. And finally, that being said, we love, oh, that is a shame because that is an amazing gift. Kubernetes. So, this talk will go fast. There's lots of demos. I'll do my best. A prayer to the demo gods and an offering. Let's go. We all know this, right? We shouldn't run privileged, we should not run UID zero, and the Docker socket should not be mounted inside a container. Why is that, I hear you ask? Uh, let's see. Okay, where are we? Uh, we are here, hopefully. Nope. We are here. Okay. So, we are spinning up a container with a double high-privileged flag. What happens when we run privileged? Do you see anything in that list of mounted devices that we shouldn't do? Yes, that is the host device dev/vda0 mounted with etc/hosts on top of it. Why is that a bad thing? Because you can do as root inside the container, and importantly, there is no boundary between root inside and outside the container unless we have user namespaces enabled. What can we do? We can mount dev/vda1. And we can put that inside our container at any old mount point. And then what do we have there? Well, that not that alias. We have the host's root file system. That is not a good look. Can it get any worse? Well, yes, indeed it can. Let's just leave evidence that Andy indeed was at one stage here. All right, and then we're back on the host. What are we going to do now? Have a look at the roots, and Andy was there. Who was that owned by and when did that turn up? 14:16 looks relatively recent. And that's not how you spell stat. Yeah, so that's a bad day. Do not run privileged. Um, having UID zero inside container is not a vulnerability in and of itself, but it leads to a much easier privilege escalation if I can get inside your container. Okay. And those were misconfigurations, right? That's that's something that we can actually defend in the pipeline.

A slide with bullet points about Docker and Kubernetes, accompanied by an illustration of an octopus. A slide with bullet points about container evolution, with a video of evolving creatures. A GIF of multiple cats in cardboard boxes. A slide with 'LET'S GO' and bullet points on what not to do. A terminal demo showing a privileged Docker container mounting the host's root filesystem. The 'What Not to Do' slide reappears.

So, it's all very well talking about the problem. Let's talk about the fix as well. In the container's life cycle, unit tests are kind of run in isolation. And I'm taking the test pyramid and I'm superimposing my own view over the top. In this case, unit tests are static or dynamic analysis on the container itself. Integration tests are probably more dynamic analysis inside the container, maybe akin to actually testing the public APIs of the application inside the container. Why is that important? Because the configuration changes with each environment that we promote it through. The container stays the same, but its behavior is dependent upon environment variables or config that we mount in, so we have to test. And end-to-end test, well, that's the full system essentially. So, what can we do for static analysis for Dockerfiles? We can lint them. And we can determine whether or not we have done certain things wrong. Something that we can't do, or that is in fact very difficult to do, is identify which user was running, because that is a runtime construct, and we can switch users in the entry point, for example. So, we probably, oh, also you can do Kubernetes with uh, Sim, a sec rather, which is static analysis for Kubernetes resources. This will tell you do not mount XYZ, and it will give you a risk score to try and quantify the danger of running a particular configuration. There are lots of things that you can do in a pod YAML to break it. Right, so, what about dynamic testing? Well, we can use inspect, it's heavyweight, it's Ruby. Do you want to install Ruby inside your container? Hell no. What about server spec? Well, they're still quite nice, but again, the same problem, it's Ruby. So, what do we use? Goss. Go server spec. It is simple, declarative, highly parallelized, and written in GoLang. What does that say? Cloud native. Hooray! So, this is what Goss looks like. You have a simple YAML-based format. It runs everything by with default 50 threads or Go channels. And uh in this case, command is the type of test, and key here is the version. So, we're just making sure that our base container, we have a contract with it that it's shipping something to us. Right, obviously you can use Goss for anything and everything, and I recommend it to the house. What next? Let's break out of some containers. Who remembers Dirty COW? A copy-on-write vulnerability in the kernel, which had been there since potentially 2007, version 2.6.22. And it's a copy-on-write race condition whereby an unprivileged user is able to write into root-owned memory, execute it, and pop a shell. Exploitation of this bug does not need leave any trace of what happened. It was detected by some dude running a rolling packet catcher on his honeypot, who then pulled out the binary and recompiled it. What a guy. Okay, why is it bad? It hoses your system. Um, it containers in a default configuration at the time did not contain this bug. Containers rely on the kernel for protection. System calls from inside a container do not hit a local kernel. They are proxied onto the host. This is why the kernel isolation model is more difficult and nuanced than a VM, which has an entire full bio-simulated version of the kernel running inside it. Instead, with containers, we get this speed increase, we start our processes very quickly, but we pay this penalty of proxying system calls. If the container is reliant upon the kernel, and the kernel lets the containers guard down, we're having a bad day. There are ways around this. Let's have a look. Non-deterministic live demo. Woohoo! Okay, so, on to. Dirty COW. So, this may suffer slightly on this screen. One side. Very good already. Yes. Sorry. As usual. Okay, so what are we going to do here? Start tmux. Sweet. Uh, also, I managed to remove the U key from my keyboard, so you'll notice that I have trouble typing U. There was chocolate beneath it in a late night hack session is my excuse. Okay, and what's actually happened there? It's just rolling up to the top of this. Okay, so what are we doing? We've got Docker at the top. We have sysdig twice. We have at the bottom temp.x, which is a lock file that the exploit uses to determine whether it's been run, because it's just spraying system calls, highly parallelized. It's a copy-on-write race condition vulnerability, so we do as much as possible in order to break it. Sorry, this is going to have to go a little bit lower. Lolcat's not installed. I apologize. Okay. Let's get into. Okay, do we want to proceed? Yes, please. There's a number of exploits suggesting that actually the kernel version we're using is vulnerable, but this is. Do we want to run with AppArmor? In this case, no, because we have a specific configuration that will fix it. And off we go. So we're just firing. Deadbeef is the exploit name, wonderful choice of hex value. And we are trying to patch the VSO, the virtual dynamic shared objects, which is kind of a proxy in user space to stop us having to hit the kernel all the time. And then we're trying to ptrace it, and at the point that we gain control of the process, then we inject our own code in. That will then, you see we've got a listener running here on the logs to 1234 on the host, all the host adapters. And then once this thing kicks in, it will, so that host adapter is where the, the endpoint is in, in the VSO. And once we pop that, we will connect from inside the container to that, and then we have root control of the host. As I said, this is non-deterministic. If it doesn't finish by the time I've finished this sentence, I will come back to it because it will just happily run in the background. Okay. You will have to take my word for that, and we'll come and have a look in a moment. Okay. So, what is going on there? We're reliant upon the Ptrace system called patch to the VSO. Let's skip ahead like this. What just happened? Well, theoretically, we bypass container security, but let's just see if anything's actually occurred yet. Yeah, there we go. Hurray. So, this claims, oh no, it says one still. Okay. It is non-deterministic. Let's keep on going. All right, so, theoretically, we bypassed container container security mechanisms. If we rerun that same exploit with a slightly modified AppArmor profile, different to the default one that Docker ships, to block Ptrace calls from within the namespace, we effectively block this exploit, but the actual solution is to patch our kernels, as always, run latest versions of things.

A slide shows a 'Container Test Pyramid'. A slide lists two GitHub projects for Dockerfile linters. A slide titled 'Static Analysis for Kubernetes Resources' shows 'kubesec.io'. A slide shows a JSON output from kubesec.io. A slide lists 'Pod YAML - Isolation-Breaking Configs'. A slide shows an example of a 'goss' test configuration. A slide with the text 'breaking out of containers'. A slide introduces 'DirtyCOW' with a cartoon cow image. A slide shows a quote about how the Dirty COW exploit was discovered. A slide titled 'Why is it bad?' explains the Dirty COW vulnerability. A slide announces a 'Nondeterministic Live Demo' for Dirty COW. A terminal window shows the speaker attempting to start a demo. A terminal window with multiple panes is displayed, showing tmux configuration errors. The terminal output scrolls rapidly, displaying provisioning steps and an error. The terminal shows `tmux` commands being executed, and the 'dirtycow demo' starts. The demo continues in the terminal, showing prompts for proceeding with the exploit. The terminal displays a rapid stream of `ptrace` requests and `bind`/`listen` responses. The screen transitions to a slide titled 'Dirty COW (2016)' with a cow graphic.

What can we do around this? There's various things here, but ultimately this is sandboxing. There we go. Hooray. Okay, so theoretically this has worked. You can see it's got patch two of two. Because of the number of system calls we made, the second window will keep on spooling. But here we should be root. Sweet. You can see here that we've actually, uh, that is one of the host at the bottom. Now, because we were inside a container at this point, we shouldn't be able to see anything on the host. So what should we look for? Yes. Let's see if runc is there. If we are going to grep, we should probably use the word grep. And no, because it's older than that. Let's grep for the Docker daemon. There we go. So we should not be, what is it called? Yeah. Okay. Uh, obviously this kernel version is a few years old and I've had to keep this VM around. But what we see there is we're inside a container and we're doing stuff on the host. It's a bad day. Container isolation is broken. Let's continue. So, what do we do? We modify AppArmor and seccomp profiles. We should be fine-tuning these things. Tools like Jesse Frazelle's Bane, there's loads of new eBPF-based seccomp tracing stuff that will extract seccomp profiles from running applications. This is all that the big container security tooling does for you anyway. Some bells and whistles, uh, or as I heard it described, Belgian whistles, around that. Um, and of course we want to write effectively. These slides are available for posterity afterwards. 13 minutes. Okay, bypassing controls. What else are we going to do? Uh, well, oh yes. Thanks Jim. Some of these are just not loading in time, are they? Uh, bypass container security controls. Oh well, that does move eventually. Um, so there was an AppArmor bypass recently. This was quite interesting because Ah, these are just not loading. Okay, we'll get that. Cool. So what we've seen here is we've just created a volume and put it over part of the proc file system. We shouldn't be able to do that. It was a bug, but this means that when the application looks for information as to what AppArmor configuration is configured, we've overwritten it and there is none. Slight little bypass. I don't know how long this was around for, probably a good long time. Um, yeah. Proc self exe. This one is fun. This is uh taking a sim link or a pointer perhaps back to the uh Docker run C binary from inside the container, overwriting it and popping a shell. And I will attempt to demo this one.

A slide with a cartoon image of two people connected by a cloud. A slide titled 'Security Contexts'. A terminal screen showing command output, including `ptrace` calls and the `id` command output confirming root access. The terminal screen continues to show command output as the speaker types `ps faux | grep runc` and `ps faux | grep docker-daemon`. A slide titled 'Docker's Default seccomp Profile'. A slide titled 'Writing Effective Seccomp Profiles'. A slide with the text 'bypass container security controls'. A slide with the text 'AppArmor bypass'. A slide with the text 'AppArmor bypass'. A slide titled 'Minimal PoC' showing a code snippet that demonstrates an AppArmor bypass. A webpage with text about an AppArmor bypass vulnerability. A white slide with the text "/proc/self/exe".

And this as I said will carry on. Yeah, yeah. And this is saving the bit off. Okay, where are we now? So let's go here. Um, so what have we got? We have a Dockerfile. And in that Dockerfile, you can see we have compiled some exploits. We're actually patching set comp inside the container and then um, we're sim linking proc self exe to the entry point of the container. Um, I should point out that none of these exploits are mine. I'm standing on the shoulders of giants, of course, and all of these are publicly available. So, what happens? We build this Dockerfile. Um, and actually what we've done in the exploit stage here is actually it's in the other stage. Is um, I didn't. Is right, here we go. So we write the string CVE 2019 onto the end of the run C binary. What would we actually do? Well, we just replace it with a malicious payload, like a file or a bash script that did something that we wanted. So, how do we prove this hasn't yet worked? Let's have a look at where run C is. And we see at the end there is nothing there. And then if we just run this container that I built earlier, you'll have to trust me. Uh then what are we going to see? There we go. So these are the return codes from system calls. And we have appended our string. So again, what's happened? We're inside a container, a theoretical isolation boundary, and we've been able to influence or impact things that sit on the host. This is a bad day. This is how we break out of containers. What is the fix here? Don't run old versions of software. Really, really easy. But as we know, everything is a people problem and our organizations probably mean that it's very difficult to keep things patched in a timely manner. This is immutable infrastructure. This is aggressive builds and pipelines for all of our work, for all of our service. Onwards. Right, what have we next? Uh, yeah, we've done that one. Yeah, here is the lesson. Patch your hosts. We can use. Yeah, so we can use Goss to uh test the kernel parameters, to test for the output of bash scripts, to test for everything, and we can use it in a sly way. Um, let's and one of my favorite quotes from our esteemed track hosts, containers are a user space fiction. I I love that and it's you will notice, containers don't really exist. Let's not forget we are still in the host kernel. We are still poking around on the same machine. We don't have this uh nested, we don't have a virtualization as we would do with other containers. Um, other. Yeah, okay. Collection of stimuli and restrictions born from unintelligent design and years of evolution, a lot like consciousness. Is there a lesson here? No. But everything on the internet and in our organizations is held together with string and sticky tape. We should test everything because when it gets changed, we need to maintain the same behavior as we had before. We're doing this for ourselves, we're doing this for future us, we're doing this for the maintainers of the system that don't even know who we were. We're doing this because we are good open source citizens and colleagues. And we're putting a security test suite in place to help it. So, testing is a dark art. It is a software engineering discipline. We need rigor, we need objectivity. Anything can be a security test. What do we do? Arrange, act, assert. Prepare the environment, uh, perform some sort of execution and capture the result, and then assert that it's actually worked. Prove it fails as expected. This is very, very important when writing tests, otherwise you've just got a green test suite that doesn't actually catch anything. And beware acceptance testing. Okay. DevSecOps. Hooray. Yeah, this just carry on because we're a bit pressed for time. Testing out of date versions. All right, Goss again. This is an example of how to paste an animated GIF as a GIF. All right. Well, uh, never mind. There is a link there to Goss. Basically, it's very easy to build that Goss test suites. I love them. Testing is cool. It's the only way we can prove that we're secure, but we're not proving the absence of bugs. We're not proving that the system is actually secure. We're just saying for our particular model of it and our understanding at this point in time, it conforms to some level of security.

The screen shows a terminal window with various commands and their outputs. The terminal displays the content of a Dockerfile. The terminal shows C code for `stage2.c` and `stage1.c`. The terminal shows commands to inspect the `docker-runc` binary and then runs a container. A white slide with the text "/proc/self/exe". A slide titled "patch your hosts" with bullet points and an image of a chained and padlocked gate. A white slide with the title "containers are a userspace fiction". An animated GIF showing a character from Portal creating portals. A slide titled "Security Test Suite!". An animated GIF showing two stylized figures fighting with swords. A slide titled "testing: out of date versions". A slide titled "goss". A slide titled "Security Testing with Goss". A slide titled "TESTING IS COOL" with bullet points.

Okay. But that was too easy, right? Uh, let's find some public clusters and prone those. Right. And in clusters in the wild. How many insecure Kubernetes hosts do you think we can find in a few seconds? 10? 100? 1000? Very good. It'll be a lot. Let's go. Okay, this is. Yeah, right. I have my head of security to thank for that. Uh, right, this is Binary Edge. Binary Edge is Showdan for infrastructure. This search term finds open Kubernetes. It's not a nice day if you're on this list. Uh, which uh, where are we? This is actually a web page. Let's uh, let's go back to this. Uh, this may not be legal in your jurisdiction. Alert. What Binary Edge does is it connects to unauthenticated API endpoints. That is a gray, gray area. So take this as you will, but the platform has already scanned the IP4 address space for us and then poked at what it has found. Uh, we can see up here that is the query. And down here, this is some Chinese honey pots, actual cluster, who knows, I wouldn't touch it with a barge pole. So, here is one I pwned earlier. Let's go down here and where are we now? Okay. So, let's uh, still on these. So let's test the API server and see if it is leaking anything useful. Um, this is an Nmap script that runs in the Nmap scripting engine. Let's go back up here to the. Um, and so Nmap Q API server. Um, I run this with a little bit. Yeah, so all we're doing here is. Uh, not that. What's that? I can't even read it. Okay. So, what we've done there is we've looked for a certain port, we check for an HTTP response and run a regular expression over it. The regular expression matches get version, get commit and we can see here that the API server, the Kubernetes API server is leaking its version information. You would not do this with Nginx. You would not do it with Apache. We learned this lesson a long time ago, but evidently Kubernetes is better than us. Uh, where are we now? Uh, so, well, what do we think we can do about that version? Any ideas if we can attack it? Well, 1114 has uh one of the mother of all TVs associated with it. It is error it's poor error handling in in the MTLS server. Essentially what happens is a web socket connection is instantiated, that is bundled in a TLS pipe, if you like, so the encryption is established and then the web socket communication goes over that. There is incorrect handling of the web socket error code, so the tunnel would stay open and we could then send whatever commands we wanted through. This was initially the most extreme remote code execution, actually it's scoped to reflective APIs and it's a little bit more difficult to exploit than we thought about, but let's try anyway. Okay, so we've done what we have here. Okay. So first of all, we will just run this. So we can see we're not actually running the exploits and as such we have. And all we're doing here is opening a socket and sending this web socket upgrade connection six times. But it's not handled correctly because we keep on sending it in. If we then uh, say that we will exploit this to run that piece of code, you'll see that what we have at the bottom, which is a 403 Forbidden, magically becomes a 200 OK. As we have an unauthenticated request handled by the API server through that connection. This is exploitable, but a bit more difficult and uh, I will leave that as an exercise to the reader. Seven minutes. Okay, we can now watch it burn. We can run some in it if we actually got pod deployment access, we could deploy nano miners or change plus to creds, or delete all the nodes. So more lessons, don't run a public API server endpoint on the internet. It's a privileged API and zero trust does not mean just trust everything is infinitely secure and all our authentication and authorization mechanisms just work. They don't, we would layer ourselves in. So more lessons, don't run a public API server endpoint on the internet. It's a privileged API and zero trust does not mean just trust everything is infinitely secure and all our authentication and authorization mechanisms just work. They don't, we would layer ourselves in.

A slide with the text 'too easy'. A slide with 'public clusters drive by'. A slide displays a diagram titled 'hunting clusters in the wild'. The screen switches to a web browser showing the BinaryEdge platform. The screen briefly returns to the 'hunting clusters in the wild' slide. The screen shows the BinaryEdge website again, with warnings about legal jurisdiction. A slide changes to 'Here's one I pwnd earlier' with an image of a 'Blue Peter' book cover. The screen switches to a terminal window. A slide changes to 'The Banner Banhammer'. The screen returns to the terminal, showing `nmap-kube-apiserver` commands. The terminal displays the output of the Nmap scan. The screen returns to 'The Banner Banhammer' slide. The slide changes to 'API server Pwnage' with a Matrix-like visual. The screen shows a terminal running a `hack.rb` script, indicating 'NOT EXPLOITING'. The terminal displays the Ruby code for the `hack.rb` script. The terminal shows the `hack.rb` script being run again, with the exploit enabled, showing '200 OK'. The screen returns to the 'API server Pwnage' slide. The slide changes to 'Watch It Burn'. The slide changes to 'Here cometh the Lesson' with an image of a tin foil hat.

not mean. Just trust everything is infinitely secure and all our authentication and authorization mechanisms just work. They don't. We would layer ourselves in any other situation. Tin foil hats are cool. and Moss knows best. If you need another reason why not to run a public API server, you may have heard of this attack. Thanks to Ro and Bradman and Ian Coldwater for bringing the honk on this one. This is a essentially it's a fork bomb for YAML. Um it's like a zip bomb. It's eternal recursion. It will exhaust our API. It should be noted that any string will do in data key A, but honk has been emphatically recommended by the authors. Another open API moral. Zero trust, trust but verify, mutual cryptographic authentication does not preclude the existence of other bugs. And of course, we update and we keep ourselves the hell offline. Okay, let's see if this one will done in minutes I have left. Uh, where am I? Oh, that's another API server test. So, try that again. I don't know why that failed. Uh yeah, of course, we can't write tests for everything and anything that we can programmatically, whoops, do and this is just uh a test for the presence of that. And we didn't care so much because we're now on the billion of loss. Okay, so what do we have here? Uh, so this is my nice server and I'm looking for something that I've now lost. Here we go. So we'll run the time to do this. We'll run that data so we can actually see something going on. Uh, we'll check the API server logs, we'll watch events, and we will, uh, basically just fire loads of, uh, what's question? I can't actually see what that says. Uh, so we've got an exploit here which is just going to run this, um, actually, I can't get that as well. Uh, so all we're doing is sending this self-subject access review, which says, as an authenticated user, if I was this user, what would I be able to do? That is not an authenticated API call, we send our payload, billions of honks, and we are just doing it again and again and again, which is what the try medium prefix is on this. Let's now source it. Okay. So we'll come back to this in a minute, but what we will start to see is the API server exhausting its threads, failing its health checks and restarting because we keep so many sockets open doing this, that is the end of the API servers. Even if they're low balanced, there is an asymmetrical data flow in this attack. They cannot handle the amount of recission necessarily to infinitely recurs, I'm surprisingly. So, uh, we will keep on going. If indeed we have the time. That unfortunately I just started again, that's not what you want to see.

A slide with bullet points and an image of a tin foil hat. A slide displays a scene from 'The IT Crowd' featuring Moss with a burning computer. A slide with the title 'Billion Laughs'. A video clip of numerous white geese walking in a line. A slide displays a block of YAML code titled 'Self Destructive SelfSubjectAccessReview'. A slide presents details about CVE-2019-11253. The slide updates to show recommendations for CVE-2019-11253. A slide displays the word 'HONK!' with goose images. The screen transitions to a terminal window showing commands being executed. The terminal window shows a series of failed attempts to run a script. The terminal window shows a successful execution of a script. The speaker is at a desk, looking at a laptop, with a terminal window showing commands. The screen now shows multiple terminal windows, arranged in a grid. The speaker continues to interact with the terminals. The screen now shows the content of the `CVE-2019-11253` exploit script. The speaker explains the impact of the attack on the API server. The screen briefly shows a timer and then a slide with a list of commands and a large 'HONK!' text.

testing testing. Range act to surf network infrastructure. Well, we've built a tool at control play, is highly paralleled and map. These slides are available later. We love back core. I am the only maintainer left on this project. Please join me. It's useful assertions for bash. Uh we've built some extensive and expansive test suite with that. It's brilliant. Who runs in the room? A few people, this is for you. especially you sir. Okay. Uh, we did a lot of trouble around this. There's lots of stuff that go wrong with. We don't use the security policy because it doesn't let us configure it. So we have to use OPA. Let's attack the mesh in the last two minutes. Okay. So what have we got here? number we on. server. minutes. Ah, okay. So, what have we got here? I think I may actually be out of time, but uh, suffice to say, there is no endpoint security. You can hit localhost 1500 triple quits, uh, you can post to it, and this issue will explain how you knock yourself off the mesh. It is going to be fixed. Let's just start by making the admin listener a real listener. and recursively correct ourselves. Um, we are almost there. How do we evade detection? We stop the API server emitting its audit logs. We blackhole traffic, we get in the way of the endpoint, we deny of service the endpoint, we root the cluster and we turn them off. Nice and easy. De-auditing Kubernetes. Hacking in a safe space. How do we teach everybody the extreme amount of content I've packed into 30 minutes? With this tool, my time is up, it teaches you all of this stuff, and it's really great. This is all wonderful. Test everything. Prepare for the unexpected. Thank you very much.

A slide titled 'Testing testing 123'. A slide titled 'K8S Network Policy Testing'. A slide now focuses on 'bats-core'. The slide continues with more code examples related to `bats-core`. A new slide appears with the title 'break out of service meshes'. The slide lists 'Istio Threats'. A slide with the title 'Attacking the Mesh' shows a blue, interconnected mesh diagram. The screen returns to terminal output. The terminal output continues, displaying information related to the API server. The video shows a slide titled 'Attacking the Mesh'. The screen displays a GitHub issue page titled 'Admin endpoint security #2763'. The screen shows a GitHub issue page with comments discussing proposed fixes. A slide titled 'Lesson' is displayed. A slide with the text 'evade detection' is shown. The slide changes to 'De-auditing K8S' then to 'hacking in a safe space'. A slide introduces 'kubesim.io - K8S Hacking and Hardening Simulator'. A slide titled 'K8S Breach Lessons' is shown. The slide changes to 'How to Train your Red Team for Cloud Native'. A new slide appears, titled 'Layers of Security Testing'. The slide changes to 'Getting the House in Order' and 'Continuous Security (ish)'. A black slide with the word 'FIN' in white text appears. The final slide is titled 'Prepare for the Unexpected'.

The screen goes black with a 'NO SIGNAL' message in blue text. The final slide displays the FOSDEM 20 logo and various sponsor logos on balloons.