1 00:00:00,000 --> 00:00:05,680 Hello, in this episode I have a guest. I will interview Louis from PentesterLab 2 00:00:05,680 --> 00:00:12,320 and he has an incredible experience because as a creator of PentesterLab he has insight into 3 00:00:12,320 --> 00:00:17,840 what problems do people face, what challenges are easy, what are difficult, where people get stuck 4 00:00:17,840 --> 00:00:23,680 and we all, me and you, we can use this knowledge to basically speed up our learning journey. 5 00:00:23,680 --> 00:00:28,559 So hello Louis, thank you so much for joining me today and can you please tell people who don't 6 00:00:28,559 --> 00:00:32,560 know you maybe what is your background and how did you get to the point you're currently at? 7 00:00:32,560 --> 00:00:40,160 Okay, so my name is Louis, I'm the founder CEO of PentesterLab. So I started by going to 8 00:00:40,160 --> 00:00:45,680 university and doing an engineering degree in computer architecture, then at the same time I 9 00:00:45,680 --> 00:00:52,080 did a master in security back in France, then I was lucky enough to get an internship doing some 10 00:00:52,080 --> 00:00:58,959 Windows kernel programming. After that I got a job starting as a security consultant, moved to 11 00:00:58,959 --> 00:01:05,599 Australia, became like a Pentester and then Code Review and then AppSec and at the same time I 12 00:01:05,599 --> 00:01:13,519 started doing like training at conferences and I had all this amazing content that I created for the 13 00:01:13,519 --> 00:01:19,599 training and I was only using them like once or twice a year so I decided to launch PentesterLab 14 00:01:20,320 --> 00:01:22,800 and the rest is history I guess. 15 00:01:22,800 --> 00:01:28,400 The rest is history, even for me PentesterLab was one of the first resources I used for learning 16 00:01:28,400 --> 00:01:34,239 and I must admit I haven't logged in there for a while but my profile, I believe at least from 17 00:01:34,239 --> 00:01:39,440 the early days I have a lot of the flags so definitely thank you for this. These days, 18 00:01:39,440 --> 00:01:43,919 do you even have the time to hack for fun or do you mostly spend time on PentesterLab? 19 00:01:44,559 --> 00:01:53,199 So PentesterLab is still like a really small business and I'm lucky enough or I force myself to 20 00:01:53,760 --> 00:01:58,559 keep it in a position where I can still be doing some hacking because that's what I'm really 21 00:01:58,559 --> 00:02:03,760 passionate about. As much as it's cool to manage young people hacking or other people hacking, 22 00:02:03,760 --> 00:02:09,839 I like to do Code Review, Pentest and things like that. I don't Pentest for clients but I 23 00:02:09,839 --> 00:02:12,639 like to do like vulnerability research, find bugs and things like that. 24 00:02:13,199 --> 00:02:14,720 What are your favorite bug classes? 25 00:02:16,960 --> 00:02:26,399 I don't really have a favorite bug classes. I like complexity, I like details, I like things 26 00:02:26,399 --> 00:02:31,199 where I'm like, oh, I didn't know that. That's really what interests me. So you can have boring 27 00:02:31,199 --> 00:02:39,360 bug classes with amazing bugs or boring, amazing bug classes but the exploit is really, really 28 00:02:39,360 --> 00:02:48,160 interesting. I like the details. I think all of us that do CTFs, do some labs, we all like the 29 00:02:48,160 --> 00:02:52,639 edge cases where it's really, really difficult and it really shouldn't work but it does work 30 00:02:52,639 --> 00:02:58,720 after all. I think it's definitely, especially it's fun when it happens in real life as well, 31 00:02:58,720 --> 00:03:01,279 then it's super, super satisfying. Exactly. 32 00:03:01,839 --> 00:03:08,080 How about from the creator of lab standpoints? Which bug classes are fun to create labs that 33 00:03:08,320 --> 00:03:13,039 just spend a few hours and you have the lab ready? And on the other hand, which are super 34 00:03:13,039 --> 00:03:18,639 hard where you have to spend a lot of time to do the whole setup to create the lab? 35 00:03:18,639 --> 00:03:24,720 So little labs where it's like a simple bugs or just like a CV are pretty easy to create 36 00:03:24,720 --> 00:03:30,399 because for CV, you just can find the Docker container with a vulnerable version. What is 37 00:03:30,399 --> 00:03:37,279 really hard to create is labs around OAuth and SAML, like all these protocol where you have a 38 00:03:37,279 --> 00:03:42,240 lot of little moving pieces and you need to have them just right, especially since I'm trying to 39 00:03:42,240 --> 00:03:48,240 get labs where you only have one bug. So it's really easy to have like a lab with 20 bugs, 40 00:03:48,240 --> 00:03:51,360 but only one bug and one way to exploit it, it's a lot harder. 41 00:03:51,360 --> 00:03:56,240 How does the infrastructure work? Is it like Kubernetes under the hood or is it, how does it 42 00:03:56,240 --> 00:04:00,240 work? Just some Docker and like some magic around it and yeah. 43 00:04:00,240 --> 00:04:05,839 Okay. What do you do when there's like an RCE and you have to make sure that one user does not 44 00:04:06,800 --> 00:04:08,080 affect other users? 45 00:04:08,080 --> 00:04:11,360 Oh, it's just like lockdown or at the Docker level. Yeah. 46 00:04:11,360 --> 00:04:13,440 Do you have separate containers for separate users? 47 00:04:13,440 --> 00:04:13,919 Yeah. 48 00:04:13,919 --> 00:04:20,480 Okay. Okay. I imagine this, when I think of creating labs, because also we do as a CTF team, 49 00:04:20,480 --> 00:04:25,040 we also do, even this weekend, we organized onsite finals. So I do sometimes create labs 50 00:04:25,760 --> 00:04:32,880 and two like challenges are how to make sure that different users do not affect each other. Of 51 00:04:32,880 --> 00:04:39,519 course, you can just spin up instances for each user. And when you have an RCE, we are forced to 52 00:04:39,519 --> 00:04:44,799 do this. So we do it, but also like you don't want to do it for a backlash if you don't need it 53 00:04:45,839 --> 00:04:50,160 because it generates costs. So this is one challenge. And also the other challenge is 54 00:04:50,160 --> 00:04:54,559 the client-side bugs where you need the interaction from the user. I think, 55 00:04:54,559 --> 00:04:57,200 do you use headless browsers when you have a client-side? 56 00:04:57,200 --> 00:05:07,359 Yes. A lot of Puppeteer and historically a bit of PhantomJS as well. And all of that is in AWS, 57 00:05:07,359 --> 00:05:11,040 so in Lambda. So this way it's pretty isolated from everything else. 58 00:05:11,679 --> 00:05:16,880 Yeah. But for me, it's still like, I usually just copy paste some code of the Chromium with 59 00:05:16,880 --> 00:05:22,320 different Chromium options, which I admit, I don't understand 100%. And I'm worried if I 60 00:05:22,320 --> 00:05:29,519 won't use a vulnerable Chrome version and be vulnerable this way. But from my perspective, 61 00:05:29,519 --> 00:05:34,640 when I created the labs, the easiest ones are the ones where it's like a server-side bug that 62 00:05:34,640 --> 00:05:40,640 does not affect other users, like a file read. Yeah. And that's why, for example, 63 00:05:41,600 --> 00:05:47,359 you need both all the moving pieces, but also you need the client-side, like the fake client 64 00:05:47,359 --> 00:05:51,119 visiting everything and with the right redirect and everything like that. So 65 00:05:51,679 --> 00:05:57,519 it's challenging, but it's good fun. Yeah. Yeah, for sure. How do you, these days, 66 00:05:57,519 --> 00:06:03,440 after you've modeled so many bugs, so many labs, how do you still find ideas for new labs? 67 00:06:04,239 --> 00:06:10,880 Research. It's really easy to create a lot of labs. What is really hard is to create the right 68 00:06:11,519 --> 00:06:18,079 amount of labs and keep people learning. And that's a big part of what I do is curation, 69 00:06:18,640 --> 00:06:26,160 like looking at CVs, probably like 50 to 100 CVs a week, looking at, I have a bit of automation 70 00:06:26,160 --> 00:06:31,279 around that, looking at bug bounty reports, looking at research, looking at all these 71 00:06:31,279 --> 00:06:36,239 things and trying to feel like, okay, this is new, or this is like a variation of something, 72 00:06:36,239 --> 00:06:42,320 and this variation matters. Okay. Once you have the CVE, how do you, what's the process? 73 00:06:42,320 --> 00:06:47,839 There's no public exploit, of course. What's the process to finding out what is the bug? 74 00:06:48,720 --> 00:06:54,079 So you get the vulnerable version, you get the patch version based on the CVE information, 75 00:06:54,799 --> 00:06:58,559 you do a diff, and then you try to find the bug and exploit it. 76 00:06:58,559 --> 00:07:01,760 Yeah. What do you do when, because some companies, when they 77 00:07:02,399 --> 00:07:09,119 commit a fix, the comment will have like fix security bug, SSRF, whatever. But some will 78 00:07:09,119 --> 00:07:13,519 intentionally name commits, like I'll have the commit message that does not say it's a 79 00:07:13,519 --> 00:07:20,640 vulnerability. What are your top tips for finding out which commits are sort of hidden vulnerability 80 00:07:20,640 --> 00:07:28,880 fixes? I think the main thing is to really carefully read the advisory, because in every 81 00:07:28,959 --> 00:07:35,440 advisory, you got some tiny details that tell you like what is going wrong. And then it's just like 82 00:07:36,480 --> 00:07:41,040 spending time reading source code and yeah, and trying and trying and trying. 83 00:07:41,040 --> 00:07:48,480 Okay. How about, cause I know that it's the easiest when the company publishes tests for, 84 00:07:48,480 --> 00:07:53,359 and the test sometimes has the exact payload. Are there more tricks similar to this one that 85 00:07:53,359 --> 00:07:56,480 I don't know about maybe? So yeah, test cases really, 86 00:07:56,480 --> 00:08:03,359 really help with that. Comments in source code, changes, looking at some function, for example, 87 00:08:03,359 --> 00:08:08,880 a lot of vulnerabilities nowadays are around like people making mistakes around using substring to 88 00:08:08,880 --> 00:08:13,440 compare things or they're missing a check. So they're using, they're adding an if statement 89 00:08:14,000 --> 00:08:20,640 or they're modifying like a check somewhere. And then it's just, yeah, a lot of times. 90 00:08:21,440 --> 00:08:27,359 A lot of times, yeah. Like the test cases is good, but often like you need to be lucky to like, 91 00:08:27,359 --> 00:08:30,160 or you need someone to have written the test cases and yeah. 92 00:08:30,160 --> 00:08:33,760 Yeah. It's, it's, I don't say it's common, but when it is, it's very satisfying. 93 00:08:35,119 --> 00:08:41,599 And I also saw some people, there was one case, one CTF task where I spent a lot of time on 94 00:08:41,599 --> 00:08:48,719 something and it was an XSS based task. And one of my friends just took a payload from 95 00:08:48,719 --> 00:08:54,880 Don't Purify test cases and it works straight away. And it was like a soul, such a long payload. 96 00:08:54,880 --> 00:08:59,119 And I was like, how did you come up with this? Oh, I just copied it from Don't Purify. 97 00:08:59,119 --> 00:09:02,239 Yeah. Like good or like sometimes as well. 98 00:09:02,239 --> 00:09:08,239 Well, it wasn't like it was his skill and he's, he's thinking that he came up with this idea and 99 00:09:08,239 --> 00:09:15,039 I didn't, but it was, it was funny. You said you have the automation around CVEs? 100 00:09:15,039 --> 00:09:15,440 Yeah. 101 00:09:15,440 --> 00:09:16,239 What is this? 102 00:09:16,239 --> 00:09:22,640 Oh, more like you get like a script, you give it like the Git repository, the patch version, 103 00:09:22,640 --> 00:09:27,200 the vulnerable version, and you get like all the diff automatically and things like that. So that 104 00:09:27,200 --> 00:09:33,359 way you don't have to do all that manually. Like just like five or 10 Git commands, but at least 105 00:09:33,359 --> 00:09:34,239 it's done. Yeah. 106 00:09:34,239 --> 00:09:38,080 Oh, okay. So then, okay. So then you still have to like see yourself. 107 00:09:38,080 --> 00:09:38,640 Yeah. 108 00:09:38,640 --> 00:09:39,119 Yeah. 109 00:09:39,119 --> 00:09:44,400 Okay. I stopped you like automatically find out which CVEs are worth. 110 00:09:44,400 --> 00:09:49,520 I wish I had that. So I tried to get that output from the diff to a chat GPT, for example, 111 00:09:50,159 --> 00:09:54,640 it's terrible. Like giving it like the advisory and the patch, chat GPT is just like, Oh, 112 00:09:55,359 --> 00:09:59,679 creating like weird finance and things like that and making it, making up things. So. 113 00:10:00,239 --> 00:10:01,440 It doesn't really work. 114 00:10:01,440 --> 00:10:02,159 Not yet. 115 00:10:02,159 --> 00:10:08,080 No. Do you have any other automations or tools that helped you in the process? 116 00:10:08,080 --> 00:10:13,840 I got a bit of like rules around same grabs and things like that to like look at code basis. But 117 00:10:13,840 --> 00:10:18,159 for that part, not yet. I need to spend more time. It's always a balance between like 118 00:10:18,159 --> 00:10:20,400 more automation, less automation and yeah. 119 00:10:21,119 --> 00:10:24,880 Yeah. And then when you create automation, you have to invest more time to maintain the 120 00:10:24,880 --> 00:10:25,679 automation. 121 00:10:25,679 --> 00:10:26,159 Exactly. 122 00:10:26,159 --> 00:10:30,960 That's, that's one of the reasons why I don't maintain my end automations and I'm just, 123 00:10:30,960 --> 00:10:36,559 I decided I'm a manual hacker and I live with, cause yeah, I had some tools in the past and it 124 00:10:36,559 --> 00:10:40,000 was consuming more time than actually saving. 125 00:10:40,000 --> 00:10:40,239 Yeah. 126 00:10:40,880 --> 00:10:48,559 How about authentication related bugs? Cause we will jump into, into particular bug cases 127 00:10:48,559 --> 00:10:56,080 because I know you have a lot of labs on JWT and SAML. And the question is what are the attacks 128 00:10:56,080 --> 00:11:01,840 which where you have to use custom tooling? Cause we have the tools in burp, we have extensions, 129 00:11:01,840 --> 00:11:06,880 we have some other tools, but are there some, some attacks that require us to set up some, 130 00:11:06,880 --> 00:11:07,760 something on our own? 131 00:11:08,400 --> 00:11:15,599 For SAML, SAML Raider is pretty good, but some exploitation requires you like are not supported 132 00:11:15,599 --> 00:11:20,880 by SAML Raider. Basically like as a signature wrapping attacks, not all of them are in SAML 133 00:11:20,880 --> 00:11:26,080 Raider. So sometime you need to like write your own script. It's not that complicated. It's a bit 134 00:11:26,080 --> 00:11:33,919 of XML parsing and rearranging information for JWT. I haven't looked at JWT extension because I 135 00:11:33,919 --> 00:11:37,039 write my own tool and that's why, yeah, that's why I'm trying. 136 00:11:37,440 --> 00:11:44,000 That's what I'm trying to teach people as well is it's good to rely on tool when you want to be 137 00:11:44,000 --> 00:11:48,880 fast, when you want to be accurate as well. But most of the time, what you want to do, 138 00:11:48,880 --> 00:11:55,119 especially when you're learning is learning to write your own tool. It may look like a waste of 139 00:11:55,119 --> 00:12:01,200 time at the start and it may be, but at one point you're going to end up like, okay, this protocol, 140 00:12:01,200 --> 00:12:07,039 these things, there is no tool for it. And if you don't know how to write your own tool, 141 00:12:07,039 --> 00:12:10,799 you're just going to be stuck forever. And you're going to have to write, to wait for someone to 142 00:12:10,799 --> 00:12:19,679 write the tool for you, which isn't great. And for example, there's a tool, JWT tool, I think, 143 00:12:19,679 --> 00:12:26,799 that a lot of people use to create, to attack JWT. It was written by someone who use, I don't 144 00:12:26,880 --> 00:12:31,760 know if it's still like a Pentest or a customer and all the tools were based on the labs and yeah. 145 00:12:31,760 --> 00:12:33,440 And I think that's something I'm really proud of. 146 00:12:33,440 --> 00:12:38,239 Yeah. Nice. Yeah. Well, actually I remember I have my own repo with Pentest or AppSource 147 00:12:38,239 --> 00:12:43,599 somewhere. It's private, obviously. And I remember there were like different Python 148 00:12:43,599 --> 00:12:48,960 scripts for like JWT, something. I remember it was a two digit number. So, so actually you can 149 00:12:48,960 --> 00:12:53,440 just probably compile all this to one good tool, add some, some more and it's probably. 150 00:12:54,000 --> 00:12:59,679 Exactly. And the learning you have from that, like being able to automate, like sending requests, 151 00:12:59,679 --> 00:13:04,159 doing like a bit of crypto, like HMAC signature and things like that, that's going to always, 152 00:13:04,159 --> 00:13:09,280 always be valuable. Like once you hit that level where like, oh, there is no tool to do that. 153 00:13:09,280 --> 00:13:10,719 I need to write my own. 154 00:13:10,719 --> 00:13:14,960 Yeah. Which, which is your language of choice these days, if you have to write a tool? 155 00:13:14,960 --> 00:13:21,760 Oh, Ruby. Always, always has been, always will be like, everyone is different. And I think I, 156 00:13:21,760 --> 00:13:26,640 I made a video on that actually. I'm not recommending a language to a person. I'm 157 00:13:26,640 --> 00:13:34,000 telling people like, okay, try a few and see how it goes. My brain with Ruby just work. Magic. 158 00:13:34,719 --> 00:13:40,320 Other people, I don't know why, Python work for them. You have to find the one that work for you. 159 00:13:40,320 --> 00:13:45,520 And once you got that one, like go like really hard on it and get really, really good at it. 160 00:13:45,520 --> 00:13:49,039 And for me, I spent too much time on Ruby, so I can't go back. 161 00:13:49,760 --> 00:13:55,039 I don't use it a lot, but it was one of the first languages where I learned to create a web 162 00:13:55,039 --> 00:14:00,159 application because I had it at the university. And I really liked it back then to have like 163 00:14:00,159 --> 00:14:05,200 different controllers because it was generate the whole application or the files. I'm talking 164 00:14:05,200 --> 00:14:12,880 about race, obviously. And this year I think I created the CTF challenge in Go, but I think 165 00:14:12,880 --> 00:14:18,080 last year I used, I use Ruby. I use race for it because I just liked it. But then there are so 166 00:14:18,080 --> 00:14:23,760 many files. If you generate the app with Rails, I'm like, and I didn't touch most of them. 167 00:14:24,479 --> 00:14:32,400 Then I'm a bit afraid of what is in my application. Yeah. It's as a trade-off, you get a lot for free, 168 00:14:32,400 --> 00:14:36,880 but you get a lot of for free. So you don't never necessarily know what the attack surface is, 169 00:14:37,520 --> 00:14:43,280 but yeah, right. It's really good for productivity. Yeah. Well, there's some hacking tools 170 00:14:43,280 --> 00:14:48,320 for Rails because hacking, sorry, hacking tips for race. I mean, because one of, one of the 171 00:14:48,320 --> 00:14:54,159 things I know it does automatically at developers may not know about this is it adds the dot Jason 172 00:14:54,159 --> 00:14:59,119 route implicitly. What other tricks you, you commonly use with, with phrase. 173 00:15:00,239 --> 00:15:07,119 So a lot of people forget to put authentication properly. So authorization properly. So I think 174 00:15:07,119 --> 00:15:13,200 it's always worth, like when you look at a Rails application to compare all the authorization 175 00:15:13,200 --> 00:15:18,559 check, because often people will put it for like the show for the index, but they will forget to 176 00:15:18,559 --> 00:15:25,200 put it in place for update or edit or delete. Mass assignment is a good example as well, where you can 177 00:15:25,200 --> 00:15:30,080 add parameter to the HTTP request. And this parameter may be mapped to the object, 178 00:15:30,080 --> 00:15:35,520 to the underlying object. So you use, for example, you add admin equals one in the URL 179 00:15:35,520 --> 00:15:39,280 or in the post request, in the body of the request, and then you'll become admin when 180 00:15:39,280 --> 00:15:48,719 you create an account. Yeah. A lot of things, but what else Ruby, no race removed XML processing by 181 00:15:48,719 --> 00:15:55,039 default a while back. Yeah. Mostly around authorization. Yeah. I think authentication 182 00:15:55,039 --> 00:16:01,760 is most of the time handled by device, which is pretty good. Yeah. There was a bug in device 183 00:16:02,400 --> 00:16:08,960 last year in James Cuddle research. Am I correct? Yes. Yeah. Was it really exploitable in the wild? 184 00:16:08,960 --> 00:16:13,760 I think it was. Yeah. Okay. Nice. Yeah. I haven't, you need the right timing and everything, but yeah. 185 00:16:14,400 --> 00:16:19,760 Yeah. It's cool. Have you played with this year's research from guys from Portswiger? 186 00:16:19,760 --> 00:16:26,080 Not yet. Not yet. I need to like, look if like right now I'm like deep diving into source code 187 00:16:26,080 --> 00:16:30,960 review and like creating like a big, I created like already a big training, but trying to improve 188 00:16:30,960 --> 00:16:34,479 it, improve it, improve it. So I don't do as much hacking. I do a lot of code review. 189 00:16:35,039 --> 00:16:43,200 Okay. Well, there's some, you're like best code review tips that, that you give to people. 190 00:16:43,840 --> 00:16:50,000 Read the source code. People just grep, grep, grep. And then they say like, oh, I suck at code 191 00:16:50,000 --> 00:16:55,840 review. I'm like, are you actually reading code? And most of the time people are like, ah, actually 192 00:16:55,840 --> 00:16:59,840 no, I'm just greping and looking at result. And that's not what code review is about. 193 00:16:59,840 --> 00:17:03,679 Are you a sources to sync or sync to sources type of reviewer? 194 00:17:04,319 --> 00:17:09,760 I mostly use like different approach, not necessarily like, like that source to sync 195 00:17:09,760 --> 00:17:16,800 or sync. I'm trying, I like to look at sync first, but I'm more like around, like looking 196 00:17:16,800 --> 00:17:22,479 at differences, like comparing like two code bases, doing the same thing, looking at one 197 00:17:22,479 --> 00:17:28,000 functionality in depth and things like that. Yeah. Okay. Yeah. I know there's no one 198 00:17:28,000 --> 00:17:32,319 definitive answer to this question. I wouldn't give a definitive answer myself, but I still ask 199 00:17:32,319 --> 00:17:38,400 it to, to, to see because people, people often expect there's just one good way to do something. 200 00:17:38,400 --> 00:17:41,439 No, like programming finds a way that works for you. 201 00:17:41,439 --> 00:17:47,119 Yeah. What do you say to people that I'm, that ask another very common question is, 202 00:17:47,119 --> 00:17:52,959 do you need to be able to code to be a hacker? That's a very good question. I think you don't 203 00:17:53,040 --> 00:17:56,640 have to, but oh my God, you're missing something. 204 00:17:56,640 --> 00:17:57,520 Yeah, for sure. 205 00:17:57,520 --> 00:18:02,719 Like it's amazing. You write something and bang, you ran it. And it does, the computer does 206 00:18:02,719 --> 00:18:08,160 whatever you tell it to do. It's amazing. Like you don't need it to be a hacker, but you're 207 00:18:08,160 --> 00:18:12,959 missing on something amazing. It's like, yeah, it's like you've got this amazing part of computer 208 00:18:12,959 --> 00:18:17,040 science and you're not doing it. And it's like, you don't need to be the best programmer in the 209 00:18:17,040 --> 00:18:21,760 world, but like, yeah, you need to code just because it's amazing feeling. 210 00:18:21,760 --> 00:18:26,479 It's also like, there are some things that you see as a, when you, when you just use the 211 00:18:26,479 --> 00:18:32,319 code as a, as a developer that I think would be hard to pick up just with black box approach all 212 00:18:32,319 --> 00:18:40,319 the time. I'm actually planning to just create my own like SAML and SSO servers to like control 213 00:18:40,319 --> 00:18:45,119 absolutely everything and be able to do some attacks that I couldn't do with like existing 214 00:18:45,119 --> 00:18:52,319 IDPs. So, so I'm planning to do it, but because I know also I will probably find out some things, 215 00:18:52,319 --> 00:18:58,959 some ideas for attacks that I, I don't know about them yet. So I definitely want to do it. 216 00:18:58,959 --> 00:19:04,640 That's why you need to read code as well, instead of graphing, because you want to find those 217 00:19:04,640 --> 00:19:10,400 unknown unknowns, things you wouldn't think of. And that's where the bugs are. Like if you're 218 00:19:10,959 --> 00:19:14,719 doing hard targets or like bug bounty on things that a lot of people have tested, 219 00:19:15,439 --> 00:19:20,319 sure, everyone runs like run SAML radar, like three, three times and things like that. 220 00:19:20,319 --> 00:19:22,160 So you need to come up with your own thing. Right. 221 00:19:22,959 --> 00:19:27,760 Yeah. If you, if you were supposed, if you are supposed to create a new web application from 222 00:19:27,760 --> 00:19:33,920 scratch, do you use a lot of AI for, obviously for some reproducing, some vulnerability, 223 00:19:33,920 --> 00:19:37,680 do you use a lot of AI to, to, to support you writing the code? 224 00:19:37,680 --> 00:19:43,680 A little bit for like the boiler plating part, like getting the things to work together. 225 00:19:44,959 --> 00:19:48,880 But most of the time I need to write, like the interesting part, I write myself because yeah. 226 00:19:48,880 --> 00:19:51,280 Okay. So, so how do you use it to boilerplate? 227 00:19:51,920 --> 00:19:56,800 Mostly like to create, like, Oh, I need like a simple web application in Python 228 00:19:56,800 --> 00:20:00,640 and give me like the structure and I got everything working. Or if I come across 229 00:20:00,640 --> 00:20:04,800 an error message, like. So you can set GPT for it or something else? 230 00:20:04,800 --> 00:20:06,400 Have you tried Courser? 231 00:20:06,479 --> 00:20:07,199 Not yet. 232 00:20:07,199 --> 00:20:09,839 I haven't, I haven't either, but it sounds really good. 233 00:20:09,839 --> 00:20:11,280 Yeah. Like, yeah. 234 00:20:11,280 --> 00:20:17,599 This co-pilot, but I'm the same. I just ask chat GPT. Although my conversations are quite 235 00:20:17,599 --> 00:20:21,680 long with chat GPT because then I ask it to do something more and the changes. And 236 00:20:21,680 --> 00:20:27,359 it's actually not, not bad, especially for, there is some, some point where it's, 237 00:20:27,359 --> 00:20:31,680 it starts to get so complex. It does hallucinate a bit, or maybe not hallucinate, 238 00:20:31,680 --> 00:20:36,079 but just make mistakes. But for starting, it's definitely, definitely nice. 239 00:20:36,560 --> 00:20:41,680 Yeah. It definitely hallucinate on CV. I can tell you that like creating CV numbers 240 00:20:41,680 --> 00:20:47,119 out of thin air. I just, Oh, I'm looking for a bug in VAT with VAT. Do you remember? 241 00:20:47,920 --> 00:20:52,800 Like there is this bug that I read about, like, I don't know, 10 years ago and I can't find it. 242 00:20:53,680 --> 00:20:57,520 And I asked chat GPT to try to find it for me and couldn't find it. I mean, 243 00:20:57,520 --> 00:21:01,680 chat GPT made up like 10 different CVs and all of them were wrong. 244 00:21:01,680 --> 00:21:05,920 Oh, that's interesting. I would think this is the perfect use case for it. 245 00:21:05,920 --> 00:21:07,040 Exactly. But no. 246 00:21:08,479 --> 00:21:15,280 Well, I think for me as a manual hacker, it's not bad. I mean, it's inevitable that AI will, 247 00:21:16,079 --> 00:21:20,400 will be used for hacking and probably hacking agents or hacking assistant or whatever will, 248 00:21:20,400 --> 00:21:24,880 will, will happen at some point, but I'm not complaining if it's a bit slow. 249 00:21:24,880 --> 00:21:26,319 That's why you need more code review. 250 00:21:26,959 --> 00:21:32,239 Yeah. What, what, what do you think are bad classes that are so complex that even the 251 00:21:32,319 --> 00:21:36,640 hacking agents of the future will not be able to find the bugs? 252 00:21:37,920 --> 00:21:46,160 That's a good question. I think you need to look at what automated scanner don't find today. 253 00:21:46,160 --> 00:21:46,640 Yeah. 254 00:21:46,640 --> 00:21:54,560 Like all the crypto attacks, all the interactions between components, all of those things where 255 00:21:54,560 --> 00:22:00,400 you ever need to know like small, really small details that you can only have from code review 256 00:22:01,040 --> 00:22:05,920 or things where you need to aggregate a lot of different things, like chaining vulnerabilities, 257 00:22:05,920 --> 00:22:10,959 for example, would be another one where the AI may struggle a bit more than human, but yeah. 258 00:22:10,959 --> 00:22:14,640 Yeah. I don't think we will lack work anytime soon. 259 00:22:14,640 --> 00:22:15,140 No. 260 00:22:16,239 --> 00:22:20,319 From learning or from observing different people learning, 261 00:22:20,880 --> 00:22:24,000 what are the most common problems that, that people face? 262 00:22:24,959 --> 00:22:29,520 The second one first. The second one is people don't read error messages. 263 00:22:30,959 --> 00:22:35,280 People don't read error messages. Trust me on that. They just see an error and think like it 264 00:22:35,280 --> 00:22:41,359 doesn't work. Okay. Or people assume like where, when there is an error, it mean it doesn't, 265 00:22:41,359 --> 00:22:46,640 it didn't work. Let's say you exploiting like a peak old serialization bug. At one point, 266 00:22:46,640 --> 00:22:50,640 the application will crash because you sent an object that is not the right object. 267 00:22:50,640 --> 00:22:51,119 Yeah. 268 00:22:51,119 --> 00:22:56,560 But the code execution happens before that. And people see the error message and say like, okay, 269 00:22:56,560 --> 00:23:03,839 that didn't work. The first one is a lot of people don't know what a public IP address is. 270 00:23:05,359 --> 00:23:11,839 And that's because a lot of environment, training environment are using like a jump host 271 00:23:12,400 --> 00:23:17,680 where you're in the same like private network as a target. But when you're attacking like real 272 00:23:17,680 --> 00:23:22,560 target on the internet, you need like an IP address or you need to use ngrok and things 273 00:23:22,560 --> 00:23:27,199 like that. And a lot of people email support or email me saying, oh, that doesn't work, 274 00:23:27,199 --> 00:23:30,719 but they're using like a private IP address for the connect back and things like that. 275 00:23:31,439 --> 00:23:36,560 I remember in the early days, you had some labs as virtual machines. 276 00:23:36,560 --> 00:23:37,119 Yeah. 277 00:23:37,119 --> 00:23:41,680 And I remember also struggling. It was like, I didn't even know networking back then. 278 00:23:41,680 --> 00:23:48,000 Struggling to know on which IP address I can access the web app that's inside the VM and I 279 00:23:48,000 --> 00:23:54,959 can access outside the VM. I remember having problems with this and it's a hard topic for 280 00:23:54,959 --> 00:23:55,599 general, but. 281 00:23:56,479 --> 00:24:00,719 But I think, yeah, the problem is that people learn one way and then they don't realize that 282 00:24:00,719 --> 00:24:05,040 that's not what they should be using when they go on a hard time, like on a real target on 283 00:24:05,040 --> 00:24:09,280 the internet. And I'm pretty sure a lot of people are missing bugs because they're just like, oh, 284 00:24:09,280 --> 00:24:13,520 I don't have a connect back. But yeah, because you put like your private IP address, 285 00:24:13,520 --> 00:24:14,640 so it's never going to happen. 286 00:24:15,520 --> 00:24:16,020 Yeah. 287 00:24:16,479 --> 00:24:21,199 Where do you think people can learn these things? Because they are not directly related 288 00:24:21,199 --> 00:24:25,599 to any vulnerability. Where do you think someone can learn about this stuff? 289 00:24:25,599 --> 00:24:30,880 So that's very interesting. Back in the day, we were learning that because that's the only way we 290 00:24:30,880 --> 00:24:36,239 could like install like a mail server, an HTTP server at home or a DNS server. But nowadays, 291 00:24:36,239 --> 00:24:39,839 like everything is services online, so you don't need that knowledge as much. 292 00:24:39,839 --> 00:24:45,040 I think you need to learn as you go and yeah, just go and learn about public and private IP 293 00:24:45,040 --> 00:24:51,280 address. It's going to take like an hour, but you may find more bugs or not miss bugs at least. 294 00:24:51,839 --> 00:24:56,959 Yeah. This is the hard part because when you have a problem, you know that you need to look 295 00:24:56,959 --> 00:25:01,680 for the solution for the problem. But in the case you gave the example with somebody not getting 296 00:25:01,680 --> 00:25:06,319 the connect back from the server, they may not be aware they have the problem and they may not 297 00:25:06,319 --> 00:25:11,520 find out until a few years later. So this is the hard part, no? 298 00:25:11,520 --> 00:25:17,599 But it's a bit easier with a Burp collaborator now, so people don't have that problem as much. 299 00:25:19,839 --> 00:25:24,880 You kind of mentioned it a little bit, but back in the day, there were way fewer resources, 300 00:25:24,880 --> 00:25:33,199 but also I guess the entry barrier was lower. Now I think there are so many resources, 301 00:25:33,199 --> 00:25:36,800 it might be problematic to know which ones are the good ones. 302 00:25:37,520 --> 00:25:40,079 So how do you think people have to navigate this? 303 00:25:41,680 --> 00:25:45,680 It depends where you're at in your journey as well. It's easier and easier to find the good 304 00:25:45,680 --> 00:25:49,920 resources once you know what you're looking for. At the beginning, I think you need to find 305 00:25:49,920 --> 00:25:53,839 somewhere where you have good content and curation, so you don't end up doing the same 306 00:25:53,839 --> 00:25:58,880 thing again and again and again. That's one thing that is hard with creating content for Pentestal 307 00:25:59,599 --> 00:26:05,199 is not repeating myself, same lab again and again with 50 variations of the same thing, 308 00:26:05,839 --> 00:26:13,760 but also trying to make it harder and harder, but not big steps, just small steps, smaller steps, 309 00:26:13,760 --> 00:26:19,760 small steps all the time. I think the important things when you're learning, and I did a keynote 310 00:26:19,760 --> 00:26:26,800 on that, is you don't want... So what people do usually, they say, okay, I'm going to be a 311 00:26:26,800 --> 00:26:30,479 beginner at Linux, beginner at Windows, beginner at web, beginner at reversing. 312 00:26:31,040 --> 00:26:36,079 Then once they got this basic knowledge, they move to medium, like intermediate, 313 00:26:36,079 --> 00:26:43,520 intermediate web, intermediate Linux, intermediate Windows, but too hard to do all those things. 314 00:26:43,520 --> 00:26:50,560 What you need to do is pick one, get really good at Windows, or get really good at Linux, 315 00:26:50,560 --> 00:26:55,359 or get really good at reversing, whatever. And early on, get something, pick something, 316 00:26:55,359 --> 00:27:00,079 and get really, really good at that thing. And in that thing, take something that is even smaller 317 00:27:00,079 --> 00:27:06,800 and get even better at that thing, like Tomcat, or MySQL, or whatever, and get really, really good 318 00:27:06,800 --> 00:27:13,760 at that thing. Because the lesson you're going to learn, the hard lesson, when you learn hard things, 319 00:27:13,760 --> 00:27:19,920 then you can reproduce that on all the other things. And because that last part, getting 320 00:27:19,920 --> 00:27:26,400 really, really good at something, that's when you do your own research, you can present the research, 321 00:27:26,400 --> 00:27:31,359 and you are not using someone else's tool, you are not using or following someone else's research, 322 00:27:31,359 --> 00:27:35,280 you are doing your own. And that's probably the hardest part. And once you know how to do that, 323 00:27:35,280 --> 00:27:39,520 it's really easy to replicate that to other technologies. Yeah, that's really, really smart. 324 00:27:42,800 --> 00:27:46,479 Yeah, because it's impossible at some point to learn everything. And 325 00:27:47,439 --> 00:27:54,319 also, when you get to a certain level, once you're a beginner, there are so many resources everywhere. 326 00:27:54,319 --> 00:28:00,160 But once you get intermediate or advanced, it gets harder and harder. So how to still 327 00:28:00,160 --> 00:28:06,160 keep learning new things if we are more advanced at something? I think you need to look at 328 00:28:07,520 --> 00:28:13,920 things that are around the task you want to achieve that will help you go deeper or be faster. 329 00:28:14,640 --> 00:28:20,640 Let's say you want to look at SAML, you want to be really good at SAML, you need to get better 330 00:28:20,640 --> 00:28:25,280 at code review, you need to get better at debugging, you need to get better at scripting, 331 00:28:25,280 --> 00:28:30,560 you need to get better at crypto, you need to get better at XML signature, all those things 332 00:28:30,560 --> 00:28:36,319 that are around SAML and will make you will enable you to find those SAML bugs. 333 00:28:37,199 --> 00:28:44,560 Okay. So it's more like solving problems in time where you try to reproduce some bug and 334 00:28:44,560 --> 00:28:50,400 you learn something else? Or would you like consciously sit and think what things around 335 00:28:50,400 --> 00:28:56,079 the SAML I have to learn and only then learn these things and then learn the one in the middle? 336 00:28:56,079 --> 00:29:02,479 Oh, it's more like looking at other tasks around SAML, like trying to see like, 337 00:29:02,479 --> 00:29:09,760 what do I need to know to find things that maybe other people missed, or to be more efficient, 338 00:29:09,760 --> 00:29:17,920 more accurate, or do new research on that subject. And often that's, okay, I need to debug 339 00:29:17,920 --> 00:29:21,599 and like look at a debugger for hours, looking at exactly what SAML is doing, 340 00:29:22,560 --> 00:29:26,560 looking at the code review for like looking at SAML source code and go like through like 10 341 00:29:26,560 --> 00:29:32,719 implementations of it, and things like that. And all of that won't, are tasks that are not 342 00:29:32,719 --> 00:29:37,920 necessarily linked to the SAML, but they will help you like go to the next level. 343 00:29:37,920 --> 00:29:44,959 Yeah. So this is at this point, it's mostly just learning with the code that exists and with 344 00:29:44,959 --> 00:29:48,719 our experience, and it's not no longer looking for resources online. 345 00:29:48,719 --> 00:29:54,160 No, it's, that's often when people get stuck. And that's often when people move to something else, 346 00:29:54,160 --> 00:30:01,119 because it's hard. It's not comfortable. It's, you're wasting your time because you're like, 347 00:30:01,119 --> 00:30:04,239 okay, or you feel like you're wasting your time because sometimes you're going to look at 348 00:30:04,239 --> 00:30:08,479 something and it's, you're not going to yield anything. You're just not going to find anything, 349 00:30:08,479 --> 00:30:14,239 but you need to go through that journey to get better at one point. Otherwise you just get stuck 350 00:30:14,239 --> 00:30:18,560 and move to like, okay, I'm going to buy another certification or I'm going to buy like a new 351 00:30:18,560 --> 00:30:22,079 training. But at one point you need to start suffering on your own, I guess. 352 00:30:24,479 --> 00:30:31,359 Yeah. How should people know which is the moment to learn more advanced topics from the area we're 353 00:30:31,359 --> 00:30:35,760 already quite good at and where is the time to learn something new? 354 00:30:35,760 --> 00:30:42,640 I think when you start to find that all the training you can find on one topic you're 355 00:30:42,640 --> 00:30:49,439 interested in are things you already know, or like small variations of something you already 356 00:30:49,439 --> 00:30:54,880 know, or the research you look at is like something, okay, I didn't find it, but I could 357 00:30:54,880 --> 00:31:00,719 have found it because it's now that I see it, it's obvious. I never looked at that code base, 358 00:31:00,719 --> 00:31:05,280 but now that I, now that I read that blog post, I think I could have found that bug. 359 00:31:05,280 --> 00:31:09,119 And I think that's the point where you're ready maybe to like go to the next level 360 00:31:10,079 --> 00:31:15,839 and to move to something else. I think it's, sometimes it's good to look at something else 361 00:31:15,839 --> 00:31:20,719 to get better at that initial thing as well. So if you get stuck in your journey here, like 362 00:31:21,520 --> 00:31:25,280 you're learning some old and you're like, after a while you get stuck at some old, 363 00:31:26,160 --> 00:31:31,280 maybe it's time to move to something else. Maybe look at Windows Federation, for example. 364 00:31:31,280 --> 00:31:31,599 Yeah. 365 00:31:31,599 --> 00:31:34,479 And things like that to maybe, and then you come back to someone and you 366 00:31:34,479 --> 00:31:37,119 may have like a new vision, a new way to look at things. 367 00:31:37,920 --> 00:31:44,400 Yeah. How about people that think about more general topics? Let's say they feel quite good 368 00:31:44,400 --> 00:31:49,520 with web. Of course, there's still a lot to learn because web is huge. And within web, 369 00:31:49,520 --> 00:31:53,280 you have small niches, like you just mentioned some, we keep mentioning it. 370 00:31:54,239 --> 00:31:59,760 How do they know if they should stick with web or maybe learn Linux, learn cloud, learn things? 371 00:31:59,760 --> 00:32:05,760 Because I think a lot of people get to the point in their career where they are not sure if they 372 00:32:05,760 --> 00:32:10,400 have to learn something new to have a better resume or they should just master one area. 373 00:32:10,959 --> 00:32:14,959 So I think it depends on what interests you and what's your long-term goals as well. 374 00:32:15,520 --> 00:32:22,079 If, let's say you're like at that level, you're pretty good and you tell yourself, 375 00:32:22,079 --> 00:32:26,560 like, I want to be a CISO in 10 years, or I want to be a manager in two years or three years, 376 00:32:26,560 --> 00:32:31,760 you probably want to look at other things. If your goal is, I want to be a security researcher 377 00:32:32,479 --> 00:32:36,719 or I want to go to point to own or things like that, you probably want to stick to that one 378 00:32:36,719 --> 00:32:44,079 thing. Yeah. Yeah, I agree. For context of bug bounty, it's probably almost always better to 379 00:32:44,079 --> 00:32:52,800 specialize in as small thing as possible because you just need the very, I'm a very, very good 380 00:32:52,800 --> 00:32:58,560 example. I'm really good at web, but I don't have a lot of like cloud experience. I have never done 381 00:32:58,959 --> 00:33:06,319 Windows pen testing, not even like Linux pen testing. I maybe it's okay. I could do something, 382 00:33:06,319 --> 00:33:12,479 but I've never actually done a pen test on my own. So because I always knew I want to do bug bounty 383 00:33:12,479 --> 00:33:16,640 and I was just learning about more and more advanced things from bug bounty. 384 00:33:18,400 --> 00:33:23,520 And I think for me, it works really well. But I think if I were for hypothetically, 385 00:33:24,079 --> 00:33:29,920 if now I were to find a new job, my resume wouldn't look that impressive because there 386 00:33:29,920 --> 00:33:34,640 would be like web skills. Okay. They would be really strong. But then probably the company 387 00:33:34,640 --> 00:33:40,640 would prefer somebody that I don't know, knows G GCP and AWS and maybe big part of, 388 00:33:40,640 --> 00:33:47,439 of like my web knowledge. A bad manager will, a good manager will understand that your knowledge 389 00:33:48,079 --> 00:33:52,959 can be passed to other people in your team and their knowledge can be passed to you. 390 00:33:53,280 --> 00:33:57,760 And that's easily to transfer that knowledge. That's a lot easier to transfer knowledge to 391 00:33:57,760 --> 00:34:02,239 someone who is really good in one thing and having this person transferring knowledge to other people 392 00:34:02,239 --> 00:34:06,079 than just having a team of people who are really average and just know the same thing. 393 00:34:06,800 --> 00:34:09,280 Yeah. Do you think there are a lot of managers like this? 394 00:34:09,280 --> 00:34:10,000 I hope so. 395 00:34:10,000 --> 00:34:10,560 Really? 396 00:34:10,560 --> 00:34:12,879 Yeah. Smart managers. 397 00:34:12,879 --> 00:34:16,879 Yeah. I'm not planning to just, just that's the way I think about it. 398 00:34:16,879 --> 00:34:20,479 For bug bounty, it completely makes sense to invest in things. If you look at 399 00:34:21,360 --> 00:34:27,199 people finding good bugs, they usually spend a lot of time specializing on one or two things. 400 00:34:27,199 --> 00:34:27,439 Yeah. 401 00:34:27,439 --> 00:34:31,600 Like they get really good at OAuth. They get really good at timing issues. They like, 402 00:34:32,239 --> 00:34:37,600 you need to find something that is hard. So everyone else before you gave up. 403 00:34:37,600 --> 00:34:44,159 Yeah. Yeah. Especially if you get to like, uh, events with, with hacking where you just have 404 00:34:44,159 --> 00:34:49,199 to compete with everyone else and you have to somehow find bugs that they didn't find 405 00:34:49,520 --> 00:34:53,760 because otherwise it doesn't make sense. So that's when the fun starts. Cause I think it's 406 00:34:53,760 --> 00:35:01,600 for bug bounty, you can still earn fairly, fairly good wage for finding pretty simple bugs, 407 00:35:01,600 --> 00:35:07,199 but finding a lot of them on, on, uh, lower paying programs. But when you get to, 408 00:35:07,199 --> 00:35:12,080 when you want to, to be at the events and the big public programs and things like this, 409 00:35:12,879 --> 00:35:18,080 we start to, to like need to be more specialized and detailed and things like this. 410 00:35:18,080 --> 00:35:24,719 So yeah, it definitely pays off in bug bounty to just be, be a specialist. 411 00:35:24,719 --> 00:35:25,760 Definitely. 412 00:35:25,760 --> 00:35:30,159 And even, even to the level, not only speaking about back classes, even specialists that 413 00:35:30,159 --> 00:35:35,199 a particular program, there are a lot of people that are just specializing at one program and 414 00:35:35,199 --> 00:35:40,159 that's a skill of its own. That's also probably fairly transferable to other things. It's not 415 00:35:40,159 --> 00:35:47,439 like you learn one website or one company and then the skills from this are irrelevant, but 416 00:35:48,320 --> 00:35:52,239 yeah, it's, it's a very, very, very specialized industry, I would say. 417 00:35:52,239 --> 00:35:57,520 Yeah. Because when you're really good at these few things or these few programs, 418 00:35:58,159 --> 00:36:02,800 you can think in terms of patterns and then map those patterns back to other programs, 419 00:36:02,800 --> 00:36:04,560 other technology and things like that. 420 00:36:04,560 --> 00:36:10,080 Yeah. I even thought about maybe just spending a lot of time on one of the, let's say password 421 00:36:10,080 --> 00:36:15,280 managers and then trying the same things and just, you know, six, seven of, of, of the most 422 00:36:15,280 --> 00:36:18,239 common one. And I haven't done this, but maybe I will. 423 00:36:18,879 --> 00:36:22,879 That's one of the thing I like to do in code review. And that's one of the thing I teach is 424 00:36:22,879 --> 00:36:26,719 place, but the difference, you know, like you've got two images and you need to find 425 00:36:26,719 --> 00:36:31,439 like the differences between two, you take two code base, doing exact two code bases, 426 00:36:31,439 --> 00:36:36,000 doing exactly the same thing. And you compare them. You just read, okay. They're doing check 427 00:36:36,000 --> 00:36:40,879 one, two, three, they're going check three, four, five. Oh, it's that different. Either 428 00:36:40,879 --> 00:36:44,719 does the order matter? What is the same? What is different? And it's a lot easier 429 00:36:44,719 --> 00:36:48,320 because you don't need to know what you're looking for. You just need to compare things. 430 00:36:48,320 --> 00:36:52,399 Yeah. Do you, are you doing it as a conscious process that you, 431 00:36:52,399 --> 00:36:54,879 you literally have to code basis next to each other? Really? 432 00:36:54,879 --> 00:36:59,600 Yeah. Oh, not, not, not necessarily at the same time, except for like small snippets. 433 00:36:59,600 --> 00:37:00,080 Yeah. 434 00:37:00,080 --> 00:37:06,159 But, um, like I found a bug in a play or a session injection in play. I was like taking 435 00:37:06,159 --> 00:37:10,080 the train every day to go to work. And one day I decided, oh, I'm going to look at, uh, 436 00:37:10,080 --> 00:37:13,040 five or 10 implementation of session in Java applications. 437 00:37:13,040 --> 00:37:13,360 Yeah. 438 00:37:14,000 --> 00:37:20,000 They all look the same except play very different. And I was like, okay, I'm going to investigate. 439 00:37:20,000 --> 00:37:23,919 Yeah. And you found a bug because of this. Can you say what was the bug? 440 00:37:23,919 --> 00:37:30,080 Oh, it was, um, so basically it's a science session, like Ruby on Rails. So you can't 441 00:37:30,080 --> 00:37:35,600 modify the content of the session without modifying the signature and you don't have 442 00:37:35,600 --> 00:37:41,840 the secret. So you can't forge a signature. But what you could do is you got like a key value, 443 00:37:41,840 --> 00:37:47,360 key value in the session. You could inject in the value to add more elements in the session. 444 00:37:47,360 --> 00:37:50,959 Also, we could just, uh, close out the bracket. 445 00:37:50,959 --> 00:37:51,760 Close out the quotes. 446 00:37:51,760 --> 00:37:55,919 Exactly. So they weren't using JSON. They were using their own format and that's what the bug 447 00:37:55,919 --> 00:38:02,560 was. They were using like a, so null byte, key, colon, value, null byte, null byte, key, another 448 00:38:02,560 --> 00:38:07,760 key, colon, another value. And you could inject a null byte in your username, for example, and 449 00:38:08,479 --> 00:38:10,560 then add whatever you wanted. 450 00:38:10,560 --> 00:38:16,639 Yeah, that's so nice. I had the CTF test quite similar to this some time ago. There was some 451 00:38:16,639 --> 00:38:21,840 very strange, also assigned science session with some attributes, but also there were 452 00:38:23,199 --> 00:38:33,520 the binary format had the length before the value. And, um, and you had to create, uh, 453 00:38:34,239 --> 00:38:40,399 wait, what was how? Okay. You had to, the challenge was that the app used the same 454 00:38:41,520 --> 00:38:47,360 private key for signing these cookies and for SSH authentication. So I had to create my own 455 00:38:47,360 --> 00:38:54,719 SSH challenge that would sort of match this binary format for the session. So then the process of the 456 00:38:54,719 --> 00:38:58,959 challenge of SSH would give me the valid signature that I could use in the HTTP request. 457 00:38:58,959 --> 00:38:59,600 Nice. 458 00:38:59,600 --> 00:39:05,760 It was so nice. I don't think I could ever find this without the access to the source code, 459 00:39:05,760 --> 00:39:07,120 but it was so fun. 460 00:39:07,120 --> 00:39:10,000 Yeah, that's, yeah. That's why I like about source code review. 461 00:39:10,719 --> 00:39:17,280 What are the other bad classes that you think are quite common in the world, but are not 462 00:39:17,280 --> 00:39:21,280 discovered only because most people don't have access to the source code? 463 00:39:21,919 --> 00:39:30,560 Missing randomness, uh, crypto, crypto, crypto, and, um, some level of authorization. 464 00:39:31,439 --> 00:39:35,760 It's interesting actually, because, um, so I'm looking at a lot of CV regularly 465 00:39:36,320 --> 00:39:42,000 and one thing you never know is how the bug was found. Yeah. And sometimes you find, 466 00:39:42,000 --> 00:39:48,239 you see bugs like, okay, if it's, if this person who found it using code review, that's amazing. 467 00:39:48,239 --> 00:39:53,439 If I found it using, uh, just like fuzzing, it's pretty boring and you're like, Oh, I wish I knew. 468 00:39:54,239 --> 00:40:00,239 Yeah. Yeah. It's a shame that, that from, uh, a lot of time, even in write-ups, 469 00:40:00,239 --> 00:40:04,320 when somebody writes a blog post, we don't actually know how they discovered the bug. 470 00:40:04,320 --> 00:40:09,840 I really, really like it when somebody not only describes the bug shops is really good at this or 471 00:40:09,840 --> 00:40:14,560 asset node in general, they always write the steps, how they discovered it, how it actually works, 472 00:40:14,639 --> 00:40:20,320 why it works. Then they only, then they get to the back classes also all with like code snippets. 473 00:40:20,320 --> 00:40:25,679 Yeah. So it's, it's a really, really like good, good, good thing to read, but it's, it's quite 474 00:40:25,679 --> 00:40:30,320 rare. No. Yeah. Like explaining some mindset, the thought process and all of that thing is really, 475 00:40:30,320 --> 00:40:37,360 really valuable. Yeah. You also do read a lot of write-ups now. Yeah. So how do you sort of optimize 476 00:40:38,080 --> 00:40:41,600 like, you know, that's a lot of write-ups are not really interesting for you. 477 00:40:42,239 --> 00:40:46,879 And I assume you just have a way to like first scan the write-up to see if it's interesting or not. 478 00:40:46,879 --> 00:40:53,120 I just like read it quickly. Okay. Uh, friends of mine created something named talk back and 479 00:40:53,120 --> 00:40:59,439 the aggregate, uh, write-ups and things like that. Uh, like all security research and rate it like 480 00:40:59,439 --> 00:41:04,399 automatically rate it, export like a keywords using like AI and things like that. It's really, 481 00:41:04,399 --> 00:41:08,800 really good. Yeah. That's nice. And it's free. So yeah. Really? Yeah. What's the name? 482 00:41:08,800 --> 00:41:14,800 Talk back, talk back. Okay. I will make sure to, to add this later to the description 483 00:41:15,360 --> 00:41:21,760 because it would be really interesting for me for the case studies because I do also read the 484 00:41:21,760 --> 00:41:28,239 write-ups when I do the case study and I usually scroll to steps to reproduce. And this is like 485 00:41:28,239 --> 00:41:34,239 the, the, the most valuable thing of the writer, but this, this is really going to really help me. 486 00:41:35,120 --> 00:41:38,320 Yeah, no, it's pretty good. But it's, uh, if they don't do bug bounty write-ups, 487 00:41:38,320 --> 00:41:42,959 they mostly do like security research outside of that. So probably not as useful, but maybe 488 00:41:42,959 --> 00:41:47,280 they would be happy to add it as well. Is the tool itself public or just the database? 489 00:41:47,280 --> 00:41:51,840 Just, uh, data. So maybe it's not open source, but maybe I can talk with them. 490 00:41:53,280 --> 00:41:57,919 You're really, really cool. What are other obstacles? You mentioned people don't read 491 00:41:57,919 --> 00:42:01,919 errors. You mentioned people, uh, don't know public and private IP addresses. 492 00:42:01,919 --> 00:42:05,760 What are some obstacles? Maybe not as specific ones, but in general, 493 00:42:05,760 --> 00:42:09,760 things that just hold people back and they can't advance or can solve a challenge. 494 00:42:10,479 --> 00:42:17,840 So I think a lot of people tend to give up at the same time. And that's very like, uh, applicable to 495 00:42:17,840 --> 00:42:24,479 bug bounty is that it feels like everyone is trying, trying, trying, and just give up at the 496 00:42:24,479 --> 00:42:29,439 same time. And I see people like struggling exactly at the same point on this challenge. 497 00:42:30,000 --> 00:42:35,760 And there's this image of like someone digging and there's like, um, a diamond and we're just 498 00:42:35,760 --> 00:42:40,639 like that close to like all the diamonds and we just give up and like turn around and go back. 499 00:42:40,639 --> 00:42:44,800 And that's always something I have in mind when I do like research or like trying to find bugs, 500 00:42:44,800 --> 00:42:50,080 it's like, am I as a guy, like, uh, but like turning down and like being so close. And I 501 00:42:50,080 --> 00:42:55,679 think people need to like, okay, when they're learning, when they're trying to find bugs, 502 00:42:55,679 --> 00:43:00,639 when they're exploiting bugs, they need to like, okay, take a deep breath, go for a walk, 503 00:43:00,639 --> 00:43:05,760 come back and spend like two hours, five hours, two days more. And they definitely 504 00:43:05,760 --> 00:43:12,879 going to find more bugs. Like everyone gave up too early. Really? Yeah. How do you know you're 505 00:43:12,879 --> 00:43:18,000 not spending too much time on something that's not vulnerable? You may be spending too much time, 506 00:43:18,560 --> 00:43:24,080 but most of the time it's worth spending too much time. Okay. Because no one else did. 507 00:43:24,959 --> 00:43:30,479 Yeah. That's interesting. How do you have some sort of visualization of the data in 508 00:43:30,479 --> 00:43:36,639 PentesterLab where you see where people, um, where people give up or something like this? 509 00:43:36,639 --> 00:43:40,399 No, I don't have that, but I do, I do the support. So it's pretty easy to know, 510 00:43:40,399 --> 00:43:45,360 like exactly what's the, so you know it from like people's messages. Exactly. Okay. Yeah. 511 00:43:45,360 --> 00:43:49,919 That's interesting. It's like you have people like you so closely, like one type of way you're 512 00:43:49,919 --> 00:43:55,439 like, and yeah, but it's hard. Like people get frustrated, people don't understand, people are 513 00:43:55,439 --> 00:43:59,280 learning. So it's hard, like learning to hack is really hard. Like it's a lot of work, but 514 00:43:59,919 --> 00:44:05,120 most of the time it's like, yeah. And that's as well, why I like to do support is like trying to 515 00:44:05,919 --> 00:44:11,280 help people, but not too much because like, if you want them to remember something, 516 00:44:11,280 --> 00:44:15,600 you need them to have this moment where like, Oh, I found it. Yeah. 517 00:44:15,600 --> 00:44:21,600 Yeah. That's, that's always scary to me that I will miss a bug because of a typo or something. 518 00:44:21,600 --> 00:44:26,639 Cause you know, once in a while I work on something and it doesn't work. And then after, 519 00:44:26,639 --> 00:44:32,959 sometimes after an hour, for example, I notice a mistake as stupid as a typo or just the system. 520 00:44:32,959 --> 00:44:38,800 Sometimes it was random. Like yesterday I wrote a writeup where somebody had SSRF 521 00:44:38,800 --> 00:44:47,600 of DNS rebinding that worked one in 30 attempts. So, and I also had had this year, some bugs that 522 00:44:47,600 --> 00:44:55,679 like, um, I had, um, it was a sort of a client side bug. Um, okay. I can't disclose too many 523 00:44:55,679 --> 00:45:00,639 details, but in general I had just had the loop that would send the same request over and over 524 00:45:00,639 --> 00:45:05,360 again because it would just randomly sometimes work and sometimes not with exactly the same 525 00:45:05,360 --> 00:45:12,479 payload. And knowing that, um, these things happened to me and I resolved them probably 526 00:45:12,479 --> 00:45:17,760 there were, there must be things that were similarly stupid problems or randomness problems 527 00:45:17,760 --> 00:45:22,560 or my type of problems. And I didn't find out about them and I missed bugs because of this. 528 00:45:22,560 --> 00:45:27,120 This is the scary part for me. I know a lot of people like you spent a lot for that. 529 00:45:27,679 --> 00:45:32,479 Just keep solving like the new labs all the time or go back to all labs for two things, 530 00:45:32,479 --> 00:45:39,040 speed and accuracy, like getting better, like getting more accurate and getting faster and 531 00:45:39,040 --> 00:45:46,959 getting faster at doing the thing. And also getting faster at detecting if, uh, it's not 532 00:45:46,959 --> 00:45:53,040 exploitable or they made a mistake. And again, like reading error messages is trying to like 533 00:45:53,040 --> 00:45:58,320 explore behaviors. It's like, um, am I doing it wrong or it's not exploitable. 534 00:45:58,800 --> 00:46:03,120 Yeah. But with, with labs and with CTFs, it's that much easier that, you know, 535 00:46:03,919 --> 00:46:08,560 you're probably, you're doing something wrong with real world. This is the real challenge. 536 00:46:08,560 --> 00:46:10,320 Yeah, definitely. 537 00:46:10,320 --> 00:46:15,040 How about the challenges itself? I assume you have some stats on which are the most popular 538 00:46:15,040 --> 00:46:20,159 one, like most popularly solved or attempted by people, which are the most popular. 539 00:46:20,159 --> 00:46:22,719 I was the early one and the easiest one. Yeah. 540 00:46:23,439 --> 00:46:28,000 About, I know that probably the, the, the free ones are the most popular by numbers. 541 00:46:28,399 --> 00:46:33,600 But out of, let's say the, the, the paid section, what, what is something that people choose a lot? 542 00:46:33,600 --> 00:46:37,280 So we have a path to tell people like that's the direction you should follow. So usually 543 00:46:37,280 --> 00:46:43,040 like people give up at one point. So that's the early one obviously is a, uh, most, 544 00:46:44,399 --> 00:46:52,000 like the most solved one. Um, I think in terms of like what people really enjoy, uh, is a JWT one 545 00:46:52,080 --> 00:46:57,360 and the crypto one. Yeah. Like people look at those and it's, it feels like magic. 546 00:46:57,919 --> 00:46:58,719 Yeah, it does. 547 00:46:58,719 --> 00:47:05,280 Like when you crack like a CBC or an ECB or like a CBC Mac, all these like things that 548 00:47:05,280 --> 00:47:11,919 are encrypted and it's like, Oh, it's just like basic 64 random crap. And you play with that. 549 00:47:11,919 --> 00:47:15,919 And then you get something else and people are amazed by that. 550 00:47:15,919 --> 00:47:19,439 Yeah. It's funny you mentioned it. I would probably say this was my favorite challenge 551 00:47:19,439 --> 00:47:24,399 with like one of the CBC ones because I never saw something like this in real life. I'm not 552 00:47:24,399 --> 00:47:31,280 a crypto guy and just having something like this was, was very, very fun. What's on the other hand, 553 00:47:31,280 --> 00:47:35,919 are there some challenges that people skip for some reason, even though they are on the path? 554 00:47:35,919 --> 00:47:42,000 Um, not necessarily because you need to finish them to have the badge. I think, um, 555 00:47:42,800 --> 00:47:48,879 we have like a lot of code review challenges now and it's hard to do scoring for code review. So 556 00:47:48,879 --> 00:47:54,560 people often get frustrated because of that. Um, because what we do is we ask people like to pin 557 00:47:54,560 --> 00:48:01,040 like a file name and a line number and the type of bug. And sometimes it's not that accurate. So 558 00:48:01,040 --> 00:48:06,159 it's like best effort kind of thing. So often like people get frustrated because they are reading 559 00:48:06,159 --> 00:48:11,919 source code and it's already hard. And then like, Oh, it doesn't work. And but now people are 560 00:48:11,919 --> 00:48:17,199 pretty happy. And yeah, it's not, it's not like, um, easy, like they're not, 561 00:48:18,080 --> 00:48:22,399 they are easy labs, but it's not like, um, an easy thing to do to learn all of that. 562 00:48:22,399 --> 00:48:26,479 It's a lot of content to go through. And yeah. Yeah. And then some challenges are, you know, 563 00:48:27,280 --> 00:48:32,719 a few hours of work. I remember this. So to finish, are there many people that have all 564 00:48:32,719 --> 00:48:37,360 the challenges solved? Yeah. Really? Yeah. Like, Oh, I'd say probably like, 565 00:48:38,159 --> 00:48:43,600 I know that at least like 12 to 15 people, like if there's a challenge is out the next day, 566 00:48:43,600 --> 00:48:49,760 it's solved. Like, yeah. Like loving it. Yeah. Yeah. That's nice. Congrats to you. 567 00:48:50,639 --> 00:48:58,399 If there's you. Yeah. Also about a PentesterLab, cause, uh, you, you just, you mentioned that you 568 00:48:58,399 --> 00:49:02,800 still do the customer support. So you intentionally run it as a small business. 569 00:49:03,360 --> 00:49:10,399 Yeah. Like I think doing customer support is great because, um, it helps me make the content 570 00:49:10,399 --> 00:49:16,000 better and understand what people struggle with. And I really enjoy hacking and things 571 00:49:16,000 --> 00:49:20,239 like that. So I don't, I don't like, I enjoy doing like a lot of different things as well. 572 00:49:20,239 --> 00:49:24,800 So that's why keeping it small really helps. And, uh, I'm having a lot of fun. I can't believe it's 573 00:49:24,800 --> 00:49:29,439 my job basically. Are you still doing it alone or do you have some people? 574 00:49:29,439 --> 00:49:35,439 I have people helping me with design, with, uh, looking at CVS, like mostly like the extraction 575 00:49:35,439 --> 00:49:41,040 of data for CVS and things like that. Yeah. And yeah. Yeah. I look up to it because I think 576 00:49:42,080 --> 00:49:46,879 it will be easy or a lot of people in your position, having pens to be so popular, 577 00:49:46,879 --> 00:49:52,719 they would just try to scale it up and, and be perhaps more income. Cause it would be easy to 578 00:49:52,719 --> 00:49:57,600 like scale it up. It'd be more popular, have large income and not, not necessarily big profit. And I 579 00:49:57,600 --> 00:50:03,520 think you, you just run it to, so it's fun for you. So we don't have to have like problems of 580 00:50:03,520 --> 00:50:09,520 big company and you would basically be a full-time manager if you did this. And I have a similar 581 00:50:09,520 --> 00:50:14,080 approach at least at the moment that I don't want to make the channel. I could just hire 582 00:50:14,080 --> 00:50:19,280 people and do different stuff and be big and publish videos every day and stuff like this. 583 00:50:19,280 --> 00:50:25,360 And I just don't want, so, so I do definitely look up to it because, uh, I'm also happier. 584 00:50:25,360 --> 00:50:31,679 I think doing this, I don't want to be the CEO. I know some people that became CEOs and they miss 585 00:50:31,679 --> 00:50:37,199 hacking. So, um, so I'm definitely looking up to, to like your approach basically. 586 00:50:37,199 --> 00:50:41,520 Yeah. I've been like a manager in a previous life and I didn't enjoy it much. So 587 00:50:42,159 --> 00:50:46,080 previous life, you, you, you, you now treat being employed as the previous life. 588 00:50:46,080 --> 00:50:55,120 Yeah. No, but, um, I think I, I have the right balance. Like it's not easy, but it's a good 589 00:50:55,120 --> 00:51:01,919 balance. Yeah. What are some tips to me? Cause I'm, I'm way younger entrepreneur than you are. 590 00:51:01,919 --> 00:51:08,399 So you definitely have some, some tips that you can give me, um, trying to have fun. Yeah. Because 591 00:51:08,399 --> 00:51:14,479 it's a long, long road. So if you're just in it for like making money and things like that, 592 00:51:14,479 --> 00:51:19,600 that's not going to work. So you need to keep the fun. Um, you've got to find like a lot of 593 00:51:19,600 --> 00:51:25,439 people who are more successful than you, uh, doing more stuff, bigger company, more employees, 594 00:51:25,439 --> 00:51:31,120 doesn't mean they are happier than you. Yeah. So that's because for diverse reasons and find 595 00:51:31,120 --> 00:51:34,879 something that makes you happy, I think. And people like kind of feel it. I think people 596 00:51:34,879 --> 00:51:39,760 like Pentestal because all people who know me know that I really enjoy what I'm doing 597 00:51:39,760 --> 00:51:45,520 and I'm really passionate about it. And for example, um, doing security code review, uh, 598 00:51:45,520 --> 00:51:52,560 content is not necessarily where, um, customers are, but it's something I really enjoy. And I 599 00:51:52,560 --> 00:51:56,560 think that's what matters right now for the industry and for what people should be learning. 600 00:51:57,360 --> 00:52:02,639 So if I were like trying to get more profit and things like that, I should probably do like some 601 00:52:02,639 --> 00:52:07,199 blue team content or things like that. Yeah. But security code review is so good. 602 00:52:09,840 --> 00:52:14,159 That's so nice. What are the, your plans for the platform for let's say upcoming year? 603 00:52:14,800 --> 00:52:22,000 Um, a lot of code review, a lot of code, like every single language. I want to be able to 604 00:52:22,000 --> 00:52:28,239 have someone who's like, okay, I got a new job as an app sec person in a Golang, uh, shop. 605 00:52:28,879 --> 00:52:33,120 They only do Golang. I need to have this path and say like, okay, do those labs, 606 00:52:33,120 --> 00:52:39,360 lab, lab, lab, lab, those labs. And you're ready. Okay. And, um, more presentation, more talks, 607 00:52:40,000 --> 00:52:46,000 um, public speaking, more, uh, training. So I do like life training now. So that's, 608 00:52:46,000 --> 00:52:49,520 I really enjoy that. It's good to have, like, instead of like just being in my corner, 609 00:52:49,520 --> 00:52:56,159 like talking to people, it's pretty hardcore, but yeah. Like, um, as in, I did like, uh, 610 00:52:56,159 --> 00:53:00,479 two days in a row and like the second, I could barely talk because it's like me talking for 12 611 00:53:00,479 --> 00:53:07,760 hours and, uh, going for like 972 slides or something. 972 slides. Yeah. There's a lot 612 00:53:07,760 --> 00:53:13,439 in like four times, three hours. So it's pretty, yeah. And, but yeah, like trying to grow, 613 00:53:13,439 --> 00:53:18,560 but with the right clients, I think that's what matters and what is a great client, right client, 614 00:53:19,199 --> 00:53:24,399 someone who really enjoy hacking. Okay. Like someone who really enjoy like technical 615 00:53:24,399 --> 00:53:31,199 nitpick, technical details. Someone would spend like, uh, two days on some little details or email 616 00:53:31,199 --> 00:53:35,840 me like, Oh, we, have you thought about that? That's great. Like when I started that lab, 617 00:53:35,840 --> 00:53:39,199 I thought about like this other way that you don't show in the video. Like, that's amazing. 618 00:53:39,919 --> 00:53:45,360 And yeah. And having fun and yeah. But like, yeah, there's so much content I want to create. 619 00:53:45,360 --> 00:53:51,120 Like I want to do something, uh, more content for, uh, beginners because I feel like a lot of content 620 00:53:51,120 --> 00:53:58,719 around beginners is not really well done. Okay. Like I want to approach it with more like a 621 00:53:58,719 --> 00:54:06,080 security engineering point of view, explaining the details, like, um, people teach DNS. 622 00:54:07,120 --> 00:54:11,840 It's interesting to teach DNS to security people, but in DNS, you have like security mechanisms in 623 00:54:11,840 --> 00:54:17,360 place and explaining both security mechanism, how they work, why they are there and things like that 624 00:54:17,919 --> 00:54:23,520 to build not only the knowledge, but the threat models around technology and things like that. 625 00:54:24,320 --> 00:54:29,040 Okay. And same, I want to explain TLS, like the number of like infographics on TLS 626 00:54:29,040 --> 00:54:36,639 online that are wrong, that drives me nuts. Like, yeah, like, yeah, I wouldn't, 627 00:54:36,639 --> 00:54:42,080 I wouldn't draw a correct one. Yeah. That's not a problem. Don't do infographics if you can't. 628 00:54:42,800 --> 00:54:49,199 Awesome. Thank you so much for joining me today. No worries. Thank you for having me like, yeah, 629 00:54:49,760 --> 00:54:56,000 that's great. Yeah. Thank you. Uh, and thank you for listening to the episode. If you enjoyed it, 630 00:54:56,000 --> 00:55:02,159 uh, we spoke a lot about code review. If you want to, if you're inspired to do more code review, 631 00:55:02,159 --> 00:55:06,080 uh, check out the episode with shops that's linked in the description. 632 00:55:06,080 --> 00:55:15,520 And for now, thank you for listening and goodbye.