James Jardine  0:00  
Hi, everyone, and welcome to episode 32 of the developsec podcast. I'm your host, James Jardine. And today, I want to talk a little bit about dynamic analysis. Previously, back in one of the earlier episodes, actually Episode 25, we talked about static analysis, which as a quick recap, was analyzing the source or the binary of an application, not an application sexual RUNNING state, whereas with dynamic analysis, we're going to analyze the application as it's running. And so it has some advantages and disadvantages compared to static analysis. Let's start with some of the advantages. You know, some of the nice things that we see with dynamic analysis is that we can actually see what parts of the program or being used and how they will actually react in real time. Some of the limitations we see in static analysis is that sometimes it may not take into consideration the type of server that the code is going to be running on if we take a ASP dotnet as an example, typically, we're going to run that niasse. But you know, it is possible to run it under Apache or some other servers, and those may act differently. But we're pretty consistent under IAS, but then we have different versions of iOS, whether we're on an older version, like I six for the rep and seven 7.5. If we're all the way up to eight, right, they may react a little bit differently for how the application works. And then also we have things like how do our config settings, change the way that our application works? For example, if we have web config settings, that may or may not affect how code actually executes? Where if I look at something, and I just look at that one snippet of code? Well, it doesn't, you know, that looks like that could be vulnerable. But maybe there's a framework piece of setting, maybe there's a piece in IAS where, just like we talked about, in Episode 31, response splitting and header injection, where even I asked will take into account CRLF injection, and it won't allow that carriage return line feed to come down in a response header. So we can actually see with dynamic analysis how the application will actually run in its current environment. So that does give us a couple advantages. Also, with our dynamic analysis, we can actually see when things happen. So for cross site scripting, rather than just saying the code looks like it'll do that, here, we can actually prove out that we can get cross site scripting to work that something isn't encoding it that we didn't see. So that's actually very useful for us,

as we're going through our testing, now, there are some drawbacks, because we don't get line numbers, right. With static analysis, I know exactly where in the code, they're saying the problem is, it's, you know, line 253, of test page dot ASP x.cs. That's where they're saying the cross site scripting as because we're not encoding our output. With Dynamic Assessment, actually, what we're seeing is we actually get the request the response, so we actually get quite a bit of data, we just don't get the exact line number. And typically, we're able to, to determine or extract out where that is in the code, if we really understand our applications. Very often, and most developers know this that, you know, show me something on a specific page, and I can tell you where that's generated. So I can quickly go to that and see what the issue is. Now, with dynamic analysis, we're not going to potentially hit every function in the website, whereas static code analysis can hit everything. We can't see things like clear text, passwords, low entropy, some of the other cryptographic type finding stuff that you'd actually have to have the source code for. To see that, right. From a dynamic standpoint, it's really more of a hey, what could somebody that sat down with this app do? And how's that going to react in the application as we go? Now, it doesn't mean it's going to find everything, right? Every automated scanner out there, they have their limitations. So usually, dynamic scanners are not going to be good at finding authorization errors, you may find some that it can actually find authorization errors, if you run them under multiple accounts, they may be able to Spider the application under a specific account, spider it under a different account and do a diff of the pages that was able to find, but that's usually a piece where somebody manually is going to have to go back and verify. Yeah, they should be able to see that no, they shouldn't be able to see that. Even looking at these results. So we could automate the process of identifying all the pages. But usually that's still going to be a manual identification process of whether authorization really is an issue. Same thing with Can I access that record versus another record, right, we talk all the time I use, always use the example ID equals five, change it to ID equals six and see if you have access, well, we have to understand whether we should have access or not. Because just because I change it to another ID doesn't necessarily mean that I didn't have access to that maybe I did have access to that. So you have to have knowledge of the application. And typically, people are the only ones that are going to be able to find that knowledge and be able to apply it unless you have strict cases where you can build your own automation around it really, where you know that, hey, the test database has these parameters. And so if I do a five here, I know that the user I'm logged in with has access to it. And if I pick 25, I know the user doesn't have access to that. And if that never changes, then you can start helping look for the authorization issues that way. But typically, you're not picking up authorization issues, you're not picking up CSRF is actually fairly difficult for an automated scanner to identify. Because there's critical components to CSRF to really understand it, right? One of those being it has to be a sensitive transaction. Is it something I even care about? You know, is it okay? If somebody had triggered that to happen, versus something where you're changing a password, you're transferring money, you're doing something that truly is a sensitive transaction. So we're not going to find those as well, either. As I mentioned, with static analysis, you know, you're getting a lot of queued level stuff, or we're not getting that in that dynamic area. So there's lots of things to think about as we look at these. Now, there's a few different options. And I won't hit on every option out there. But some of the more popular options we see out there. As far as dynamic analysis, you've got white hat security, who has does both static and dynamic, you have HP Web inspect, which is another tool you can run. IBM has app scan, and I've links to all these in the show notes. Veracode has dynamic analysis, and kinetics as another one, actually, we used to see it at my old job, randomly, people would scan us with actionetics, I encourage you not to just scan sites, you don't have permission to scan. You know, even if you have a license to one of these scanners, you know, make sure you have permission to scan sites, because you never know where you're going to end up. And you may find yourself in some hot water. If somebody doesn't really like what you did. And there are some records out there, some people getting popped a little bit and getting in a little bit of trouble. For scan, even just doing something as simple as scanning a site without kinetics can get you in trouble. So to be on the safe side, scan your own stuff, make sure you have permission to do it. You know, we want to make sure we're on the up and up. Now, a couple other options we have when we're talking about dynamic analysis, if you're looking to implement this in your company, there, there are two main types, right, you have the managed services, just like when we had static analysis, right? If managed services, which is really like your white hat, maybe a couple other ones Veracode will do the manage services. And then you have the in house, right, I'm going to buy the product, and I'm going to install it locally. And we're going to manage it completely some perks to being a managed service, as a lot of times, you really don't have to do a lot, you give them credentials. And that pretty much turns into the management part you do. And then they handle making sure the scanners working properly. They make sure the scans are running, right, you can go into the schedule the scans to make sure you know, hey, I wanted to go between these hours on these days. But go through they set up the scans. And then typically, they'll actually have analysts on staff that will review a lot of those results to make sure you're not getting a whole bunch of false positives, that what you're getting his actionable items. As opposed to when you're not dealing with a managed service. If you're dealing with just bringing the software in house. Well now it's a little bit more difficult because you have to run the tool, you have to analyze the results, you're getting raw results, you're not getting anybody else, kind of that middleman to intercept those, review them and try to weed out some of the stuff that really isn't worth looking at or is truly a false positive. A lot of the in house ones we don't get that right. So, you know, typically what I say is a lot of it depends on your skill level. What type of knowledge and skill sets do you have in house and resources available to be able to handle these type of products? Or, you know, are you in a situation where you really can't handle it and you know, going with a managed servers where basically we pay the bill, we put in some credentials and and everything else is kind of managed, we get the results as well as they come in, and we can manage them that way. So it's really up to how your company is set up and what you can handle. You don't want to get too far over your head. Typically there are some people to being in house versus manage just as the or vice versa. But from an in house standpoint, as we already covered, really the the managed perks in the house, typically it's easier to build into your continuous integration type of systems, kick them off, customize them. So if you've got specific rule sets you want to run, you may be able to do that if you've actually got the code base where you can write some of your own rules. Typically, you see that more with static analysis than you do dynamic analysis. But it's possible that you might be able to do that. Now remember that dynamic analysis typically happens in the verification phase, if we're thinking about the software development lifecycle, we're talking about the verification phase. So it's after we've really done the coding, you want to get it as close as possible to where we're coding, but typically static analysis, what we're going to do during the actual coding phase, and then as we get into verification, normally, what we do is we offset some of our or augment, I guess, some of our manual testing, with this dynamic testing. So as we're going through, and we've got testers that are testing the application, and they're browsing through it with the browser, and they've got a proxy going in there, they're testing different things. They're also running that dynamic scanner. And that dynamic scanner, which is good at finding things like cross site scripting, SQL injection, command injection, that type of stuff, dynamic scanners are good at. And they can do it really fast, right? And they're doing it consistently. Whereas a manual tester, well, if I'm, you know, really into it today, maybe I'll find something. And if I'm not really into it, I might kind of skip over some stuff or not do all my full tasks, right. And that's one of the downsides to manual testing, as it is so limited to your own skill set of the tester. And honestly, the mood you're in that day, whether you really feel pumped about doing it, or you just kind of having a blog day and, and I'll kind of all struggled through this, and I'll find some stuff. So that's kind of the nice thing about that dynamic analysis is it's consistent, you know, you can count on it for the for the specific things over and over, and we can start handling those. But then we're also doing the manual testing, because remember, all these pieces are just pieces to the full security puzzle, right, we want to have that static analysis, because of, you know, the view that it has into the code and, and how things work in the framework, you want to have that dynamic testing, because it's going to run the application, it's going to run through it and look for vulnerabilities as attackers who actually would be using them, while we don't actually attack during a dynamic analysis. You know, it is sending some of those payloads to test and see, hey, is this here, right, it's going to send a single quote, and if it gets some sort of SQL error message or error message back, it's going to make the assumption that there's SQL injection, there doesn't necessarily mean there is because again, it doesn't go through and fully exploit it. It's just going through to help identify where to look. And then we have our manual testing, which is really picking up all the stuff that none of the scanners are really that good at right, it's going through and checking, authorization and authentication and CSRF and logic type flaws that we can't do with any of the type of scanners. So it is a good solution. Now picking the right one that can be difficult to time. So make sure you do your research,

I've got more experience dealing with the white hat scanner, I've seen some of the stuff from webinspect, I really haven't dealt a lot with some of the other ones. So I can't really give good detail or in depth analysis of any of them. But really go out, see if you can get a trial or something. Give them a try. See what you're looking for. I'd be curious to know if anybody is currently using some if you want to share that or share any of your experiences, feel free to shoot that over to me. You know, I'd love to get some information on what other people are using and how they work. You know, I mean, I know how white hat works. I know how web inspect works. I've actually seen some of actionetics. So but not having the experience with others. I'd be curious to know who really what's the most popular one that people are using out there in how you feel about it, you know, sometimes we get thrown into a tool that we really don't have a choice on. And you know, we were kind of stuck with it right? Somebody in the company said that's what you're using and now you're stuck using that tool, which you know if it works, it works. But but you know, what are the things you like, what are the things you don't like and you know, feel free to you know, send back at Twitter at develop sec. If you don't want to share it, because I know some places don't want to share. You know what tools they use because it might be secret information. or proprietary, you know, feel free to shoot me an email, I won't share it with everybody else, like who's using what have, you know, just some of your experiences, it'd be nice to have some conversation of what people are using, what they like, what they don't like, I feel like from at least the dev side, we often sit in the silo, where we don't really talk a lot about what we're doing for security, we have a lot of talk of how we should do security. And, uh, yeah, we have to do secure coding. But we don't have a lot of talk of, Hey, guys, we're using this, and this is how it's working. And this is great. Or, hey, we're using this other one. And I, you know, it really doesn't work. Like I'd like it to I wish I could do this with it. Where are we missing stuff. When we think about these different tool sets, that maybe there's an opportunity where we can voice that and get some things change, make it better, for example, some of these systems, you know, I'm a big proponent of storing all your bugs in one system, you know, don't store regular bugs and TFS. And then you're storing security bugs and some other system that's not connected to TFS, right, everything should be in that one system. So if there's integrations that are missing, maybe you're using a tool that, you know, you spend all your day in Visual Studio, or you spend all your day in Eclipse and has no plugin to make that tie NZ constantly have to log in to multiple accounts, or multiple systems to handle it. And that's a real pain. You know, what is it that you're getting? What you're not getting? You know, let's start opening up some of those discussions and see how it goes. And I think it'd really be helpful for everybody, because it's tough, right? I mean, I sit here and say, Hey, go get dynamic analysis. It's good, here's what it's going to do. But I really can't recommend the best one. And you know, nobody really knows. Right? So now everybody does the same thing of well, what does this one do? What does that one do, which one's going to be best for me, and everybody has to do that analysis. And why you should be doing that analysis, it'd be nice if we had more information available to help compare some of those. But that is dynamic analysis, kind of in a nutshell, it's kind of that intro level.

piece that's going to cover what we're talking about. And just introduce people to the idea of dynamic analysis. A lot of places aren't using yet some are. But it is just another one of those tool sets in our arsenal, we should think of them as you know, application development tools, not necessarily developer tools, I would lean towards static analysis as being more of a developer tool. But they should be development tools, right? That means hopefully, QA is running some of these tools, somebody is running these tools, where we're getting that feedback immediately. So we can start addressing some of these issues. Now keep in mind, there are going to be things that are going to be false positives, you may find that some of the results aren't rated quite like they should be for your environment. For example, I see somewhere, a hot topic now is HTTP only cookies, setting the secure flag on the cookies, some systems will rank those as very low. Whereas in your organization, if you're using your session tokens stored in a cookie, then maybe it's a little bit more critical that you have those flag set to help protect your site against things for session hijacking, or something like that. So maybe that's a little bit more important to you. So don't just take everything face value, this is one of the problems we have with every scanner out there. It doesn't matter what it is dynamic, static, you name it, right? While it is an unbiased review of your system. But at the same point, when we talk about doing risk analysis, we have to understand our systems, we have to understand our company, our business, our business models, what's important to us. And as we figure that out, then as we go through and start triaging these flaws and say, Hey, we've got a cross site scripting, well, for us that's high. Or for us, that's critical. Or maybe it's a medium, depending on what your systems do, or where that that item is found. Take the time to triage the flaws, and work with them. So that way you understand which ones take highest priority, because again, we want to fix security. We want to have secure applications, but we don't want to waste time on stuff that is really low priority stuff. Right? Let's slide that aside. We'll get to it when we can get to it. Focus on the stuff that's going to be critical, right? I'm much more concerned about a cross scripting flaw than I am about a response header that gives me a server banner that tells me it's iOS 7.5. Right. There's lots of other ways we can figure that out. It's not directly exploitable. Yes, it is information. But focus on the ones that are critical. So that way we can still maintain this balance of fixing current security flaws that have been passed us before we started putting in security into our SDLC and creating new functionality, which while we're creating the new functionality We're implementing these new security standards. So we're getting less flaws going through anyway. So it's difficult but very critical for us to be able to maintain that balance. And as we work through it, we should see those flaws start dropping off, and less and less being identified on each iteration. And if we're not, then we need to start looking back at our processes of where are we falling through? And where can we refine that process to make it better. So that's my quick rant on dynamic analysis, we'll have some more rants coming soon, some more information, hopefully

stuff that you guys are finding it helpful. Again, please, you know, feel free to reach out share this with other people, I do really want it to be more towards the development teams. And as developers, business analysts, project managers, application owners, you know, everybody along that line, you know, this is where we're trying to reach. So, you know, if you are a security out there listening to this, and you know, I appreciate that too. And I ask that you hopefully pass this along to some of your development teams, right, because most security folks are going to find this kind of basic level, I would, I would assume, some may not if you're just starting to get into the industry. But feel free to share this with the other side, I'm trying not to make it too technical, or to kind of over anybody's head. Because I want it to be able to reach a large audience, I want people to start getting involved. And I want it to be helpful. So feel free to share it again, at develop sec, on Twitter, or you can go to develop sec.com. And I've got all the stuff there. I actually just updated the podcast piece, all the podcasts actually listed there, even though they still reach out to Lipson. And that's where the blog is. That's where some of the information is, and I've got some other stuff planned. You can also subscribe on there, to what is listed as a newsletter. But basically, if you subscribe, I'll actually send out when I say a message when we have new podcasts or a new blog post. It's not spammy type. I'm not marketing through it. I'm not selling it to anybody else. It's strictly for me to be able to come up with another way to help notify people that, hey, here's this new thing. Here's a quick summary. And if you're interested in listening, here's the link. That's it. Nothing more. Nice and simple. So thanks again for listening. And we'll be looking forward to Episode 33 coming out shortly. Thanks again.