0

I have implemented a packet analyzer in Java. Now I want to detect whether the packets captured are malicious or not. For example, how do I detect a DOS attack using Java code? What is the legal size of a TCP packet? If the size is above the legal size, can the packet be considered malicious?

moonshadow
  • 75,857
  • 7
  • 78
  • 116
priya
  • 1
  • 1
  • 1
  • see http://stackoverflow.com/questions/2613734/maximum-packet-size-for-a-tcp-connection for packet size info – Brandon Frohbieter Mar 05 '11 at 19:15
  • 6
    check the evil bit -- http://www.ietf.org/rfc/rfc3514.txt – Jumbogram Mar 05 '11 at 19:18
  • 3
    Let me rewrite your question in another way, "how do you know if a file is malicious?" You don't, which is why anti-viruses exist. How do *they* know? Years of experience; via virus signatures (amongst other techniques). Same thing with packets, you would need a signature DB. With regards to DOS attacks, it's not really possible, which is why DOS attacks are really fixed by using good hardware, the right software patterns (eg: caching) and the like. I like to think of DDOS as there is nothing to stop a big enough DDOS. – Christian Mar 05 '11 at 19:19
  • Jumbogram - Sorry for being quite the spoilsport ;) – Christian Mar 05 '11 at 19:20

2 Answers2

5

Test for the evil bit!

Seriously, there's no shortcut unfortunately. It's a bit like asking how you detect a terrorist at an airport checkpoint.

You need to read the RFCs, research the sorts of attacks that are possible, and decide which of these you want to try and detect; given this information, working out the mechanics of how to detect any particular kind of attack should be straightforward, and if you do run into problems you will be able to ask a more specific question here.

Some links to get you started:

  • IP datagram format
  • Snort is an intrusion detection system - i.e. a piece of software that does what you are attempting: captures traffic and checks it against a large list of known malicious patterns. It is open source: you can look at the source to see how it does things, and look at their database to see what sorts of things it checks for.

As with virus detection, building a sufficiently large database of patterns of malicious activity is the bulk of the difficulty / expense in such a project and is what will make the difference between your tool being useful and not. To produce a tool useful for any purpose other than personal research / educating yourself on how such tools work will likely take many man-years. Your best way forward would likely be to make use of an existing open database such as Snort's, or simply contribute your time to their effort.

moonshadow
  • 75,857
  • 7
  • 78
  • 116
1

You secretly have two questions in one:

  1. What sequence of events would be categorized as malicious
  2. How do I detect such a sequence of events.

(2) is fairly straightforward given (1), but answering (1) is difficult without years of experience (or borrowing from other's experience), as Christian Sciberras was alluding to.

Jumbogram
  • 2,249
  • 1
  • 19
  • 24