Re: [Full-disclosure] High performance exception/traceback reporting system

["Cal Leeming [Simplicity Media Ltd]" <cal.leeming@xxxxxxxxxxxxxxxxxxxxxxxx>]

Oh, having support for XMPP/AMQP would be extremely nice, ideally I'd want
to try and make it compatible with as many different messaging systems as
possible.

On top of this, each supported language would have a library containing
out-of-the-box functions which has every type of messaging/transport option
available. This means the developer can;

   - Choose to use the system out-of-the-box, using the self contained
   messaging system binaries, and simply copy and paste the exception handling
   library / code.

   - Choose to optimize the system by not using the self contained binaries
   (for example, specifying an external database / messaging system to use
   etc).

   - Modify the exception handling code to suit there needs, rather than
   having to write one up from scratch.

As for the fingerprint hashes, this is a really good idea and would make the
reporting aspect very efficient, so I'll be sure to include this.

I'd never heard of SIEM before, after looking on wikipedia I came across
"NitroSecurity" SIEM which sure does look interesting. I'm gonna have a
flick through some of these sites for some inspiration, this may end up
turning in quite a big project!

On Mon, Feb 14, 2011 at 7:54 AM, Daniël W. Crompton <
daniel.crompton@xxxxxxxxx> wrote:

>
> Hi Cal,
>
> I've been thinking of this issue over the weekend and imagined one of the
> solutions to have a messaging system which accepted inputs from different
> languages and transforms these into a report which can be put into a bug
> reporting tool which could use hashes to fingerprint the bugs so you can see
> which are the errors which are causing the most problems.
>
> Sounds like a good plugin for a SIEM too.
>
> D.
>
>
> On 13 February 2011 17:50, Cal Leeming [Simplicity Media Ltd] <
> cal.leeming@xxxxxxxxxxxxxxxxxxxxxxxx> wrote:
>
>> Hi,
>>
>> I haven't started development on this yet, I will post the location of the
>> project once I've begun (hopefully next week!)
>>
>> Cal
>>
>>
>> On Sun, Feb 13, 2011 at 4:36 PM, Daniël W. Crompton <
>> daniel.crompton@xxxxxxxxx> wrote:
>>
>>>
>>> Is there any place I can retrieve the code?
>>>
>>> D.
>>>
>>>
>>> On 11 February 2011 18:17, Cal Leeming [Simplicity Media Ltd] <
>>> cal.leeming@xxxxxxxxxxxxxxxxxxxxxxxx> wrote:
>>>
>>>> Hey all,
>>>>
>>>> For the last two years, I've been meaning to write a reporting server
>>>> which allows webapps to post their exception tracebacks, which are then
>>>> viewable from a centralized location. After having Thunderbird corrupt my
>>>> mailbox due to over 250 thousand debug emails, this project has now been
>>>> given a bit more priority ;)
>>>>
>>>> The current prototype stores basic exception information (the file path,
>>>> line number, exception type, exception value, originating webapp, node
>>>> hostname etc) in the database, and the traceback details are then
>>>> serialized, dumped into a file, and the path to that file stored against 
>>>> the
>>>> row. A web interface then allows you to browse through these exceptions
>>>> (currently via Django admin), and view them using the same prettified
>>>> exception page which it shows for actual exceptions. This prettified page
>>>> also shows the variables within each frame in the stack, which is very
>>>> handy!
>>>>
>>>> From a developers point of view, this makes life extremely easy, because
>>>> all your webapps report to a single place, you can do sphinx searches,
>>>> alerts, custom reports etc, and it looks pretty lol.
>>>>
>>>> The entire thing is going to be open source, and will eventually be a
>>>> one-click install with a set up page etc.
>>>>
>>>> Here are some of the features I am planning on adding, but if anyone has
>>>> any suggestions as to what they would like to see in this, please feel free
>>>> to mention them!
>>>>
>>>>    - Tracebacks can be sent to the server primarily via POST request,
>>>>    but custom plugins will allow it to pull in via other means (such as 
>>>> mail
>>>>    attachments)
>>>>    - Alerts can be given different classifications (for example, you
>>>>    could configure specific nodes, webapps, or exception types to alert 
>>>> you via
>>>>    BulkSMS)
>>>>    - Prettified traceback page should initially support Python/PHP,
>>>>    other languages can be added as and when.
>>>>    - Basic authentication / IP restrictions for the admin login
>>>>    - Authentication support for when the tracebacks are POST'd to the
>>>>    server
>>>>    - Tar source should pre-package a lightweight nginx/uwsgi/python
>>>>    environment, so it is self sufficient (this will need to be security
>>>>    maintained obviously).
>>>>    - A nice, pretty, easy to use interface, because this just makes
>>>>    people feel all nice and warm inside ^_^
>>>>
>>>> I don't want to go as far as to say that it should be used to collect
>>>> error_log outputs, I think that would be going a bit too far, the main
>>>> reason for having a system like this is simply due to the sheer amount of
>>>> information usually contained within a traceback dump, and the Django
>>>> prettifier makes it so much easier to debug with!
>>>>
>>>> Thoughts/criticisms welcome!
>>>>
>>>> Cal
>>>>
>>>> _______________________________________________
>>>> Full-Disclosure - We believe in it.
>>>> Charter: http://lists.grok.org.uk/full-disclosure-charter.html
>>>> Hosted and sponsored by Secunia - http://secunia.com/
>>>>
>>>
>>>
>>>
>>> --
>>> blaze your trail
>>>
>>> --
>>> Daniël W. Crompton <daniel.crompton@xxxxxxxxx>
>>>
>>> <http://specialbrands.net/>
>>>
>>> <http://specialbrands.net/>
>>> http://specialbrands.net/
>>> <http://twitter.com/webhat> 
>>> <http://www.facebook.com/webhat><http://plancast.com/webhat><http://www.linkedin.com/in/redhat>
>>>
>>>
>>
>
>
> --
> blaze your trail
>
> --
> Daniël W. Crompton <daniel.crompton@xxxxxxxxx>
>
> <http://specialbrands.net/>
>
> <http://specialbrands.net/>
> http://specialbrands.net/
> <http://twitter.com/webhat> 
> <http://www.facebook.com/webhat><http://plancast.com/webhat><http://www.linkedin.com/in/redhat>
>
>

_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.grok.org.uk/full-disclosure-charter.html
Hosted and sponsored by Secunia - http://secunia.com/