[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Full-disclosure] one of my servers has been compromized
- To: mitchell <mitchell@xxxxxx>
- Subject: Re: [Full-disclosure] one of my servers has been compromized
- From: "Larry W. Cashdollar" <larry0@xxxxxx>
- Date: Mon, 05 Dec 2011 14:15:09 +0000 (GMT)
<html><body><div>I'd check these
too:<br><br>http://virtuemart.net/security-bulletins<br></div><div><br>On Dec
05, 2011, at 05:35 AM, mitchell <mitchell@xxxxxx>
wrote:<br><br><div><blockquote type="cite"><div class="msg-quote"><div><span
style="color: #222222; font-family: arial,sans-serif;" data-mce-style="color:
#222222; font-family: arial,sans-serif;" color="#222222" face="arial,
sans-serif">Hi,</span></div><div><span style="color: #222222; font-family:
arial,sans-serif;" data-mce-style="color: #222222; font-family:
arial,sans-serif;" color="#222222" face="arial,
sans-serif"><br></span></div><div><span style="color: #222222; font-family:
arial,sans-serif;" data-mce-style="color: #222222; font-family:
arial,sans-serif;" color="#222222" face="arial, sans-serif">Here is what you
generally need to do in such cases.</span></div><div><span style="color:
#222222; font-family: arial,sans-serif;" data-mce-style="color: #222222;
font-family: arial,sans-serif;" color="#222222" face="arial, sans-serif">1.
Suspend the webapp until you investigate.</span></div><div><span style="color:
#222222; font-family: arial,sans-serif;" data-mce-style="color: #222222;
font-family: arial,sans-serif;" color="#222222" face="arial, sans-serif">2.
Check the web server logs for unusual entries and identify the entry point. You
should have noticed the timestamp of the Perl script in the /tmp directory. Try
looking for entries around this timestamp. Usually, the timestamp would be your
starting point. The files created in the /tmp/.m directory were most probably
put there via an HTTP request.</span></div><div><span style="color: #222222;
font-family: arial,sans-serif;" data-mce-style="color: #222222; font-family:
arial,sans-serif;" color="#222222" face="arial, sans-serif">3. Find all
locations writable by www-data.</span></div><div><span style="color: #222222;
font-family: arial,sans-serif;" data-mce-style="color: #222222; font-family:
arial,sans-serif;" color="#222222" face="arial, sans-serif">4. Touch a file
with a timestamp = the date of the incident.</span></div><div><span
style="color: #222222; font-family: arial,sans-serif;" data-mce-style="color:
#222222; font-family: arial,sans-serif;" color="#222222" face="arial,
sans-serif">5. Find all files -newer than the file you `touch`-ed in the
locations writable by www-data.</span></div><div><span style="color: #222222;
font-family: arial,sans-serif;" data-mce-style="color: #222222; font-family:
arial,sans-serif;" color="#222222" face="arial, sans-serif">6. Identify any
malicious files in the returned listing.</span></div><div><span style="color:
#222222; font-family: arial,sans-serif;" data-mce-style="color: #222222;
font-family: arial,sans-serif;" color="#222222" face="arial, sans-serif">7.
Stat the malicious files and log the data.</span></div><div><span style="color:
#222222; font-family: arial,sans-serif;" data-mce-style="color: #222222;
font-family: arial,sans-serif;" color="#222222" face="arial, sans-serif">8.
Quarantine / remove the malicious file(s).</span></div><div><span style="color:
#222222; font-family: arial,sans-serif;" data-mce-style="color: #222222;
font-family: arial,sans-serif;" color="#222222" face="arial, sans-serif">9.
*Patch* the Web application.</span></div><div><span style="color: #222222;
font-family: arial,sans-serif;" data-mce-style="color: #222222; font-family:
arial,sans-serif;" color="#222222" face="arial, sans-serif">10. Check the
application code for other vulnerabilities.</span></div><div><span
style="color: #222222; font-family: arial,sans-serif;" data-mce-style="color:
#222222; font-family: arial,sans-serif;" color="#222222" face="arial,
sans-serif">11. Allow access to the Webapp.</span></div><div><span
style="color: #222222; font-family: arial,sans-serif;" data-mce-style="color:
#222222; font-family: arial,sans-serif;" color="#222222" face="arial,
sans-serif">12. Check for updates for the application regularly and apply fixes
for any security issues if full upgrade is not possible.</span></div><div><span
style="color: #222222; font-family: arial,sans-serif;" data-mce-style="color:
#222222; font-family: arial,sans-serif;" color="#222222" face="arial,
sans-serif"><br></span></div><div><span style="color: #222222; font-family:
arial,sans-serif;" data-mce-style="color: #222222; font-family:
arial,sans-serif;" color="#222222" face="arial, sans-serif">Unless you patch
the application, the issue will most certainly re-occur.</span></div><div><span
style="color: #222222; font-family: arial,sans-serif;" data-mce-style="color:
#222222; font-family: arial,sans-serif;" color="#222222" face="arial,
sans-serif"><br> </span></div><div><span style="color: #222222; font-family:
arial,sans-serif;" data-mce-style="color: #222222; font-family:
arial,sans-serif;" color="#222222" face="arial,
sans-serif">Regards,</span></div><div><span style="color: #222222; font-family:
arial,sans-serif;" data-mce-style="color: #222222; font-family:
arial,sans-serif;" color="#222222" face="arial, sans-serif"><br></span>--<br>#
m</div><br><div class="gmail_quote">On Mon, Dec 5, 2011 at 12:44, Lucio Crusca
<span dir="ltr"><<a href="mailto:lucio@xxxxxxxxxx";
data-mce-href="mailto:lucio@xxxxxxxxxx";>lucio@xxxxxxxxxx</a>></span>
wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex" data-mce-style="margin: 0 0 0
.8ex; border-left: 1px #ccc solid; padding-left: 1ex;">Hello *,<br> <br> I'm
not new here, but I've mostly lurked all the time through gmane. I never<br>
believed it could happen to me until it actually happened: they compromized<br>
one of my servers. It's a Ubuntu 10.04 server with all security patches<br>
regularly applied. I'm inclined to believe they used some hole in the web<br>
application, which is a old customized Virtuemart version (1.1.3), which is<br>
not upgradable because of the invasive code customizations (I'm not the<br>
author of that code, so I have no clue about what had been changed back<br>
then).<br> <br> Now the problem for me is to track down the security hole. Here
is the email<br> my provider received and forwarded to me:<br> <br> >
Subject: ISP Report; botnet activity on <a href="http://irc.undernet.org";
data-mce-href="http://irc.undernet.org";>irc.undernet.org</a><br> > [...]<br>
><br> > Hello, I am an operator on the irc chat network,<br> > <a
href="http://irc.undernet.org";
data-mce-href="http://irc.undernet.org";>irc.undernet.org</a> and i would like
you to investigate the<br> > owner of the Ip addresses that are listed at
the foot of this<br> > email.<br> ><br> > This/These host(s) have
likely been compromised, and had an<br> > altered/rogue process installed on
it, and was part of a botnet<br> > that was found on our network.<br>
><br> > The exploit or compromise running on this system is likely<br>
> to be an irc bot. Can you please alert the person who is<br> >
responsible, for its security to patch/upgrade, remove the<br> > irc process
and secure their system.<br> ><br> > = Unix System owners =<br> > A
favourite place for hiding the bot(s) is in tmp<br> > and in /var/tmp/ or
/dev/shm/ or in a users home directory<br> > sometimes it may be hidden like
/tmp/". ."/ or similar.<br> ><br> > The bot files can usually be
found by running these one line<br> > commands as the root user.<br>
><br> > find / -exec grep -l "undernet" {} +<br> > find / -exec grep
-l "sybnc" {} +<br> > find / -name "*.set" | perl -pe
's/.\/\w+-(\w+)-.*/$1/' | sort | uniq<br> > find / -name "inst" | perl -pe
's/.\/\w+-(\w+)-.*/$1/' | sort | uniq<br> ><br> > netstat -tanp<br> >
lsof -i tcp:<Port number><br> ><br> > *netstat looking for
connections to remote port 6667 or the<br> > range of ports between
6660-7000 once you find the port you<br> > can use the command, lsof -i
tcp:portnumber to determine<br> > which process/user it is running under,
and terminate it.<br> ><br> > = Windows System Owners =<br> > most
windows bots are mIRC scripted bots and generally<br> > need a file called
mirc.ini to run, you should search for<br> > this file. Run a good antivirus
scanner and firewall.<br> ><br> > This Ip/host may be removed from our
Irc network due to the<br> > risks it presents to our users.<br> ><br>
> Should you need any help with removing the files or bot<br> > process,
feel free to contact me by mail or on our network,<br> > which you connect
to using any irc client and issuing<br> > /server <a
href="http://irc.undernet.org";
data-mce-href="http://irc.undernet.org";>irc.undernet.org</a><br> ><br> >
I look forward to your reply<br> > Scot<br> ><br> > * Affected
host/IPs, capture time is GMT+1: United kingdom<br> > and servers they were
connected to.<br> ><br> > Please note: when resolving server names to IP
Addresses<br> > that all our servers end with .<a href="http://undernet.org";
data-mce-href="http://undernet.org";>undernet.org</a> (for example)<br> > <a
href="http://Tampa.FL.US"; data-mce-href="http://Tampa.FL.US";>Tampa.FL.US</a>.
is actually <a href="http://Tampa.FL.US.undernet.org";
data-mce-href="http://Tampa.FL.US.undernet.org";>Tampa.FL.US.undernet.org</a><br>
><br> > Important: If you reply to this mail needing further<br> >
information, please leave this mail intact, or supply us<br> > with the IP
Address(es) in question, as we reference these<br> > mails by the unique IP
Address<br> ><br> > Time of Capture: DECEMBER 3, 2011 10:03:48 PM<br>
><br> > List of IP address(es) and server it connected to:<br> >
my.server.ip.address (<a href="http://CHICAGO.IL.US";
data-mce-href="http://CHICAGO.IL.US";>CHICAGO.IL.US</a><br> ><br> > <a
href="http://BUDAPEST.HU.EU";
data-mce-href="http://BUDAPEST.HU.EU";>BUDAPEST.HU.EU</a><br> ><br> > <a
href="http://MONTREAL.QC.CA.undernet.org";
data-mce-href="http://MONTREAL.QC.CA.undernet.org";>MONTREAL.QC.CA.undernet.org</a>)<br>
><br> <br> I've run the "find" commands and found a number of file with the
first<br> "find", under /tmp/.m<br> <br> Deleted them, looked up remote
connections with netstat, killed perl<br> processes that where trying to
connect to port 6959 (only trying because<br> I've now set up iptables so that
they actually can't), but those processes<br> kept spawning. Checked crontab of
www-data, found the launcher, removed it.<br> <br> Now the problem is: how do I
pervent further abuse? What should I search in<br> the logs (if anything) to
spot the security hole?<br> <br> TIA<br> Lucio.<br> <br> <br> <br> <br> <br>
_______________________________________________<br> Full-Disclosure - We
believe in it.<br> Charter: <a
href="http://lists.grok.org.uk/full-disclosure-charter.html";
data-mce-href="http://lists.grok.org.uk/full-disclosure-charter.html";>http://lists.grok.org.uk/full-disclosure-charter.html</a><br>
Hosted and sponsored by Secunia - <a href="http://secunia.com/";
data-mce-href="http://secunia.com/";>http://secunia.com/</a><br></blockquote></div><br><div
class="_stretch">_______________________________________________<br>
Full-Disclosure - We believe in it.<br> Charter: <a
href="http://lists.grok.org.uk/full-disclosure-charter.html";
data-mce-href="http://lists.grok.org.uk/full-disclosure-charter.html";>http://lists.grok.org.uk/full-disclosure-charter.html</a><br>
Hosted and sponsored by Secunia - <a href="http://secunia.com/";
data-mce-href="http://secunia.com/";>http://secunia.com/</a><div
style="width:0px;
height:0px;"> </div></div></div></blockquote></div></div></body></html>_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.grok.org.uk/full-disclosure-charter.html
Hosted and sponsored by Secunia - http://secunia.com/