Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "EGI CSIRT:Incident reporting"

From EGIWiki
Jump to navigation Jump to search
({{From OSCT wiki|http://osct.web.cern.ch/osct/incident-reporting.html}})
 
(33 intermediate revisions by 10 users not shown)
Line 1: Line 1:
{{Under construction}}
<!--{{Egi-csirt-header}}-->
{{Egi-csirt-header}}
{{New-Egi-csirt-header}}


OSCT developed an [https://edms.cern.ch/document/867454 EGEE Incident Response Procedure] that must be used as the roadmap for the incident response actions.
==How to report a security incident==


For you convinience, we had reproduced incident reporting templates from the original document at this page.
Please follow the [[SEC01|EGI Security Incident Handling Procedure]] to report a security incident to <span style="color:#ff0000"> '''abuse at egi.eu'''</span> ([https://wiki.egi.eu/wiki/EGI_CSIRT:PGP PGP key]).
Below you will find some explanations about that incident response procedure.
 
Sites must report an incident or possible incident to abuse at egi.eu (at least within 4 hours after the suspected incident has been discovered).
 
You will find it useful to print the [[File:SEC01-RC.pdf|Resource Center Checklist]]
 
 
There is also a [[Forensic Howto]] page.


== Initial HEADS-UP message ==
== Initial HEADS-UP message ==


This template is aimed at notifying the grid participants soon after the incident has been discovered (heads-up), as described in Step 3 of the incident response procedure.
The initial HEADS-UP, which you should aim to send as soon as the incident has been discovered, should contain the minimum information that would allow the EGI CSIRT to notify all members of the EGI Infrastructure and close collaborations about the incident, in order to contain it. This email will, in most cases, be forwarded as-is (plus EGI case number) to all security contacts.
 
<pre><nowiki>
<pre><nowiki>
From: <YOUR_EMAIL_ADDRESS@YOUR_ORGANISATION>
FROM: <you>
To: project-egee-security-csirts@in2p3.fr
TO: abuse@egi.eu
Subject: Security incident suspected at <YOUR SITE>
SUBJECT: [TLP:AMBER] Security incident suspected at <site>  
** AMBER Information – Limited Distribution                        **
** This may be shared with trusted security teams on a need-to-know basis **
** see https://wiki.egi.eu/wiki/EGI_CSIRT:TLP for distribution restrictions **
Dear EGI CSIRT,


** PLEASE DO NOT REDISTRIBUTE ** EGEE-<DATE> (ex: EGEE-20090531)
A suspected security incident has been detected at <SiteName>.
** This message is sent to the EGEE CSIRTs and must NOT be publicly archived **


Dear CSIRTs,
It seems a security incident has been detected at <YOUR SITE>.
Summary of the information available so far:
Summary of the information available so far:
<Ex: A malicious SSH connection was detected from 012.012.012.012. The extent of the incident is
unclear for now, and more information will be published in the coming hours as forensics are
progressing at our site. However, all sites should check for successful SSH connection from
012.012.012.012 as a precautionary measure.>
</nowiki></pre>


<Ex: A malicious SSH connection was detected from 012.012.012.012.
The extent of the incident is unclear for now, and more information
will be published in the coming hours as forensics are progressing
at our site.  However, all sites should check for successful
SSH connections from 012.012.012.012 as a precautionary measure.>
</nowiki></pre>
== Follow-up message ==
== Follow-up message ==


This template can be used to provide a detailed view of the incident, and may be completed and reposted as the investigation progresses, as described in Step 5 of the incident response procedure.
 
This template can be used to provide a detailed view of the incident, and may be completed and resent as the investigation progresses.
The data in this email will, in most cases, be forwarded to all security contacts, but some filtering might be applied if deemed necessary
 
<pre><nowiki>
<pre><nowiki>
From: <YOUR_EMAIL_ADDRESS@YOUR_ORGANISATION>
FROM: <you>
To: project-egee-security-csirts@in2p3.fr
TO: abuse@egi.eu
Subject: Security incident suspected at <YOUR SITE>
SUBJECT: [TLP:AMBER] Security incident suspected at <site>
** AMBER Information – Limited Distribution                        **
** This may be shared with trusted security teams on a need-to-know basis **
** see https://wiki.egi.eu/wiki/EGI_CSIRT:TLP for distribution restrictions **


** PLEASE DO NOT REDISTRIBUTE ** EGEE-<DATE> (ex: EGEE-20090531)
Dear EGI CSIRT,
** This message is sent to the EGEE CSIRTs and must NOT be publicly archived **


Dear CSIRTs,
A security incident has been detected at <SiteName>.
It seems a security incident has been detected at <YOUR SITE>.


- Short summary of the incident
- Short summary of the incident
<Provide a high level overview of the incident>
<Provide a high-level overview of the incident>


- Host(s) affected
- Host(s) affected
< List of compromised hosts and/or hosts running suspicious user code.
<List of compromised hosts and/or hosts running suspicious user code.
ex: grid-worker-node-124.mysite.org (123.123.123.123)>
ex: grid-worker-node-124.mysite.org (123.123.123.123)>


- Host(s) used as a local entry point to the site (ex: UI or WMS IP
- Host(s) used as a local entry point to the site (ex: UI or WMS IP address)
  address)
<The host that the attacker is likely to have used to access the site.
<The host that the attacker is likely to have used to access the site.
ex: grid-ui-101.mysite.org (123.123.123.124)>
ex: grid-ui-101.mysite.org (123.123.123.124)>
Line 57: Line 69:
ex: 123.adsl.somecorp.com (012.012.012.012)>
ex: 123.adsl.somecorp.com (012.012.012.012)>


- Evidence of the compromise, including timestamps (ex: suspicious files
- Evidence of the compromise, including timestamps (ex: suspicious files  
  or log entry)
or log entry) <Ex: the attacker logged in has root from 123.adsl.somecorp.com.  
<Ex: the attacker logged in has root from 123.adsl.somecorp.com. Times
Times are UTC:
are UTC: Mar 24 12:00:09 grid-ui-101 sshd[13896]: Accepted password for
Mar 24 12:00:09 grid-ui-101 sshd[13896]: Accepted password for root
root from 012.012.012.012>
from 012.012.012.012>


- What was lost, details of the attack
- What was lost, details of the attack
< Provide available details on the extent of the compromise. For ex:
<Provide available details on the extent of the compromise. Ex:
System logs revealed the attacker guested the root password of
System logs revealed the attacker guessed the root password of  
grid-ui-101 on Mar 24 12:00:09 (UTC) after hundreds of attempts. Then,
grid-ui-101 on Mar 24 12:00:09
the attacker [...] etc.>
(UTC) after hundreds of attempts. Then, the attacker [...] etc.>


- If available and relevant, the list of other sites possibly affected
- If available and relevant, the list of other sites possibly affected
<Ex: firewall logs reveals suspicious SSH connections from the
<Ex: firewall logs reveal suspicious SSH connections from the compromised node to grid-
compromised node to grid-ui.friendlysite.org on Mar 24 13:01:03 (UTC).
ui.friendlysite.org on Mar 24 13:01:03 (UTC). friendlysite.org has been contacted.>
friendlysite.org has been contacted.>


- Possible vulnerabilities exploited by the attacker
- Possible vulnerabilities exploited by the attacker
<Ex: the attacker exploited a weak root password and gained further
<Ex: the attacker exploited a weak root password and gained further access by exploiting CVE-2009-
access by exploiting CVE-20090123 against [...] etc.>
1234 against [...] etc.>


- The actions taken to resolve the incident
- Actions taken to resolve the incident
<Ex: Disk images have been saved, hosts have been reinstalled from
<Ex: Disk images have been saved, hosts have been reinstalled from scratch with new, strong root
scratched with new, strong root passwords, and SSH has been configured
passwords, and SSH has been configured to prevent "root" logins with password.>
to prevent "root" logins with password.>


- Recommendations for other sites, actions suggested
- Recommendations for other sites, actions suggested
<Ex: Sites should check and report any successful SSH connection
<Ex: Sites should check and report any successful SSH connection from grid-ui-101 between Mar 24
grid-ui-101 between Mar 24 12:00:09 (UTC) and Mar 24 17:00:00 (UTC). It
12:00:09 (UTC) and Mar 24 17:00:00 (UTC).
is also recommended to avoid direct SSH access, and to configure sshd
It is also recommended to avoid direct SSH access, and to configure sshd with "PermitRootLogin
with "PermitRootLogin without-password".>
without-password".>


- Timeline of the incident
- Timeline of the incident
<Ex:
<Ex:
2009-03-24 09:12:43 Multiple SSH connection attempts from 012.012.012.012
2009-03-24 09:12:43 UTC Multiple SSH connection attempts from 12.012.012.012
2009-03-24 12:00:09 Attacker connects as root on grid-ui-101.mysite.org
2009-03-24 12:00:09 UTC Attacker connects as root on grid-ui-101.mysite.org from 012.012.012.012
                    from 012.012.012.012
2009-03-24 13:01:03 UTC SSH scan from grid-ui-101 against grid-ui.friendlysite.org
2009-03-24 13:01:03 SSH scan from grid-ui-101 against
                    grid-ui.friendlysite.org
[...]
[...]
2009-03-24 15:00:00 Site security team investigating
2009-03-24 15:00:00 UTC Site security team investigating
2009-03-24 15:34:00 EGEE CSIRTs informed via
2009-03-24 15:34:00 UTC EGI security contacts informed [...]>
                    project-egee-security-csirts@in2p3.fr
[...]>


</nowiki></pre>
</nowiki></pre>


{{From OSCT wiki|http://osct.web.cern.ch/osct/incident-reporting.html}}
==About the EGI security incident handling procedure==
 
EGI-CSIRT developed the [[SEC01|EGI Security Incident Handling Procedure]].
The document have been approved by EGI OMB and PMB. EGI sites must follow this procedure when handling security incident.
 
The "Security Incident Handling Procedure" define site and incident coordinator responsibilities when handling Grid-related security incident.
We strongly encourage our security contacts and system administrators to have a printing copy of this procedure.

Revision as of 16:37, 14 June 2017

EGI-CSIRT web site EGI-CSIRT Public wiki EGI-CSIRT Contacts EGI-CSIRT Activities EGI-CSIRT Private wiki


How to report a security incident

Please follow the EGI Security Incident Handling Procedure to report a security incident to abuse at egi.eu (PGP key). Below you will find some explanations about that incident response procedure.

Sites must report an incident or possible incident to abuse at egi.eu (at least within 4 hours after the suspected incident has been discovered).

You will find it useful to print the File:SEC01-RC.pdf


There is also a Forensic Howto page.

Initial HEADS-UP message

The initial HEADS-UP, which you should aim to send as soon as the incident has been discovered, should contain the minimum information that would allow the EGI CSIRT to notify all members of the EGI Infrastructure and close collaborations about the incident, in order to contain it. This email will, in most cases, be forwarded as-is (plus EGI case number) to all security contacts.

FROM: <you>
TO: abuse@egi.eu
SUBJECT: [TLP:AMBER] Security incident suspected at <site> 
** AMBER Information – Limited Distribution                        **
** This may be shared with trusted security teams on a need-to-know basis **
** see https://wiki.egi.eu/wiki/EGI_CSIRT:TLP for distribution restrictions **
Dear EGI CSIRT,

A suspected security incident has been detected at <SiteName>.

Summary of the information available so far:
<Ex: A malicious SSH connection was detected from 012.012.012.012. The extent of the incident is
unclear for now, and more information will be published in the coming hours as forensics are
progressing at our site. However, all sites should check for successful SSH connection from
012.012.012.012 as a precautionary measure.>

Follow-up message

This template can be used to provide a detailed view of the incident, and may be completed and resent as the investigation progresses. The data in this email will, in most cases, be forwarded to all security contacts, but some filtering might be applied if deemed necessary

FROM: <you>
TO: abuse@egi.eu
SUBJECT: [TLP:AMBER] Security incident suspected at <site>
** AMBER Information – Limited Distribution                        **
** This may be shared with trusted security teams on a need-to-know basis **
** see https://wiki.egi.eu/wiki/EGI_CSIRT:TLP for distribution restrictions **

Dear EGI CSIRT,

A security incident has been detected at <SiteName>.

- Short summary of the incident
<Provide a high-level overview of the incident>

- Host(s) affected
<List of compromised hosts and/or hosts running suspicious user code.
ex: grid-worker-node-124.mysite.org (123.123.123.123)>

- Host(s) used as a local entry point to the site (ex: UI or WMS IP address)
<The host that the attacker is likely to have used to access the site.
ex: grid-ui-101.mysite.org (123.123.123.124)>

- Remote IP address(es) of the attacker
<The remote host from where the attacker is likely to have connected from.
ex: 123.adsl.somecorp.com (012.012.012.012)>

- Evidence of the compromise, including timestamps (ex: suspicious files 
or log entry) <Ex: the attacker logged in has root from 123.adsl.somecorp.com. 
Times are UTC:
Mar 24 12:00:09 grid-ui-101 sshd[13896]: Accepted password for root 
from 012.012.012.012>

- What was lost, details of the attack
<Provide available details on the extent of the compromise. Ex:
System logs revealed the attacker guessed the root password of 
grid-ui-101 on Mar 24 12:00:09
(UTC) after hundreds of attempts. Then, the attacker [...] etc.>

- If available and relevant, the list of other sites possibly affected
<Ex: firewall logs reveal suspicious SSH connections from the compromised node to grid-
ui.friendlysite.org on Mar 24 13:01:03 (UTC). friendlysite.org has been contacted.>

- Possible vulnerabilities exploited by the attacker
<Ex: the attacker exploited a weak root password and gained further access by exploiting CVE-2009-
1234 against [...] etc.>

- Actions taken to resolve the incident
<Ex: Disk images have been saved, hosts have been reinstalled from scratch with new, strong root
passwords, and SSH has been configured to prevent "root" logins with password.>

- Recommendations for other sites, actions suggested
<Ex: Sites should check and report any successful SSH connection from grid-ui-101 between Mar 24
12:00:09 (UTC) and Mar 24 17:00:00 (UTC).
It is also recommended to avoid direct SSH access, and to configure sshd with "PermitRootLogin
without-password".>

- Timeline of the incident
<Ex:
2009-03-24 09:12:43 UTC Multiple SSH connection attempts from 12.012.012.012
2009-03-24 12:00:09 UTC Attacker connects as root on grid-ui-101.mysite.org from 012.012.012.012
2009-03-24 13:01:03 UTC SSH scan from grid-ui-101 against grid-ui.friendlysite.org
[...]
2009-03-24 15:00:00 UTC Site security team investigating
2009-03-24 15:34:00 UTC EGI security contacts informed [...]>

About the EGI security incident handling procedure

EGI-CSIRT developed the EGI Security Incident Handling Procedure. The document have been approved by EGI OMB and PMB. EGI sites must follow this procedure when handling security incident.

The "Security Incident Handling Procedure" define site and incident coordinator responsibilities when handling Grid-related security incident. We strongly encourage our security contacts and system administrators to have a printing copy of this procedure.