Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "FAQ: lcg-ce to cream-ce"

From EGIWiki
Jump to navigation Jump to search
m (Protected "FAQ: lcg-ce to cream-ce" ([edit=sysop] (indefinite) [move=sysop] (indefinite)))
 
(10 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{Template:Op menubar}}
{{Template:Op menubar}} {{Template:Doc_menubar}} {{TOC_right}}
The FAQ provides the questions and answers those may arise for the user communities during the transition from lcg-ce to cream-ce. It does not replace the CREAM User Guide (see the references section), but it provides a list of pointers to relevant documentation.


'''Q1: What is cream-ce? '''
The FAQ provides the questions and answers those may arise for the user communities during the transition from lcg-ce to cream-ce. It does not replace the CREAM User Guide (see the references section), but it provides a list of pointers to relevant documentation.


The [https://twiki.cern.ch/twiki/bin/view/EMI/EMIServicesDocs#gLite_Job_Management CREAM] (Computing Resource Execution And Management) Service is a simple, lightweight service that implements all the operations at the Computing Element (CE) level; its well-defined Web Service-based interface and its implementation as an extension of the Java-Axis servlet (running inside the Apache Tomcat container) provide interoperability with clients written in any programming language and running on any computer platform.
== '''Q1: What is cream-ce? '''  ==
 
The [https://twiki.cern.ch/twiki/bin/view/EMI/EMIServicesDocs#gLite_Job_Management CREAM] (Computing Resource Execution And Management) Service is a simple, lightweight service that implements all the operations at the Computing Element (CE) level; its well-defined Web Service-based interface and its implementation as an extension of the Java-Axis servlet (running inside the Apache Tomcat container) provide interoperability with clients written in any programming language and running on any computer platform.  


The CREAM interface is defined in Web Service Description Language (WSDL); anyone can generate his/her CREAM client to interact with the service by filling in the stub code generated by WSDL parser (gSOAP for C/C++, Axis for Java, Perl module for perl).  
The CREAM interface is defined in Web Service Description Language (WSDL); anyone can generate his/her CREAM client to interact with the service by filling in the stub code generated by WSDL parser (gSOAP for C/C++, Axis for Java, Perl module for perl).  


The interface exposes a set of operations which can be classified in three groups.  
The interface exposes a set of operations which can be classified in three groups.  
*''Lease management'': allows users to define and manage leases associated with jobs. The lease mechanism ensure that all jobs get eventually properly managed, even if the CREAM loses connection with the client application.
 
*''Job management'':operations are provided to create and subsequently manage jobs.
*''Job management'':operations are provided to create and subsequently manage jobs.
*''Lease management'': allows users to define and manage leases associated with jobs. The lease mechanism ensures that jobs are managed even if the CREAM service loses connection with the client application.  
*''Service Management'': operations that deal with the whole CREAM service.
*''Service Management'': operations that deal with the whole CREAM service.


'''Q2: What batch systems are supported by CREAM?'''
== '''Q2: Which batch systems are supported by CREAM?''' ==
 
CREAM supports different batch systems, through the concept of LRMS connectors. Currently CREAM supports all the batch system supported by BLAHPD (Batch Local Ascii Helper). Support is provided for the following batch systems:


CREAM supports different batch systems, through the concept of LRMS connectors. Currently CREAM supports all the batch system supported by BLAHPD (Batch Local Ascii Helper).
*Torque/PBS  
Support is provided for the following batch systems:
*LSF  
* Torque/PBS
*SGE
* LSF
* SGE  


'''Q3: How does proxy credential delegation and proxy renewal work in CREAM?
== '''Q3: How does proxy credential delegation and proxy renewal work in CREAM?'''  ==


Detailed documentation about security-related aspects of CREAM are available at the CREAM [https://edms.cern.ch/file/595770/1/CREAM-guide.pdf User Guide].
Detailed documentation about security-related aspects of CREAM are available at the CREAM [https://edms.cern.ch/file/595770/1/CREAM-guide.pdf User Guide].  


'''Q4: how are jobs submitted to CREAM?'''
== '''Q4: how can jobs be submitted to CREAM?''' ==


Jobs can be submitted to CREAM based CE:
Jobs can be submitted to CREAM based CE:  
* directly, using the CREAM CLI documented in the [https://edms.cern.ch/document/595770 CREAM user's guide]
* directly, using a custom client that can be implemented using a Web Services framework
* through the gLite WMS component
* through the CondorG system


CREAM has ability to accept either direct Jobs from gLite-UI, SOAP clients or any other developed UIs based on web-services or Jobs sent through WMS. Jobs can be controlled through operations such as suspend, resume and delete. CREAM supports the execution of normal/serial batch jobs, and parallel (MPI) jobs. Jobs can also be submitted to CREAM through Condor.
*directly, using the CREAM Command Line Interface (CLI) documented in the [https://edms.cern.ch/document/595770 CREAM user's guide]
*directly, using a custom client that can be implemented using a Web Services framework
*through the gLite WMS component
*through the CondorG system


In case of submission through the Workload Management System (WMS), the ICE (Interface to CREAM Environment) component of the WMS is used. To select the CREAM instances to which the job can be submitted, submission criteria can be defined by means of the Job Description Language - JDL (see the [https://edms.cern.ch/document/592336 CREAM JDL Guide]).
CREAM has ability to accept either direct Jobs from gLite-UI, SOAP clients or any other developed UIs based on web-services or Jobs sent through WMS. Jobs can be controlled through operations such as suspend, resume and delete. CREAM supports the execution of normal/serial batch jobs, and parallel (MPI) jobs. Jobs can also be submitted to CREAM through Condor.  


Intructions for the management of input and output files are available
In case of submission through the Workload Management System (WMS), the ICE (Interface to CREAM Environment) component of the WMS is used. To select the CREAM instances to which the job can be submitted, submission criteria can be defined using the Job Description Language (JDL) (see the [https://edms.cern.ch/document/592336 CREAM JDL Guide]).  
[http://grid.pd.infn.it/cream/field.php?n=Main.SandboxManagement here].


CREAM implements an interface which is different from the ''GRAM'' interface used in the ''LCG-CE''. This means that it is not possible to interact with the CREAM CE using the Globus related tools and commands (globusrun, globus-job-submit, etc.). To interact directly (i.e. not through a high level service) with the CREAM CE, it is possible to use the CREAM CLI documented in the [https://edms.cern.ch/document/595770 CREAM user's guide] or via a custom client.
Intructions for the management of input and output files are available [http://grid.pd.infn.it/cream/field.php?n=Main.SandboxManagement here].  


Execution of ''DAG'' jobs on CREAM based CE through the gLite WMS is not implemented yet.
''NOTE:'' The interface of CREAM is different from the ''GRAM'' interface used by the ''LCG-CE''. This means that it is not possible to interact with the CREAM CE using Globus related tools and commands (globusrun, globus-job-submit, etc.). The CREAM CLI (documented in the [https://edms.cern.ch/document/595770 CREAM user's guide]) or the CREAM Web services interface can be used To interact directly with CREAM CEs (i.e. not through the WMS or other high level service).  


'''Q5: is the performance of CREAM comparable to the one of the lcg-ce?'''
Execution of ''DAG'' jobs on CREAM based CE through the gLite WMS is not implemented yet. Executing ''parametric'' and ''collection'' job types is already possible.


CREAM has been intentionally designed to deliver higher performance than lcg-ce.
== '''Q5: is the performance of CREAM comparable to the one of the lcg-ce?'''  ==


<!-- provide more information about these sections when available
CREAM has been intentionally designed to deliver higher performance than lcg-ce. <!-- provide more information about these sections when available
== Benchmarks ==
== Benchmarks ==


Line 63: Line 63:
=== CREAM-CE Performance ===
=== CREAM-CE Performance ===
-->
-->
'''Q6: What client errors are returned by CREAM?'''


These are documented here: [http://grid.pd.infn.it/cream/field.php?n=Main.ErrorMessagesReportedByCREAMToClient Cream Client Error Messages]
== '''Q6: What client errors are returned by CREAM?'''  ==
 
These are documented here: [http://grid.pd.infn.it/cream/field.php?n=Main.ErrorMessagesReportedByCREAMToClient Cream Client Error Messages]  
 
== '''Q7: Where can I find more information about the known issues of CREAM?'''  ==


'''Q7: Where can I find more information about the known issues of CREAM?'''
The known issues of current CREAM software in production release are documented here:


The known issues of current CREAM software in production release are documented here:
*[http://grid.pd.infn.it/cream/field.php?n=Main.KnownIssues Known Issues of CREAM-CE]
*[http://grid.pd.infn.it/cream/field.php?n=Main.KnownIssues Known Issues of CREAM-CE]


'''Q8: When is the end-of-support of lcg-CE scheduled?'''
== '''Q8: When is the end-of-support of lcg-CE scheduled?''' ==
 
The exact schedule is under discussion, but lcg-ce will likely reach end of support during the summer of 2011. More information about this will be circulated to the user communities as soon as possible.
 
== '''Q9: Why do the EGI sites replace lcg-CEs with CREAM-CEs?'''  ==
 
The functionality of the lcg-CE is gradually incorporated into the CREAM-CE software. Moreover CREAM-CE provides several new features (such as the cemon function, a more general Web service interface) and better scalability than lcg-CEs. A more scalable and a more feature-rich computing service can better serve EGI user communities.
 
== '''Q10: How many CREAM CEs are available in the infrastructure and what is the distribution of these among the NGIs?'''  ==
 
The number of sites that provide CREAM services is continuously changes. You can see the number of instances deployed per NGI at: http://gridmap.cern.ch/gm/#topo=regions&amp;layout=tc&amp;vo=OPS&amp;serv=CREAMCE (Position the mouse on a green rectangle to get the list of the CREAM instances available.)
 
The user that is interested in knowing if its VO is enabled on a given cream instance, can check this from the operations portal;
 
== '''Q11: How can I find out whether a given CREAM site supports my Virtual Organisation or not?'''  ==
 
You can check which VOs are supported by a given EGI site in the operation portal: http://operations-portal.in2p3.fr/vo/rd
 
== '''Q12: How can I get support about CREAM?'''  ==
 
You are recommended to submit a ticket to the EGI Helpdesk system ([https://gus.fzk.de/pages/home.php GGUS]). <!--*cream-support AT lists.infn.it-->
 
== '''Q13: Where can I find more documentation about CREAM?'''  ==


The exact schedule is under discussion, but lcg-ce will likely reqch end of support in Summer 2011. More information will be made available in this FAQ as soon as possible.
*[http://grid.pd.infn.it/cream/ CREAM Homepage]
*[http://grid.pd.infn.it/cream/field.php?n=Main.MovingFromLCG-CEToCREAM lcg-CE to CREAM wiki maintained by CREAM team]


'''Q9: How can I get support about CREAM?'''
== '''Q14: Is extra NGI_specific information available?''' ==


You are recommended to submit a ticket to the EGI Helpdesk system ([https://gus.fzk.de/pages/home.php GGUS]).
Some NGI instruction pages exist:  
<!--*cream-support AT lists.infn.it-->


'''Q10: Where can I find more documentation about CREAM?'''
*Italy: [http://wiki.italiangrid.org/twiki/bin/view/Installation/InstallationAndConfigurationOfACREAMCE How to configure and install CREAM CE]. Note: this page is only relevant to sites using the infn-grid middleware release.


* [http://grid.pd.infn.it/cream/ CREAM Homepage]
[[Category:Operations_Manuals]]
* [http://grid.pd.infn.it/cream/field.php?n=Main.MovingFromLCG-CEToCREAM lcg-CE to CREAM wiki maintained by CREAM team]

Latest revision as of 09:55, 14 May 2013

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators



The FAQ provides the questions and answers those may arise for the user communities during the transition from lcg-ce to cream-ce. It does not replace the CREAM User Guide (see the references section), but it provides a list of pointers to relevant documentation.

Q1: What is cream-ce?

The CREAM (Computing Resource Execution And Management) Service is a simple, lightweight service that implements all the operations at the Computing Element (CE) level; its well-defined Web Service-based interface and its implementation as an extension of the Java-Axis servlet (running inside the Apache Tomcat container) provide interoperability with clients written in any programming language and running on any computer platform.

The CREAM interface is defined in Web Service Description Language (WSDL); anyone can generate his/her CREAM client to interact with the service by filling in the stub code generated by WSDL parser (gSOAP for C/C++, Axis for Java, Perl module for perl).

The interface exposes a set of operations which can be classified in three groups.

  • Job management:operations are provided to create and subsequently manage jobs.
  • Lease management: allows users to define and manage leases associated with jobs. The lease mechanism ensures that jobs are managed even if the CREAM service loses connection with the client application.
  • Service Management: operations that deal with the whole CREAM service.

Q2: Which batch systems are supported by CREAM?

CREAM supports different batch systems, through the concept of LRMS connectors. Currently CREAM supports all the batch system supported by BLAHPD (Batch Local Ascii Helper). Support is provided for the following batch systems:

  • Torque/PBS
  • LSF
  • SGE

Q3: How does proxy credential delegation and proxy renewal work in CREAM?

Detailed documentation about security-related aspects of CREAM are available at the CREAM User Guide.

Q4: how can jobs be submitted to CREAM?

Jobs can be submitted to CREAM based CE:

  • directly, using the CREAM Command Line Interface (CLI) documented in the CREAM user's guide
  • directly, using a custom client that can be implemented using a Web Services framework
  • through the gLite WMS component
  • through the CondorG system

CREAM has ability to accept either direct Jobs from gLite-UI, SOAP clients or any other developed UIs based on web-services or Jobs sent through WMS. Jobs can be controlled through operations such as suspend, resume and delete. CREAM supports the execution of normal/serial batch jobs, and parallel (MPI) jobs. Jobs can also be submitted to CREAM through Condor.

In case of submission through the Workload Management System (WMS), the ICE (Interface to CREAM Environment) component of the WMS is used. To select the CREAM instances to which the job can be submitted, submission criteria can be defined using the Job Description Language (JDL) (see the CREAM JDL Guide).

Intructions for the management of input and output files are available here.

NOTE: The interface of CREAM is different from the GRAM interface used by the LCG-CE. This means that it is not possible to interact with the CREAM CE using Globus related tools and commands (globusrun, globus-job-submit, etc.). The CREAM CLI (documented in the CREAM user's guide) or the CREAM Web services interface can be used To interact directly with CREAM CEs (i.e. not through the WMS or other high level service).

Execution of DAG jobs on CREAM based CE through the gLite WMS is not implemented yet. Executing parametric and collection job types is already possible.

Q5: is the performance of CREAM comparable to the one of the lcg-ce?

CREAM has been intentionally designed to deliver higher performance than lcg-ce.

Q6: What client errors are returned by CREAM?

These are documented here: Cream Client Error Messages

Q7: Where can I find more information about the known issues of CREAM?

The known issues of current CREAM software in production release are documented here:

Q8: When is the end-of-support of lcg-CE scheduled?

The exact schedule is under discussion, but lcg-ce will likely reach end of support during the summer of 2011. More information about this will be circulated to the user communities as soon as possible.

Q9: Why do the EGI sites replace lcg-CEs with CREAM-CEs?

The functionality of the lcg-CE is gradually incorporated into the CREAM-CE software. Moreover CREAM-CE provides several new features (such as the cemon function, a more general Web service interface) and better scalability than lcg-CEs. A more scalable and a more feature-rich computing service can better serve EGI user communities.

Q10: How many CREAM CEs are available in the infrastructure and what is the distribution of these among the NGIs?

The number of sites that provide CREAM services is continuously changes. You can see the number of instances deployed per NGI at: http://gridmap.cern.ch/gm/#topo=regions&layout=tc&vo=OPS&serv=CREAMCE (Position the mouse on a green rectangle to get the list of the CREAM instances available.)

The user that is interested in knowing if its VO is enabled on a given cream instance, can check this from the operations portal;

Q11: How can I find out whether a given CREAM site supports my Virtual Organisation or not?

You can check which VOs are supported by a given EGI site in the operation portal: http://operations-portal.in2p3.fr/vo/rd

Q12: How can I get support about CREAM?

You are recommended to submit a ticket to the EGI Helpdesk system (GGUS).

Q13: Where can I find more documentation about CREAM?

Q14: Is extra NGI_specific information available?

Some NGI instruction pages exist: