By default Lithops works on Localhost if no configuration is provided. To run workloads on the Cloud, you must configure both a compute and a storage backend. Failing to configure them properly will prevent Lithops to submit workloads. Lithops configuration can be provided either in a configuration file or in runtime via a Python dictionary.
To configure Lithops through a configuration file you have multiple options:
-
Create a new file called
configin the~/.lithopsfolder (i.e:~/.lithops/config). -
Create a new file called
.lithops_configin the root directory of your project from where you will execute your Lithops scripts. -
Create a new file called
configin the/etc/lithops/folder (i.e:/etc/lithops/config). Useful for sharing the config file on multi-user machines. -
Create the config file in any other location and configure the
LITHOPS_CONFIG_FILEsystem environment variable:LITHOPS_CONFIG_FILE=<CONFIG_FILE_LOCATION>
An alternative mode of configuration is to use a python dictionary. This option allows to pass all the configuration details as part of the Lithops invocation in runtime. An entire list of sections and keys is here
Choose your compute and storage engines from the table below
|
Compute Backends |
Storage Backends |
|---|---|
|
Serverless (FaaS) Backends: Serverless (CaaS) Backends: Standalone Backends: |
Object Storage:
In-Memory Storage: |
Test if Lithops is working properly:
import lithops
def hello_world(name):
return f'Hello {name}!'
if __name__ == '__main__':
fexec = lithops.FunctionExecutor()
fexec.call_async(hello_world, 'World')
print(fexec.get_result())Example of providing configuration keys for IBM Code Engine and IBM Cloud Object Storage
import lithops
config = {
'lithops': {
'backend': 'code_engine',
'storage': 'ibm_cos'
},
'ibm': {
'region': 'REGION',
'iam_api_key': 'IAM_API_KEY',
'resource_group_id': 'RESOURCE_GROUP_ID'
},
'ibm_cos': {
'storage_bucket': 'STORAGE_BUCKET'
}
}
def hello_world(number):
return f'Hello {number}!'
if __name__ == '__main__':
fexec = lithops.FunctionExecutor(config=config)
fexec.map(hello_world, [1, 2, 3, 4])
print(fexec.get_result())| Group | Key | Default | Mandatory | Additional info |
|---|---|---|---|---|
| lithops | backend | aws_lambda | no | Compute backend implementation. localhost is the default if no config or config file is provided. |
| lithops | storage | aws_s3 | no | Storage backend implementation. localhost is the default if no config or config file is provided. |
| lithops | data_cleaner | True | no | If True, automatically deletes temporary data written to storage_bucket/lithops.jobs. |
| lithops | monitoring | storage | no | Monitoring system implementation. Options: storage or rabbitmq. |
| lithops | monitoring_interval | 2 | no | Interval in seconds for monitoring checks when using storage monitoring. |
| lithops | data_limit | 4 | no | Maximum size (in MB) for iterator data chunks. Set to False for unlimited size. |
| lithops | execution_timeout | 1800 | no | Maximum execution time in seconds for functions. Functions exceeding this time are terminated. Can also be set per call via the timeout parameter. |
| lithops | include_modules | [] | no | List of dependencies to explicitly include for pickling. If empty, all required dependencies are included. If set to None, no dependencies are included. |
| lithops | exclude_modules | [] | no | List of dependencies to exclude from pickling. Ignored if include_modules is set. |
| lithops | log_level | INFO | no | Logging level. Options: WARNING, INFO, DEBUG, ERROR, CRITICAL. Set to None to disable logging. |
| lithops | log_format | "%(asctime)s [%(levelname)s] %(name)s -- %(message)s" | no | Format string for log messages. |
| lithops | log_stream | ext://sys.stderr | no | Logging output stream, e.g., ext://sys.stderr or ext://sys.stdout. |
| lithops | log_filename | (empty) | no | File path for logging output. Overrides log_stream if set. |
| lithops | retries | 0 | no | Number of retries for failed function invocations when using the RetryingFunctionExecutor. Default is 0. Can be overridden per API call. |