Preparing apps for actions
IBM Cloud® Functions is deprecated. Existing Functions entities such as actions, triggers, or sequences will continue to run, but as of 28 December 2023, you can’t create new Functions entities. Existing Functions entities are supported until October 2024. Any Functions entities that still exist on that date will be deleted. For more information, see Deprecation overview.
Whether you bring an app with you, or you write a script specifically to respond to an event, your code must meet certain requirements before you create an action from it.
Each programming language has specific requirements to run, but most have the following general requirements:
-
The expected name for the entry point into the code is
main
by default. If your entry point is notmain
, a custom name can be specified when the action is created, so take note of that name. -
Input parameters into your app and output results from your app must be formatted into a specific structure that can be passed between entities. The structure depends on your code language. For example, with Python apps, the input parameters must be a dictionary and the result of your app must be structured as a dictionary. Because you can also pass parameters in a structured object to your action. In JSON, for example, you might structure your code to expect an input parameter with JSON values from certain fields, like
name
andplace
.JSON input example
{"name": "Dorothy", "place": "Kansas"}
JavaScript example
function main(params) { return {payload: 'Hello, ' + params.person.name + ' from ' + params.person.place}; }
-
If your app contains multiple files, they must be combined into one single file to be used in an action. You can either rewrite your code into one file or you can package the files and dependencies into a single archive file. If your runtime is not supported, you can package your app in a Docker image.
Code compilation is not required, but if possible for your runtime, compiling your code in advance can improve performance.
Preparing JavaScript apps
Before you create an action, get your JavaScript code ready. Confirm that your code is structured properly, then decide whether it needs packaged.
Structuring JavaScript code
- The expected name for the entry point function is
main
. If the function in your code is notmain
, take note of the name to specify it when the action is created. - The input parameters are passed as a JSON object.
- The result of a successful activation is also a JSON object but is returned differently depending on whether the action is synchronous or asynchronous.
Example
function main() {
return {payload: 'Hello world'};
}
Example with multiple functions
function main() {
return { payload: helper() }
}
function helper() {
return new Date();
}
Structuring JavaScript code with synchronous behavior
The JavaScript activation is synchronous when the main function either exits without executing a return
statement or exits by executing a return
statement that returns any value except a promise.
Example of synchronous code
// each path results in a synchronous activation
function main(params) {
if (params.payload == 0) {
return;
} else if (params.payload == 1) {
return {payload: 'Hello, World!'};
} else if (params.payload == 2) {
return {error: 'payload must be 0 or 1'};
}
}
Structuring JavaScript code with asynchronous behavior
JavaScript functions can continue to run in a callback function even after a return. The JavaScript activation is asynchronous if the main function exits by returning a promise. In this case, the system assumes that the action is still running
until the promise is fulfilled or rejected. JavaScript functions that run asynchronously can return the activation result after the main
function returns by returning a promise in your action.
Start by instantiating a new promise object and passing a callback function. The callback takes two arguments, resolve and reject, which are both functions. All your asynchronous code goes inside that callback. The action handler can have
any name that conforms to the conventional signature of accepting an object and returning an object (or a Promise
of an object).
In the following example, you can see how to fulfill a promise by calling the resolve function.
function main(args) {
return new Promise(function(resolve, reject) {
setTimeout(function() {
resolve({ done: true });
}, 2000);
})
}
This example shows how to reject a promise by calling the reject function.
function main(args) {
return new Promise(function(resolve, reject) {
setTimeout(function() {
reject({ done: true });
}, 2000);
})
}
In the examples, the following details are executed.
- The
main
function returns a promise. The promise indicates that the activation isn't completed yet but is expected to in the future. - The
setTimeout()
JavaScript function waits for 2 seconds before it calls the promise's callback function, which represents the asynchronous code. - The promise's callback accepts the arguments
resolve
andreject
, which are both functions.- The call to
resolve()
fulfills the promise and indicates that the activation completes normally. - A call to
reject()
can be used to reject the promise and signal that the activation completes abnormally.
- The call to
Structuring JavaScript code with synchronous and asynchronous behavior
An action can be synchronous on some inputs and asynchronous on others, as shown in the following example.
function main(params) {
if (params.payload) {
// asynchronous activation
return new Promise(function(resolve, reject) {
setTimeout(function() {
resolve({ done: true });
}, 2000);
})
} else {
// synchronous activation
return {done: true};
}
}
Example: Calling an external API with JavaScript
The following example invokes the external API for the NASA Astronomy Picture of the Day (APOD) service, which provides a unique image of our universe every day.
let rp = require('request-promise')
function main(params) {
const options = {
uri: "https://api.nasa.gov/planetary/apod?api_key=NNKOjkoul8n1CH18TWA9gwngW1s1SmjESPjNoUFo",
json: true
}
return rp(options)
.then(res => {
return { response: res }
})
}
A call is made to the NASA APOD API, and fields are extracted from the JSON result.
Next, create, and invoke the action to test it. The following example object is returned.
{
"copyright": "Eric Houck",
"date": "2018-03-28",
"explanation": "Does an alignment like this occur only once in a blue moon? ...",
"hdurl": "https://apod.nasa.gov/apod/image/1803/MoonTree_Houck_1799.jpg",
"media_type": "image",
"service_version": "v1",
"title": "Blue Moon Tree",
"url": "https://apod.nasa.gov/apod/image/1803/MoonTree_Houck_960.jpg"
}
Packaging JavaScript code with the webpack
module
Before you begin, review the packages that are included with the JavaScript runtime to see whether a dependency of your app is already included with the runtime. If your dependency is not included, you must package it with your app. The following steps assume that you are running the commands on a Linux-based distribution on a processor with AMD64-based architecture.
-
Create a
package.json
file. Addwebpack
as a development dependency.{ "name": "my-action", "main": "dist/bundle.js", "scripts": { "prebuild": "NODE_ENV=development npm install", "build": "webpack --config webpack.config.js ", "deploy": "ibmcloud fn action update my-action dist/bundle.js --kind nodejs:20", "clean": "rm -rf node_modules package-lock.json dist" }, "dependencies": { "left-pad": "1.1.3" }, "devDependencies": { "webpack": "^5.72.0", "webpack-cli": "^4.9.2" } }
-
Save the following
webpack
configuration code in a file namedwebpack.config.js
.var path = require('path'); module.exports = { entry: './index.js', output: { path: path.resolve(__dirname, 'dist'), filename: 'bundle.js' }, target: 'node' };
-
Prepare your app code. In this example, which you can save as a file that is named
index.js
, the variableglobal.main
is set as the main function of the app.Example
function myAction(args) { const leftPad = require("left-pad") const lines = args.lines || []; return { padded: lines.map(l => leftPad(l, 30, ".")) } } global.main = myAction;
-
Install all dependencies locally.
npm install
npm run prebuild
While most
npm
packages install JavaScript sources onnpm install
, some packages also install and compile platform-dependent binary file artifacts. Because this environment is Linux AMD64-based, thenpm install
must be executed on a similar platform. Otherwise the action invocations might not succeed. -
Build the
webpack
bundle locally.npm run build
The file
dist/bundle.js
is created and deploys as the action source code.To avoid compatibility issues, you can use the runtime to build the
webpack
. Use the following command in the source directory to run steps 4 and 5 inside the container.docker run --rm -it --entrypoint "/bin/bash" -v $PWD:/nodejsAction ibmfunctions/action-nodejs-v20:1.0.0 -c "npm run prebuild && npm run build"
-
Create the action by using the
npm
script or theibmcloud fn action update
CLI.-
Run the following
npm
script.npm run deploy
-
Or run the following IBM Cloud CLI command.
ibmcloud fn action update my-action dist/bundle.js --kind nodejs:20
The bundle file that is built by
webpack
supports only JavaScript dependencies. Action invocations might fail if the bundle has other dependencies because these dependencies are not included with the filebundle.js
. -
-
You can clean up the generated artifacts
package-lock.json
,node_modules
anddist
by using one of the following options.npm run clean
or
docker run --rm -it --entrypoint "/bin/bash" -v $PWD:/nodejsAction ibmfunctions/action-nodejs-v20:1.0.0 -c "npm run clean"
Packaging JavaScript code as NPM files
As an alternative to writing all your action code in a single JavaScript source file, you can package your code as a npm
package in a compressed file.
Before you begin, review the packages that are included with the JavaScript runtime to see whether a dependency of your app is already included with the runtime. If your dependency is not included, you must package it with your app. The following steps assume that you are running the commands on a Linux-based distribution on a processor with AMD64-based architecture.
-
In the root directory, create the
package.json
andmy-action.js
file.Example
package.json
{ "name": "my-action", "main": "my-action.js", "dependencies" : { "left-pad" : "1.1.3" } }
my-action.js
function myAction(args) { const leftPad = require("left-pad") const lines = args.lines || []; return { padded: lines.map(l => leftPad(l, 30, ".")) } } exports.main = myAction;
-
Install all dependencies locally.
npm install <dependency>
While most
npm
packages install JavaScript sources onnpm install
, some packages also install and compile platform-dependent binary file artifacts. Because this environment is Linux AMD64-based, thenpm install
must be executed on a similar platform. Otherwise the action invocations might not succeed. -
Create an archive that contains all files, including all dependencies.
zip -r action.zip *
Windows users Using the Windows Explorer action for creating the compressed file results in an incorrect file structure. Cloud Functions archive actions must have
package.json
at the root of the archive, but Windows Explorer places it inside a nested folder. Use thezip
command instead.
NPM libraries with native dependencies
Node.js libraries can depend on native modules that are compiled during installation for the local runtime by using the npm install
command. IBM Cloud runtimes are based on a Linux AMD64 platform, which requires that the native
modules are compiled for that platform. If you are not using a Linux AMD64-based operating system, you can use the nodejs runtime
Docker container to install the dependencies.
Zipped actions for Cloud Functions are limited to 48MB
. If your zipped action exceeds this limit, you must create a custom docker image for your action.
-
Run the following command to fetch the Node.js modules, compile the native dependencies and Create the zipped action code, including the
node_modules
directory.docker run --rm -it --entrypoint "/bin/bash" -v $PWD:/nodejsAction ibmfunctions/action-nodejs-v20:1.0.0 -c "npm install && zip action.zip -r *"
-
Create the action by using the Cloud Functions CLI.
ibmcloud fn action create my-action action.zip --kind nodejs:20
Using ES6 module in your action
If you want to use new ES6 modules in your Actions you will have to create a wrapper function to import the new modules. This wrapper consists of a global variable for your module you want to import and a function which imports the module as a promise.
To load extra more use promise chaining to load further ES6 modules. Add the variable name (let module_name
) and the import code to add another ES6 module (.then( () => return import('other_module').then( module => module_name = module.<obj_to_load>))
).
-
Create the
my-action.js
file with the following content.let uuidv5; function main(params) { const MY_NAMESPACE = '1b671a64-40d5-491e-99b0-da01ff1f3341'; console.log( "uuidv5 = " + uuidv5) const triggerID = uuidv5('Hello, World!', MY_NAMESPACE); console.log("----> uuid module successfully loaded and trigger calculated = ", triggerID) return { message: 'Hello World with id = ' + triggerID }; } function main_wrapper(params) { return import('uuid').then(module => uuidv5 = module.v5) //.then( () => return import('xxxxx').then( module => global_var_yy = module.<obj_to_load>)) .then( () => { return main( params ) }) }
-
Create the action and set the main action to
main_wrapper
.ibmcloud fn action create my-action my-action.js --main main_wrapper
If you are creating a action with the Cloud Functions UI you must rename the
main_wrapper
function tomain
and themain
function to something else, as the entry function in the UI is alwaysmain
.When you create a zipped action, follow the previous guide and specify the entry point with
exports.main = main_wrapper
-
Invoke your action.
ibmcloud fn action invoke my-action
How do I package my Python app for deployment in Cloud Functions
Structuring Python code
Python apps must consume a dictionary and produce a dictionary. The expected name for the entry point method is main
. If the function in your code is not main
, take note of the name to specify it when the action is
created.
Example
def main(args):
name = args.get("name", "stranger")
greeting = "Hello " + name + "!"
print(greeting)
return {"greeting": greeting}
Packaging multiple Python files into an archive
When to use this method
Your app uses multiple Python files, but does not require any dependencies or packages outside of the packages that are included with the base Python runtime. You can create a compressed file that includes your Python files and deploy the compressed file when you create your action.
Example command
ibmcloud fn action create <action_name> <compressed_python_files.zip> --kind python:3.11
For more information, see Packaging multiple Python files into a compressed file.
Packaging Python code with a local virtual environment in a compressed file
When to use this method
If your app requires dependencies that are not included with the base Cloud Functions Python runtime, you can install those dependencies into a virtualenv
folder and then compress into a compressed file to deploy in Cloud Functions. Your compressed file must be smaller than the maximum codeSize
as described in the Action Limits.
Example command
ibmcloud fn action create <action_name> <compressed_python_virtualenv.zip> --kind python:3.11
For more information, see Packaging Python code with a local virtual environment in a compressed file.
Packaging Python code with a Docker virtual environment in a compressed file
When to use this method
If your app requires dependencies that are not included with the base Cloud Functions Python runtime, you can install those dependencies into a virtualenv
folder by using the Python environment inside the Cloud Functions Python runtime image. You can then compress the folder into a compressed file to deploy in Cloud Functions. Your compressed file must be smaller than the maximum codeSize
as described in the Action Limits.
Example command
You can use the compressed file to create an action. Replace <file_path>
with the file path to your compressed file.
ibmcloud fn action create <action_name> <compressed_python_virtualenv.zip> --kind python:3.11
For more information, see Packaging Python code with a Docker virtual environment in a compressed file.
Packaging large Python dependencies in a custom Docker image
When to use this method
Your app requires dependencies that are not included with the base Cloud Functions Python runtime. You can specify a base Cloud Functions image in your Dockerfile and also specify the dependencies to install when you build your Docker image. You can then specify your custom Docker image when you deploy your app in Cloud Functions. Note that only public Docker images are supported.
Example command
ibmcloud fn action create <action_name> <app_code.py> --docker <dockerhub_username>/<repo_name>:<tag_name>
If your app code is larger than the maximum codeSize
described in the Action Limits, you can combine this method with Packaging multiple Python files into a compressed file.
For more information, see Packaging large Python dependencies in a custom Docker image.
Packaging your app within a custom Docker image
When to use this method
Your app requires dependencies that are not included with the base Cloud Functions Python runtime and your app code, even when it is compressed into a
.zip file, is still larger than the maximum codeSize
as described in the Action Limits. If so, you can install those dependencies into a custom Docker image
and include your app code in the action/exec
folder of the Docker skeleton. You can then specify your custom Docker image when you deploy your app in Cloud Functions. During deployment, you do not need to specify for your app
code. Note that only public Docker images are supported.
Example command
ibmcloud fn action create <action_name> --docker <dockerhub_username>/<repo_name>:<tag_name>
Packaging Python code
The following sections provide tutorials for how to package your Python app for deployment with Cloud Functions.
Before you begin
Review How do I package my Python app for deployment in Cloud Functions?.
Packaging multiple Python files into a compressed file
Package Python code and dependent modules in a compressed file. In this example, the source file that contains the entry point is __main__.py
and the helper modules are in a file called helper.py
.
Before you begin, review the packages that are included with the Python runtime to see whether a dependency of your app is already included with the runtime. If your dependency is not included, you must package it with your app.
-
Create a
test
directory on your desktop. -
Save the following code as a file called
__main__.py
in yourtest
directory.from helper import helper def main(args): return helper(args)
-
Save the following code as a file called
helper.py
in yourtest
directory. This code accepts aname
parameter and returns a greeting. If noname
is specified, thename
that is returned isstranger
.def helper(dict): if 'name' in dict: name = dict['name'] else: name = "stranger" greeting = "Hello from helper.py, " + name + "!" return {"greeting": greeting}
-
To package your app as a compressed file,
cd
to yourtest
directory and run the following command. In this example, the archive is calledstranger.zip
.zip -r stranger.zip __main__.py helper.py
-
You can use then use the compressed file to create an action called
hello
. Replace<file_path>
with the file path to your compressed file.ibmcloud fn action create hello <file_path>/test/stranger.zip --kind python:3.11
-
Test the action.
ibmcloud fn action invoke hello --result
Example output
{ "greeting": "Hello from helper.py, stranger!" }
-
Test the action again and specify the
name
parameter. Replace the<your_name>
parameter with your name.ibmcloud fn action invoke hello --result --param name <your_name>
Example output
{ "greeting": "Hello from helper.py, <your_name>!" }
Packaging Python code with a local virtual environment in a compressed file
You can package Python dependencies by using a virtual environment, virtualenv
. With the virtual environment, you can link additional packages that can be installed by using pip
.
The setup of the local Python environment has a major impact on the compressed action file that is created. Some configurations (for example, non-default installation paths or mixed Python installations) can make the compressed action file fail during execution.
To minimize these dependencies on your local environment, use the Packaging Python code with a Docker virtual environment in a compressed file approach. This approach creates the compressed action file, but also leverages the Python environment inside the Cloud Functions Python runtime image itself so that both the generated action compressed file and the later execution environment fully match.
Before you begin
-
The following steps assume that you are running the commands on a Linux-based distribution on a processor with AMD64-based architecture.
-
Review the packages that are included with the Python runtime to see whether a dependency of your app is already included with the runtime. If your dependency is not included, you must package it with your app.
-
Make sure that the locally installed Python version to create the compressed action file (for example, Python 3.11.x) matches the Cloud Functions kind that is chosen to later create the action (
--kind python:3.11
). -
Install the
virtualenv
Python package.pip install virtualenv
To package your app:
-
Create a directory that you can use to create your virtual environment. In this example, a
jokes
directory is created on the desktop. After you create thejokes
directory,cd
to it.cd desktop; mkdir jokes; cd jokes
-
From the
jokes
directory, create a virtual environment namedvirtualenv
.The virtual environment must be named
virtualenv
.virtualenv virtualenv
Example output
created virtual environment CPython3.9.10.final.0-64 in 398ms creator CPython3Posix(dest=/action/Projects/python/virtualenv, clear=False, no_vcs_ignore=False, global=False) seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv) added seed packages: pip==21.2.4, setuptools==58.0.4, wheel==0.37.0 activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
-
From your
jokes
directory, activate yourvirtualenv
virtual environment.source virtualenv/bin/activate
-
Ensure the Python version inside the virtual environment matches the version as specified with the
--kind
option when you create the action later on (for example,python:3.11
). To check the actual version,python --version
-
Install the
pyjokes
module.(virtualenv) $ pip install pyjokes
Example output
Collecting pyjokes Using cached pyjokes-0.6.0-py2.py3-none-any.whl (26 kB) Installing collected packages: pyjokes Successfully installed pyjokes-0.6.0
-
Stop the
virtualenv
virtual environment.(virtualenv) $ deactivate
-
Copy the following code and save it into a file called
__main__.py
in yourjokes
directory.import pyjokes def joke(params): return {"joke": pyjokes.get_joke()}
-
From your
jokes
directory, create an archive of thevirtualenv
folder and your__main__.py
file. These files must be in the top level of your.zip
file.zip -r jokes.zip virtualenv __main__.py
Example output
... adding: virtualenv (stored 0%) adding: __main__.py (deflated 18%) ...
-
Create an action called
jokes
that uses yourjokes.zip
file. You must also specify the entry point asjokes
. You must also specify the--kind
flag for the runtime.ibmcloud fn action create jokes </path/to/file/>jokes.zip --kind python:3.11 --main joke
Example output
ok: created action jokes
-
Invoke the action to verify it is working. Include the
--result
flag to return the result in the command line.ibmcloud fn action invoke jokes --result
Example output
{ "joke": "A QA engineer walks into a bar. Runs into a bar. Crawls into a bar. Dances into a bar. Tiptoes into a bar. Rams a bar. Jumps into a bar." }
You can use this method to extend the functionality of Cloud Functions actions by using other Python packages.
Packaging Python code with a Docker virtual environment in an archive
You can package Python dependencies by using a virtual environment, virtualenv
. By using the virtual environment, you can link more packages that can be installed by using pip
.
This approach is recommended when you want to add additional required python packages. It ensures that the generated compressed action file is compatible with the Python runtime used for later execution of the action.
Before you begin
- Review the packages that are included with the Python runtime to see whether a dependency of your app is already included with the runtime. If your dependency is not included, you must package it with your app.
- The following steps assume that you are running the commands on a Linux-based distribution on a processor with AMD64-based architecture.
Package your app by completing the following steps.
-
Create a directory that you can use to create your virtual environment. In this example, a
test
directory is created on the desktop. After you create thetest
directory,cd
to it.cd desktop; mkdir test; cd test
-
Create a requirements.txt in your file in your
test
directory that contains thepip
modules and versions to install.touch requirements.txt
-
Use vim to edit the
requirements.txt
file. Enter the names of thepip
modules and versions you want to install. In this example,pyjokes
is the module that is used.vim requirements.txt
Example requirements.txt
pyjokes
-
Press
ESC
, then:wq
to save and close yourrequirements.txt
file.To keep the
virtualenv
to a minimum size, add only the modules that are not part of the selected runtime environment to therequirements.txt
. For more information about the packages that are included in Python runtimes, see the Python runtime reference.- For
python:3.11
, use the Docker imageibmfunctions/action-python-v3.11
. - For
python:3.9
, use the Docker imageibmfunctions/action-python-v3.9
. - For
python:3.7
, use the Docker imageibmfunctions/action-python-v3.7
. - For
python:3.6
, use the Docker imageibmfunctions/action-python-v3.6
.
Example
docker pull ibmfunctions/action-python-v3.9:1.0.0
Example output
Using default tag: latest latest: Pulling from ibmfunctions/action-python-v3.9:1.0.0
- For
-
Create a virtual environment and install the additional Python packages.
The virtual environment directory must be named
virtualenv
to create avirtualenv
folder in thetest
directory.docker run --rm -v "$PWD:/tmp" --entrypoint "/bin/bash" ibmfunctions/action-python-v3.9:1.0.0 -c "cd /tmp && virtualenv virtualenv && source virtualenv/bin/activate && pip install -r requirements.txt"
This command instantiates a container (
docker run
) based on the runtime image selected and mounts the current working directory ($PWD
) as/tmp
into the container (-v "$PWD:/tmp"
). Inside the container, it then changes to the/tmp
directorycd /tmp
, creates and activates thevirtualenv
(virtualenv virtualenv && source virtualenv/bin/activate
), and runs thepip install
to add the selected packages. The container is deleted when the command completes (--rm
). The directory structure of the createdvirtualenv
and finally, the installed packages can be found in the foldervirtualenv
in your current directory.Example output
created virtual environment CPython3.9.10.final.0-64 in 3291ms creator CPython3Posix(dest=/tmp/virtualenv, clear=False, no_vcs_ignore=False, global=False) seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv) added seed packages: pip==21.3.1, setuptools==60.2.0, wheel==0.37.1 activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator Collecting pyjokes Downloading pyjokes-0.6.0-py2.py3-none-any.whl (26 kB) Installing collected packages: pyjokes Successfully installed pyjokes-0.6.0
-
Save the following code as
__main__.py
in yourtest
directory. When you create actions with a compressed file, the source file that contains the entry point must be named__main__.py
.import pyjokes def joke(params): return {"joke": pyjokes.get_joke()}
-
To deploy this code as an action, you must create a compressed file of the
virtualenv
folder and the__main__.py
file.Sometimes, the resulting compressed file is larger than the maximum
codeSize
as described in the Action Limits allowed by Cloud Functions. To reduce the size of the compressed file, select only the dependencies that you need, rather than selecting the entirevirtualenv
folder. The packages that you need can be found in thesite-packages
directory within thevirtualenv
folder. Note that you must also include theactivate_this.py
file from thebin
directory of yourvirtualenv
folder in your compressed file.zip -r pyjoke.zip virtualenv __main__.py
-
Create an action called
pyjoke
by using thepyjoke.zip
file. Make sure to use the--kind
corresponding to the runtime image used to create the compressed action file. Otherwise, the action fails to execute during invoke.ibmcloud fn action create pyjoke <file_path>/pyjoke.zip --kind python:3.11
-
Invoke the action to test that the
pyjoke
module is working.ibmcloud fn action invoke pyjoke --result
Example output
{ "joke": "A QA engineer walks into a bar. Runs into a bar. Crawls into a bar. Dances into a bar. Tiptoes into a bar. Rams a bar. Jumps into a bar." }
Packaging large Python dependencies in a custom Docker image
Cloud Functions has a size limit for the app code, see maximum codeSize
described in the Action Limits. However, you can install large packages and dependencies
into a custom Docker image and deploy it with your app code when you create an action. You can then import the packages at run time.
In this example, install large Python packages such as matplotlib
and seaborn
to build a Cloud Functions web action that generates a PNG file of a joint plot with seaborn
.
Before you begin
- Review the packages that are included with the Python runtime to see whether a dependency of your app is already included with the runtime. If your dependency is not included, you must package it with your app.
- The following steps assume that you are running the commands on a Linux-based distribution on a processor with AMD64-based architecture.
Only public Docker images are supported.
Package the app in a custom Docker image by completing the following steps.
-
Create a directory that you can use to create your Dockerfile. In this example, a
functions
directory is created on the desktop. After you create thefunctions
directory,cd
to it.cd desktop; mkdir functions; cd functions
-
Create a Dockerfile in your
functions
directory.touch Dockerfile
-
Use vim to edit the
Dockerfile
file. Enter the names of thepip
modules and versions you want to install. In this example, several additional Python modules are installed.vim Dockerfile
-
Paste or enter the following text in your
Dockerfile
.FROM ibmfunctions/action-python-v3.9:1.0.0 RUN pip install \ --upgrade pip \ matplotlib \ seaborn \ pandas \ statsmodels
-
Press
ESC
, then:wq
andEnter
to save and close your Dockerfile. -
From your
functions
directory, you can rundocker build
to build a Docker image by using yourDockerfile
.docker build -t <dockerhub_username>/<repo_name>:<tag_name> .
You can also run
docker build
commands with this format:docker build . -t <dockerhub_username>/<repo_name>:<tag_name>
-
The image builds and installs the dependencies that you specified in your Dockerfile.
[+] Building 0.1s (6/6) FINISHED => [internal] load build definition from Dockerfile.ml 0.0s => => transferring dockerfile: 40B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/ibmfunctions/action-python-v3.9:1.0.0 0.0s => [1/2] FROM docker.io/ibmfunctions/action-python-v3.9:1.0.0 0.0s => CACHED [2/2] RUN pip install --upgrade pip matplotlib seaborn pandas statsmodels 0.0s => exporting to image 0.0s => => exporting layers 0.0s => => writing image sha256:4a0d140e65fc379d8c25d18fce9aedd580203f768f43da011149993cd57565d4 0.0s => => naming to docker.io/docker-username/repo-name:tag ... ...
-
Push your image to Docker Hub.
docker push <dockerhub_username>/<repo_name>:<tag_name>
Be sure to log in to Docker Hub before you attempt to push your image.
-
Save the following code as
seaborn.py
in yourfunctions
directory. This code generates a joint plot inseaborn
that uses random data. You can then create a web action with Cloud Functions to return the plot to a Cloud Functions endpoint.# import modules import base64 import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns # optional: set seaborn style sns.set(style="dark") def main(args): #generate a jointplot from random data. x, y = np.random.randn(2, 300) g = (sns.jointplot(x,y, kind="hex", color="#9b59b6") .set_axis_labels("x", "y")) # save the plot as a .png so that it can be base64 encoded/decoded. plt.savefig('output.png') # open, read, and encode the image. image = base64.b64encode(open("output.png", "rb").read()) # decode the image into a string. data = image.decode('utf-8') # return the string as a JSON web request. return { 'body': data, 'statusCode': 200, 'isBase64Encoded': 'true', 'headers': {'Content-Type': 'image/png'} }
-
Create a web action called
seaborn
by using the custom Docker image that you created that contains the required Python dependencies to run a joint plot.ibmcloud fn action create seaborn --docker <dockerhub_username>/<repo_name>:<tag_name> seaborn.py --web true
Example output
ok: created action seaborn
-
Invoke the action to test it. Invoking the action returns the base64 string for the generated joint plot.
ibmcloud fn action invoke seaborn --result
Example output
<base64_string>, "headers": { "Content-Type": "image/png" }, "isBase64Encoded": "true", "statusCode": 200 }
-
Because this action is a web action, you can use the
action get
command to return the URL.ibmcloud fn action get seaborn --url
Example output
ok: got action seaborn https://us-south.functions.cloud.ibm.com/api/v1/web/<namespace_ID>/default/seaborn
-
Copy and paste the URL into your browser to see the generated joint plot. Refresh the page to invoke the action and generate a new plot.
You can use this method of building custom Docker images to install large dependencies rather than packaging them with your app.
Preparing apps in Docker images
With Cloud Functions, you can write your app in any language and package it as a Docker image.
You can use images from public registries only, such as an image that is publicly available on Docker Hub. Private registries are not supported. For more information about some possible workarounds, see Large Applications on OpenWhisk and Large (Java) Applications on Apache OpenWhisk.
Before you begin
- You must have a Docker Hub account. You can set up a free Docker ID and account on Docker Hub.
- Install Docker.
- Review the requirements for the Docker runtime.
- The following steps assume that you are running the commands on a Linux-based distribution on a processor with AMD64-based architecture.
Creating a custom Docker image for your action
In a Dockerfile, you can specify a Cloud Functions base runtime image by using the FROM
instruction. You can use the RUN
instruction to specify dependencies and packages to install in your Docker image. For more information
about creating a Dockerfile, see the Dockerfile reference.
You can see a list of ibmfunctions
Docker base images on Docker Hub.
Cloud Functions limits app code to the maximum codeSize
as described in the Action Limits.
-
Create a
test
directory on your Desktop andcd
to it.cd desktop; mkdir test; cd test
-
Create a Dockerfile in your
test
directory and open it invim
.touch Dockerfile; vim Dockerfile
-
Press the
i
key to edit your Dockerfile. -
Specify a Cloud Functions base image with the
FROM
argument in your Dockerfile. -
Install any packages and dependencies by specifying them
RUN
argument followed by the installation commands for your dependencies. -
Press
ESC
, then:wq
andEnter
to save and close your Dockerfile.Example Dockerfile for installing Python dependencies
The following example uses
ibmfunctions/action-python-v3.7
as a base image and installs the Python modules:matplotlib
,pandas
, andstatsmodels
.FROM ibmfunctions/action-python-v3.7 RUN pip install \ --upgrade pip \ matplotlib \ pandas \ statsmodels
-
Build your custom Docker image.
docker build -t <dockerhub_username>/<repo_name>:<tag_name>
-
Push your image to Docker Hub.
docker push <dockerhub_username>/<repo_name>:<tag_name>
Be sure to log in to Docker Hub before you attempt to push your image.
Deploying an action with a custom Docker image
When you create your Cloud Functions action, you can combine your app file with a public Docker image to create a custom runtime environment. The action is invoked with the Docker image.
Run the action create
command and include the --docker
flag to specify a Docker image for your app to use.
ibmcloud fn action create <action_name> --docker <dockerhub_username>/<image_name> <app_file>
You can also deploy a compressed file with a Docker image to create an action. You can use the previous command and replace <app_file>
with your compressed file. You can use this method to deploy large app files or incorporate
large dependencies.
To see an example deployment of a custom Docker image with a Cloud Functions action, see Packaging large Python dependencies in a custom Docker image.
Preparing Go Apps
You can create Actions by using Golang.
Use a single file for quick testing or development purposes. For production apps, pre-compile your Go actions into an executable file for better performance. To deploy actions made up of multiple source files and including third-party libraries,
package them as compressed file and deploy the file. When deploying a compressed file, specify the runtime by using the kind
parameter (--kind=go:1.21
)
Although you can create a compressed file on any Go platform by cross-compiling with GOOS=Linux
and GOARCH=amd64
, use the pre-compilation feature that is embedded in the runtime container image(docker run -i openwhisk/action-golang-v1.21:nightly ...
).
You can package multiple source files or vendor libraries.
The following steps assume that you are running the commands on a Linux-based distribution on a processor with AMD64-based architecture. You must install the ibmcloud cli
to run the commands. Note that some examples also require
Docker.
Structuring Go Apps
When you structure your Go code, note that the expected name for the entry point package is main
. If the package in your code is not main
, take note of the name to specify it when the action is created (--name <your name>
).
The package must also be public (start with an upper-case letter).
This example creates a simple Hello World
action in Go.
package main
import "fmt"
// Main is the function implementing the action
func Main(params map[string]interface{}) map[string]interface{} {
// parse the input JSON
name, ok := params["name"].(string)
if !ok {
name = "World"
}
msg := make(map[string]interface{})
msg["body"] = "Hello " + name + "!"
// can optionally log to stdout (or stderr)
fmt.Println("hello Go action")
// return the output JSON
return msg
}
Actions that are written in Go can be deployed as source code or as pre-compiled executable files in a compressed format. If your actions require only one source file, you can edit its contents directly in the Functions action window in the
IBM Cloud console if you create the action without pre-compiling it.
Use the following steps to create actions that use Go.
- Create the function that you want to deploy.
- (
optional
) If you have more than one file, package the files as a compressed file, otherwise skip this step (see the following examples) - (
optional
) Compile thego/zip
file by using the Docker image (docker run -i openwhisk/action-golang-v1.21:nightly -compile ...
). This step returns a compressed file that contains the executable file. - Create the action by using the
ibmcloud cli
.
These steps are used in each of the following examples.
Creating a simple Golang Action
You can create a simple action in Go by creating a file that contains a Go function.
-
Create the Go file that contains the function. The default entry name is
Main
. You can change this name to any different public function name (Uppercase first character
) if it is specified by using the--main
parameter with theibmcloud fn action create <action_name> <action_file> --main <function_name>
command; for example,--main hello
if the function is namedHello
.main.go
package main import "fmt" // Main is the function implementing the action func Main(params map[string]interface{}) map[string]interface{} { // parse the input JSON name, ok := params["name"].(string) if !ok { name = "World" } msg := make(map[string]interface{}) msg["body"] = "Hello " + name + "!" // can optionally log to stdout (or stderr) fmt.Println("hello Go action") // return the output JSON return msg }
-
(
optional
) If you want to pre-compile the function to an executable file that is stored in a compressed format first,docker run -i openwhisk/action-golang-v1.21:nightly -compile main <main.go >main-bin.zip
<
and>
are bash input output redirects and are part of the command.Specify the generated compressed file (
main-bin.zip
) as the file for theaction create
command. -
Create an action by using the Cloud Functions managed
go:1.21
runtime. If your action is not calledmain
, specify the function name with--name <your action name>
.With the source code (
main.go
),ibmcloud fn action create simple-action main.go
With the pre-compiled compressed file (
main-bin.zip
),ibmcloud fn action create simple-action main-bin.zip
Alternatively if you want to pin the runtime image to a fixed runtime image version, use the
--docker
tag.ibmcloud fn action create simple-action main.go --docker openwhisk/action-golang-v1.21:nightly
If you pin the action to a fixed runtime, it cannot change or receive security fixes.
Create a Golang action made up of multiple packages
You can create an action that includes multiple Go packages. Each package must include a go.mod
file.
.
├── go.mod
├── hello
│ ├── go.mod
│ └── hello.go
└── main.go
-
Create the following example files as shown in the following examples.
main.go
package main import ( "fmt" "hello" ) func Main(args map[string]interface{}) map[string]interface{} { fmt.Println("Main") return hello.Hello(args) }
go.mod
module action go 1.21 replace hello => ./hello require hello v0.0.0
hello/hello.go
package hello import ( "fmt" ) func Hello(args map[string]interface{}) map[string]interface{} { msg := make(map[string]interface{}) greetings := "world" name, ok := args["name"].(string) if ok { greetings = name } msg["msg"] = "Hello, " + greetings fmt.Printf("Hello, %s\n", greetings) return msg }
hello/go.mod
module hello go 1.19
-
Compress the source code files into a compressed file that is called
src.zip
.zip -r src.zip main.go go.mod hello/hello.go hello/go.mod
This command compresses the files
main.go go.mod hello/hello.go hello/go.mod
intosrc.zip
. For more information about thezip
command, useman zip
.If you are pre-compiling your code you might have to generate a go sum by running
go mod tidy
and adding it to your zip -
(
Optional
) If you want to pre-compile the code, you can compile your compressed source code with the Docker runtime image using-compile
Compile the function to an executable file that is stored in a compressed format and uses the go runtime itself.
docker run -i openwhisk/action-golang-v1.21:nightly -compile main <src.zip >main-bin.zip
<
and>
are bash input output redirects and are part of the command. -
Create the action. Note that the runtime must be specified as
--kind=go:1.21
.With
src.zip
ibmcloud fn action create multiple-packag-action src.zip --kind=go:1.21
With pre-compiled code (
main-bin.zip
)ibmcloud fn action create multiple-packag-action main-bin.zip --kind=go:1.21
Alternatively, if you want to pin the runtime image to a fixed runtime image version, use the
--docker
tag.ibmcloud fn action create multiple-packag-action src.zip --docker openwhisk/action-golang-v1.21:nightly
If you pin the action to a fixed runtime, the runtime cannot change or receive security fixes.
Create an action by using external libraries with Go modules
You can create an action by using third-party libraries with Go modules. For more information about Go modules, see Go module doc.
If the action has not been pre-compiled, then the libraries are downloaded at the action execution time. If you pre-compile the action, then the libraries are already packaged into the binary and don't need to be downloaded during the action execution time.
.
├── go.mod
└── main.go
-
Create the function.
main.go
package main import ( "github.com/rs/zerolog" "github.com/rs/zerolog/log" ) func init() { zerolog.TimeFieldFormat = "" } // Main function for the action func Main(obj map[string]interface{}) map[string]interface{} { name, ok := obj["name"].(string) if !ok { name = "world" } log.Debug().Str("name", name).Msg("Hello") msg := make(map[string]interface{}) msg["module-main"] = "Hello, " + name + "!" return msg }
go.mod
module action go 1.21 require github.com/rs/zerolog v1.19.0
-
Compress the source code to a compressed file that is called
src.zip
.zip -r src.zip main.go go.mod
This example compresses
main.go
andgo.mod
files tosrc.zip
.If you are pre-compiling your code you might have to generate a go sum by running
go mod tidy
and adding it to your zip -
If you want to pre-compile the code, use the compressed source code (
src.zip
) and compile it with the docker runtime image with the-compile
command.-
Compile the function to an executable file that is stored in a compressed format (
main-bin.zip
).docker run -i openwhisk/action-golang-v1.21:nightly -compile main <src.zip >main-bin.zip
<
and>
are bash input output redirects and are part of the command. -
Specify the compressed file (
main-bin.zip
) as the file for theaction create
command. The runtimekind
must be specified when you use a compressed file; for example,--kind=go:1.19
.
-
-
Create the action. The runtime must be specified with
--kind=go:1.21
.With
src.zip
ibmcloud fn action create module-action src.zip --kind=go:1.21
With pre-compiled code (
main-bin.zip
)ibmcloud fn action create module-action main-bin.zip --kind=go:1.21
Alternatively if you want to pin the runtime image to a fixed runtime image version, use the
--docker
tag.ibmcloud fn action create module-action src.zip --docker openwhisk/action-golang-v1.21:nightly
If you pin the action to a fixed runtime, the runtime cannot change or receive security fixes.
Preparing PHP apps
Before you create an action, get your PHP code ready.
Structuring PHP code
When you structure your code, note that the expected name for the entry point function is main
. If the function in your code is not main
, take note of the name to specify it when the action is created. PHP actions
always consume an associative array and return an associative array.
Example
<?php
function main(array $args) : array
{
$name = $args["name"] ?? "stranger";
$greeting = "Hello $name!";
echo $greeting;
return ["greeting" => $greeting];
}
?>
Packaging PHP code
You can package PHP files or dependent packages in a compressed file.
Before you begin, review the packages that are included with the PHP runtime to see whether a dependency of your app is already included with the runtime. If your dependency is not included, you must package it with your app.
To package your app, run the following command.
zip -r <archive_name>.zip <file_1>.php <file_2>.php
For example, package the following code examples.
-
Create the code examples.
index.php
<?php include 'helper.php'; function main(array $args) : array { $name = $args["name"] ?? "stranger"; $help = help($name); return ["help" => $help]; } ?>
helper.php
<?php function help($name) { return "Hello " . $name . " the answer to life the universe and everything is 42"; } ?>
-
Compress the code examples.
zip -r helloPHP.zip index.php helper.php
-
Create the action.
ibmcloud fn action create packagePHP helloPHP.zip --kind=php:7.4
Packaging Composer modules
You can package extra Composer modules into your action
Before you begin, review the packages that are included with the php runtime to see whether a dependency of your app is already included with the runtime. If your dependency is not included, you must package it with your app. The following steps assume that you are running the commands on a Linux-based distribution on a processor with AMD64-based architecture.
-
In the root directory, create a
index.php
andcomposer.json
file.index.php
<?php use Mpociot\ChuckNorrisJokes\JokeFactory; function main(array $args) : array { $jokes = new JokeFactory(); $joke = $jokes->getRandomJoke(); return ["joke" => $joke]; } ?>
composer.json
{ "require": { "mpociot/chuck-norris-jokes": "^1.0" } }
-
Install all dependencies locally.
composer install
While most
php
packages install PHP sources oncomposer install
, some packages also install and compile platform-dependent binary file artifacts. Because this environment is Linux AMD64-based, thephp install
must be executed on a similar platform. Otherwise the action invocations might not succeed. -
Create an archive that contains all files, including all dependencies.
zip -r action.zip *
Windows users Using the Windows Explorer action for creating the compressed file results in an incorrect file structure. Cloud Functions archive actions must have
index.php
andcomposer.json
at the root of the archive, but Windows Explorer places it inside a nested folder. Use thezip
command instead. -
Create the Function
ibmcloud fn action create chuckJoke action.zip --kind=php:7.4
Preparing Java apps
Before you create an action, get your Java code ready.
Structuring Java code
A Java action is a Java program with a method called main
. main
must have the following signature.
public static com.google.gson.JsonObject main(com.google.gson.JsonObject);
- You must specify the name of the main class by using
--main
. An eligible main class is one that implements a staticmain
method. If the class is not in the default package, use the Java fully qualified class name, for example,--main com.example.MyMain
. - You can customize the method name of your Java action by specifying the fully qualified method name of your action, for example,
--main com.example.MyMain#methodName
.
Packaging Java code
Package your code by creating a .jar
file.
Before you begin
You must have JDK 8 installed locally. This example uses the google-gson-2.9.0.jar
.
If you are working with a JDK version other than JDK 8, you must specify --release 8
when you compile your code with the javac
command.
To create a Java action, complete the following steps.
-
Save the following code in a file named
Hello.java
.import com.google.gson.JsonObject; public class Hello { public static JsonObject main(JsonObject args) { String name = "stranger"; if (args.has("name")) name = args.getAsJsonPrimitive("name").getAsString(); JsonObject response = new JsonObject(); response.addProperty("greeting", "Hello, " + name + "!"); return response; } }
-
Download the
gson-2.9.0.jar
. -
Add the
gson-2.9.0.jar
to yourClASSPATH
. This example usesgson-2.9.0.jar
, which is saved in atest
folder in theDesktop
directory.export CLASSPATH=$CLASSPATH:/Users/Desktop/test/gson-2.9.0.jar
-
Add the
bin
folder of your JDK to yourCLASSPATH
. This example usesopenjdk-8
.export CLASSPATH=$CLASSPATH:/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/bin
-
Verify the JDK
bin
folder andgson-2.9.0.jar
are in yourCLASSPATH
.echo $CLASSPATH
Example output
/Desktop/test/gson-2.9.0.jar:/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/bin
-
Navigate to the folder where your
Hello.java
file is stored. In this example, theHello.java
file is saved to theDesktop/test
folder.cd Desktop/test
-
Compile your
Hello.java
file into a class file.javac Hello.java
-
Compress the class file into a
.jar
file namedhello.jar
.jar cvf hello.jar Hello.class
You can create an action with your hello.jar
. Because the class file you created does not use the default name main
, you must set the --main
flag to Hello
when you create your action. The
--main
flag must match your Java class
. For more information, see Creating actions.
When you update your Java code, you must repeat these steps to recompile your code into a new .jar
file.
Packaging Java code with Gradle
Instead of compiling from the command line, you can use a build a tool such as Gradle to fetch the libraries from a repository like Maven Central. You can use Gradle to fetch and build a final .jar archive that includes your code and all dependencies.
The following example uses Gradle to build a Java action that leverages the library com.google.zxing
that provides the functionality to generate a QR code image.
-
Create a file that is named
build.gradle
and specify the dependencies.apply plugin: 'java' version = '1.0' repositories { mavenCentral() } configurations { provided compile.extendsFrom provided } dependencies { provided 'com.google.code.gson:gson:2.9.0' compile 'com.google.zxing:core:3.3.0' compile 'com.google.zxing:javase:3.3.0' } jar { dependsOn configurations.runtime from { (configurations.runtime - configurations.provided).collect { it.isDirectory() ? it : zipTree(it) } } }
-
Run the command
gradle jar
, which generates a .jar archive in the directorybuild/libs/
.
For more information, read the Gradle documentation Declaring Dependencies.
Packaging Java code by using the Java runtime with Docker
Package your code with Docker by creating a .jar
file inside the Java runtime.
Before you begin, you must have Docker installed locally.
You won´t need Java installed locally as everything is supplied by the Java runtime.
To create a Java action by using Docker, complete the following steps.
-
Create
Hello.java
fileimport com.google.gson.JsonObject; public class Hello { public static JsonObject main(JsonObject args) { String name = "stranger"; if (args.has("name")) name = args.getAsJsonPrimitive("name").getAsString(); JsonObject response = new JsonObject(); response.addProperty("greeting", "Hello, " + name + "!"); return response; } }
-
Navigate to the folder that contains the
Hello.java
file and run the Docker Java runtime container.docker run --rm -it --entrypoint "/bin/bash" -v $PWD:/tmp openwhisk/java8action:nightly
Use the
nightly
tag for the latest runtime version or a specific tag. -
Set up the container.
-
Navigate to the
/tmp
folder that contains your action code mounted from the host system.cd /tmp
-
Install
curl
to download the dependencies.apt update && apt install curl -y
-
Download the
gson
dependency.curl -L -o gson-2.9.0.jar https://repo1.maven.org/maven2/com/google/code/gson/gson/2.8.5/gson-2.9.0.jar
-
Export the path and add
gson
to it.export CLASSPATH=$CLASSPATH:$PWD/gson-2.9.0.jar export CLASSPATH=$CLASSPATH:/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/bin
-
-
Compile the code
- Compile the Java code to a class
javac Hello.java
- Package the Java class to a deployable
.jar
file.jar cvf Hello.jar Hello.class
- Compile the Java code to a class
-
Exit the runtime container.
exit
-
Create action called
hello-java
.ibmcloud fn action create hello-java Hello.jar --main Hello
-
Invoke the action
ibmcloud fn action invoke hello-java -b
Packaging Java code with Maven inside Docker
-
Create a maven Project
-
Navigate to the folder that contains the maven Project and run the Docker Java runtime container.
docker run --rm -it --entrypoint "/bin/bash" -v $PWD:/tmp openwhisk/java8action:nightly
This command run the java runtime container and map the current directory inside the container to
/tmp
Use the
nightly
tag for the latest runtime version or a specific tag. -
Set up the container.
Navigate to the
/tmp
folder that contains your action code mounted from the host system.cd /tmp
-
Install
curl
andunzip
to install Maven.apt update && apt install curl unzip -y
-
Download
Maven
curl -L -o apache-maven-3.8.5-bin.zip https://dlcdn.apache.org/maven/maven-3/3.8.5/binaries/apache-maven-3.8.5-bin.zip
-
Extract Maven
unzip apache-maven-3.8.5-bin.zip
-
Add Maven to path
export PATH=$PATH:/apache-maven-3.8.5/bin
-
Test Maven installation
mvn -v
-
Package the action code with maven
mvn package
-
Exit the runtime container.
exit
At this point, the current directory contains a
traget
directory containing the action.jar
file. -
Create action called .
ibmcloud fn action create <actionname> <Jar name>.jar --main <package name and class name seperated by dots>
The main name is something similar to
com.package.example.Hello
. -
Invoke the action
ibmcloud fn action invoke <actionname> -b