IBM Cloud Docs
Getting started tutorial

Getting started tutorial

IBM Analytics Engine Serverless instance is allocated to compute and memory resources on demand when Spark workloads are deployed. When an application is not in running state, no computing resources are allocated to the IBM Analytics Engine instance. Pricing is based on the actual amount of resources consumed by the instance, billed on a per second basis.

Getting started using serverless IBM Analytics Engine instances

The IBM Analytics Engine Standard Serverless plan for Apache Spark offers the ability to spin up IBM Analytics Engine serverless instances within seconds, customize them with library packages of your choice, and run your Spark workloads.

Currently, you can create IBM Analytics Engine serverless instances only in the US South region.

Before you begin

To start running Spark applications in IBM Analytics Engine, you need:

  • An IBM Cloud® account.
  • Instance home storage in IBM Cloud Object Storage that is referenced from the IBM Analytics Engine instance. This storage is used to store Spark History events, which are created by your applications and any custom library sets, which need to be made available to your Spark applications.
  • An IBM Analytics Engine serverless instance.

Provision an instance and create a cluster

To provision an IBM Analytics Engine instance:

  1. Get a basic understanding of the architecture and key concepts. See Serverless instance architecture and concepts.
  2. Provision a serverless instance

Run applications

To run Spark applications in a serverless IBM Analytics Engine instance:

  1. Optionally, give users access to the provisioned instance to enable collaboration. See Managing user access to share instances.
  2. Optionally, customize the instance to fit the requirements of your applications. See Customizing the instance.
  3. Submit your Spark application by using the Spark application REST API. See Running Spark batch applications.
  4. Submit your Spark application by using the Livy batch API. See Running Spark batch applications using the Livy API.

End-to-end scenario using the Analytics Engine serverless CLI

To help you get started quickly and simply with provisioning an Analytics Engine instance and submitting Spark applications, you can use the Analytics Engine serverless CLI.

For an end-to-end scenario of the steps you need to take, from creating the services that are required, to submitting and managing your Spark applications by using the Analytics Engine CLI, see Create service instances and submit applications using the CLI.