What is InstructLab?

InstructLab is a private, secure generative AI solution powered by Red Hat Enterprise Linux AI, available on IBM Cloud. It allows users to retain ownership of their data and models, leverage unique business data for innovation, and minimize the risk of catastrophic forgetting.

Why should I use InstructLab for my generative AI solution?

InstructLab offers several benefits for your generative AI solution. First, it allows you to retain ownership of both the data and the model, giving you control over how your data is used and how your model performs. Second, it enables you to leverage unique business data to unlock efficiencies and drive innovation by creating AI-powered solutions. Third, it minimizes the risk of catastrophic forgetting by using built-in Granite models as a foundation for learning new skills and knowledge. Fourth, it is available as a service on IBM Cloud, allowing you to reduce unnecessary costs by paying just for what you need and optimize IT expenditures by delivering simpler, faster, and more economical models.

What are the benefits of InstructLab on IBM Cloud?

InstructLab on IBM Cloud offers several benefits, including:

Data ownership

Users retain ownership of both the data and the model, allowing them to control their data and model.

Leveraging unique business data

Users can unlock efficiencies and drive innovation by creating AI-powered solutions using their unique business data.

Minimizing the risk of catastrophic forgetting

InstructLab uses Granite models as a foundation for learning new skills and knowledge, minimizing the risk of losing previously learned information when learning new information.

Secure, up-to-date, and available

InstructLab is available as a service on IBM Cloud, allowing users to reduce unnecessary costs and optimize IT expenditures.

Data portability

Users can export their content and configuration to other infrastructures.

Enterprise-grade cloud infrastructure

InstructLab uses IBM Cloud's robust and secure infrastructure, designed to meet the stringent requirements of business critical workloads.

Flexibility

InstructLab offers access to a wide variety of hardware profiles, VMware compute accelerators, and the capability for new capacity expansion within the hour.

Advanced Cloud Services

IBM Cloud provides access to the latest GPUs and IBM watsonx services gen AI, inferencing, and machine learning to fast-track innovation into business processes.

What are Granite models?

Fit for purpose and open source, these enterprise-ready, multimodal models deliver exceptional performance against safety benchmarks and across a wide range of enterprise tasks from cybersecurity to RAG.

Which Granite model does InstructLab use?

InstructLab uses the granite-3.1-8b-starter-v1 model.

How does billing work?

Costs are incurred by the usage of both Red Hat AI InstructLab and the IBM Cloud® Object Storage service, which is used as a storage location.

If you choose to deploy the model on another service, additional charges can come from that service as well.

How is cost calculated in Red Hat AI InstructLab?

The cost from Red Hat AI InstructLab usage is based on two metrics that are measured in tokens. Each token corresponds to a specific amount of computational power that is required for the processing tasks. The total number of tokens consumed directly influences the scale of data generation or model fine-tuning. This metric serves as a basis for our billing system, enabling users to monitor and control their costs according to the computational resources utilized. The tokens that are processed for Synthetic Data Generation (SDG) and Model Alignment are billed separately.

Synthetic Data Generation (SDG)

Output tokens (SYN-DATA-TOKEN) are calculated by the volume of generated data produced by the service from the entire input taxonomy. The text is tokenized by using Hugging Face's tokenizer library with the tokenization information for the Mistral teacher model.

Model alignment training

Input tokens (MODEL-TRAIN-TOKEN) are calculated based on the amount of data fed that into the system for model alignment training, as well as the Granite base knowledge that is used to increase accuracy without knowledge loss. Because of the foundational knowledge that is used, there is a minimum cost.

How do I find and track cost information as I go?

1. Before you begin running anything in Red Hat AI InstructLab, you can use the cost estimator to get an estimate of what the cost might be.

2. When you upload your taxonomy, look for the cost estimate based on synthetic data tokens. Then run the data generation job.

3. After the synthetic data generation completes, but before you begin the training, look for the estimated amount of tokens for the training costs. Then, begin training.

Are failed operations billed?

Failed operations are not billed. Successful operations and user canceled operations are billed, though user canceled operations are prorated based on the processing that completed.

What is data generation?

Data generation is the process of generating questions and answers based on the questions and answers that you included in the QNA files.

What is model training?

Training is the process of learning the questions and answers. The training begins with knowledge and foundational skills, then moves on to compositional skills.

How long does it take to run?

Data generation and model training both take significant time to complete. You can find general estimates in the console when you start the processes.

Factors that impact completion time:

• The contents of the knowledge documents
• The number of other jobs in the queue

How long does data generation take?

For data generation, the general formula is to take the number of tokens, divided by about 5380.5 tokens per second, divided by 60 seconds per minute, and divided by 60 minutes in an hour.

Tokens / 5380.5 / 60 / 60 = Number of hours

How long does model training take?

For model training, the general formula is to take the number of tokens, divided by about 4115 tokens per second, divided by 60 seconds per minute, and divided by 60 minutes in an hour.

Tokens / 4115 / 60 / 60 = Number of hours