Fine Tuning Models Part 3

This article walks through preparing JSON Lines (JSONL) training data, launching a fine-tuning (customization) job with Amazon Bedrock, and invoking hosted models for inference. It includes practical boto3 examples, recommended hyperparameters, hosting options, and expected outcomes from fine-tuning.

Training data format (JSONL)

Fine-tuning data for Bedrock is typically a JSON Lines (JSONL) file where each line is a standalone JSON object. Each example usually contains a prompt and a completion that demonstrate the desired input → output behavior. Example JSONL (each line is a separate JSON object):

{"prompt": "Where is my order?", "completion": "Thank you for your question. Please provide your order number so I can help."}
{"prompt": "Can I return an item?", "completion": "Yes. Please share your order number and I can explain the return options."}
{"prompt": "My package is late", "completion": "I'm sorry about the delay. Please provide your order number so I can check the shipment status."}

You can provide a few dozen examples for basic customization or thousands for deeper domain alignment. The quality, diversity, and representativeness of examples have a direct impact on the fine-tuned model’s behavior.

Preparing S3 locations and IAM role

When using the Bedrock console or SDK, you will point the customization job at:

Training data (S3): e.g., s3://mybucket/customer-tone/train.jsonl
Output location (S3): e.g., s3://mybucket/customer-tone/output/
IAM role: a role with permissions to read the training S3 object and write job output

Example values you might supply in the console:

s3://mybucket/customer-tone/train.jsonl
s3://bucket/path-to-your-data/
s3://mybucket/customer-tone/output/
fine_tune_iam_role

The IAM role must include read access to the training S3 URI and write access to the output S3 bucket.

Creating a customization (fine-tuning) job with boto3

Use the Bedrock control-plane client (bedrock) to create the customization job. For inference, use the Bedrock Runtime client (bedrock-runtime) — see the callout below.

When creating the SDK client, use the Bedrock control-plane API (service name bedrock) for operations such as creating customization jobs. Use the Bedrock Runtime API (service name bedrock-runtime) only for inference requests.

Example: create a model customization job with boto3:

import time
import boto3

bedrock = boto3.client("bedrock", region_name="us-east-1")

role_arn = "arn:aws:iam::123456789012:role/MyCustomizationRole"
training_s3_uri = "s3://my-training-bucket/train.jsonl"
output_s3_uri = "s3://my-output-bucket/fine-tuning-output"

response = bedrock.create_model_customization_job(
    jobName="customer-support-ft-job",
    customModelName="customer-support-ft-model",
    roleArn=role_arn,
    baseModelIdentifier="arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-text-express-v1",
    hyperParameters={
        "epochCount": "1",
        "batchSize": "1",
        "learningRate": "0.0001",
        "learningRateWarmupSteps": "0"
    },
    trainingDataConfig={"s3Uri": training_s3_uri},
    outputDataConfig={"s3Uri": output_s3_uri}
)

job_arn = response["jobArn"]
print("Started job:", job_arn)

# Poll for status until the job reaches a terminal state
while True:
    status_response = bedrock.get_model_customization_job(jobIdentifier=job_arn)
    status = status_response["status"]
    print("Status:", status)

    if status in ("Completed", "Failed", "Stopped"):
        break

    time.sleep(60)

print(status_response)

Key customization parameters

Parameter	Purpose	Example / Notes
`jobName`	Friendly name for the job	`"customer-support-ft-job"`
`customModelName`	Name for your resulting custom model	`"customer-support-ft-model"`
`roleArn`	IAM role ARN used by Bedrock to access S3	`arn:aws:iam::123456789012:role/MyCustomizationRole`
`baseModelIdentifier`	ARN of the foundation model to customize	`arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-text-express-v1`
`hyperParameters`	Training dynamics	Example: `{"epochCount":"1","batchSize":"1","learningRate":"0.0001","learningRateWarmupSteps":"0"}`
`trainingDataConfig`	Where your JSONL lives	`{"s3Uri": training_s3_uri}`
`outputDataConfig`	Where to write the customized model artifacts	`{"s3Uri": output_s3_uri}`

Notes on hyperparameters:

epochCount: number of passes over the dataset.
batchSize, learningRate, learningRateWarmupSteps: tune to control convergence and stability.
The best values depend on dataset size, base model, and desired trade-offs (quality vs. cost/time).

Hosting options and ARNs

After successful customization, your model artifacts appear in your account under custom models (not the public/shared catalog). You can host the model in two primary ways:

Serverless (on-demand): call the custom model ARN directly for inference. Ideal for variable, low-volume usage.
Provisioned throughput: deploy a provisioned model contract and use the provisioned model ARN when invoking inference. This provides predictable capacity for production workloads. Note: you are billed for provisioned capacity whether or not it is used.

Table: hosting options

Hosting Mode	How to invoke	When to use
Serverless (on-demand)	Use the custom model ARN with Bedrock Runtime	Low/variable throughput; pay-per-use
Provisioned throughput	Use the provisioned model ARN with Bedrock Runtime	Production workloads requiring predictable latency and throughput

After hosting, use the Bedrock Runtime API for inference requests (e.g., invoke_model).

Invoking a hosted (provisioned) model with boto3

Example using the Bedrock Runtime client. Replace provisioned_model_arn with your provisioned model ARN (or use the custom model ARN for on-demand).

import json
import boto3

bedrock_runtime = boto3.client("bedrock-runtime", region_name="us-east-1")

# Example provisioned model ARN (replace with your actual provisioned model ARN)
provisioned_model_arn = "arn:aws:bedrock:us-east-1:123456789012:provisioned-model/my-provisioned-model"

body = json.dumps({
    "inputText": "Where is my order?",
    "textGenerationConfig": {
        "temperature": 0.2
    }
})

response = bedrock_runtime.invoke_model(
    modelId=provisioned_model_arn,
    body=body
)

result = json.loads(response["body"].read())
print(result)

When invoking:

Use lower temperature for more deterministic responses.
Optionally include generation constraints (max tokens, stop sequences) in the textGenerationConfig depending on the runtime schema.

What to expect from fine-tuning

Benefits of fine-tuning include:

More consistent outputs: the model learns the mapping you provided in training examples.
Reduced prompt complexity: fewer prompt-engineering tricks are often needed.
Better alignment with business needs: top-level behavior and style can be internalized.
Improved user experience: outputs more closely match real-world expectations.

A presentation slide titled "Results" with four numbered panels, each containing a circular icon. The panels read: "More consistent outputs," "Reduced prompt complexity," "Better alignment with business needs," and "Improved user experience."

To summarize succinctly: fine-tuning teaches the model; prompting only guides it. Fine-tuning updates (a subset of) model weights so the behavior demonstrated in your training examples becomes internalized. High-quality training data yields better outcomes.

A presentation slide titled "Key Takeaway" showing the point: "Fine-tuning teaches the model, prompting only guides it." The slide has a dark blue left panel and a light gray area with a blue "01" marker.

Invest time in curating training examples that are representative, diverse, and labeled with the exact tone and behaviors you want the model to reproduce.

Next steps: multimodal models (text + images)

The next lesson explores multimodal capabilities: combining text prompts with image inputs, constructing multimodal examples, and running inference across text + image modalities.

We will include practical code snippets for handling image inputs, tips for multimodal JSONL formatting, and examples of combining text and visual context to improve downstream application behavior.

Links and references

Bedrock boto3 client (control plane): https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock.html
Bedrock Runtime (inference): https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock-runtime.html
Amazon Bedrock documentation: https://docs.aws.amazon.com/bedrock/
Best practices for fine-tuning and prompt design: AWS developer guides and model vendor docs

Tip: Start with a small dataset and conservative hyperparameters to validate workflows and end-to-end permissions before scaling to larger training sets and longer epoch counts.

Introduction to Gen AI and LLMs

Introduction to Amazon Bedrock

Getting Started With Amazon Bedrock

Foundation Models in Amazon Bedrock

Basic Concepts of Prompt Engineering

Introduction to Text Generation With Amazon Bedrock

How Foundation Models Process Input

Extract Insights Using Prompt Engineering

Working With Different Task Types

Integrating Amazon Bedrock With Other AWS Services

Build a Marketing Email Generation Application

Introduction to Retrieval Augmentation With Bedrock Knowledge Bases

Best Practices and Optimization

Governance, Safety in Responsible AI

Security

Monitoring and Logging

Taking action with Bedrock Agents

Advanced Topics (Optional)

Final Project

Wrap Up

Fine Tuning Models Part 3

Training data format (JSONL)

Preparing S3 locations and IAM role

Creating a customization (fine-tuning) job with boto3

Key customization parameters

Hosting options and ARNs

Invoking a hosted (provisioned) model with boto3

What to expect from fine-tuning

Next steps: multimodal models (text + images)

Links and references

Watch Video

​Training data format (JSONL)

​Preparing S3 locations and IAM role

​Creating a customization (fine-tuning) job with boto3

​Key customization parameters

​Hosting options and ARNs

​Invoking a hosted (provisioned) model with boto3

​What to expect from fine-tuning

​Next steps: multimodal models (text + images)

​Links and references

Watch Video

Training data format (JSONL)

Preparing S3 locations and IAM role

Creating a customization (fine-tuning) job with boto3

Key customization parameters

Hosting options and ARNs

Invoking a hosted (provisioned) model with boto3

What to expect from fine-tuning

Next steps: multimodal models (text + images)

Links and references