Skip to main content
Welcome back — this is Michael Foster. In this lesson/article we’ll cover practical tools for identifying responsible AI features: how to assess whether a model behaves responsibly and what to look for when evaluating fairness, explainability, and trustworthiness. We’ll focus on two core AWS services used for these tasks and explain where each fits into a responsible AI workflow. Responsible AI depends on three interrelated pillars: bias detection, transparency (explainability), and runtime trust controls. Below we outline how Amazon SageMaker Clarify and Amazon Bedrock Guardrails address these needs.
A presentation slide titled "Responsible AI with AWS — Introduction" summarizing that responsible AI practices ensure models are fair, explainable, and trustworthy, and that bias, trustworthiness, and transparency are critical evaluation factors. Icons of people and a balance scale accompany the text.
Overview: two complementary AWS services
  • Amazon SageMaker Clarify — for bias detection and explainability across data prep, training, and deployment/monitoring.
  • Amazon Bedrock Guardrails — for runtime enforcement to limit, filter, or flag model outputs in production.

​
SageMaker Clarify

Amazon SageMaker Clarify helps you identify potential fairness issues and produce explainability artifacts that show which features influenced model decisions. Clarify supports analysis at multiple stages: before training (data preparation), after training (model evaluation), and during deployment (monitoring).
Slide titled "Amazon SageMaker Clarify — Introduction" explaining that SageMaker Clarify is AWS's tool for bias detection and model explainability. It notes the tool supports bias detection at different stages: data preparation, post-training, and deployment.
Bias detection during data preparation
  • Clarify analyzes dataset balance and disparity across sensitive attributes (e.g., age groups, gender, ethnicity).
  • Early detection helps you rebalance data, collect more samples, or apply preprocessing to reduce downstream bias.
Example: A loan dataset skewed toward middle-aged applicants may produce models that underperform for younger or older applicants; Clarify highlights these imbalances so you can act before training.
A dark presentation slide titled "SageMaker Clarify — Bias Detection During Data Preparation." It features two highlighted boxes reading "Detect potential biases in your dataset before training" and "Analyze dataset balance and identify disparities."
Bias detection after model training
  • Clarify computes bias metrics using model predictions to surface performance differences across demographic groups.
  • Common metrics include demographic disparity, recall difference (true positive rate differences), and accuracy difference.
Use these metrics to detect disparate impact — for example, if a model disproportionately rejects applications for a specific gender or ethnicity.
A dark-themed slide titled "Bias Detection After Model Training" showing two rounded boxes reading "Analyze model predictions for bias" and "Use bias metrics to identify performance differences between groups." The design uses teal outlines and text on a navy background with a small © KodeKloud note.
Explainability and feature attributions
  • Clarify generates explainability outputs (treating the model as a black box) that show which input features most influenced predictions.
  • Methods like SHAP are commonly used for feature attributions, providing local and global explanations to support transparency, debugging, and audits.
Explaining decisions is essential for regulatory compliance and user trust. For example, being able to show why a loan application was denied helps meet transparency requirements.
A presentation slide titled "Explaining Model Decisions With SageMaker Clarify" showing two text boxes: one stating that model explainability helps understand how a model reaches its decisions, and the other explaining that SageMaker Clarify treats the model as a black box and analyzes inputs and outputs.
Processing jobs and S3 outputs
  • Clarify runs as SageMaker processing jobs: it reads input datasets and model predictions (often from Amazon S3), runs analyses, and writes metrics, feature attributions, and visual reports back to S3.
  • These artifacts can be downloaded, integrated into monitoring dashboards, or used as evidence for audits.
Clarify analysis runs as SageMaker processing jobs. Results (bias metrics, feature attributions, and visual reports) are saved to S3, where you can review or integrate them into downstream workflows.
Infographic titled "SageMaker Clarify Processing Jobs" with two numbered panels. Panel 01 (gear icon) says Clarify uses processing jobs to analyze data and model outputs; Panel 02 (bucket icon) says results are stored in an S3 bucket and include bias metrics and feature attributions.
Common Clarify metrics and artifacts
  • Demographic disparity (differences in outcome rates between groups)
  • Recall difference (difference in true positive rates across groups)
  • Accuracy difference (variation in overall accuracy by group)
  • Feature attributions showing which features influenced predictions
These outputs guide remediation steps: data rebalancing, changes to loss/objectives, fairness-aware algorithms, or additional monitoring.
A presentation slide titled "Metrics Used by SageMaker Clarify." It notes bias metrics (demographic disparity, recall difference, accuracy difference) and that feature attribution shows which features influenced predictions.

​
Amazon Bedrock Guardrails

Amazon Bedrock provides runtime guardrails for generative models, enabling you to restrict, filter, or flag model outputs so applications behave safely and ethically in production. Guardrails operate at the output layer to reduce the risk of harmful or non-compliant responses.
  • Common guardrail actions: filter/redact disallowed content, block or flag biased/discriminatory responses, sanitize sensitive data, and enforce response templates or constraints.
  • Guardrails are particularly useful for consumer-facing chatbots, content generation, and regulated domains (healthcare, finance, legal).
A presentation slide titled "Using Amazon Bedrock for Guardrails" showing two numbered panels: one with a bar-chart icon about demographic disparity and unequal treatment, and the other with a hand-and-dollar icon giving an example of age-based differences in loan approvals.
When to use Bedrock Guardrails
  • Use guardrails when you need production-time controls to limit model outputs—especially in sensitive or regulated applications.
  • Guardrails complement detection: Clarify discovers and measures problems; guardrails mitigate their effects in real time.
Examples:
  • Filter or redact outputs that contain disallowed keywords/topics
  • Block or flag responses that appear biased or discriminatory
  • Apply templates or constraints so outputs meet compliance and consistency requirements
A presentation slide titled "Responsible AI With AWS – Introduction." It says Amazon Bedrock provides tools to implement responsible AI guardrails and that guardrails ensure AI applications are safe and ethical, shown alongside wrench and shield icons.
Comparison: Clarify vs Bedrock Guardrails Limitations and best practices
  • Clarify identifies and quantifies potential bias and provides explainability, but it does not automatically fix bias — remediation (data collection, reweighting, model adjustments) is required.
  • Guardrails reduce harm at runtime, but they are not a substitute for upstream fairness work; combined use yields stronger responsible AI outcomes.
Guardrails mitigate harmful outputs at runtime but cannot replace upstream fixes. Use Clarify for measurement and remediation, then apply guardrails to enforce policies in production.
Recommended workflow
  1. Data preparation: run Clarify to detect dataset imbalances and address them (collect more data, resample, transform features).
  2. Model evaluation: run Clarify again on model predictions to compute bias metrics and generate feature attributions.
  3. Remediation: apply model or data-level fixes, retrain, and re-evaluate with Clarify.
  4. Production: deploy guardrails (Bedrock or custom middleware) to filter, flag, or constrain outputs; continue monitoring with Clarify for drift and emerging bias.
Resources and references

​
Summary

  • Amazon SageMaker Clarify: Detect bias during data preparation, after training, and during deployment; produce explainability artifacts (feature attributions and visual reports); runs as SageMaker processing jobs and writes results to S3.
  • Amazon Bedrock Guardrails: Implement runtime constraints, filtering, and flags to limit unsafe, biased, or non-compliant outputs.
Together, these services form a complementary responsible AI approach: Clarify provides measurement and transparency; Bedrock Guardrails enforce operational controls to keep model behavior within desired boundaries.

Watch Video