Getting Started With Information Extraction in Azure

In this lesson you will learn how Azure can convert unstructured content — documents, audio, and video — into useful structured information. We focus on Azure Content Understanding in Foundry and explain how it complements other document-analysis services. The goal is to automate extraction so applications can act on data (e.g., storing invoice totals, creating cases from calls, or indexing video transcripts) instead of relying on manual review. Key concepts covered:

Document analysis with pre-built and custom analyzers
Structured JSON outputs and why they matter
Extending extraction to audio and video
Using Content Understanding in Foundry to test and iterate

Document processing: pre-built vs custom analyzers

Organizations often receive large volumes of semi-structured documents (invoices, receipts, IDs, business cards). Storing a file is not the same as extracting the fields you need for automation (invoice number, date, total, customer ID). Pre-built analyzers

Ready to use out of the box for common scenarios (invoices, receipts, business cards, identity documents).
No training required — provide the document and get structured results.

Custom analyzers

Use when documents are specific to your business (special forms, domain-specific fields).
Require a schema (field names/types) and sample documents that show where fields appear.
The analyzer is trained on the schema and examples to extract your business-specific fields.

Typical custom analyzer workflow:

Define a schema describing the fields and expected data types.
Provide representative sample documents.
Create and train the custom analyzer.
Run new documents through the trained analyzer to extract structured fields.

Analyzer Type	Best for	Requirements	Example use case
Pre-built	Common documents and media types	None (ready-to-use)	Generic invoices, receipts, business cards
Custom	Business-specific forms and domain fields	Schema + sample documents	School registration forms, custom insurance claim forms

A presentation slide titled "Extracting Information From Documents" showing a schema table (policyNo, issued, premium), sample document icons feeding into a trained-analyzer icon. To the right it explains creating custom analyzers by defining a schema and providing sample documents, with a button labeled "Your fields, your forms."

Example: a school with a custom student registration form can train an analyzer to extract StudentID, CourseName, EnrollmentDate, and TuitionAmount exactly as defined in the schema.

Structured output — why it matters

Analyzers return structured outputs (typically JSON). Structured data enables automation: store values in a database, update records, trigger workflows, or route items for human review. Example extracted JSON (insurance policy):

{
  "policyNo": "KK-4192",
  "issued": "2026-01-01",
  "premium": 249.00
}

Benefits of structured outputs:

Machine-readable results for immediate integration
Confidence scores to guide automation vs manual review
Time-aligned transcripts and timestamps for audio/video

Note: Azure Content Understanding (Foundry) and focused document-analysis services are complementary:

Some services specialize in deep document layout, table, and form extraction.
Content Understanding in Foundry provides analyzers across content types (documents, audio, video) and enables a pattern-based approach for structured extraction.

Document analysis in practice

When analyzing documents (for example an invoice), analyzers typically:

Detect layout and form elements (company logo, invoice table, header/footer)
Extract fields (InvoiceId, InvoiceDate, CustomerName, LineItems)
Return confidence scores for each field to help you decide whether to auto-process or send for human review

Operational choices based on confidence:

High confidence → automatic downstream processing
Low confidence → queue for manual verification

A screenshot of a document-extraction interface titled "Extracting Information From Documents" showing a scanned invoice being analyzed. The left side shows the invoice image with highlighted fields and the right side lists extracted fields (CustomerName, InvoiceDate, CustomerId, PurchaseOrder, InvoiceId) and confidence scores.

Extending extraction to audio and video

Valuable information often lives in audio recordings and video (customer calls, meetings, interviews, training). Content Understanding in Foundry supports pre-built and custom analyzers for audio/video and returns structured outputs such as transcripts, summaries, entities, topics, and insights. Input flow: audio/video → analyzer (pre-built or custom) → structured output (transcripts, summaries, entities, topics, insights)

A slide titled "Extracting Information From Audio and Video" showing audio and video inputs feeding into a Content Understanding analyzer. The analyzer uses prebuilt or custom analyzers to produce structured results.

Typical structured outputs from audio/video analyzers:

Output type	Description
Transcripts	Time-aligned text of spoken words (with timestamps)
Summaries	Condensed representation of the main points
Entities	People, organizations, amounts, locations, etc.
Topics	Primary subjects discussed in the media
Insights	Higher-level patterns useful for automation and reporting

A slide titled "Extracting Information From Audio and Video" showing five labeled cards — Transcripts, Summaries, Entities, Topics, and Insights — each with a blue circular icon and a short description of the output. The slide outlines the structured results an analyzer returns from audio/video content.

Azure also offers Video Indexer (https://www.videoindexer.ai/) for in-depth video insights. Video Indexer overlaps with Content Understanding but is specifically tailored for video-rich analysis.

Example: audio analysis workflow

A recorded customer-service call can be processed to produce:

A time-aligned transcript (WEBVTT or similar)
An automatic summary of the interaction
Extracted entities (customers, companies)
Topics and metadata for categorization or SLA tracking

Sample WEBVTT transcript excerpt:

WEBVTT

00:00.080 --> 00:03.520
<v Agent 1>Good afternoon, thank you for calling Contoso, how may I help?

00:04.400 --> 00:06.880
<v Customer 1>I am looking to transfer money to my checking account.

00:07.760 --> 00:12.560
<v Customer 1>I have a $1,000 certificate of deposit that matured last month, and I have to use it.

A compact summary example:

Customer wants to transfer funds from a matured certificate of deposit to a checking account; agent confirms the transfer can be processed.

Structured outputs may include arrays, for example: companies: ["Contoso"] and people: ["Mary Smith"]. These make it easy to create cases, flag follow-ups, or trigger downstream automation.

Exploring Content Understanding in Foundry

Foundry’s portal and playground let you experiment with analyzers and inspect returned JSON (text, layout, tables, entities). Common tasks in the Foundry portal:

Browse deployments and Services to find analyzers (Document Layout Analyzer, Invoice Analyzer, Call Center Analyzer, Tax US Analyzer, etc.).
Upload sample files and run the analyzer in the playground to preview results (OCR, layout, tables).
Inspect returned JSON to integrate text, table structures, and entities into databases and workflows.

Example analyzers you may encounter:

Document Layout Analyzer: layout, paragraphs, spans
Invoice Analyzer: invoice fields and line items
Call Center Analyzer: transcripts, speakers, entities
Identity Analyzer: driver’s license, passport field extraction

Table extraction returns structured objects you can iterate and save. Example of table cell objects returned by an analyzer:

[
  {
    "kind": "content",
    "rowIndex": 0,
    "columnIndex": 0,
    "content": "Document",
    "span": { "offset": 391, "length": 8 },
    "elements": ["/paragraphs/2"]
  },
  {
    "kind": "content",
    "rowIndex": 0,
    "columnIndex": 1,
    "content": "Share with",
    "span": { "offset": 409, "length": 10 },
    "elements": ["/paragraphs/3"]
  },
  {
    "kind": "content",
    "rowIndex": 1,
    "columnIndex": 0,
    "content": "Add name(s)",
    "span": { "offset": 420, "length": 11 },
    "elements": ["/paragraphs/4"]
  }
]

A screenshot of a web-based "Content Understanding" OCR/document playground showing a resume preview in the center and the extracted text output on the right. The left sidebar shows sample files and controls to upload and run analysis.

You can also run identity-document analyzers (OCR + field extraction). Results include fields, spans, and confidence scores. Low confidence often indicates the analyzer wasn’t trained on that exact document layout or the image quality is poor.

A screenshot of an Azure/Microsoft Foundry "Content Understanding" tool displaying a scanned California driver's license. The main view shows the license image with highlighted fields and the right panel lists parsed document data and confidence scores.

Choose a pre-built analyzer when your documents or media match a common scenario. Choose a custom analyzer when you need business-specific fields or domain-specific extraction.

Summary and recommendations

Use pre-built analyzers for common document and media types — no training required and fast to deploy.
Use custom analyzers when your organization needs to extract business-specific fields; supply a schema and representative examples to train an effective model.
Content Understanding in Foundry unifies analyzers across documents, audio, and video — use it for cross-media extraction while leveraging specialized services (e.g., Video Indexer) for deeper video-centric analysis.
Structured outputs (JSON, transcripts, summaries, entities, topics, insights) enable automation, reporting, and human-in-the-loop workflows.

This lesson covered the end-to-end journey: unstructured content in → analyzer (pre-built or custom) → structured information out — ready for storage, search, or downstream automation.

Links and references

Azure Content Understanding (Foundry) documentation — search your internal Foundry docs or portal
Video Indexer — https://www.videoindexer.ai/
Kubernetes Documentation (example link for general orchestration resources)

AI in Azure

Getting Started With AI

Getting Started With Generative AI and Agents

Getting Started With Text Analysis

Getting Started With AI Speech

Getting Started With Computer Vision