Skip to main content
We’re starting with a fresh Jupyter notebook to demonstrate common text-processing tasks using a chat-based LLM: summarization, bullets, sentiment analysis, translation, and converting plain text into structured formats. These small utilities show practical patterns for building reproducible prompts and integrating model outputs into downstream workflows.
A presentation slide with the centered title "Demo: Performing Text
Processing and Analysis" on a white background. A small "© Copyright
KodeKloud" notice appears in the
bottom-left.
Overview: tasks and examples Setup
  • Load your API key from an environment variable and define a helper that wraps the ChatCompletion API. Keep API keys out of source code and follow your organization’s secret management policies.
Store sensitive credentials (like OPENAI_API_KEY) in environment variables or a secrets manager. Avoid hard-coding keys in notebooks.
Best practices
  • Separate context (source text) from instructions (the prompt). This makes prompts reusable and easier to test.
  • Delimit large contexts (e.g., using triple backticks) so the model can clearly distinguish input data from the instruction.
  • When requesting structured outputs (JSON, XML, CSV), be explicit about the required schema to minimize parsing errors.
When embedding large context into prompts, delimit it (for example with triple backticks) so the model can clearly distinguish the source content from the instruction.
Summarization
  • Keep the source text and the instruction separate. Here’s an excerpt from Steve Jobs’ 2005 Stanford commencement address. We ask for a 500-word summary, then show how to request a bullet-point summary for scannability.
500-word summary prompt and invocation:
Bullet summary (quick, scannable):
Sample bullet-form output (example):
  • Steve Jobs delivered a commencement address at Stanford University in 2005 and shared three stories from his life.
  • First story: connecting the dots — dropping out led him to learn calligraphy, which later influenced Macintosh design.
  • Second story: love and loss — getting fired from Apple enabled him to start anew (NeXT, Pixar) and eventually return.
  • Third story: death — facing mortality focused his priorities; follow your intuition and live authentically.
  • Closing advice: “Stay Hungry. Stay Foolish.” — remain curious and brave in pursuing your work.
Sentiment analysis
  • Use the same structure: pass the text as context, then instruct the model how to label each item. This pattern is useful for generating labeled datasets for downstream model training or analysis.
Expected output example:
  1. If you sometimes like to go to the movies to have fun, Wasabi is a good place to start. Sentiment: Positive
  2. An idealistic love story that brings out the latent 15-year-old romantic in everyone. Sentiment: Positive
  3. The story loses its bite in a last-minute happy ending that’s even less plausible than the rest of the picture. Sentiment: Negative
Note: LLMs can be used to generate labeled data (for example, labeling customer reviews or social media posts) which you can then use for downstream analysis or to train supervised models. Translation (poetic translation)
  • LLMs can translate and preserve tone. Provide the poem as context and request a tone-preserving English rendering.
Sample poetic translation (example): Tomorrow, at dawn’s early light, I shall depart, for I know you await. Through forest and mountain, I’ll take flight, For I cannot bear this distance, this weight. With eyes fixed on my thoughts, I’ll tread, Unseeing of the world, deaf to its sound. Alone, unknown, stooped with a heavy head, Gloomy, for me, day will be night unbound. I’ll not gaze upon the evening’s golden hue, Nor watch distant sails descend to Harfleur. And when I arrive, a bouquet I’ll bestrew, Of green holly and blooming heather pure. And there, upon your grave, my tribute laid, I’ll feel your presence, though you’ve been away. Format conversion (plain text → JSON / XML / JSONL)
  • Convert semi-structured plain text into structured formats for ingestion into pipelines and databases. Be explicit about the desired output schema (keys, types) to reduce ambiguity.
Example model-converted outputs (examples): JSON:
XML:
JSONL (JSON Lines):
Note: JSONL (also called “newline-delimited JSON” or “NDJSON”) is not the same as JSON-LD. JSONL means each line is a separate valid JSON object — convenient for streaming and line-by-line processing. Summary
  • What we covered:
    • Summarization: fixed-length summaries and bullet-style output, emphasizing separation of context and prompt.
    • Sentiment analysis: labeling text as Positive / Negative / Neutral for downstream use.
    • Translation: preserving tone (poetic translation example).
    • Format conversion: converting semi-structured text into JSON, XML, and JSONL for pipelines.
Further reading and references Embeddings and similarity search are core techniques used for retrieval-augmented generation and semantic search — topics for a follow-up lesson.

Watch Video

Practice Lab