Guidance on collecting, managing, and designing scalable telemetry ingestion into Datadog, covering telemetry types, sources, sampling, tagging, retention, and cost control practices.
This lesson explains how telemetry is collected and sent into Datadog so you can observe what’s happening across your applications and infrastructure. It covers the primary telemetry types, where they originate, and practical guidance for designing a scalable ingestion strategy.
This subdivision helps implement controls, apply security boundaries, and determine optimal collection points.In modern systems, data sources emit a large and continuous volume of telemetry. That volume can quickly become costly and noisy if not managed intentionally.
Because telemetry volume grows rapidly, treat ingestion as a design decision rather than an afterthought. Use the following practices to keep cost and noise under control while preserving observability value.
Area
Recommended practice
Why it matters
Source selection
Enable only the integrations and agents you need
Reduces noise and ingestion costs
Sampling & filtering
Sample high-volume traces; filter or redact verbose logs
Keeps signal-to-noise ratio high and controls storage
Tagging & metadata
Apply consistent tags and service names
Enables efficient querying and reduces cardinality issues
Rate limits & quotas
Configure agent-side and collector-side limits
Prevents accidental spikes from overwhelming ingestion
Retention & tiers
Use short-term high-resolution + long-term aggregated storage
Balances cost with the ability to investigate historical issues
Practical checklist:
Start with availability, performance, and error telemetry for critical services.
Validate the business value before increasing retention or instrumenting additional sources.
Apply consistent naming conventions and tags across services.
Set log sampling rules and trace sampling rates per service.
Monitor ingestion metrics (bytes ingested, events/sec, cardinality) for anomalies.
When designing ingestion, prioritize telemetry that answers your key questions: Is the service available? Is performance within SLO? Are errors rising? Begin with essential telemetry, validate its value, then expand integrations and retention as needed.
High-cardinality tags and unfiltered verbose logs are common causes of unexpected costs and query slowness. Apply tag governance and log-reduction rules early to avoid runaway ingestion bills.
That’s it for this lesson. Apply these principles when designing your telemetry ingestion pipeline to keep your observability effective and sustainable.
