What is Splunk

Splunk is a machine data analytics platform used to collect, index, search, and visualize large volumes of log data.

Organizations use Splunk to monitor:

• Applications
• Servers
• Security events
• Network devices
• IoT systems

Key Functions

1. Collect Data
2. Index Data
3. Search Data
4. Analyze Data
5. Visualize Data

2. Splunk Products

1. Splunk Enterprise

• On-premise platform
• Full features

2. Splunk Cloud

• SaaS version
• Managed by Splunk

3. Splunk Light

• Small scale log management

4. Splunk Enterprise Security (ES)

• Security monitoring platform

5. Splunk ITSI

• IT service monitoring

3. Splunk Architecture

Core Components

Forwarder

Collects logs and sends them to Splunk.

Types:

• Universal Forwarder
• Heavy Forwarder

Indexer

Stores indexed data and processes searches.

Search Head

Runs search queries and generates dashboards.

Deployment Server

Manages forwarders.

License Master

Controls license usage.

4. Splunk Installation

Requirements

• Linux / Windows
• Minimum 8GB RAM recommended

Installation Steps

1. Download Splunk
2. Install package
3. Start Splunk

Command:

Web Interface

5. Data Ingestion

Splunk collects data from multiple sources.

Types of Inputs

Example:

6. Source Types

Source types define format of data.

Examples:

• syslog
• apache_access
• mysql_error
• log4j

Purpose:

• Proper timestamp detection
• Field extraction
• Event parsing

7. Splunk Indexes

Index is where Splunk stores data.

Default indexes:

Example index creation:

8. Search Processing Language (SPL)

SPL is used to search data.

Basic syntax:

Example:

9. SPL Commands (Important)

Filtering

Stats Command

Chart Command

Timechart

Top Command

Sort

10. Fields in Splunk

Fields are extracted from events.

Examples:

• host
• source
• sourcetype
• timestamp

Field Extraction

Example:

11. Knowledge Objects

Knowledge objects enrich data.

Types:

• Fields
• Tags
• Event Types
• Lookups
• Macros
• Workflows
• Reports
• Dashboards

12. Lookups

Lookups add extra data.

Example:

Example SPL:

13. Reports

Reports are saved searches.

Example:

Daily error report.

Features:

• Scheduled execution
• Shareable
• Exportable

14. Dashboards

Dashboards display visual data.

Elements:

• Charts
• Tables
• Maps
• Metrics

Example panel query:

 

index=web_logs | stats count by status

 

---

15. Alerts

Alerts trigger actions based on conditions.

Example:

 

error count > 50

 

Alert actions:

• Email
• Webhook
• Script

---

16. Data Models

Data models organize data for analysis.

Used mainly for:

• Security analytics
• Accelerated searches

Components:

• Datasets
• Constraints
• Fields

---

17. Pivot

Pivot allows drag-and-drop analysis.

Benefits:

• No SPL required
• Fast reporting

---

18. Splunk Apps

Apps extend Splunk functionality.

Examples:

• Splunk Enterprise Security
• Splunk IT Service Intelligence
• Splunk DB Connect
• AWS monitoring app

---

19. Splunk Configuration Files (Important)

Splunk uses configuration files.

Key files:

 

inputs.conf
outputs.conf
props.conf
transforms.conf
indexes.conf
limits.conf
server.conf
authentication.conf

 

Example:

 

inputs.conf

 

 

[monitor:///var/log/messages]
index=syslog
sourcetype=syslog

 

---

20. Parsing Pipeline

Data processing stages:

 

Input
↓
Parsing
↓
Indexing
↓
Search

 

Parsing steps:

1. Line breaking
2. Timestamp detection
3. Event creation
4. Field extraction

 

21. Indexer Clustering (Advanced)

Purpose:

• High availability
• Data redundancy

Components:

• Cluster Master
• Peer Nodes

Key parameters:

 

Replication Factor (RF)
Search Factor (SF)

 

Example:

 

RF = 3
SF = 2

 

---

22. Search Head Clustering

Multiple search heads working together.

Components:

• Captain node
• Member nodes
• Deployer

Benefits:

• Load balancing
• High availability

---

23. Deployment Server

Manages configuration of forwarders.

Example tasks:

• Push apps
• Update configurations
• Manage thousands of servers

---

24. Splunk Security

Security features:

• Role based access
• Authentication
• SSL encryption
• Audit logs

Roles example:

• admin
• power user
• user

---

25. Performance Optimization

Important for large environments.

Techniques:

• Limit search time
• Use indexes
• Use summary indexing
• Use accelerated data models

---

26. Monitoring Splunk

Monitoring tools:

 

index=_internal

 

Example query:

 

index=_internal source=*metrics.log

 

Used for:

• performance monitoring
• troubleshooting

---

27. Splunk Troubleshooting

Common issues:

Forwarder not sending logs

Check:

 

splunk list forward-server

 

Index full

Check:

 

df -h

 

License exceeded

Check:

 

license usage

 

---

28. Summary Indexing

Stores summarized data for faster searches.

Example:

Daily traffic summary.

Benefits:

• Faster dashboards
• Reduced load

---

29. HTTP Event Collector (HEC)

Allows sending logs via HTTP.

Example:

 

POST /services/collector

 

Used for:

• cloud applications
• microservices
• APIs

---

30. Real Enterprise Use Cases

Splunk is used for:

Security Monitoring

Detect cyber attacks.

IT Operations

Monitor servers and apps.

Business Analytics

Customer behavior analysis.

DevOps

Application monitoring.