How to Create an Alert in Splunk
Learn how to save a Splunk search as an alert, configure thresholds, schedules, throttling, permissions, actions, and verify triggered alerts.
A Splunk alert is a saved search configured to evaluate a condition and initiate one or more actions when that condition is met. Unlike manually running a search, an alert evaluates the search according to its execution settings and can respond without someone repeatedly watching the Search page.
You can create an alert from a search run in Splunk Web, the browser-based interface for searching data and configuring Splunk objects. This lesson uses a result-count threshold: the alert fires when a search returns more than 100 results during its evaluation time range.
How Splunk alerts work
A saved search is a stored search definition that can be reused. When you save a search as an alert, Splunk retains the search and adds alert settings such as an execution schedule, trigger condition, throttle interval, permissions, and triggered actions.
The basic sequence is:
- Splunk evaluates the saved search.
- The search returns a result set for its selected time range.
- Splunk checks the configured trigger condition against that result set.
- If the condition is satisfied, Splunk runs the selected triggered actions.
For example, a search that returns 135 records in its evaluation window satisfies a condition of “result count is greater than 100.” A search returning 80 records does not.
Prepare the search to monitor
First, run or prepare the search that identifies the condition you want to monitor. The search should return the records or summary needed by the alert condition. For a result-count alert, the number of results returned by the search is the value being tested.
The search's time range is important. A time range is the period of data examined by the search. The same search can return 20 results over one hour and more than 100 results over a much longer period. The alert evaluates the result set produced for its configured time range, not an unrelated time range you happened to use during testing.
For the central example, configure an evaluation window of 300 days and a trigger condition that fires when the result count is greater than 100. Before saving the alert, run the search with the same 300-day range and check whether the result count exceeds the threshold.
Evaluation window: 300 days
Trigger rule: result count greater than 100
Create an alert from Splunk Web
- Open the search in Splunk Web and run it, or confirm that the prepared search returns the expected results.
- In the Search page, open Save As.
- Select Alert. This opens the alert configuration dialog.
- Complete the alert identity, execution, trigger, throttling, permission, and action settings.
- Save the alert.
The saved alert uses the search you started with. Review the search text and time range before saving so the alert monitors the intended data.
Configure alert identity and visibility
Title and description
Give the alert a title that states what it detects. For example, High search volume over 300 days is more useful than a name such as Alert 1.
Add a description that explains the monitored condition and expected response. A useful description could state that the alert fires when the search returns more than 100 records in the previous 300-day evaluation window and that the owner should investigate the high volume.
Permissions
Permissions determine who can view, manage, and use the alert. Keep the alert private while testing if other users should not see or modify it. When it is ready for team use, share it with users of the current app according to your organization's access model.
App sharing affects the alert's visibility and management context. An alert shared within an app is available to the users who have appropriate access to that app; a private alert remains restricted to its owner or permitted users. Verify the intended scope before saving.
| Configuration field | What it controls | Example choice |
|---|---|---|
| Title | The alert's identifying name | High search volume over 300 days |
| Description | What the alert detects and what users should do | Investigate when more than 100 records are returned |
| Permissions | Whether the alert is private or available in the current app | Private during testing; app-shared when approved |
| Alert type | When Splunk evaluates the search | Scheduled or real-time |
| Trigger condition | The rule that determines whether the alert fires | Result count greater than 100 |
| Trigger frequency | Whether actions run once or for each matching result | Once for the result set |
| Throttle interval | How long repeated triggers are suppressed | An interval appropriate for the response process |
| Triggered actions | What Splunk does after the condition is met | Record an entry on Triggered Alerts |
Choose the alert execution type
The alert type controls when Splunk evaluates the saved search and when actions can occur.
| Alert type | Evaluation timing | Best use case | Operational consideration |
|---|---|---|---|
| Scheduled alert | At defined intervals | Periodic reporting, trend checks, and conditions that do not require immediate reaction | Uses scheduled search resources and may detect a condition after the next run |
| Real-time alert | Continuously or near continuously as relevant data arrives | Conditions requiring rapid response to incoming events | Can generate frequent activity and requires careful trigger and throttle settings |
Choose a scheduled alert when periodic evaluation is sufficient. Choose a real-time alert when the response must occur as relevant data arrives. The choice should match the urgency of the condition and the amount of alert activity your environment can handle.
Set the trigger condition
A trigger condition is the rule that decides whether an alert fires. One common condition tests the result count, which is the number of records returned by the search during the evaluation time range.
For the example alert, configure the condition as:
Result count > 100
If the 300-day search returns 101 or more records, the condition is true and Splunk can execute the selected actions. If it returns 100 or fewer records, the condition is false and the actions do not run.
The time range, result set, and threshold work together:
- The time range determines which data the search examines.
- The result set is the data returned by that search for that time range.
- The threshold is the comparison value applied to the result count.
Changing any of these can change whether the alert fires. A 300-day window may exceed 100 results even when a one-day window does not.
Choose trigger frequency
Trigger frequency determines whether Splunk responds once to the search result set or separately to individual matching results.
| Trigger mode | Behavior | Appropriate scenario | Risk of notification volume |
|---|---|---|---|
| Once for the search result set | Runs the action once when the aggregate condition is met | The total number of matches is the concern | Usually lower; produces one aggregate notification per eligible firing |
| For each result | Can run alert actions separately for every matching result | Each individual event requires its own response | Potentially high; many matching events can create many actions |
For the high result-count example, choose a single trigger for the result set. The concern is that the total count exceeds 100, not that every individual record needs a separate notification.
Use per-result triggering when each matching event must be handled independently. For example, an event-by-event workflow may need a separate action for each matching record. Throttling becomes especially important in this mode because a burst of matching events can produce repeated actions.
Configure throttling
A throttle is a suppression period that limits repeated alert triggers after an alert has fired. Configure a throttle interval when the same condition could remain true across multiple scheduled runs or when many events could satisfy a real-time condition.
After the alert fires, subsequent trigger attempts during the throttle interval are suppressed according to the alert's throttling configuration. This reduces duplicate notifications and action noise; it does not change the underlying search results or permanently disable the alert.
Choose the interval based on how often the responsible team needs another notification. A short interval allows quicker reminders but may create noise. A longer interval reduces repetition but can delay awareness of a continuing or newly important condition.
Select triggered actions
A triggered action is the response performed when the alert condition is satisfied. Select one or more actions in the alert configuration dialog. The selected action runs only after the trigger condition evaluates as true and is not blocked by applicable throttling.
For this example, select the action that records the firing as an entry visible on the Triggered Alerts page. Other environments might also configure notification or integration actions, but every action should have a clear owner and purpose.
Triggered action: record the firing on the Triggered Alerts page
Save and verify the alert
- Review the saved search and confirm that it is the intended search.
- Confirm the 300-day evaluation window, or the time range appropriate for your actual use case.
- Confirm the scheduled or real-time execution type.
- Confirm the result-count rule: greater than 100.
- Confirm that trigger frequency is once for the result set rather than once per result.
- Review the throttle interval, permissions, title, description, and selected actions.
- Save the alert.
- After a search evaluation returns more than 100 records, open the Triggered Alerts page and verify that an alert event was recorded.
The expected outcome is that Splunk records a triggered alert event when the monitored search returns more than 100 records in the evaluated time range.
Troubleshoot alert behavior
The alert does not fire even though relevant data exists
- Run the underlying search with the same time range used by the alert.
- Check whether the result count actually exceeds the threshold. “Greater than 100” requires at least 101 results.
- Verify that the selected scheduled interval or real-time mode matches the required response speed.
- Check whether an active throttle period is suppressing another firing.
The alert fires too often
- Check whether trigger frequency is set to fire for every result.
- Increase the threshold if the search condition is too broad.
- Add or extend the throttle interval.
- Consider a scheduled alert instead of a real-time alert when immediate reaction is unnecessary.
Other users cannot access or manage the alert
- Review the alert's permissions.
- Confirm that it is shared with the intended users in the appropriate app rather than remaining private.
- Confirm that those users have access to the app in which the alert is shared.
No entry appears on the Triggered Alerts page
- Confirm that recording an event on the Triggered Alerts page is selected as a triggered action.
- Verify that the trigger condition was met for the alert's actual time range.
- Check whether throttling suppressed the action after an earlier firing.
Exam-relevant notes
- An alert is a saved search plus evaluation and response settings; manually running a search does not create automatic actions.
- The result count is measured for the alert search's selected time range.
- A threshold of “greater than 100” fires at 101 results, not at exactly 100.
- Scheduled alerts evaluate at intervals; real-time alerts evaluate continuously or near continuously as data arrives.
- Triggering once treats the result set as an aggregate condition, while per-result triggering can execute actions repeatedly.
- Throttling suppresses repeated activity after a firing and helps reduce duplicate notifications.
- Permissions and app sharing determine who can view, manage, and use the alert.
For related subjects, see the Splunk alerts overview, learn about the time range picker, or review how to create an alert that runs a script.