performing-log-source-onboarding-in-siem
Perform structured log source onboarding into SIEM platforms (Splunk,
v1.03 instalações+3 em 7 dias36,3 mil EstrelasApache-2.0sincronizado há 11 horas
Instalar
Ainda não tem a CLI da NotHoly? Instale primeiro.
$ curl -fsSL https://notholy.com/install.sh | sh$ notholy skills install mukul975/performing-log-source-onboarding-in-siemGrava em ~/.notholy/skills depois de conferir a assinatura e o SHA-256.
AssinaturaAssinado, ed25519
Pacote9b050acd…23216b67
9b050acdfbe9…4a0d23216b678 arquivos · 14.1 KBVarreduraMédio3 achados
- Script executável junto com a skillscripts/agent.py
- Script executável junto com a skillscripts/process.py
- Links para domínios externos
Fontegithub.com/mukul975/Anthropic-Cybersecurity-Skills/tree/main/skills/performing-log-source-onboarding-in-siem@54a7988
Este item ainda não foi escrito no seu idioma, então aparece em English.
Nesta página
Overview
Log source onboarding is the systematic process of integrating new data sources into a SIEM platform to enable security monitoring and detection. Proper onboarding requires planning data sources, configuring collection agents, building parsers, normalizing fields to a common schema, and validating data quality. According to the UK NCSC, onboarding should prioritize log sources that provide the highest security value relative to their ingestion cost.
When to Use
- When conducting security assessments that involve performing log source onboarding in siem
- When following incident response procedures for related security events
- When performing scheduled security testing or auditing activities
- When validating security controls through hands-on testing
Prerequisites
- SIEM platform deployed (Splunk, Elastic, Sentinel, QRadar, or similar)
- Network access from source systems to SIEM collectors
- Administrative access on source systems for agent installation
- Common Information Model (CIM) or equivalent schema documentation
- Change management approval for production system modifications
Log Source Priority Framework
Tier 1 - Critical (Onboard First)
| Source | Log Type | Security Value |
|---|---|---|
| Active Directory | Security Event Logs | Authentication, privilege escalation |
| Firewalls | Traffic logs | Network access, C2 detection |
| EDR/AV | Endpoint alerts | Malware, process execution |
| VPN/Remote Access | Connection logs | Unauthorized access |
| DNS Servers | Query logs | C2 beaconing, data exfiltration |
| Email Gateway | Email security logs | Phishing, BEC |
Tier 2 - High Priority
| Source | Log Type | Security Value |
|---|---|---|
| Web Proxy | HTTP/HTTPS logs | Web-based attacks, data exfiltration |
| Cloud platforms (AWS/Azure/GCP) | Audit logs | Cloud security posture |
| Database servers | Audit/query logs | Data access, SQL injection |
| DHCP/IPAM | Address allocation | Asset tracking |
| File servers | Access logs | Data access monitoring |
Tier 3 - Standard
| Source | Log Type | Security Value |
|---|---|---|
| Application servers | App logs | Application-level attacks |
| Print servers | Print logs | Data loss prevention |
| Badge/physical access | Access logs | Physical security correlation |
| Network devices (switches/routers) | Syslog | Network anomalies |
Onboarding Process
Step 1: Discovery and Assessment
1. Identify the log source:
- System type and version
- Log format (syslog, CEF, JSON, Windows Events, etc.)
- Log volume estimate (EPS - events per second)
- Network location and firewall requirements
2. Assess security value:
- What threats can this source help detect?
- Which MITRE ATT&CK techniques does it cover?
- Is there an existing SIEM parser?
3. Estimate ingestion cost:
- Daily volume in GB
- License impact (per-GB or per-EPS pricing)
- Storage retention requirementsStep 2: Configure Log Collection
Syslog-Based Collection (Firewalls, Network Devices)
# rsyslog configuration for receiving syslog
# /etc/rsyslog.d/10-siem-collection.conf
# UDP reception
module(load="imudp")
input(type="imudp" port="514" ruleset="siem_forwarding")
# TCP reception
module(load="imtcp")
input(type="imtcp" port="514" ruleset="siem_forwarding")
# TLS reception
module(load="imtcp" StreamDriver.AuthMode="x509/name"
StreamDriver.Mode="1" StreamDriver.Name="gtls")
input(type="imtcp" port="6514" ruleset="siem_forwarding")
ruleset(name="siem_forwarding") {
# Forward to SIEM
action(type="omfwd" target="siem.company.com" port="9514"
protocol="tcp" queue.type="LinkedList"
queue.filename="siem_fwd" queue.maxdiskspace="1g"
queue.saveonshutdown="on" action.resumeRetryCount="-1")
}Windows Event Log Collection (Splunk Universal Forwarder)
# inputs.conf on Splunk Universal Forwarder
[WinEventLog://Security]
disabled = 0
index = wineventlog
sourcetype = WinEventLog:Security
evt_resolve_ad_obj = 1
checkpointInterval = 5
[WinEventLog://System]
disabled = 0
index = wineventlog
sourcetype = WinEventLog:System
[WinEventLog://Microsoft-Windows-Sysmon/Operational]
disabled = 0
index = wineventlog
sourcetype = XmlWinEventLog:Microsoft-Windows-Sysmon/Operational
renderXml = true
[WinEventLog://Microsoft-Windows-PowerShell/Operational]
disabled = 0
index = wineventlog
sourcetype = XmlWinEventLog:Microsoft-Windows-PowerShell/OperationalCloud Log Collection (AWS CloudTrail)
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"CloudTrailToSIEM": {
"Type": "AWS::CloudTrail::Trail",
"Properties": {
"TrailName": "siem-cloudtrail",
"S3BucketName": "company-cloudtrail-logs",
"IsLogging": true,
"IsMultiRegionTrail": true,
"IncludeGlobalServiceEvents": true,
"EnableLogFileValidation": true,
"EventSelectors": [
{
"ReadWriteType": "All",
"IncludeManagementEvents": true,
"DataResources": [
{
"Type": "AWS::S3::Object",
"Values": ["arn:aws:s3"]
}
]
}
]
}
}
}
}Step 3: Parse and Normalize
Custom Parser Example (Splunk props.conf/transforms.conf)
# props.conf
[custom:firewall:logs]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%dT%H:%M:%S%z
MAX_TIMESTAMP_LOOKAHEAD = 30
TRANSFORMS-firewall = firewall_extract_fields
FIELDALIAS-src = src_addr AS src_ip
FIELDALIAS-dst = dst_addr AS dest_ip
EVAL-action = case(fw_action=="allow", "allowed", fw_action=="deny", "blocked", true(), "unknown")
EVAL-vendor_product = "Custom Firewall"
LOOKUP-geo = geo_ip_lookup ip AS dest_ip OUTPUT country, city, latitude, longitude
# transforms.conf
[firewall_extract_fields]
REGEX = ^(\S+)\s+(\S+)\s+action=(\w+)\s+src=(\S+):(\d+)\s+dst=(\S+):(\d+)\s+proto=(\w+)\s+bytes=(\d+)
FORMAT = timestamp::$1 hostname::$2 fw_action::$3 src_addr::$4 src_port::$5 dst_addr::$6 dst_port::$7 protocol::$8 bytes::$9CIM Field Mapping
| Raw Field | CIM Field | Data Model |
|---|---|---|
| src_addr | src_ip | Network_Traffic |
| dst_addr | dest_ip | Network_Traffic |
| dst_port | dest_port | Network_Traffic |
| fw_action | action | Network_Traffic |
| bytes_sent + bytes_recv | bytes | Network_Traffic |
| user_name | user | Authentication |
| login_result | action | Authentication |
| process_path | process | Endpoint |
Step 4: Validate Data Quality
# Verify events are arriving
index=new_source earliest=-1h
| stats count by sourcetype, host, source
# Check field extraction quality
index=new_source earliest=-1h
| stats count(src_ip) as has_src count(dest_ip) as has_dest count(action) as has_action count by sourcetype
| eval src_coverage=round(has_src/count*100,1)
| eval dest_coverage=round(has_dest/count*100,1)
| eval action_coverage=round(has_action/count*100,1)
# Verify CIM compliance
| datamodel Network_Traffic search
| search sourcetype=new_sourcetype
| stats count by source, sourcetype
# Check for timestamp parsing issues
index=new_source earliest=-1h
| eval time_diff=abs(_time - _indextime)
| stats avg(time_diff) as avg_lag max(time_diff) as max_lag by host
| where avg_lag > 300Step 5: Enable Detection Coverage
# Verify existing correlation searches work with new source
index=new_source sourcetype=new_sourcetype
| tstats count from datamodel=Authentication by _time span=1h
| timechart span=1h count
# Create source-specific detection rule
[New Source - Authentication Anomaly]
search = index=new_source sourcetype=new_sourcetype action=failure \
| stats count by src_ip, user \
| where count > 10Onboarding Checklist
- Log source assessed and approved
- Network connectivity verified
- Collection agent/method configured
- Log forwarding confirmed
- Parser/field extraction configured
- CIM compliance validated
- Data model acceleration enabled
- Volume within license budget
- Retention policy configured
- Detection rules enabled/created
- Dashboard updated
- Documentation completed
- SOC team notified
References
Arquivos desta versão
assets/11.1 KB
- template.md1.1 KB
references/34.1 KB
- api-reference.md1.6 KB
- standards.md1.2 KB
- workflows.md1.3 KB
scripts/215.5 KB
- agent.py7.6 KB
- process.py7.9 KB
- LICENSE11.0 KB
- SKILL.md9.5 KB
Permissões
| Permissão | Declarada | Encontrada na varredura |
|---|---|---|
| Lê arquivos | Não | Não |
| Grava arquivos | Não | Não |
| Roda comandos no shell | Sim | Sim |
| Usa a rede | Sim | Sim |
| Usa conectores | Não | Não |
| Controla um navegador | Não | Não |
Achados da varredura
Revisado em 13 de nov. de 2026
- MédiaArquivo · Script executável junto com a skillscripts/agent.py
- MédiaArquivo · Script executável junto com a skillscripts/process.py
- InfoLink · Links para domínios externos
Uma varredura limpa reduz o risco, mas não prova que o pacote é seguro. Leia o que você instala.
| Versão | Publicada em | Commit | SHA-256 | Assinatura | Mudança de permissão |
|---|---|---|---|---|---|
| v1.0 | 13 de nov. de 2026 | 54a7988 | 9b050acd | Assinada | – |
Mais de mukul975
Tudo deste publicador- testing-for-xml-injection-vulnerabilitiesmukul975/testing-for-xml-injection-vulnerabilities5
- testing-for-sensitive-data-exposuremukul975/testing-for-sensitive-data-exposure5
- performing-jwt-none-algorithm-attackmukul975/performing-jwt-none-algorithm-attack5
- detecting-aws-credential-exposure-with-trufflehogmukul975/detecting-aws-credential-exposure-with-trufflehog5
Mais em DFIR e threat hunting
Ver a categoria- hunting-for-command-and-control-beaconingmukul975/hunting-for-command-and-control-beaconing5
- audit-context-buildingtrailofbits/audit-context-building4
- incident-response-forensicssecurityskills/incident-response-forensics3
- extracting-credentials-from-memory-dumpmukul975/extracting-credentials-from-memory-dump3
- analyzing-browser-forensics-with-hindsightmukul975/analyzing-browser-forensics-with-hindsight3
- forensics-ctfaeondave/forensics-ctf3
Pergunte tudo o que o seu escopo permite.
Crie a conta, adicione créditos e faça a primeira pergunta no chat ou no terminal.
- 1/3
Crie a conta
Individual ou empresa. Dá para virar empresa depois sem perder saldo nem chaves.
Pessoa físicaEmpresa - 2/3
Adicione créditos
No cartão. Você paga por token, sem assinatura.
$5.00recarga mínima - 3/3
Pergunte
No chat, ou instale o CLI e rode notholy login no terminal.
$ notholy login