Log Collection

Log collection can be a daunting task, especially if massive amounts of data need to be collected. This process could be made easier by previously considering some key points.

Choosing Log Event Sources

The selection of event sources should be driven by the use cases that have been identified. For example, if the use case is the monitoring of user logins to critical resources, the event sources selected should be only those related to the critical resources in question. This may include the LDAP directory server, local servers, firewalls, network devices, and key applications.

There are many other potential event source categories, including:

• Security

• Firewalls

• Endpoint Security (EDR, AV, etc.)

• Web Proxies/Gateways

• LDAP/Active Directory

• IDS

• DNS

• DHCP

• Servers

• Workstations

• Netflow

• Ops

• Applications

• Network Devices

• Servers

• Packet Capture/Network Recorder

• DNS

• DHCP

• Email

• DevOps

• Application Logs

• Load Balancer Logs

• Automation System Logs

• Business Logic

Collection Methods

A decision must be made as to how the logs will be collected. After a list of event sources has been determined, the next step is to decide the method of collection for each source. It is critical to understand what method each event source uses and what resources may be required.

For example, if a log shipper will be required to read logs from a local file on all servers, the log shipper must be selected and tested before deployment. In other cases, proprietary APIs or software tools must be employed and integrated. In some cases, changes to the event sources themselves (security devices, network hardware, or applications) may be required. Additional planning is often required to deploy and maintain these collection methods over time.

Leona supports many input types out of the box and many more are available in LeonaLog Marketplace.