Overview
Logging and event management provide the visibility needed to support security monitoring, system reliability, incident response, and regulatory compliance across UCSF systems. Logs record system, application, and user activity, while event management helps teams collect, review, and respond to important events and potential issues.
Logging should be implemented based on the system's risk level, data sensitivity, compliance requirements, and operational importance so that monitoring efforts are aligned with the potential impact of security, privacy, or service disruptions.
Logging and Monitoring Framework
Logging and monitoring serve different purposes and often work together to support operations, security, investigations, and compliance requirements.
Operational Monitoring
Operational monitoring focuses on system health, availability, performance, and service reliability. It is typically managed by Service Owners or their delegated operational teams.
Examples include:
- Application errors and performance issues
- Infrastructure health and capacity metrics
- Service availability and uptime monitoring
- Failed integrations or workflow disruptions
- Deployment and configuration changes
Security Monitoring
Security monitoring focuses on identifying threats, suspicious activity, and potential security incidents. These activities are managed by Information Security using approved security tools and processes.
Examples include:
- Suspicious authentication activity
- Indicators of compromise or malicious behavior
- Unauthorized access attempts
- Threat detection and investigation
- Security incident response activities
Audit and Forensic Logging
Audit and forensic logging focuses on retaining records that support investigations, compliance requirements, and historical traceability. Service Owners are accountable for ensuring that appropriate logs are collected and retained.
Examples include:
- Authentication and access logs
- Operating system and server logs
- Network device logs
- Application audit trails
- Database access logs
- Security tool logs
These logs are retained to support audits, investigations, and forensic analysis.
Platform model
UCSF uses a complementary approach that separates long-term log retention from operational monitoring and alerting.
Syslog-NG: Log retention and audit
Syslog-NG is used to collect and retain comprehensive logs for audit, compliance, investigation, and forensic purposes. These logs provide historical records that can be reviewed when needed.
DataDog: Monitoring and alerting
DataDog is used to collect and analyze operationally significant events that require dashboards, monitoring, and alerting. Security monitoring use cases are managed separately by Information Security.
Examples include:
- Repeated failed login attempts
- Application failures or degraded performance
- Infrastructure threshold breaches
- Critical workflow failures
- Significant deployment or configuration changes
These events support rapid detection, notification, and response.
Implementation guidance
In most cases:
- Send comprehensive logs to Syslog-NG for retention and audit purposes.
- Send actionable events and monitoring signals to DataDog for dashboards and alerting.
- Use both platforms when a system requires both historical log retention and operational monitoring.
Service Owners should determine the appropriate logging and monitoring approach based on system risk, business criticality, compliance requirements, and operational needs.
Roles and Responsibilities
Information Security
Information Security is responsible for:
- Security event monitoring
- Threat detection
- Threat hunting
- Security incident response
- Security monitoring tools and processes
IT Incident Command does not provide Security Operations Center (SOC) services unless those responsibilities have been formally assigned.
Service Owners
Service Owners are accountable for ensuring that logging, monitoring, and log retention practices meet UCSF standards, University of California policy requirements, and applicable regulatory or contractual obligations.
A Service Owner may be an application manager, infrastructure manager, technical lead, vendor manager, business owner, or another designated individual responsible for the service.
Service Owners may delegate operational tasks but remain accountable for ensuring that logging and monitoring requirements are defined, implemented, maintained, and periodically reviewed.
Service Owners are responsible for determining:
- Which systems and services require logging
- What information should be captured in logs
- Which events require real-time monitoring and alerting
- Which events should be retained for auditing and investigation
- How long logs should be retained
Logging is expected for systems that store, process, or transmit sensitive institutional data, support critical business or clinical services, or are subject to regulatory requirements such as HIPAA or PCI DSS.
IT Incident Command Services
IT Incident Command provides centralized platforms and support for logging, monitoring, and observability.
Prerequisite: CMDB Registration
Systems and services must be registered in the ServiceNow Configuration Management Database (CMDB) before they can be onboarded to centralized logging and monitoring services.
Accurate CMDB records, including ownership information and technical identifiers such as hostnames and IP addresses, are required to support log ingestion, monitoring, alerting, and operational support.
Log retention services (Syslog-NG)
IT Incident Command provides centralized log collection and retention through Syslog-NG.
IT Incident Command provides:
- Centralized log collection and storage
- Secure log retention
- Log rotation and secure deletion
- Support for audit and forensic use cases
The standard retention period is one year unless alternate requirements have been approved.
Requests for extended retention or specialized compliance storage may require review and may incur additional costs.
Monitoring and observability services (DataDog)
IT Incident Command provides DataDog as a centralized observability platform for operational monitoring.
Capabilities include:
- Collection of selected logs, events, and metrics
- Alerting based on thresholds and anomalies
- Operational dashboards
- Correlation across logs, metrics, and services
Support services
IT Incident Command supports Service Owners by providing:
- Platform onboarding guidance
- Integration support
- Reference architectures and recommended practices
- Shared logging and monitoring standards
- Optional consulting assistance
Service Owner responsibilities
Use of centralized platforms does not transfer operational ownership.
Service Owners remain responsible for:
- Defining monitoring requirements
- Creating and maintaining monitors, alerts, and dashboards
- Validating logging and monitoring configurations
- Reviewing and responding to alerts
- Retrieving logs for operational, audit, or investigative purposes
Centralized ingestion into Syslog-NG or DataDog does not include 24x7 monitoring, alert review, or incident response unless separately defined.
Cost considerations
DataDog usage is subject to contractual and licensing limits. Expanded usage may incur additional costs that may be recovered through recharge mechanisms.
Requesting Logging and Monitoring Services
Onboarding to centralized logging and monitoring services follows a standard process.
Before you begin
Ensure that your system or application is registered in the ServiceNow CMDB with current ownership and system information.
Onboarding process
- Define logging, retention, and monitoring requirements.
- Document the service, including data classification and compliance requirements.
- Submit a request for Syslog-NG onboarding, DataDog onboarding, platform access, or a combination of services.
- Configure log forwarding and integrations.
- Validate log ingestion, monitoring, and alerting.
- Maintain and periodically review configurations to ensure ongoing effectiveness and compliance.
Request access
Access to enterprise logging capabilities is available upon request to the Enterprise Logging Service. ELS will work with Service Owners to owners to onboard systems and applications into the enterprise logging environment and to support logging configurations, retention requirements, and operational use cases.
Request logging and monitoring services by emailing [email protected].
Exceptions to Logging and Monitoring Requirements
Applications that provide native logging, monitoring, and retention capabilities that satisfy University of California and UCSF requirements may be eligible for an exception to centralized logging requirements.
To request an exception:
- Open the Security Exception Request Form.
- Under Uncommon (Additional Security Review Required), select Other.
- Describe how the application's native logging, monitoring, and retention capabilities meet applicable requirements.
- Submit the request for review.
Approval is required before opting out of centralized logging or monitoring services.
Requesting an Exception to Log Archival and Monitoring Requirements
If your application provides built-in logging tools and storage that meet UCOP policy requirements, you may request a Security Exception through the established process.
How to submit the request:
- Open the Security Exception Request form.
- In the Uncommon (Additional Security Review Required) section, select Other.
- Complete and submit the form with details on how your application’s native logging and retention meet UCOP requirements.