UCSF IT Technology
Search
Give to UCSF

Guidance for Logging, Monitoring, and Compliance

Questions? Get IT help

Overview

Logging and event management enable security visibility, operational reliability, and regulatory compliance across UCSF systems. Logging should be implemented based on system risk, data sensitivity, and operational criticality rather than treated as a uniform requirement. 

Security Operations Responsibilities 

Security event monitoring, threat detection, and incident response responsibilities remain under the designated Information Security organization and associated security tooling. 

IT Incident Command does not provide Security Operations Center (SOC) services unless otherwise formally defined. 

Security monitoring activities—including but not limited to detection of malicious behavior, threat hunting, and security incident response—are outside the scope of IT Incident Command services and are governed by Information Security policies and procedures. 

Service Owners are the subject matter experts (SMEs) for their systems and are ultimately accountable for ensuring that logging, monitoring, and retention practices meet UCSF standards, University of California Office of the President policy, and any applicable regulatory or contractual obligations. 

A Service Owner may be fulfilled by roles such as an application manager, infrastructure manager, technical lead, vendor manager, or business owner. 

Service Owners may delegate operational responsibilities (such as monitoring implementation or alert response), but remain accountable for ensuring that logging, monitoring, and retention capabilities are defined, implemented, and maintained. 

This includes determining: 

  • Which systems and services require logging 
  • What level of detail is necessary 
  • Which events should be monitored in real time versus retained for audit 
  • Appropriate retention periods based on compliance and risk 

Logging is expected for systems handling sensitive institutional data, supporting critical services, or subject to regulatory requirements such as HIPAA or PCI DSS. 

Monitoring Definitions 

  • Operational Monitoring: Focused on system health, availability, and performance. Owned by Service Owners or delegated operational teams. 
  • Security Monitoring: Focused on threat detection, anomalous behavior, and security events. Owned by Information Security. 
  • Compliance Logging (Audit / Forensic Logging): Focused on retention, traceability, and audit requirements. Accountable to Service Owners and supported by IT Incident Command platforms. 

A practical model separates logging into two complementary use cases: 

Archival, Audit, and Forensic Logging (Syslog-ng) 

These logs are comprehensive and retained to support investigations, audits, and compliance validation. 

Examples include: 

  • Authentication activity (logins, failures, privileged access, account lifecycle changes) 
  • Operating system and server logs 
  • Network infrastructure logs (firewalls, routers, switches, load balancers) 
  • Application audit logs (user activity, administrative actions, API access) 
  • Database access and query logs 
  • Security tooling logs (endpoint detection, IDS/IPS, antivirus) 

These logs are stored for historical traceability and forensic analysis. 

Monitoring and Alerting (DataDog) 

These logs represent curated, high-value signals used for operational monitoring and alerting. Security monitoring use cases are handled separately by Information Security. 

Examples include: 

  • Repeated failed logins or anomalous authentication behavior 
  • Privileged access outside expected patterns 
  • Application errors, crashes, or degraded performance 
  • Infrastructure threshold breaches (CPU, memory, disk, network) 
  • Deployment or configuration changes 
  • Failures in critical workflows or integrations 

These events enable alerting, dashboards, and rapid response. 

Decision Framework 

  • Send complete logs to syslog-ng for retention and audit 
  • Send filtered, actionable events to DataDog for monitoring 
  • Many systems will use both approaches in parallel 

Service Owners are expected to apply their expertise to determine the appropriate balance. 

IT Incident Command Logging and Monitoring Services 

Prerequisite: CMDB Registration 

Registration of systems and services in the ServiceNow CMDB is a prerequisite for onboarding into centralized logging and monitoring services. 

Accurate CMDB records (including system identifiers such as hostname, IP address, and ownership) are required to support log ingestion validation, alerting, and operational alignment. 

IT Incident Command provides centralized services for log archival and event monitoring, with clearly defined scope and operational boundaries. 

Syslog-ng (Log Archival and Retention) 

  • Centralized ingestion and storage of Enterprise IT logs 
  • Secure storage architecture to preserve integrity and availability 
  • Logs stored separately from source systems where required 
  • The standard baseline retention period is one year unless alternate retention requirements are approved 
  • Managed log rotation and secure purging aligned with institutional standards 

Requests for extended retention, additional archival, or specialized compliance storage require further review and may incur additional cost. 

Access and Retrieval 

  • Access to logging platforms is available upon request 
  • Service Owners are responsible for retrieving and supplying logs for their service 

DataDog (Event Monitoring and Observability) 

  • Real-time ingestion of selected events and metrics 
  • Alerting based on thresholds, anomalies, and defined conditions 
  • Dashboards for operational and service health visibility 
  • Correlation across logs, metrics, and services 

DataDog is provided as a centralized observability platform supporting operational monitoring use cases. 

IT Incident Command provides: 

  • Onboarding support and integration guidance 
  • Reference templates and recommended baseline monitoring 
  • Optional consulting support 
  • Shared standards for monitoring and log structure 

However: 

  • Monitoring requirements are defined by the Service Owner 
  • Creation of monitors, alerts, and dashboards is the responsibility of the Service Owner 
  • Ongoing tuning and operational management remain the responsibility of the Service Owner 

Centralized ingestion into DataDog does not imply active 24x7 review, alert response, or operational ownership by IT Incident Command. 

Cost Considerations 

  • DataDog usage is governed by contractual limits 
  • Expanded usage may incur additional costs 
  • Costs may be recovered through recharge 

Integration and Onboarding Support 

IT Incident Command provides structured support to help Service Owners integrate systems into centralized logging and monitoring services. 

Service Owners remain responsible for validating completeness, accuracy, and effectiveness of logging and monitoring configurations. 

Alerting and validation depend on log sources being registered in the ServiceNow CMDB. 

Requesting Log Archival and Monitoring Services 

Onboarding into centralized logging and monitoring follows an intake-driven process to ensure consistency, compliance, and proper integration. 

  1. Define Requirements - Service Owners identify systems, logging scope, retention needs, and monitoring use cases. 
  1. Complete Intake and Documentation - Documentation includes system description, data classification, logging scope, and compliance requirements. 
  1. Submit Request to IT Incident Command - Requests include Syslog-ng onboarding, DataDog monitoring, and access. 
  1. Configure Log Forwarding - Service Owners configure systems while IT Incident Command provides guidance. 
  1. Validation and Testing - Logging and monitoring configurations are tested and validated. 
  1. Ongoing Management - Service Owners maintain responsibility for compliance and effectiveness. 

Requesting an Exception to Log Archival and Monitoring Requirements 

If your application provides built-in logging tools and storage that meet UCOP policy requirements, you may request a Security Exception through the established process. 

How to submit the request: 

  1. Open the Security Exception Request form: https://ucsf.service-now.com/ucsfit?id=ucsf_sc_cat_item&sys_id=ab7022b113f06f0094f75e7f3244b00d&sysparm_category=c76baa05a5d51100e2dca212349e2286 
  1. In the “Uncommon (Additional Security Review Required)” section, select “Other.” 
  1. Complete and submit the form with details on how your application’s native logging and retention meet UCOP requirements. 

 

Enterprise Logging RACI Model
Enterprise Logging RACI Model
R = Responsible    A = Accountable    C = Consulted    I = Informed
1. Logging Strategy and Requirements Definition
1.1 Scope and Compliance

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Define logging scope and requirements

C

C

C

AR

Determine compliance and retention needs

C

C

C

AR

Identify log sources and event types

AR

C

C

I

2. Log Collection and Forwarding (Syslog-ng)
2.1 Forwarding Setup

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Configure systems to send logs

AR

C

I

I

Provide Syslog-ng ingestion endpoints

I

AR

I

I

Configure centralized log ingestion

I

AR

I

I

2.2 Troubleshooting

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Troubleshoot log forwarding issues

AR

C

I

I

3. Log Archival and Retention
3.1 Retention Management

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Define retention requirements

C

C

C

AR

Store logs (one-year retention)

I

AR

I

I

Manage log rotation and purging

I

AR

I

I

Request extended retention

AR

C

I

I

4. Monitoring and Alerting (DataDog)
4.1 Monitoring Requirements

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Define monitoring requirements

C

C

C

AR

Create monitors and alerts

AR

C

C

I

Tune and maintain monitoring

AR

C

C

I

4.2 Active Monitoring and Response

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Actively monitor defined services

AR

R

C

I

Respond to alerts and incidents

AR

I

C

I

4.3 Platform Support

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Provide DataDog platform

I

AR

C

I

Provide monitoring guidance

I

AR

C

I

5. Integration and Onboarding
5.1 Intake and Documentation

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Complete intake documentation

AR

C

C

I

Submit onboarding request

AR

C

I

I

5.2 Validation

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Validate log ingestion

A

R

C

I

5.3 New Source Setup

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Configure Syslog-ng for new sources

I

AR

C

I

Provide onboarding support and guidance

I

AR

C

I

6. Log Access and Retrieval
6.1 Retrieval Process

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Request logs

AR

C

I

I

Assign Access to Syslog-ng

R

A

I

I

Retrieve logs

AR

C

I

I

7. Governance, Compliance, and Oversight
7.1 Standards and Compliance

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Ensure compliance with logging standards

AR

C

C

I

Define security logging standards

I

C

AR

I

7.2 Exceptions and Oversight

Activity

Application Owners

IT Incident Command

IT Security

Executive Leadership

Risk acceptance and exceptions

A

C

R

I

Executive oversight and accountability

I

I

C

A

Service Category Security
Owner Team IT Incident Command
Service
Syslog-NG

Get more guidance

Explore step-by-step how-to guides and tips to make the most of this IT service.