Python多进程日志按时间分割：自定义TimedRotatingFileHandler实现

洛溪

发布时间：2026-10-13 | 6人浏览过

原创

本文介绍在Python多进程环境下，通过继承TimedRotatingFileHandler并重写computeRollover与doRollover方法，实现日志按整分钟、小时或天进行精确分割的技术方案。该方法解决了默认行为导致的半闭半开区间问题，确保日志文件按指定时间周期准确轮转，适用于需要严格时间控制的日志记录场景。

Python多进程日志分割背景

在Python多进程编程中，实现日志文件按时间周期自动分割是保障系统可维护性的关键需求。以下提供一种具体的实现思路与代码示例，供开发者参考。

自定义日志Handler实现原理

核心原理在于自定义日志处理类，使其继承自logging模块中的TimedRotatingFileHandler。实现的关键步骤包括重写computeRollover和doRollover两个核心函数。其中，重写computeRollover的主要目的是修正默认的时间计算逻辑，使其能够严格按照整分钟、整小时或整天的边界进行日志分割。例如，在按天分割的场景下，默认行为可能产生从日志创建时间或当前时刻到次日相同时长的半闭半开区间，这往往不符合预期的时间周期切割需求。通过自定义逻辑，可以确保日志文件在指定的整点时刻（如2018-04-10 00:00:00至2018-04-11 00:00:00）准确切换，避免时间区间错位。

具体实现代码如下：

核心代码实现


#!/usr/bin/env python
# encoding: utf-8

"""自定义日志处理类"""


import os
import time
from logging.handlers import TimedRotatingFileHandler


class MyLoggingHandler(TimedRotatingFileHandler):

  def __init__(self, filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False, atTime=None):
    TimedRotatingFileHandler.__init__(self, filename, when=when, interval=interval, backupCount=backupCount, encoding=encoding, delay=delay, utc=utc, atTime=atTime)

  def computeRollover(self, currentTime):
    # 将时间取整
    t_str = time.strftime(self.suffix, time.localtime(currentTime))
    t = time.mktime(time.strptime(t_str, self.suffix))
    return TimedRotatingFileHandler.computeRollover(self, t)

  def doRollover(self):
    """
    do a rollover; in this case, a date/time stamp is appended to the filename
    when the rollover happens. However, you want the file to be named for the
    start of the interval, not the current time. If there is a backup count,
    then we have to get a list of matching filenames, sort them and remove
    the one with the oldest suffix.
    """
    if self.stream:
      self.stream.close()
      self.stream = None
    # get the time that this sequence started at and make it a TimeTuple
    currentTime = int(time.time())
    dstNow = time.localtime(currentTime)[-1]
    t = self.rolloverAt - self.interval
    if self.utc:
      timeTuple = time.gmtime(t)
    else:
      timeTuple = time.localtime(t)
      dstThen = timeTuple[-1]
      if dstNow != dstThen:
        if dstNow:
          addend = 3600
        else:
          addend = -3600
        timeTuple = time.localtime(t + addend)
    dfn = self.rotation_filename(self.baseFilename + "." +
                   time.strftime(self.suffix, timeTuple))
    # 修改内容--开始
    # 在多进程下，若发现dfn已经存在，则表示已经有其他进程将日志文件按时间切割了，只需重新打开新的日志文件，写入当前日志；
    # 若dfn不存在，则将当前日志文件重命名，并打开新的日志文件
    if not os.path.exists(dfn):
      try:
        self.rotate(self.baseFilename, dfn)
      except FileNotFoundError:
        # 这里会出异常：未找到日志文件，原因是其他进程对该日志文件重命名了，忽略即可，当前日志不会丢失
        pass
    # 修改内容--结束
    # 原内容如下：
    """
    if os.path.exists(dfn):
      os.remove(dfn)
    self.rotate(self.baseFilename, dfn)
    """

    if self.backupCount > 0:
      for s in self.getFilesToDelete():
        os.remove(s)
    if not self.delay:
      self.stream = self._open()
    newRolloverAt = self.computeRollover(currentTime)
    while newRolloverAt <= currentTime:
      newRolloverAt = newRolloverAt + self.interval
    # If DST changes and midnight or weekly rollover, adjust for this.
    if (self.when == 'MIDNIGHT' or self.when.startswith('W')) and not self.utc:
      dstAtRollover = time.localtime(newRolloverAt)[-1]
      if dstNow != dstAtRollover:
        if not dstNow: # DST kicks in before next rollover, so we need to deduct an hour
          addend = -3600
        else:      # DST bows out before next rollover, so we need to add an hour
          addend = 3600
        newRolloverAt += addend
    self.rolloverAt = newRolloverAt