Skip to content

Race condition between Cache Dispose(bool disposing) and CleanupLoop() #2

@AnyhowStep

Description

@AnyhowStep

When running tests that use PersistentQueue, I sometimes see this,

Unhandled exception. System.ObjectDisposedException: The CancellationTokenSource has been disposed.
   at Persistent.Queue.Cache.Cache`2.CleanupLoop()
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
The active test run was aborted. Reason: Test host process crashed : Unhandled exception. System.ObjectDisposedException: The CancellationTokenSource has been disposed.
   at Persistent.Queue.Cache.Cache`2.CleanupLoop()
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()

Or,

Unhandled exception. System.ObjectDisposedException: The CancellationTokenSource has been disposed.
   at System.Threading.CancellationTokenSource.get_Token()
   at Persistent.Queue.Cache.Cache`2.CleanupLoop()
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
The active test run was aborted. Reason: Test host process crashed : Unhandled exception. System.ObjectDisposedException: The CancellationTokenSource has been disposed.
   at System.Threading.CancellationTokenSource.get_Token()
   at Persistent.Queue.Cache.Cache`2.CleanupLoop()
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()

The race condition is somewhere here,

protected virtual void Dispose(bool disposing)
{
if (disposing)
{
_cts.Cancel();
RemoveAll();
// Wait until write lock becomes available.
// If this is the case, The CleanupLoop must have finished.
_lock.EnterWriteLock();
_lock.ExitWriteLock();
_lock.Dispose();
_cts.Dispose();
}
}
private async void CleanupLoop()
{
try
{
while (!_cts.IsCancellationRequested)
{
RemoveOldItems();
await Task.Delay(_ttl / 2, _cts.Token);
}
}
catch (OperationCanceledException)
{
// ignore
}
}

  1. CleanupLoop() while-loop has not exited yet
  2. _cts.Cancel();
  3. _cts.Dispose();
  4. !_cts.IsCancellationRequested <-- ObjectDisposedException

Or,

  1. CleanupLoop() while-loop has not exited yet
  2. _cts.Cancel();
  3. _cts.Dispose();
  4. _cts.Token <-- ObjectDisposedException

Just because _lock.EnterWriteLock(); succeeds, does not mean CleanupLoop won't reference _cts again soon after

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions