Feature #2620

Keep supports I/O locking

Added by Tim Pierce over 7 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Tim Pierce
Category:
-
Start date:
04/01/2014
Due date:
% Done:

100%

Estimated time:
(Total: 26.00 h)
Story points:
2.0

Description

Notes:
  • Lock is per-volume. It's the sysadmin's problem to ensure volumes don't share a spindle.
  • Lock is in-process (not flock). It's the sysadmin's problem to ensure keep servers don't share a volume.
  • Enable per-volume lock with -serialize flag.
  • Exempt from lock behavior: HEAD, index, and the process of discovering whether a block is present during a GET or PUT. In other words, we aren't strictly limiting IO to one client per spindle: we're assuming the OS has good enough cache behavior to support hordes of readdir operations alongside a big read/write without (statistically) causing thrash.

Subtasks

Task #2695: Review 2620-keep-serialize-ioResolvedTim Pierce

Task #2629: I/O locking unit testsResolvedTim Pierce

Task #2560: Serialize Keep I/OResolvedTim Pierce

Associated revisions

Revision afe657ca (diff)
Added by Tim Pierce over 7 years ago

Reorganize Keep code in preparation for dividing into modules.

Refs #2620

Revision 8202cb5f (diff)
Added by Tim Pierce over 7 years ago

Adding a Volume interface and UnixVolume implementation.

refs #2620

Revision b66ad0b2 (diff)
Added by Tim Pierce over 7 years ago

Bug fix: GetBlock must report a CorruptError immediately.

Refs #2620.

Revision da3e84a0 (diff)
Added by Tim Pierce over 7 years ago

Added UnixVolume unit tests. (refs #2620)

Revision 93db9aec (diff)
Added by Tim Pierce over 7 years ago

Moved the MD5 verification check back to GetBlock.

Policy tests such as checksumming belong in the Keep front end. The back
end should be limited to moving data in and out of a volume. (refs #2620)

Revision 78b5bfcb (diff)
Added by Tim Pierce over 7 years ago

Updated UnixVolume unit tests (refs #2620).

Revision dbce5332 (diff)
Added by Tim Pierce over 7 years ago

Added -serialize flag.

Added IORequest and IOResponse types for communicating I/O
requests over channels with a goroutine.

New IOHandler method on UnixVolume. IOHandler receives requests on a
command channel, handles them, and delivers responses. Whenever a
UnixVolume is created with a non-nil queue, an IOHandler must be started
to handle its requests.

UnixVolume methods Get and Put now handle external I/O requests. These
methods serialize I/O requests if serialization is enabled for that
volume; otherwise they call Read and Write directly.

New unit tests: TestGetSerialized and TestPutSerialized.

Refs #2620.

Revision b860994f (diff)
Added by Tim Pierce over 7 years ago

Cleaned up unit tests. (refs #2620)

Added a MockVolume implementation to use in unit tests for the
Keep front-end handlers.

Simplified IsValidLocator and keep_test.go:setup code.

Revision e05f165a (diff)
Added by Tim Pierce over 7 years ago

Cleaned up unit tests. (refs #2620)

Added a MockVolume implementation to use in unit tests for the
Keep front-end handlers.

Simplified IsValidLocator and keep_test.go:setup code.

Revision f37f7c06 (diff)
Added by Tim Pierce over 7 years ago

Make sure an IOHandler is running when -serialize is on.

Use MakeUnixVolume to create new UnixVolumes, to ensure that an
IOHandler is started whenever necessary. (refs #2620)

Revision f74cbd15 (diff)
Added by Tim Pierce over 7 years ago

Reorganized Volume code for easier comprehension.

Refs #2620

Revision 30f834ee (diff)
Added by Tim Pierce over 7 years ago

Incorporating code review comments (refs #2620)

  • use filepath.Join to build file paths instead of fmt.Sprintf
  • use ioutil.ReadAll to ensure that we read the whole file

Revision 57b46f9b (diff)
Added by Tim Pierce over 7 years ago

Replaced KeepVolumes with a VolumeManager interface.

A VolumeManager interface may be used to set policy for writing to a set
of Keep volumes. The sole implementation at present is RRVolumeManager,
which implements simple round-robin scheduling; other implementations
could include a manager which selects the least-loaded disk first, the
fastest one, the one with the fewest pending writes, etc. etc.

Refs #2620

Revision a6439c7d
Added by Tim Pierce over 7 years ago

Merge branch '2620-keep-serialize-io' (closes #2620)

History

#1 Updated by Ward Vandewege over 7 years ago

  • Assigned To set to Tim Pierce

#2 Updated by Tim Pierce over 7 years ago

  • Tracker changed from Bug to Feature
  • Subject changed from Keep I/O locking to Keep supports I/O locking

An admin should be able to enable I/O locking: a per-spindle lock that prevents Keep from reading or writing to more than one file at a time. I/O operations on different spindles may still be executed concurrently.

#3 Updated by Tim Pierce over 7 years ago

  • Story points set to 3.0

#4 Updated by Tim Pierce over 7 years ago

  • Story points changed from 3.0 to 2.0

#5 Updated by Tim Pierce over 7 years ago

  • Target version deleted (2014-05-07 Storing and Organizing Data)

#6 Updated by Tim Pierce over 7 years ago

  • Target version set to 2014-05-07 Storing and Organizing Data

#7 Updated by Tom Clegg over 7 years ago

  • Description updated (diff)

#8 Updated by Tim Pierce over 7 years ago

  • Status changed from New to Resolved
  • % Done changed from 92 to 100

Applied in changeset arvados|commit:a6439c7ddc3407193889386051f06df2a1e74ea9.

Also available in: Atom PDF