Project

General

Profile

Idea #3761

Updated by Tom Clegg over 9 years ago

At startup, keepstore will set up: 
 * A BlockList instance called "currentPullList" (exists but currently called pullmgr) 
 * A BlockPuller instance that performs work on the current pull list (e.g., @go BlockPuller.New(currentPullList).Run()@) 

 The resulting BlockPuller goroutine will use the BlockList interface to learn what work is to be done. 

 The BlockPuller's Run method will perform the following algorithm: 

 * Get the current pull list by calling getList() on the list given during instantiation.[1] 
 * Walk the pull list. Attempt each {block,server} entry one time. Log each attempt and each success. 
 * When a pull succeeds, call removeItem() on the list. (This will delete all pull list entries for that block.)[2] 
 * Repeat. 

 "Attempt" means: 
 * Generate a random API token.[3] 
 * Generate a permission signature using the random token, timestamp 60 seconds in the future, and desired block hash. 
 * Using this token & signature, retrieve the given block from the given keepstore server. 
 * Verify checksum. 
 * Write to storage, just as if it had been provided by a client in a PUT transaction. 

 fn1. Optionally/eventually, in cases where the current pull list is empty, GetList() will block until something is available. 

 fn2. If this isn't already done, start by stubbing removeItem() as a no-op, and revisit after everything else if there's time left in this story. 

 fn3. Currently, Keep doesn't actually verify API tokens, just the permission signature, so a random token is just as effective as a real one. 

Back