[Dev] [Libretools - Bug #410] [librerelease] blocks [librestage]

[labs at parabola.nu]

Issue #410 has been updated by lukeshu.

Description changed from My solution to Bug #379 was for librerelease to grab an exclusive lock on the staging directory, while having librestage require a shared lock.

mtjm:

> I don't understand how locking would help: wouldn't it delay the second librestage until librerelease finishes, keeping the CPU idle instead of building packages for several hours in some cases?
> 
> The old solution was to make a list of staged files in librerelease once and then release them. It's safe to continue staging files until the next librerelease run releases them.

lukeshu:

> That is the case: it will block librestage.
> 
> Because sets of files can't be added to the staging directory atomically, there are a couple of race conditions so that I don't want to have the list of staged files.
> 
> Perhaps if librerelease first moved files to a temporary upload directory, and only held the lock while it was moving the files?
> 
> Although, this could also be fixed in the programs you use to build packages. Staging doesn't need to block building. treepkg/fullpkg could run the staging commands in the background.  Also, they could be learn how to do parallel builds by spawning new chroot copies.

> I'm closing this, and creating a "librerelease block librestage" bug.
 to My solution to Bug #379 was for librerelease to grab an exclusive lock on the staging directory, while having librestage require a shared lock.

mtjm:

> I don't understand how locking would help: wouldn't it delay the second librestage until librerelease finishes, keeping the CPU idle instead of building packages for several hours in some cases?
> 
> The old solution was to make a list of staged files in librerelease once and then release them. It's safe to continue staging files until the next librerelease run releases them.

lukeshu:

> That is the case: it will block librestage.
> 
> Because sets of files can't be added to the staging directory atomically, there are a couple of race conditions so that I don't want to have the list of staged files.
> 
> Perhaps if librerelease first moved files to a temporary upload directory, and only held the lock while it was moving the files?
> 
> Although, this could also be fixed in the programs you use to build packages. Staging doesn't need to block building. treepkg/fullpkg could run the staging commands in the background.  Also, they could be learn how to do parallel builds by spawning new chroot copies.
> 
> I'm closing this, and creating a "librerelease block librestage" bug.

Assignee set to lukeshu

Herp, I can just have the lock in librerelease while creating the list.  Once the list is created, I can close the lock.
----------------------------------------
Bug #410: [librerelease] blocks [librestage]
https://labs.parabola.nu/issues/410

Author: lukeshu
Status: open
Priority: feature
Assignee: lukeshu
Category: 
Target version: 


My solution to Bug #379 was for librerelease to grab an exclusive lock on the staging directory, while having librestage require a shared lock.

mtjm:

> I don't understand how locking would help: wouldn't it delay the second librestage until librerelease finishes, keeping the CPU idle instead of building packages for several hours in some cases?
> 
> The old solution was to make a list of staged files in librerelease once and then release them. It's safe to continue staging files until the next librerelease run releases them.

lukeshu:

> That is the case: it will block librestage.
> 
> Because sets of files can't be added to the staging directory atomically, there are a couple of race conditions so that I don't want to have the list of staged files.
> 
> Perhaps if librerelease first moved files to a temporary upload directory, and only held the lock while it was moving the files?
> 
> Although, this could also be fixed in the programs you use to build packages. Staging doesn't need to block building. treepkg/fullpkg could run the staging commands in the background.  Also, they could be learn how to do parallel builds by spawning new chroot copies.
> 
> I'm closing this, and creating a "librerelease block librestage" bug.




--
You have received this notification because you have either subscribed to it, or are involved in it.
To change your notification preferences, please click here: http://labs.parabola.nu/my/account