I’m developing a service in which users can add and remove content which exists as DB metadata and files-on-disk for the data. If two users add the same content the data file is shared. When adding content I don’t care about race conditions because adding the same data file twice is not a big problem.
However, when deleting content there is a chance that another user will add the same content at the same time and cause a “hole” in the on-disk data:
Thread1: DB row deleted
Thread2: DB row created (since no db rows matched the user must upload data)
Thread2: file data stored
Thread1: file data deleted (uh, oh - missing data)
I hope this simplified example makes sense.
I could solve this problem using a global lock around adding and deleting but this would create a fairly serious lock contention problem when many users are adding and removing many things. CockroachDB only supports transaction serialization errors to flag conflicts like this, but I can’t rollback the deletion of a file on disk. Is there a common pattern to deal with this type of problem?