zinger sync from should handle multiple processes writing to a Zed pool and ensure consistency of the computed kafka offsets. We need an option to the Zed lake load endpoint where we can specify a commitID that must be at HEAD in order for the load to complete. This is easy to add given the current model for the constraint check in the lake load logic.
zinger sync fromshould handle multiple processes writing to a Zed pool and ensure consistency of the computed kafka offsets. We need an option to the Zed lake load endpoint where we can specify a commitID that must be at HEAD in order for the load to complete. This is easy to add given the current model for the constraint check in the lake load logic.