Problem
CTK is currently processing DMS events sequentially, see io/kinesis/relay.py. The consequence is data loading will be slow.
Proposal
Because the ingress DMS event stream can include arbitarily-alternating event types, the data loading in CTK needs a component which synthesizes per-table batches of the same kind of DML operation before submitting them to the database.
It would be nice if this package could provide such a component, because it includes and implements relevant domain knowledge about DMS events already.
Problem
CTK is currently processing DMS events sequentially, see io/kinesis/relay.py. The consequence is data loading will be slow.
Proposal
Because the ingress DMS event stream can include arbitarily-alternating event types, the data loading in CTK needs a component which synthesizes per-table batches of the same kind of DML operation before submitting them to the database.
It would be nice if this package could provide such a component, because it includes and implements relevant domain knowledge about DMS events already.