Right now the cache aspirationally tries to do things things in parallel but since its all cpu bound work and the main process is a single thread this doesn't help much. It also seems to block the runtime, which is bad since means the server doesn't actually accept any requests until the cache is done. Change it so cache construction doesn't block the runtime so the server can begin serving requestions.
There are two possible approaches:
- make process yield (in theory the use of async should do this already, investigate what's going on)
- use worker threads and true parallelism
Right now the cache aspirationally tries to do things things in parallel but since its all cpu bound work and the main process is a single thread this doesn't help much. It also seems to block the runtime, which is bad since means the server doesn't actually accept any requests until the cache is done. Change it so cache construction doesn't block the runtime so the server can begin serving requestions.
There are two possible approaches: