The loader is pinned to redis~=3.5.3 (requirements-loader.txt), released 2020. redis-py 4/5 have faster command packing and a C parser via hiredis, and building the connection with decode_responses=False on the write path avoids decoding replies the loader never reads.
Upgrading would speed up the client side of the load and, importantly, ties into #381 (migrate the frontend off aioredis): redis-py >= 4.2 has asyncio built in, so the frontend and loader could share one client library instead of the current aioredis (frontend) + redis-py (loader) split.
Scope: bump the pin, verify the loader's pipeline/CONFIG SET/MSET usage against the new API, and re-run the loader unit + integration tests. Coordinate with #381.
Context: requirements-loader.txt, node_normalizer/loader/loader.py, node_normalizer/redis_adapter.py.
The loader is pinned to
redis~=3.5.3(requirements-loader.txt), released 2020.redis-py4/5 have faster command packing and a C parser viahiredis, and building the connection withdecode_responses=Falseon the write path avoids decoding replies the loader never reads.Upgrading would speed up the client side of the load and, importantly, ties into #381 (migrate the frontend off
aioredis):redis-py>= 4.2 has asyncio built in, so the frontend and loader could share one client library instead of the currentaioredis(frontend) +redis-py(loader) split.Scope: bump the pin, verify the loader's pipeline/
CONFIG SET/MSETusage against the new API, and re-run the loader unit + integration tests. Coordinate with #381.Context:
requirements-loader.txt,node_normalizer/loader/loader.py,node_normalizer/redis_adapter.py.