Inside VS Code's harness-level engineering: how extended prompt caching, embedding-guided tool search, and WebSockets cut agent tokens by up to 28% and idle latency by 19%.
Design a cache. We had to clarify all requirements surrounding this . Itโs a global cache for a distributed database with 100 billion records and each cache holds key value records so based on that do the sizing , distribution , latencies , read heavy, etc. Discuss all issues like replication, crashing of cache servers, cache warm up , TTL etc proactively.