Optimizing network data movement in server class microprocessors

High-bandwidth network interface cards (NICs), each capable of transferring 100s of Gigabits per second, are making inroads into the servers of next generation data centers. Such unprecedented data delivery rates impose immense pressure, especially on the server’s memory subsystem, as NICs first transfer network data to DRAM before processing. The cache hierarchy has evolved in response, supporting a direct data input-output (DDIO) technology to place network data directly in the last-level cache (LLC). Subsequently, various policies have been explored to manage such LLCs and have proven effective in reducing service latency and memory bandwidth consumption of network applications. However, the more recent evolution of the cache hierarchy decreased the size of LLC per core but significantly increased that of mid-level cache (MLC) with a non-inclusive policy. This calls for a re-examination of the DDIO as mentioned above technology and management policies. In this work, we first develop a framework for modeling 100 Gigabit networking in a cycle-accurate architectural simulator. We also identify shortcomings of the static data placement policy, placing network data to LLC first, and the non-inclusive policy with a commercial server system. We then propose an intelligent direct input-output (IDIO) technology that extends DDIO to MLC and provides three synergistic mechanisms: (1) self- invalidating I/O buffer, (2) network-driven MLC prefetching, and (3) selective direct DRAM access. Our detailed experiments using a full-system simulator show that IDIO significantly reduces data movement (up to 84% MLC and LLC write-back reduction), provides LLC isolation (up to 22% performance improvement), and improves tail latency (up to 38% reduction in 99 percentile latency) for receive-intensive network applications.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Agarwal, Siddharth

Contributors dc:contributor

• Kim, Nam Sung

• Networking
• Computer Architechutre

Statement dc:rights

• Copyright 2023 Siddharth Agarwal

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/122175

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F122175