Stretch: Scalable Multi-Threaded Execution for Distributed Machines

This thesis presents Stretch, a Distributed Shared Memory (DSM) system that transparently scales multithreaded POSIX applications across multiple physical nodes by leveraging recent Linux kernel APIs exposed to user space. Motivated by the widening gap between stagnant single-core CPU performance and rapidly improving network and memory band-width, Stretch revisits the feasibility of page-based DSM on commodity clusters. The system integrates and extends CRIU checkpoint/restore to migrate arbitrary subsets of threads to remote nodes, and leverages userfaultfd to intercept missing and write-protection faults in user space. These faults drive a centralized MSI-style write-invalidation coherence protocol, implemented by a DSM server that tracks page ownership and sharer sets. Meanwhile, DSM clients fetch, install, write-protect, and invalidate pages via process vm readv/writev and process madvise. Stretch preserves standard shared-memory and pthread semantics without requiring new APIs, offering an automatic mode for unmodified programs and a manual mode that uses mmap-based, page-aligned shared regions and DSM-aware synchronization pages to reduce false sharing and fault overhead. Stretch is evaluated on CloudLab across all seven Phoenix MapReduce benchmarks using both TCP and RDMA transports. The results show that Stretch improves performance whenever the computation per page is large enough to amortize DSM overheads, write-sharing is limited, and the machine is sufficiently overloaded to benefit from distributing threads across nodes. Among the benchmarks, Matrix Multiply achieves a speedup of up to 3.39×, and PCA achieves a speedup of up to 2.08×, illustrating cases where page faults are infrequent relative to computation. In contrast, K-means performs 0.56×–0.64× slower than vanilla across all configurations due to its fine-grained updates to shared centroid pages, which generate heavy invalidation traffic and prevent amortization of DSM costs. Overall, the results demonstrate that CRIU-based thread migration, combined with userfaultfd coherence, forms a practical foundation for transparent DSM, capable of scaling unmodified parallel applications beyond a single machine when computation-to-page-fault ratios are sufficiently high.

Author dc:creator

• Edoardo D'Alessio (24399398)

• Distibuted Systems
• Operating System

Statement dc:rights

• In Copyright

DOI dc:identifier

https://doi.org/10.25417/uic.32993828.v1

OAI identifier oai:identifier

oai:figshare.com:article/32993828