Defio: Instance-Optimized Fusion of AWS Database Services

Building large-scale data infrastructures is hard: There are often more than a single type of workloads and business requirements, but unfortunately, “one size does not fit all”. Modern database systems tend to specialize towards a specific type of workload, and thus organizations are left to integrate these differently-specialized database systems in order to achieve sufficient performance for all of their use cases and workloads. This kind of hybrid architecture—also known as Data Mesh architecture—often leads to the increasing complexity of maintaining and utilizing database services, both for the data engineers and the end users. However, we believe that some of this complexity can be abstracted away from the end users, in particular with respect to query routing, i.e. determining where to execute each individual SQL query among the multiple database engines. To overcome this challenge, we propose Defio, a unified interface to multiple specialized database engines that can intelligently handle myriads of workloads without having the end users think about the underlying execution of each query. Specifically, this thesis focuses on the design and implementation of an instance-optimized query router, which ultimately enables Defio to take advantage of the performance benefits of each specialized database in a Data Mesh architecture—resulting in what we call a fusion of database services.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2023

Author dc:creator

• Fanggohans, Dean

Advisor dc:contributor.advisor

• Kraska,Tim

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/151479

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/151479