Corums claims to make the work of developers implementing consensus algorithms easier.
To verify if this statement holds, we implemented Multi-Paxos with it. We also built an application layer around our Multi-Paxos implementation to assess whether Corums is usable for practical applications. Our experience is that Corumsdoes genuinely simplify the implementation of the Paxos protocol. We found the provided abstractions useful and clear. In addition, Corums manages other aspects of distributed programming, such as configuring network connections and handling thread interleaving, so that we are allowed to focus on the details of implementing the Paxos protocol instead.
With the contributions we have made in this thesis, we have uncovered some of the areas were Corums is lacking and thus, should get more focus. Our contributions also show that there is good use for Corums in the distributed systems field by explaining usability and compatibility of the framework. In addition, having a practical application, such as ours, implemented using Corums will make it easier to implement similar systems in the future. We predict that some of the difficulties we have had in implementing our system will become easier in the future, as more people use the framework so that more practical examples are produced and best practices takes form.
2.1 A perfect/ideal Paxos instance. . . 11
3.1 Corums’ event loop architecture. . . 20
4.1 Distributed Dictionary system architecture. . . 24
4.2 Overview of our Multi-Paxos implementation. . . 29
4.3 Failure detector execution flow. . . 30
4.4 The structure of the Corums networking implementation we implemented. 32 5.1 Latency vs. throughput with a batch size of 64. Each data point is the average of three experiments. . . 40
5.2 Latency vs. throughput with a batch size of 1024. Each data point is the average of three experiments. . . 41
47
A.1 Load-test results with a batch size of 64. . . 50 A.2 Load-test results with a batch size of 1024. . . 50
48
3.1 Stream subscription example. . . 16
3.2 Stream operators example.. . . 17
3.3 Corums’ combination of reactive and object-oriented programming. . . 18
3.4 Corums network stream example. . . 19
3.5 Sending a message with Corums. . . 19
4.1 Defining an API endpoint in an ApiController class. . . 25
4.2 Calling a method through the CorumsFacade. . . 26 4.3 Setting up an event handler for receiving prepare messages on the acceptor. 27
49
Experimental Data
This appendix contains the measurements made for the performance testing of Distributed Dictionary. This data is used to create the plots in Section 5.3.2. All latency values in the data tables are averages calculated from running three tests with the same configured throughput.
Table A.1: Load-test results with a batch size of 64.
Throughput (commits/second) Latency (ms)
Table A.2: Load-test results with a batch size of 1024.
50
Distributed Dictionary Source Code
The source code for the Distributed Dictionary service can be found in this Github repository.
51
[1] Leslie Lamport. The part-time parliament.ACM Transactions on Computer Systems 16, 2 (May 1998), 133-169. Also appeared as SRC Research Report 49. This paper was first submitted in 1990, setting a personal record for publication delay that has since been broken by [60]., May 1998. URL https://www.microsoft.com/en-us/
research/publication/part-time-parliament/. ACM SIGOPS Hall of Fame Award in 2012.
[2] Leslie Lamport. Paxos made simple. ACM SIGACT News (Distributed Com-puting Column) 32, 4 (Whole Number 121, December 2001), pages 51–58, De-cember 2001. URLhttps://www.microsoft.com/en-us/research/publication/
paxos-made-simple/.
[3] Hein Meling and Leander Jehl. Tutorial Summary: Paxos Explained from Scratch.
In David Hutchison, Takeo Kanade, Josef Kittler, Jon M. Kleinberg, Friedemann Mattern, John C. Mitchell, Moni Naor, Oscar Nierstrasz, C. Pandu Rangan, Bern-hard Steffen, Madhu Sudan, Demetri Terzopoulos, Doug Tygar, Moshe Y. Vardi, Gerhard Weikum, Roberto Baldoni, Nicolas Nisse, and Maarten van Steen, ed-itors, Principles of Distributed Systems, volume 8304, pages 1–10. Springer In-ternational Publishing, Cham, 2013. ISBN 978-3-319-03849-0 978-3-319-03850-6.
doi: 10.1007/978-3-319-03850-6_1. URL http://link.springer.com/10.1007/
978-3-319-03850-6_1. Series Title: Lecture Notes in Computer Science.
[4] Scott Addie and Tom Dykstra. Create web APIs with ASP.NET Core, February 2020.
URLhttps://docs.microsoft.com/en-us/aspnet/core/web-api/. Library Cat-alog: docs.microsoft.com.
[5] IBM Global Services. Improving systems availability, August 1999. URL http:
//www.cs.cmu.edu/~priya/hawht.pdf.
[6] Per Christensson. Scalable definition, Januaury 2011. URL https://techterms.
com/definition/scalable.
52
[7] Coinbundle Team. Consensus algorithms, September 2018. URL https://medium.
com/coinbundle/consensus-algorithms-dfa4f355259d.
[8] Mark Brooker. Leader election in distributed systems, 2019. URL https://aws.
amazon.com/builders-library/leader-election-in-distributed-systems/.
Library Catalog: aws.amazon.com.
[9] Naohiro Hayashibara, Xavier Défago, Rami Yared, and Takuya Katayama. The ph accrual failure detector. Unknown, 01 2004. doi: 10.1109/RELDIS.2004.1353004.
[10] Tushar Chandra and Sam Toueg. Unreliable failure detectors for reliable distributed systems. Journal of the ACM, 43, 07 1999. doi: 10.1145/226643.226647.
[11] Nuno Santos and André Schiper. Optimizing paxos with batching and pipelining.
Theoretical Computer Science, 496:170–183, 07 2013. doi: 10.1016/j.tcs.2012.10.002.
[12] Thomas Stidsborg Sylvest, Hein Meling, Leander Jehl, and Veronica Estrada-Galiñanes. Corums - simplifying implementation of consensus algorithms. Unpub-lished, 12 2018.
[13] Yanhong Liu, Scott Stoller, and Bo Lin. High-level executable specifications of distributed algorithms. In Unknown, volume 7596, pages 95–110, 10 2012. doi:
10.1007/978-3-642-33536-5_11.
[14] What is .NET? An open-source developer platform., 2020. URL https://
dotnet.microsoft.com/learn/dotnet/what-is-dotnet. Library Catalog: dot-net.microsoft.com.
[15] T. E. Lea, L. Jehl, and H. Meling. Towards new abstractions for implementing quorum-based systems. In2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pages 2380–2385, 2017.
[16] D. Ongaro, J.K. Ousterhout, D.F. Mazières, M. Rosenblum, and Stanford University.
Computer Science Department. Consensus: Bridging Theory and Practice. Stanford University, 2014. URLhttps://books.google.no/books?id=xfLynQAACAAJ.
[17] Sebastian Pedersen, Hein Meling, and Leander Jehl. An analysis of quorum-based abstractions: A case study using gorums to implement raft. In Unknown, pages 29–35, 07 2018. doi: 10.1145/3231104.3231957.
[18] Clement Escoffier. 5 Things to Know About Reactive Programming, June 2017. URL https://developers.redhat.com/blog/2017/06/30/
5-things-to-know-about-reactive-programming/.
[19] dotnet bot. Task class (system.threading.tasks), 2020. URL https://
docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task. Li-brary Catalog: docs.microsoft.com.
[20] Bill Wagner. lock statement - C# reference, February 2020. URL https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/
keywords/lock-statement. Library Catalog: docs.microsoft.com.
[21] Christian Cachin, Rachid Guerraoui, and Luís Rodrigues. Introduction To Reliable And Secure Distributed Programming. Springer-Verlag Berlin Heidelberg, 2011. URL
https://www.springer.com/gp/book/9783642152597.
[22] Srikanth Sastry and Scott Pike. Eventually perfect failure detectors using add channels. InUnknown, pages 483–496, 08 2007. doi: 10.1007/978-3-540-74742-0_44.
[23] Why Docker? | Docker, 2020. URLhttps://www.docker.com/why-docker. Library Catalog: www.docker.com.
[24] .NET Core SDK - Docker Hub, 2020. URL https://hub.docker.com/_/
microsoft-dotnet-core-sdk/.
[25] ASP.NET Core 2.1/3.1 Runtime - Docker Hub, 2020. URL https://hub.docker.
com/_/microsoft-dotnet-core-aspnet/.
[26] Overview of Docker Compose, June 2020. URL https://docs.docker.com/
compose/. Library Catalog: docs.docker.com.
[27] ISO/IEC 25010. ISO/IEC 25010:2011, systems and software engineering — systems and software quality requirements and evaluation (square) — system and software quality models. Technical report, International Organization for Standardization, 2011. URL https://www.iso.org/standard/35733.html.
[28] G. Salvaneschi, S. Proksch, S. Amann, S. Nadi, and M. Mezini. On the positive effect of reactive programming on software comprehension: An empirical study. IEEE Transactions on Software Engineering, 43(12):1125–1143, 2017.
[29] Jan Kończak, Nuno Filipe de Sousa Santos, Tomasz Żurkowski, Paweł T. Woj-ciechowski, and André Schiper. Jpaxos: State machine replication based on the paxos protocol. Unknown, page 38, 2011. URL http://infoscience.epfl.ch/
record/167765.
[30] Nuno Santos, Jan Kończak, and Tomasz Zurkowski. JPaxos/JPaxos, April 2020.
URLhttps://github.com/JPaxos/JPaxos. original-date: 2011-05-16T07:00:54Z.
[31] Vaida. C# vs Java Tutorial: Find Out Difference Between C# and Java, May 2019. URLhttps://www.bitdegree.org/tutorials/c-sharp-vs-java/. Library Catalog: www.bitdegree.org Section: Comparisons.
[32] C# vs Java: Differences you should Know, June 2020. URL https://hackr.io/
blog/c-sharp-vs-java. Library Catalog: hackr.io.
[33] Google. The Go Programming Language, 2020. URLhttps://golang.org/.
[34] Naveen Ramanathan. Goroutines - Concurrency in Golang, July 2017. URL https://golangbot.com/goroutines/. Library Catalog: golangbot.com.
[35] Microsoft. What is NuGet and what does it do?, 2020. URL https://docs.microsoft.com/en-us/nuget/what-is-nuget. Library Catalog:
docs.microsoft.com.
[36] Stack Overflow. Stack Overflow Developer Survey 2020, 2020. URL https:
//insights.stackoverflow.com/survey/2020/?utm_source=social-share&
utm_medium=social&utm_campaign=dev-survey-2020. Library Catalog: in-sights.stackoverflow.com.
[37] Apache JMeter - Apache JMeter™, 2020. URLhttps://jmeter.apache.org/.
[38] Sebastian Mæland Pedersen. A practical analysis of the gorums framework: A case study on replicated services with raft. Master’s thesis, University of Stavanger, 2017.
URLhttp://hdl.handle.net/11250/2455424.
[39] Amazon. Amazon EC2, 2020. URL https://aws.amazon.com/ec2/. Library Catalog: aws.amazon.com.
[40] Symfony. Ten criteria for choosing the correct framework, 2020. URL https:
//symfony.com/ten-criteria. Library Catalog: symfony.com.
[41] Christian Varisco. How to choose a framework | Hacker Noon, October 2016. URL https://hackernoon.com/how-to-choose-a-framework-ea8b5b1e1f44. Library Catalog: hackernoon.com.