Joint work with Dr. Bettina Kemme and Dr. Jörg Kienzle (McGill University)
Cloud-Based Publish/Subscribe systems are very useful when it comes to delivering real-time information quickly to interested parties. Publish/subscribe allows for decoupling content producers (publishers) from content consumers (subscribers).
In topic-based publish/subscribe [1], publishers send publication messages to a given publication server, tagged with a specific topic. All subscribers subscribing to that specific topic then receive the publication.
Large-scale topic-based publish/subscribe have wide practical applications in many areas such as social networks, push notification infrastructures and chat frameworks, amongst others. The flexibility of such systems make them easily adaptable to a various scenarios, such as large-to-epic-scale games,since specific areas of the world can be represented as « topics », and interested players and other in-game entities can publish and receive relevant state updates. A desired property is for these systems to be able to scale in real-time according to the current load. Using the Cloud allows for vertical scalability while paying only for the resources that are used.
Dynamoth: Cloud-based Scalable Publish/Subscribe
Dynamoth [2,3], a scalable, Cloud-based publish/subscribe platform that automatically scales in real-time to react to variations in experienced load. Dynamoth is tailored towards latency-constrained applications such as immersive large-scale games. It supports “per-topic”, “per-client”, fine-grained real-time load monitoring and load balancing, in order to make efficient use of Cloud resources and scale up and down whenever needed. Dynamoth also supports different horizontal scalability models in order to support arbitrarily popular topics (for instance, a very popular area in a game or a very popular microblogging feed).
Extensive experiments were run on the Amazon EC2 Cloud, in the context of large-scale massive multiplayer online games, with over 1000 AI-controled players and several thousands publish/subscribe topics, in order to assess the scalability of our approach while respecting the tight constraints of latency-constrained applications.
[1] Eugster, P.T., Felber, P.A., Guerraoui, R., Kermarrec, A.M.: The many faces of publish/subscribe. ACM Comput. Surv. 35(2), 114–131 (2003)
[2] Gascon-Samson, J., Garcia, F.-P., Kemme, B., Kienzle, J. (2015) Dynamoth: A Scalable Pub/Sub Middleware for Latency-Constrained Applications in the Cloud, International Conference on Distributed Computing Systems (ICDCS 2015), Columbus, USA [Acceptance ratio: 13%]
[3] Gascon-Samson J., Kemme, B., Kienzle, J. (2013) Lamoth: A Message Dissemination Middleware for MMOGs in the Cloud, Annual Workshop on Network and Systems Support for Games (NetGames 2013), Denver, USA
MultiPub: Latency and Cost-Aware Global-Scale Cloud Publish/Subscribe
Many Cloud providers have data centers located in different geographical regions. Supporting global-scale applications with low-latency needs (such as world-scale games) relying on a publish/subscribe paradigm brings many challenges.
MultiPub [1] aims at proposing a fine-grained model for minimizing latencies in the context of wide-area topic-based pub/sub applications. MultiPub takes advantage of multiple “Clouds” in different geographical regions in order to respect a predefined delivery time bound while minimizing Cloud-based costs. MultiPub includes a prediction model for latencies between arbitrary pairs of clients and publication servers. Latency optimization is done on a per-topic basis where topics are seamlessly migrated or replicated between Clouds, whenever needed.
A simulator tool was built in order to perform large-scale experiments, which we plan to open-source in the near-future. Furthermore, a Cloud-based implementation taking advantage of multiple Amazon EC2 regions was also built in order to assess the validity of our results in a real multi-Cloud setting. Our results showed that MultiPub was able to significantly reduce bandwidth-related costs while respecting predefined delivery time constraints, and that it performed better than other approaches.
[1] Gascon-Samson, J., Kemme, B., Kienzle, J. (2017)MultiPub: Latency and Cost-Aware Global-Scale Cloud Publish/Subscribe, ICDCS 2017, Atlanta, USA