Cloud Computing (Spring 2020)

In the modern Internet, virtually all large Web services run atop multiple geographically distributed data centers: Google, Yahoo, Facebook, iTunes, Amazon, eBay, Bing, etc. Services must scale across thousands of machines, tolerate faults, and support thousands of concurrent requests. Increasingly, the major providers (including Amazon, Google, Microsoft, HP, and IBM) are looking at "hosting" third-party applications in their data centers - forming so-called "cloud computing" services. A significant number of these services also process "streaming" data: geocoding information from cell phones, tweets, streaming video, etc.

This course, aimed at a sophomore with exposure to basic programming within the context of a single machine, focuses on the issues and programming models related to such cloud and distributed data processing technologies: data partitioning, storage schemes, stream processing, and "mostly shared-nothing" parallel algorithms.

Date	Topic	Details	Reading	Homework
Week 2	Introduction Google Cloud Part 1	Course overview Google File System(GFS): distributed storage system MapReduce: a programming model and an associated implementation for processing and generating large data sets with a parallel, distributed algorithm on a cluster Chubby: lock service for loosely-coupled distributed systems	Case Studies: NY Times article Edge Computing: Vision and Challenges (Presentation 1) Armbrust et al.: A View of Cloud Computing (Presentation 2) Sanjay et al.: The Google File System Dean and Ghemawat: MapReduce: Simplified Data Processing on Large Clusters
Week 3	Google Cloud Part 2 Presentation and Discussion	Bigtable: a distributed storage system for structured data; scalable, highly available storage system Megastore: distributed systems tracing infrastructure	Fay et al.: Bigtable: A Distributed Storage System for Structured Data (Presentation 3)
Week 4	Google Cloud Part 3 Presentation and Discussion	Dapper: a Large-Scale Distributed Systems Tracing Infrastructure Google App Engine: a platform for building web apps and mobile backends that automatically scale	Benjamin et al.: Dapper, a Large-Scale Distributed Systems Tracing Infrastructure (Presentation 4)
Week 5	Amazon Cloud Presentation and Discussion Experiment 1: set up Hadoop	Dynamo: a highly available key-value storage system EC2, S3 and SQS Simple DB: a simple database storage solution that allows developers to simply store & query data items via web services requests, saving time solutions	Subramanian et al.:f4: Facebook’s Warm BLOB Storage System(Presentation 5)
Week 6	Hadoop Presentation and Discussion Experiment 2: HDFS commands & APIs	Basics: Data types, drivers, mappers, reducers HDFS; dataflow in Hadoop Fault tolerance in Hadoop Programming model of Spark	Matei et al.:Spark: Cluster Computing with Working Sets(Presentation 6) Spanner: Google’s Globally-Distributed Database, OSDI 2012 (Presentation 7)
Week 7	Experiment 3: Process data with Hadoop/Mapreduce Presentation and Discussion Spark	Set up Hadoop Learn how to use HDFS	Kim et al.:Database High Availability Using SHADOW Systems(Presentation 8)	HW1
Week 8	Virtualization Experiment 4: Spark Presentation and Discussion	Learn how to use Spark	Zohreh et al.:Heterogeneity in Mobile Cloud Computing: Taxonomy and Open Challenges(Presentation 9)	HW2
Week 9	OpenStack Presentation and Discussion	Virtualization: Concepts and Technologies Introduction to OpenStack	Wei et al.:Managed Communication and Consistency for Fast Data-Parallel Iterative Analytics (Presentation 10) Alsheikh et al.:Mobile Big Data Analytics Using Deep Learning and Apache Spark (Presentation 11) Moritz et al.:SPARKNET: TRAINING DEEP NETWORKS IN SPARK (Presentation 12) UDDIN et al.:Human Action Recognition Using Adaptive Local Motion Descriptor in Spark (Presentation 13)