Cloud Computing Concepts: Part 2

About this course: Cloud computing systems today, whether open-source or used inside companies, are built using a common set of core techniques, algorithms, and design philosophies – all centered around distributed systems. Learn about such fundamental distributed computing "concepts" for cloud computing. Some of these concepts include: clouds, MapReduce, key-value/NoSQL stores, classical distributed algorithms, widely-used distributed algorithms, scalability, trending areas, and much, much more! Know how these systems work from the inside out. Get your hands dirty using these concepts with provided homework exercises. In the programming assignments, implement some of these concepts in template code (programs) provided in the C++ programming language. Prior experience with C++ is required. The course also features interviews with leading researchers and managers, from both industry and academia. This course builds on the material covered in the Cloud Computing Concepts, Part 1 course.

Who is this class for: Who this class is for: This course is intended for students with similar backgrounds as junior or senior undergraduates in computer science. This course will teach you basic algorithmic and design concepts for distributed systems, as used in today’s cloud systems. Much of the course, including quizzes, is conceptual and not programming oriented. The programming assignment assumes some knowledge of C++ (if you have only Java experience, you should be able to pick up C++ quickly may suffice), and allow you to write distributed algorithms in an emulated distributed system on your own machine. To ensure you have the necessary prerequisites, you need to take the prerequisite quiz and achieve a high passing score (at least 90%, preferably 100%). Based on prior student experiences, the Linux/Unix environment may work better for the programming assignments than Windows. Who this class is NOT for: This course is NOT intended for those: (1) wishing to get a high level overview of cloud computing (you can use Wikipedia for that); (2) wishing to do detailed programming in a real cloud (the Cloud Capstone and Cloud Applications MOOCs provide you that); (3) who are averse to theoretical and algorithmic concepts; (4) with little or no prior programming experience in C++ or Java; (5) who expect to see industry-quality code in programming assignments (these are play programming assignments focused on allowing you to implement concepts you learn from lectures, and are not intended for immediate deployment); (6) those not familiar with setting up IDEs/compilers, etc., especially for C++; or (7) those who want to rush through lectures and/or videos and attempt quizzes in haste (quizzes are hard, so make sure you view lectures completely and comprehensively before attempting quizzes).

Created by:  University of Illinois at Urbana-Champaign

• Taught by:  Indranil Gupta, Associate Professor

  Department of Computer Science

Basic Info Course 2 of 6 in the Cloud Computing Specialization Commitment 5 weeks of study, 5 - 10 hours/week Language English How To Pass Pass all graded assignments to complete the course. User Ratings 4.6 stars Average User Rating 4.6See what learners said Coursework

Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.

Help from your peers

Connect with thousands of other learners and debate ideas, discuss course material, and get help mastering concepts.

Certificates

Earn official recognition for your work, and share your success with friends, colleagues, and employers.

University of Illinois at Urbana-Champaign The University of Illinois at Urbana-Champaign is a world leader in research, teaching and public engagement, distinguished by the breadth of its programs, broad academic excellence, and internationally renowned faculty and alumni. Illinois serves the world by creating knowledge, preparing students for lives of impact, and finding solutions to critical societal needs.

Syllabus


WEEK 1


Week 1: Course Orientation and Classical Distributed Algorithms Continued



Lesson 1: To coordinate machines in a distributed system, this module first looks at classical algorithms for electing a leader, including the Ring algorithm and Bully algorithm. We also cover how Google’s Chubby and Apache Zookeeper solve leader election. Lesson 2: This module covers solutions to the problem of mutual exclusion, which is important for correctness in distributed systems with shared resources. We cover classical algorithms, including Ricart-Agrawala’s algorithm and Maekawa’s algorithm. We also cover Google’s Chubby support for mutual exclusion.


11 videos, 7 readings, 1 practice quiz expand

1. Reading: Orientation Overview
2. Video: Introduction to Cloud Computing Concepts, Part 2
3. Reading: Syllabus
4. Reading: About the Discussion Forums
5. Practice Quiz: Orientation Quiz
6. Video: Orientation Towards Cloud Computing Concepts: Some Basic Computer Science Fundamentals
7. Discussion Prompt: Getting to Know Your Classmates
8. Reading: Course Learning Community and Social Media
9. Reading: Week 1 Overview
10. Video: Week 1 Introduction
11. Video: 1.1. The Election Problem
12. Video: 1.2. Ring Leader Election
13. Video: 1.3. Election in Chubby and ZooKeeper
14. Video: 1.4. Bully Algorithm
15. Video: 2.1. Introduction and Basics
16. Video: 2.2. Distributed Mutual Exclusion
17. Video: 2.3. Ricart-Agrawala's Algorithm
18. Video: 2.4. Maekawa's Algorithm and Wrap-Up
19. Reading: Homework 1 Instructions
20. Discussion Prompt: Homework 1 Discussion
21. Reading: Programming Assignment Instructions

Graded: Homework 1

WEEK 2


Week 2: Concurrency and Replication Control



Lesson 1: Transactions are an important component of many cloud systems today. This module presents building blocks to ensure transactions work as intended, from Remote Procedure Calls (RPCs), to serial equivalence for transactions, to optimistic and pessimistic approaches to concurrency control, to deadlock avoidance/prevention. Lesson 2: This module covers how replication – maintaining copies of the same data at different locations – is used to provide many nines of availability in distributed systems, as well as different techniques for replication and for ensuring transactions commit correctly in spite of replication.


8 videos, 2 readings expand

1. Reading: Week 2 Overview
2. Video: Week 2 Introduction
3. Video: 1.1. RPCs
4. Video: 1.2. Transactions
5. Video: 1.3. Serial Equivalence
6. Video: 1.4. Pessimistic Concurrency
7. Video: 1.5. Optimistic Concurrency Control
8. Video: 2.1. Replication
9. Video: 2.2. Two-Phase Commit
10. Reading: Homework 2 Instructions
11. Discussion Prompt: Homework 2 Discussion

Graded: Homework 2

WEEK 3


Week 3: Emerging Paradigms



Lesson 1: We study the emerging area of stream processing, touching on key design aspects of Apache Storm. Lesson 2: We study how enormous graphs can be processed in clouds. Lesson 3: We study various types of networks/graphs that are both natural and artificial, and their surprising commonalities. Lesson 4: This module presents classical scheduling algorithms that have been used in operating systems since the inception of computers. We then cover two popular scheduling algorithms for Hadoop.


8 videos, 2 readings expand

1. Reading: Week 3 Overview
2. Video: Week 3 Introduction
3. Video: Stream Processing in Storm
4. Video: Distributed Graph Processing
5. Video: Structure of Networks
6. Video: 4.1. Single-processor Scheduling
7. Video: 4.2. Hadoop Scheduling
8. Video: 4.3. Dominant-Resource Fair Scheduling
9. Video: Storm Demo
10. Reading: Homework 3 Instructions
11. Discussion Prompt: Homework 3 Discussion

Graded: Homework 3

WEEK 4


Week 4: Classical Systems



Lesson 1: When files and directories are stored/accessed over the network, it is called a distributed file system. This module covers the working of distributed file systems like NFS and AFS. Lesson 2: This module covers Distributed Shared Memory systems, their techniques, and pros/cons. Lesson 3: This module looks at the area of sensor networks, starting from what’s inside a sensor mote and how networks of them work.


6 videos, 2 readings expand

1. Reading: Week 4 Overview
2. Video: Week 4 Introduction
3. Video: 1.1. File System Abstraction
4. Video: 1.2. NFS and AFS
5. Video: Distributed Shared Memory
6. Video: Sensor and Their Networks
7. Video: Interview with Brighten Godfrey
8. Reading: Homework 4 Instructions
9. Discussion Prompt: Homework 4 Discussion

Graded: Homework 4

WEEK 5


Week 5: Real-Life Behaviors



Lesson 1: This module is a primer on basic security concepts, not just applied to distributed systems, but also more generally. We study various policies and mechanisms, including encryption, authentication, and authorization. Lesson 2: This module presents case studies of real datacenter outages, and attempts to draw lessons on how to prevent them and how to better prepare for them.


11 videos, 3 readings expand

1. Reading: Week 5 Overview
2. Video: Week 5 Introduction
3. Video: 1.1. Basic Security Concepts
4. Video: 1.2. Basic Cryptography Concepts
5. Video: 1.3. Implementing Mechanism using Cryptography
6. Video: 2.1. What Causes Disasters?
7. Video: 2.2. AWS Outage
8. Video: 2.3. Facebook Outage
9. Video: 2.4. The Planet Outage
10. Video: 2.5. Wrap-Up
11. Video: Interview with Paul Kwiat
12. Reading: Homework 5 Instructions
13. Discussion Prompt: Homework 5 Discussion
14. Video: Conclusion to Cloud Computing Concepts, Part 2
15. Reading: Final Exam Instructions
16. Discussion Prompt: Final Exam Discussion
17. Discussion Prompt: Final Reflection

Graded: Homework 5
Graded: Fault-Tolerant Key-Value Store
Graded: Final Exam