Introduction to Distributed Programming

Definitions

Distributed Computing is computing on a distributed system.

A Distributed System is a system of computers communicating via messages over a network so as to cooperate on a task or tasks. There’s no physical shared memory in a distributed system, though algorithms can simulate such a thing.

Areas of Study

Here are a few areas we’ll cover:

• Networks and Internets
• Distributed Algorithms and Transactions
• Paradigms
• Enterprise Computing
• Grid Computing
• Programming Languages

Networks and Internets

This is a big topic, so there’s a separate page of notes.

Distributed Algorithms

Distributed algorithms are designed for programming distributed systems. Unlike centralized algorithms, distributed algorithms are unaware of any global state or a global time frame. When designing such algorithms we have to consider:

• Modeling: transition systems, state diagrams, temporal logic
• Communication, Timing, and Synchronization
• Routing
• Virtual Circuits vs. Packet Switching
• Paradigms: wave algorithms, traversal algorithms, election algorithms, snapshot algorithms
• Transactions
• Distributed Termination Detection
• Distributed Deadlock Detection
• Distributed Failure Detection
• Fault Tolerance
• Stabilization

Distributed Computing Paradigms

Paradigms are the different approaches to structuring and programming distributed systems. Here are a few:

• Client-server
• Multi-tier
• Peer-to-peer
• Publish/subscribe
• RPC
• Distributed Objects
• Object Spaces
• Mobile Agents
• Network Services
• Groupware

Enterprise Computing

Enterprise applications are applications that run on large servers with multiple (simultaneous) users communicating over a network via clients like web browsers, PDAs, cell phones, or desktop applications. These applications generally read from and write to big databases.

Some people say enterprise applications are only for business functions (accounting, customer management, product tracking, etc.); some say any big distributed application counts as “enterprise”.

Enterprise Computing Platforms

In the old days, “big systems” like IMS and CICS were run on big mainframes, often programmed in COBOL. In the 1990s and early 2000s a couple new big players emerged: Java EE and .NET, with these features:

Enterprise Architectures

In the old days, and today for the most trivial of applications, we see client-server organizations.

Two tier architectures are almost always way too fragile. They soon gave way to three-tier architectures:

The idea here is that any one of the three layers can be completely re-implemented without affecting the others.

The middle layer completely isolates the front end from any knowledge of the database. The UI doesn’t even know what the data source is. It just makes calls like fetchCustomerById(24337).

Software running in the middle tier is called middleware. Middleware products are also called containers, since they host and manage the business objects. They can manage lifecycles, transactions, memory, authentication, concurrency, distribution, security, sessions, resource pooling, logging and lots of other “system-level plumbing things” so developers only have to concentrate on business logic.

There’s no need to stop at three tiers. You’ll often hear the term n-tier.

Sometimes applications are classified by the complexity of the client:

Grid Computing

The term grid computing refers to the computation of highly compute-intensive algorithms (protein folding, SETI, earthquake simulation, climate modeling) over many computers across administrative domains. Most of the computers run similar code; they’re all contributing bits toward the overall solution.

More at Wikipedia’s article on Grid Computing.

Language-Specific Distributed programming

This paper by Henri Bal, Jennifer Steiner, and Andrew Tannenbaum from 1989, covers dozens of languages. A good read!

Here is how they categorize the covered languages:

TODO TABLE

Naturally, a lot of new languages have arrived since then!

Mainstream languages in wide use tend to provide support for distributed programming both within the language itself and in libraries. Here is where to look:

Summary