Home

Distributed Hash Table

In Networks & Distributed Systems class, we were assigned to write a Distributed Hash Table (DHT) in Python . The DHT protocol was provided by Dr. Nathan Backman of Buena Vista University.

Originally written with @jackmford

DHT

DHT chord route This is easily in my top three favorite projects/assignments at BVU.

Post college, I spent time refining the code and creating a Python interface to the main DHT code.

Best part

Implementing the ownership algorithm (how the nodes determine who owns a specific hash) was the most challenging but rewarding piece.

So, what is it??

TL;DR

Decentralized storage using ( Key, Value ) pairs.

Usage

The entire ring is reachable from any entry point (a node [computer participating in the DHT]). Then the user can store ( Key, Value ) pairs within the system that can be retrieved again later.

As mentioned on the Chat Channels page, benefits of a DHT are:

  • Data persists as long as the DHT does
    • being decentralized, data shifts owners as nodes enter and exit the DHT
  • Decentralization means no single computer is in charge of the system
    • nodes communicate and adjust ownership accordingly
  • K-Safety (not implemented here)
    • storing backups of data with predecessors in case sudden removal of a node

Explanation

I think Hazelcast’s definition explains it best:

Hazelcast dht explanation

Visualization

dht visualization

Data Types

Data types our DHT adhered to.

  • Key/hash: SHA-1, 160-bit, 20-byte UTF-8
    • Key used to lookup values in the DHT
  • valSize: 8-byte, big endian
    • valSize is the size of the data being stored
  • peerAddress: [4-byte ip][2-byte port big endian]
  • peerHash/key: SHA-1 hash of “IP:Port”
    • Example: SHA1( “192.168.0.1:5000” )

Operations

A few of our DHT operations:

Insert - Add/update/replace an item to/in the DHT
Remove - Removes an item from the DHT
Get - Retrieves an item from the DHT
Exists - Check to see if an item exists within the DHT
Owns - Asks a peer who they know to be the nearest owner to a key
...

Thanks

Thanks for reading about my experience with the Distributed Hash Table.

Best.

Python icon by Icons8
DHT chord visualization
DHT node visualization