Technical Details
MAD is a storage cluster built using GlusterFS
- Each storage node or "brick" is built on ThinkMate commodity Storage servers running CentOS Linux
- A combined storage and server platform
- Runs Linux on x86 hardware in each brick
- High-speed storage controllers used to achieve raid-6 level redundancy and unification of the storage disks
- Gateway nodes running CentOS Linux
- Provides account management
- File sharing via SCP/SFTP/Samba and rsync
- Dedicated gateway nodes, one for replicated the other for non-replicated shares
- Nodes share a dedicated infiniband switch for high-speed/low-latency back end communications
- No central metadata server(s) to fail or maintain
- Metadata is stored at the storage node file system level
- Performance should scale with the system
- As more storage nodes are added, the average work per storage node decreases
- More gateways can easily be supported as the number of storage nodes increases
- The nodes run the Gluster server, the gateway runs the Gluster client to mount the storage volume and share it out to end users
- Hard disk failure statistics
Disaster Recovery
- When disks fail, raid-6 prevents data loss
- Bad disks are replaced and raid-6 recovers from parity information
- The system can sustain two disk failures in one disk group without data loss
- There are 2 disk groups within each brick
Node Failure
- Typically node failures do not result in disk failures
- During node repair, non-replicated shares will be unavailable
Gateway Failure
- Gateway failure results in a service interruption until the gateway is restored
- No data loss occurs, even when a complete rebuild of the gateway node is required
Performance
Typical performance for clients connecting to gigabit network links has been between 25 and 60 megabytes per second. This is typically limited by client system performance and file size: larger files tend to transfer faster than a large number of smaller files