Smart home as a service: ninja style
Smart homes are becoming more and more popular with the introduction of a high number of Internet of Things (IoT) applications (heating, ventilation, multimedia and alarm systems, lighting, home appliances etc.) and respective smart devices.
Many of them being connected directly to the internet for making remote management feasible. Connectivity to the internet together with limited processor capacity of smart home devices and increased popularity of such systems introduces serious challenges to security, privacy and to efficient management of the smart home.
The devices used for smart home implementation usually have limited hardware resources, computing power and memory. This makes it very difficult or impossible to implement complex algorithms that ensure security and management.
- From my point of view the main design assumptions of centralised smart home management system includes:
- The ability to remotely configure and manage smart home devices using a unified programming interface.
- Possibility to configure, control, collect and store data from end nodes.
- Ensuring system security and continuous availability of smart home devices from Internet.
- Using protocols that ensure security and privacy, and user authentication should be carried out in a way that prevents access to the system by unauthorised persons.
All the requirements mentioned above make the construction of hardware for smart home management exciting. But let's make it more exciting! I genuinely believe that the measure of technical improvements is related to the level how an engineer is respecting own free time. The improvement of home to be smart are bring about a lot of devices. All these devices are requiring support. Thinking over I came up with the idea to consolidate the hardware for centralised smart home management with home media centre. Maybe it sounds strange but it isn't!
- The main requirements for both hardware’s are:
- Allow you to continuously access and manage your data/media content.
- Redundancy, the possibility to store data on more than one drives.
- The ability to access for different level of users on safe way from internet.
We live in era of a data-driven culture. When we need data, we need them right away, whether it is last month’s electricity consumption or photos from last year's vacation. At both case the easy accessibility is desirable.
Writing to two or more disks at the same time. Having the same data stored on separate disks enables the data to be recovered in the event of a disk failure without resorting to expensive data recovery techniques.
User management describes the ability for us to manage access to various data such as media content like movies, family albums or smart home applications and more. User management is a core part to any service and is a basic security essential for any home (home as organisation).
- Looking ahead there are some additional, technical, requirements:
- Maintenance, repair and operations have to consume a possible minimal time.
- Rebuilding of device, in case if needed, it has to be easy and possible on low cost. So, any special hardware is not acceptable.
- The file systems have to be fault tolerant, stripe and replicate data over multiple drives for high performance and to maintain data integrity.
- Possibility of taking snapshot of important data.
So, the task is given let's see how I solved it!
Considering the above presented requirements such as easy maintenance and reproducibility of hardware also taking price and compute power into account, I decided to buy a used HPE MicroServer. The compatibility, power and possibilities to expand the devices are impressed me.
As first step I removed the physical RAID card because any special hardware is not acceptable. Also, the requirement for file system are forcing solution without physical controller. I decided to use ZFS.
ZFS protects data by enabling volume management on file-system level. This feature makes “Copy on Write” technology possible. When a block of data is altered, it will change its current location on the disk before the new write is finished. If your system crashes or loses power in the process, that data would be lost or damaged. ZFS does not change the location of the data until the write is completed and verified, thus keeping your data safe in case of a system crash. To verify data integrity, ZFS uses check-sums to ensure that the data remains original from write to write.
ZFS also allows to send writes to individual physical disks, instead of just the RAID volume. Because of this, ZFS can stripe writes across RAID volumes, which is speeding up write performance. In the case you need to sync mirrors with only a little bit of information, you do not have to wait for it to sync any of the empty disk space, which can take a good amount of time. ZFS incorporates algorithms to ensure that your most recently used and most frequently used data are kept in the fastest system storage media. Spinning disks are known to be slow and SSD drives come at a very high price compared to regular disks. By using these algorithms in combination with flash-based ZFS write cache and L2ARC read cache devices, you can speed up your performance. Other great feature of ZFS are the intelligently designed snapshot, clone, and replication functions. ZFS snapshots only update based on what has changed since the last snapshot. This means that clone and replication tasks are less time consuming compared to traditional replication technology.
- A configured ZFS pool can easily be changed in its size to accommodate a growing need for more storage. The pool can be upgraded step by step with larger disks, without compromising the systematise or complicated procedures. Hard disks can even be added on different physical ports or in a changed order in a new computer system, as long as the ZFS version on the target system is the same or higher. The current rule of thumb when making a ZFS RAID is:
- MIRROR - requires two to four disks or more.
- RAIDZ-1 - requires five disks or more.
- RAIDZ-2 - requires six disk or more.
- RAIDZ-3 - requires eleven disks or more.
- The HPE MicroServer contains four slots for SATA drives. But there are some more possibilities such as place for optical drive and a hidden USB 3.0 port on motherboard. It means there are six places for drives. The main details of hardware in my case are:
- Intel Xeon E3-1260L
- 2 x 4TB WD Red HDD
- 4 x 120GB SSD
- 16GB RAM
Considering the rule of thumb for ZFS RAID and the possibility to speed up the read, I made following decision:
- 1st slot: SSD for main system (yellow)
- 2nd slot: one of two SSD for ZIL and L2ARC (blue)
- 3rd and 4th slots: HDD for storage (red)
- Optibay caddy: another SSD for ZIL and L2ARC (blue)
- HDD case with USB 3.0: mirror of main system (yellow)
The core system is installed on SSD in 1st slot and it is mirrored with SSD which is connected over USB (yellow drives). I know, over USB the boot is slower, but my opinion is that the slower boot is better than lose time when the main SSD breaks.
- Using whole physical disks is the recommended way to create ZFS storage pools. But I think it is expensive to waste an entire SSD for logs and cache. Before building of NAS I made some tests and ZFS worked fine in case when I created partitions. So, on both 'blue' SSDs I created the below described three partitions:
- The smallest partition is for log - ZIL
- The second partition is for cache - L2ARC
- The remaining part of disk is for Docker containers and Jails.
- The sizing of ZIL and L2ARC partitions are an exciting tech challenge. After studying the documentations of ZFS I made following decision:
- The ZIL size will be half of RAM size. In my case it is 8~10GB.
- L2ARC will be 4*ZIL. In my case it is ~40GB.
The partitions for ZIL are mirrored. It’s given more security. The partitions for L2ARC are striped. So, it’s meant the full size of cache in my case is 80GB. The remaining part of volume is mirrored (60GB). Later this part will be dedicated for services such as Plex and OpenHUB. In ZFS, you can enable compression at the file-system level. This will store the data in compressed format, which will save lot of disk space. LZ4 provides the best compression with the lowest CPU overhead. In my case the partitions of both “blue” drives are:
root@NAS[~]# gpart show ada1 => 40 234441568 ada1 GPT (112G) 40 125829120 1 freebsd-zfs (60G) 125829160 83886080 2 freebsd-zfs (40G) 209715240 20971520 3 freebsd-zfs (10G) 230686760 3754848 - free - (1.8G) root@NAS[~]# zpool create -o altroot=/mnt compute mirror ada1p1 ada4p1 root@NAS[~]# zfs set compress=lz4 compute
Creation of ZFS pool for storage is similar than it was for compute. The only difference is that ZIL and L2ARC are added to improve the performance. This is the storage of NAS it is named as ”storage”:
root@NAS[~]# zpool create -o altroot=/mnt storage mirror ada2 ada3 root@NAS[~]# zfs set compress=lz4 storage root@NAS[~]# zpool add storage log mirror ada1p3 ada4p3 root@NAS[~]# zpool add storage cache ada1p2 ada4p2
- Verifying the installation, you will see a similar output with following ZFS pools:
- freenas-boot - main system
- compute - dedicated for services
- storage - the storage with logs and cache
root@NAS[~]# zpool status pool: freenas-boot state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM freenas-boot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada0p2 ONLINE 0 0 0 da0p2 ONLINE 0 0 0 errors: No known data errors pool: compute state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM compute ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada1p1 ONLINE 0 0 0 ada4p1 ONLINE 0 0 0 errors: No known data errors pool: storage state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada2 ONLINE 0 0 0 ada3 ONLINE 0 0 0 logs mirror-1 ONLINE 0 0 0 ada1p3 ONLINE 0 0 0 ada4p3 ONLINE 0 0 0 cache ada1p2 ONLINE 0 0 0 ada4p2 ONLINE 0 0 0 errors: No known data errors
The above presented architecture has a redundant storage with improved read performance. In case of emergency both “red” data disks could be removed and easily attached to any other hardware while keeping your data completely secure.
The storage pool named as “compute” has no improved read performance. It does not need it because it used for containers such as Jail and Docker. This is the place where the services for smart home and media centre are live.
The main OS is FreeNAS. It is FreeBSD based system. FreeNAS is a free and open source network attached storage (NAS) software appliance. This means that you can share data over file-based sharing protocols, including CIFS, NFS and AFP. FreeNAS uses the ZFS file system to store, manage, and protect data.
FreeNAS has capability to install additional components for other purposes, such as media streaming servers (Plex) or platform to run containers (Rancher) and cloud backup services. Also, it has capability to create virtual machines. So, it can fulfil many roles of a home server as well. I take advantage of these roles for implementation of centralised management solution for smart home.
Rancher implements a portable layer of infrastructure services designed specifically to power containerised applications. Rancher infrastructure services include networking, storage, load balancer, DNS, and security. Rancher infrastructure services are typically deployed as containers themselves.
The Jail mechanism is an implementation of FreeBSD's OS-level virtualisation that allows partitioning a FreeBSD-derived computer system into several independent mini-systems called jails, all sharing the same kernel, with very little overhead.
I prefer containers instead of virtual machines. For home management platform I choose Docker based solutions. OpenHUB is implemented as centralised management solution of smart home devices. OpenHUB is an open source home automation platform. It is vendor and technology agnostic. It is developed in Java, which gives you endless devices where you can run the system. Its architecture is based on bindings that bring support for different smart home devices.
Also, inside of Rancher I deployed Traefik and Shinobi. Traefik is an open-source Edge Router that makes publishing your services on Internet easy experience. It receives requests on behalf of your system and finds out which components are responsible for handling them. Further advantage of Traefik is that use ACME provider such as Let's Encrypt for automatic certificate generation. Shinobi is open source, written in Node.js, and really easy to use. It is a powerful solution for CCTV and NVR. The security recordings are saved on the storage pool.
Regarding the media server I solved it with FreeNAS plugin for Plex. It runs inside a Jail container on compute pool and over NFS are reaching the media content on storage pool. The reason why media server is in Jail is that my goal was to totally separate the two functionalities of each other. It made improvement of solutions for smart home more convenient.
Keep calm and enjoy
The first benefit of using a NAS is that all your important files can be stored in a central location, allowing you to access them from multiple devices simultaneously, while also keeping your backup and redundancy resources in one place.
The possibilities of HPE MicroServer made the maintenance of physical hardware easy and effective. The management port and remote console access are huge advantage. It means that hardware maintenance is possible over network from any place of the World. So, the upgrade of core system could be performed from beach as well.
ZFS are doing a great job regarding data security. ZFS natively manages both disks and volumes, with solutions similar to those provided by hardware RAID technologies. Integrity and reliability are the pillars of the ZFS. Snapshots, which are natively available, are a very useful feature as well.
The upgrade of FreeNAS is also work out of the box. The whole process is well thought out and work as a charm. The architecture design, the separated pool for system, the improved read performance and the mirrored data disks are further improving system reliability. So, the time required for maintenance can be minimised and planned. Thanks to the architecture the continuous availability of services is guaranteed. The management with endpoints and users is also handled on efficient way. There are bunch of possibilities to grant access to different data and services on the secure way.
And finally, the whole system is built with open source solutions without any special device. In case that you need to rebuild the system, you can solve it on cost efficient way without any dependency.
Smart home as a service: ninja style