Scalability in Network Monitoring and the Internet of things

The internet of things refers to the growing number of network connected devices (for better or worse) that have not traditionally been connected. Today our offices have network enabled telephones, thermostats, door locks, even coffee machines. As more and more devices are added to our networks, the network monitoring tools we use to make sure our devices are working the way they should need to be able to accept these new, and often novel, additions.

Networks are becoming increasingly more complex

When it comes to network devices, the internet-standard protocol for communicating with them is the Simple Network Management Protocol (SNMP). Despite being introduced in 1988 it is still going strong. With the advent of the more secure and robust SNMPv3, as well as the long lifecycles of network devices, it’s here to stay. Luckily, SNMP is fairly efficient and when implemented properly, a network monitoring solution need not create a significant footprint when communicating with your devices. Devices that use SNMP use files known as MIBs (Management Information Base) to identify the various variables that can be read or set, called Object Identifiers (OIDs). It is expected that MIBs from major manufacturers are included in a monitoring system to support the most common devices, but being able to compile MIBs to read OIDs is critical to being Internet of Things capable. Manually entering OIDs, which can often be poorly described, is a painstaking and often frustrating task.

There are of course other means of communication as well, for example by reading the content of log files and reacting, or accepting information via API. The modern network monitoring solution needs to offer a framework to monitor all the devices in your network, not just a select few.

Self-healing and automation are critical to network monitoring

As the number of interconnected devices increases, so do the number of potential problems that a systems administrator faces. Your network monitoring solution should allow you to be proactive instead of reactive, alert you before a problem happens, and give you the tools to let you automate as much of the corrections as possible. Doing things like running scripts, restarting services, and rebooting machines automatically can cut needless hours of work. Imagine you have a CRM service running on one of your servers. If that service were to go down you would want your monitoring service to restart it. If that doesn’t solve the issue, it could then reboot the server. Finally, if that still hasn’t solved the issue, then it would notify you of the problem.

Don’t invest in monitoring that won’t scale with you

Of course, as each device is added to your network, and the more you interact with it, the more strain that puts on your network as well as your monitoring solution. Typical solutions these days are limited to being able to monitor approximately 10,000 sensors, with some dropping even as low as 2,500 if you run the service on a virtual machine. For the small to medium sized business, this has long been sufficient, ad hoc work-arounds with remote collectors have so far been used to forestall the inevitable overload. However, if your remote sensors are feeding data into a central server with a central database, the bottleneck still remains at the central server and its database. Having separate servers with their own databases increases costs, network strain, and presents the issue of how to create and manage via a unified and coherent dashboard.

The NetCrunch network monitoring system’s main Atlas dashboard

The NetCrunch network monitoring suite presents an all-in-one solution to the monitoring and management requirements of the modern business, from small to enterprise. We have been monitoring the Internet of Things before it was even called that. Our clients use NetCrunch to monitor turnstiles at stadiums, refrigeration systems at food processing plants, and even underwater scientific equipment thousands of meters below the sea.

The University of Hawaii uses NetCrunch to monitor the Aloha Cabled Observatory

NetCrunch supports SNMP up to and including SNMPv3, contains thousands of pre-compiled MIBs, and most importantly, features a built-in MIB compiler as well. For operating systems and servers we use the on-board features like WMI in Windows or SSH in Linux. On top of that, our Open Monitor feature lets you feed data in from anywhere into NetCrunch and react at will. Our External Events window lets you send traps or logs directly to NetCrunch, and then decide how to react to each one specifically.

Alerting via SMS or E-Mail is of course standard, and expected of any modern network monitoring solution, but NetCrunch strives to stop alert floods and doesn’t want to silence the real problems in a sea of alert noise. We give you a powerful automation framework to customize that ensures you’re only alerted when you’re truly needed.

NetCrunch network monitoring in a multi-screen NOC setting

NetCrunch network monitoring has been built from the ground-up to focus on scale and efficiency, and our all-in-one design ensures all of our features work together seamlessly. We have our own embedded database to store your data, and keep recent data in memory to ensure dynamic, real-time, and lighting fast dashboards.

All of the views available in NetCrunch are also dockable, and the software is optimized for multi-screen NOC room settings. With 4K capable monitors, the view is especially impressive.

Testing NetCrunch on Mini PCs

So how well does it scale? While other solutions out there are limited to the low tens-of-thousands in terms of scalability, NetCrunch is in a completely different league altogether. A single server installation is tested to support over 650,000 individual monitored elements. No agents. No remote collectors. No disjointed databases. No noisy footprint. One server, one database, and a solution that leaves the rest of the market in the dust. This, like all of our development and testing, was also done on a virtual machine and doing so offers no performance limitations. In fact, we encourage running on a virtual machine.

While of course most installations won’t come near that, it does underline the incredible efficiency of the software. We tested NetCrunch in more typical installations and ran it on Mini PCs just to reinforce the point. Our testing showed that for a 250-node network, with approximately 7,000 sensors, NetCrunch only used 11% of an Intel i3 on average, and only 22% of the wholly pedestrian Celeron in a Gigabyte BRIX. While approaching the limits of many solutions, we didn’t even break a sweat.

The network monitoring Renaissance is here with NetCrunch

We’ve put over a decade of expertise into building the NetCrunch network monitoring suite. More than we could ever write about in a single article. Our all-in-one network, server, application, file, log and web monitoring suite is a monument to efficiency usability. We encourage you to give it a try, and see what network monitoring should look like today. We are Cisco, VMWare and Microsoft partners, and support hundreds of devices and applications out-of-the-box.

Setup only takes minutes. Download our trial and get not only 30 days of installation assistance for free, but we’ll also help you set it up and get a proof-of-concept going via live webinar with a technician. Your network will thank you.

AdRem Software makes award-winning commercial and freeware network monitoring & management software trusted by thousands of admins worldwide

AdRem Software makes award-winning commercial and freeware network monitoring & management software trusted by thousands of admins worldwide